Segmenting Chinese words for the index of a book

Question

I am trying to make a strict index of individual words for a large work of literature, using Chinese word segmentation, such as are found at the back of reference books. . e.g. "大星............ found on pages 55,65,109,2202", etc.

A few sections left unsegmented by the programme were:

1. Numbers, e.g. 一千三百三十五. In English, in words this would be indexed as: "One" "thousand" "three" "hundred" "thirty" "five". Are large numbers considered a separate 'word', or should these be separated into characters?

2. Words with repeated characters. I have read a few good posts about repeated characters. Are these considered separate words, or a variant of the original? So, would 孤孤單單 be listed as a separate word to 孤單, and 子子孫孫 separately to 子孫?

Cheers.

Answer 1

I think it is up to you that how you wish the index to lead the reader to find a word, or a phrase quickly. Usually, I don't think you need to index a number or its parts. But if you feel it is necessary, you can break 一千三百三十五 down to:

"一千", "三百", "三十五", or

"一", "千", "三", "百", "十", "五".

For phrase with a repeated word:

"孤孤單單", "孤單", "孤", "單".