I am doing some work with scanning Chinese text with my computer and I have run into a problem that I do not have with other languages.

Because there are no spaces between characters, I cannot tell when one word ends and another one begins. Is there any easy way to do this? For example in English I am telling my computer to split things like this:

sentence = "My name is John"
the computer will return a list of words.
computer = ['my', 'name', 'is', 'John']
  • 1
    No easy way, no. Sometimes a character can belong to the character before it, be on its own, or belong to the character after it, and it will change the meaning ...
    – Ming
    Dec 2, 2015 at 2:09
  • 1
    You'd probably need to match it against some sort of database.
    – Mou某
    Dec 2, 2015 at 2:31
  • 6
    Chinese word segmentation (中文分词) is a difficult problem in the field of natural language processing (NLP) that has been studying for a long time. There're a few solutions such as Stanford Word Segmenter, MMSEG, and Sphhinx.
    – Stan
    Dec 2, 2015 at 6:53

1 Answer 1


Short of a lexicographic analysis, you can't. There is a little punctuation in Chinese, so you can split sentences into subgroups. But after that, there's no easy way to say where a word stops.

Even very common words like 的 (genitive marker, appears right after the possessor, eg 我的朋友: me, I; genitive; friend: my friend) cannot be relied on 100% to be always just that. 'Taxi' is written 的士, and in this case 的 would not be a single character delimiting two words, but the first character of a word.

And Chinese is not the only language that doesn't have spaces. You'll run into more problems like this.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.