2

Where to find an authoritative and/or verified list of examples for correct (and incorrect) Chinese to Pinyin conversions for words and sentences?

I am specifically looking for corner cases when automated Chinese (simplified/traditional) conversion to pinyin might fail so that I can build a test case to verify correctness.

For example, the character could be translated to pinyin as any of ne5,ni2,na4, or ne4 and it looks like some libraries are getting it wrong in certain contexts.

2

An idea coming from here suggests to take the HSK vocabulary list annotated with pinyin and put the words list to Google translate. The post's author found 11% of words incorrectly converted for HSK level 1-3 words.

The issue is that the official HSK website provides only a words-list without pinyin in an excel format. However, you can find respectively here and here words-lists annotated with pinyin in a pdf or on a website which you can later programmatically convert in a suitable format.

EDIT, in comments:

3
  • 2
    great idea about HSK! pdf is a tricky format to extract the text from as it often mangles the characters and spacing, but this is definitely a useful direction to investigate; this looks to be in a more accessible format
    – ccpizza
    Jul 25 '20 at 16:51
  • Nice! Seems like you found the right resource. Jul 25 '20 at 17:00
  • @FirminMartin: took the HSK word list and validated it against a couple of popular pinyin-conversion libraries, specifically pinyin4j (java) and pypinyin (python) and out of the 5000 hsk words around over 300 were wrong (mostly the tones). That would be below 10 percent. That's considering that both conversion libraries are using maps for both character and phrases. Even using the large_pinyin.txt map from pypinyin (~9MB) did not significantly improve the error rate.
    – ccpizza
    Jul 26 '20 at 10:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.