12

I have been wondering how come the HSK list be so different from the list of the most used Chinese characters.

The only list I know about character frequency is Jun Da's list. On the first 2635 most common characters of the list (I picked 2635, because it's the total number of the HSK characters), there are exactly 305 characters that are not listed on any HSK.

On the rest of Jun Da's list (a total of 7000 characters) there are another 304 characters that are on the HSK list, but are way way farther than the most used characters (the last two rank at #4527 and #4771).

Questions:

  1. What are the criteria for the character choice on HSK?
  2. Why is the HSK list so different from the most used characters list?

Additional comments are very welcome.

  • What do you mean by "I picked 2635, because it's the total number of the HSK characters"? I googled by "hsk number of characters 2635" and this post showed at the top of the result, so I'm sure the figure is not authoritative at all. – Blaszard Jul 9 '18 at 8:55
  • @Blaszard As far as I know, there's no official list with the total number of HSK characters. I just picked the official HSK vocabulary list (since it's official, I'd call it authoritative), removed the repeated characters and came with that number: 2635 individual characters. I also counted how many extra pronunciations there were, which adds another 101 characters to the list. – Enrico Brasil Jul 10 '18 at 16:27
  • I realised a while ago that there are plenty of useful characters that aren't in the HSK lists. It could be useful for students who have already gone through the HSK lists to have a list of the top, say, 1000 characters in Jun Da's list that aren't in the HSK lists. – goPlayerJuggler Aug 30 '18 at 15:23
3

To answer your second question:

From a mathematical perspective, it will depend on the texts that are sampled to compile any list and the variances associated with those samples. Languages evolve and there is no true character frequency, only an approximation. Spoken Chinese will have a different character frequency to written Chinese because the spoken language has a high entropy. It is likely that the HSK test had a panel of experts select characters based on what they deemed to be suitable, whereas the list you referenced was mass sampled by some kind of web crawler.

It is inconceivable that every single electronic Chinese character on the internet has been sampled, and if it was that would be very biased as that would ignore all of the written works not uploaded to the internet. Depending on the format of the inputs there will be a huge difference, if lots of text was taken from 微薄, then the character 哈 might rank as common, purely from comments that are like "hahahaha" for example. If classical Chinese was sampled heavily then characters like 曰 might feature much more frequently than 说.

Some of the HSK 5 and 6 words are very political and business oriented, but there are not many scientific words.

| improve this answer | |
2

To give some partial information regarding your questions, Olle Linge at Hacking Chinese has a recent (July 2020) article What important words are missing from HSK?. One thing he mentions is that:

It should be clear that HSK is not meant to be a representation of the most commonly used Chinese words. This is very obvious in the lower levels, where words like “train station” and “bus” are part of HSK1, which has only 150 words in total. Those words are nowhere near the top 150 words in Chinese in general, but they are of course important for foreigners visiting and travelling in China, which probably is why they are included.

Overall, I think the lower levels of HSK match the needs of foreign students quite well.

Thus, one possibility is that the creators wanted to make sure that foreigners learned what they needed to get around in China when visiting/studying there. This would bias the most frequently used words away from more general frequency lists.

This being said, Olle also came up with a list of words from a frequency list gleaned from movie and TV subtitles (the SUBTLEX-CH corpus [Cai and Brysbaert, 2010]) that either do not appear in HSK or are delayed (ie. they are in HSK but appear in a higher HSK level than expected, based on their frequency). Based on this analysis, Olle found that these types of words seemed to be missing from the HSK lists:

  • Many single-character words are missing
  • Names of places and countries are missing
  • Regional variants are missing
  • Profanity is missing entirely
  • Foreign things are mostly missing
  • Particles in informal language are missing

Seeing what is missing from HSK could help give some insight into the decision making process behind making the lists. Olle goes into more detail about the types of words missing in his article, if you want to know more.

Also, Olle did the same analysis for TOCFL (though with a different frequency list): What important words are missing or delayed in TOCFL?

An important practical note: if you want to learn the words that are "missing" from the HSK (or TOCFL) lists, Olle has generated the word lists and placed them at the end of the article so that people can use them freely for their own study.

Frequency List Reference:

Cai, Q., & Brysbaert, M. (2010). SUBTLEX-CH: Chinese word and character frequencies based on film subtitles. PloS one, 5(6), e10729.

| improve this answer | |
  • 1
    Some extra info: I chose to do the analysis you refer to based on movie subtitles because that's closer to what foreigners want to talk about. I also did (but never published) an analysis comparing HSK to Junda's frequency list. That's not very interesting, because it's basically a list of formal or written Chinese that obviously is very common in Chinese, but not suitable for beginners. This is not terribly interesting, so I focused on the SUBTLEX-CH corpus, which turned out to be quite interesting! – Olle Linge Aug 20 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.