Take the 2-minute tour ×
Chinese Language Stack Exchange is a question and answer site for students, teachers, and linguists wanting to discuss the finer points of the Chinese language. It's 100% free, no registration required.

The relationship between simplified and traditional characters can be a headache for organizations using both orthographies.

"A number of surveys, such as [Xiandai 1986], have demonstrated that the 2000 most frequent SC characters account for approximately 97% of all characters occurring in contemporary SC corpora. Of these, 238 simplified forms, or almost 12%, are polygraphic; that is, they map to two or more traditional forms."

Nevertheless, it's a task that can one day be done relatively perfectly by a computer. Google Translate can do it for you more or less. But if you are converting a short story or a novel, the task is a little more difficult. That's why I actually prefer to use the Mediawiki converter.

Nevertheless, I don't really know how it works. I have a feeling the SCIM pinyin input tables are used somehow, and the tediously underdocumented non-bijective-mappings between the two orthographies coded for manually. Is that right?

share|improve this question
2  
Hello Magnetar. Thanks for posting but I don't think your question is on topic, I voted to close. It's more related to how some web-software works rather than the Chinese language. I think it's more fit in sites like Superuser or similar, I'm not sure which one, honestly... :D –  Alenanno Dec 25 '11 at 21:18
    
Do you have a link to all of the source code? Is this project considered completely finished? –  Village Dec 26 '11 at 6:16
    
Would you support this question, @Alenanno, if it didn't focus exclusively on the Mediawiki converter? Something like, "What techniques can software use to convert simplified characters to traditional characters when simplified characters can map to more than one? For example, Mediawiki seems to work well." –  Don Kirkby Dec 26 '11 at 7:17
1  
@DonKirkby Uhm It still seems too related to software. Anyway, I wanted to be clear that this questions is not a bad one (and as you can see I didn't down vote), it's an interesting question to me as well, but I think that the knowledge required to answer is more on the computer side than the chinese side. –  Alenanno Dec 26 '11 at 11:17
    
@magnetar - Can you confirm if you want a technical or non-technical answer to your question? Are you looking for someone to explain how this can be done in code or just a general explanation to how it is done? –  xiaohouzi79 Jan 12 '12 at 1:23
show 2 more comments

2 Answers 2

There is another converter at http://mandarintools.com/zhcode.html. I lack the capacity to judge its quality. Is that a suitable alternative? It may reveal how the conversion is done.

I may have overlooked some parts, but here is my understanding of it:

  • It seems to have a large two-column list of words (hcutf8.txt). The first column contains simplified, the second column traditional. The second column occasionally contains two or more characters, in situations where one simplified character equals more than one traditional character.
  • When converting from simplified to traditional, it replaces all occurrences of words found in the first column with words in the second column.
  • When converting from traditional to simplified, it replaces all occurrences of words found in the second column with words in the first column.

Perhaps there is more in the code that I did not notice. Are additional steps needed for this conversion to be successful?

share|improve this answer
add comment

The Mediawiki converter uses a combination of automatic information from the Unicode standard, SCIM tables, and other sources plus manual tweaks to build a set of translation tables. When going from Traditional to Simplified, some characters have been condensed into one. Translating back from Simplified to Traditional requires context that a computer is not likely to understand unless you implement a full translation engine, not just a text replacement engine. This is where the manual enhancements come into play.

Mediawiki has a build system which automatically combines the automatic and manual information into a PHP source file of translations between many different types of Chinese. It appears to cover dialectical differences for mainland Chinese, Hong Kong, Taiwan, and Singapore. The information is then changed out with PHP's strtr() function which replaces all occurrences of items in a given array with matches in the supplied string, longest matches first. This longest matching is the key when switching out characters because you have more context from the source to make a better replacement.

Source: Code which uses mediawiki's conversion functions: https://github.com/tszming/mediawiki-zhconverter/blob/master/mediawiki-zhconverter.inc.php

Build file to combine automatic conversion tables with manual tweaks: http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/includes/zhtable/Makefile?revision=50768&view=markup

Compiled translation tables (large file, 17k lines): http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/includes/ZhConversion.php?view=log

Conversion tool to load tables for different conversions (used by first source link) http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/languages/classes/LanguageZh.php?revision=89118&view=markup

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.