Is there any way to detect dialect automatically?

Question

For current task I need to create a solution how to detect if some video or audio of Chinese speech is Mandarin or Cantonese (for now I do not need others dialects).

For example, there is almost 100% solution to search for radio/TV in USA/Honk Kong or others locations where Cantonese is main dialect. Canada doesn't work due it has 50/50 for Mandarin/Cantonese. For Mandarin is simpler, just search sources from Mainland China.

But there is also another part - YouTube videos. I didn't find a way to detect dialect of speech in video. Auto-subtitles shows only text version which is in Simplified Chinese (written).

Also, there was an idea about searching for -p, -t or -k endings in speech (due Mandarin does not have them any more). I think it is quite good solution but not the best - others dialects have them too.

So my question is - Do you know or maybe have an idea how to detect Cantonese/Mandarin dialect in speech video automatically or using subtitles? Maybe there is a list of Mandarin/Cantonese only characters?

Answer 1

I can think of a few ways for this to work, each could have varying degrees of difficulty in implementation.

---

Word Choice

As @Marko mentioned in comments, Cantonese usually has a very different choice of word for common expressions. For example:

• 點(trad.)/点(simp.) in Cantonese vs. 怎 in Mandarin
• 邊/边 vs. 哪
• 係/系 vs. 是
• 唔 vs. 不
• 嘅 vs. 的
• 喺 vs. 在
• 咁 vs. 这么

sentence-final particles:

• 咯 (Cantonese only)
• 咩 (Cantonese only)
• 啫 (Cantonese only)

• 㗎 (Cantonese only)

• 吗 (Mandarin only)

This will be easier to implement if the subtitle is an exact transcription of what's being said (I say this because sometimes they are not, instead, they are in written/more-Mandarin form). Frequency analysis of the appearance of these syllables could also be possible, I guess? Note: they work better on casual conversations than formal speeches or readings of written material.

This Wiktionary category page could be a gem if you decide to use this approach, but not every entry in this category is Cantonese-specific.

---

Tones

I'm thinking of a pitch analysis. And Cantonese has more tones - 9 in total - than their Mandarin counterpart - 5. If you can devise a way to count the total tones used in the audio, or to match each syllable to a tone and see which model fits best, it could be possible to tell the two apart.

---

Other phonetic features

Other distinctions exist, to name a few:

• 儿化(Erhua) in Mandarin (actually I'd not call them standard Mandarin, but northern Chinese dialect)
• the -p, -t, -k, -m sound in Cantonese

However, I'm not sure if/how you can detect them automatically.