What are some 成语 that mean something like "little dogs always bark the loudest", or "little dogs are mean"?
In a direct translation, "咬人的狗不叫", but it's not a 成语, it's a slang.
虚张声势: pretend to be strong to make others fear.
深藏不露: very talented or strong, but hiding from the others.
不露锋芒: same meaning as above.
韬光养晦: Hide one's capacity and bide one's time(Not sure it's a good match but similar with 深藏不露)
Here's some supplement to Kevman's answer.
色厉内荏："A bully is always a coward，" it literally means that looking tough outside but actually stuffed with straw.
外强中干："outwardly strong and inwardly weak。"
Actually, I think the English saying is: All bark and no bite, which is 光说不练
I recommend "满瓶不动半瓶摇", which means a bottle full of water is quiet when you shake it, while a bottle with a little water makes sound when being shook. Alluding that people with rich knowledges always keep a low profile, while those people with superficial knowledge like showing off.
I think this is similar but a bit different from "Barking dog never bites". The Chinese phrase 丑人多作怪 would be a good translation