This word exists in Japanese as well as 綿蛮, but as far as I can tell the compound just carries the meaning of "cotton barbarian". I am completely stumped as to how this in any way means "a small bird" or "chirping". Does anyone know what's going on?
Firstly, some conceptual advice on interpreting characters: Please keep in mind that characters are not words; they only represent words. Character phonology is very important when interpreting pre-Qin texts, of which the Shijing is one.
You should always be mindful that a character has both meaning and sound, and sometimes, characters are only used for sound.
If you're a Japanese learner, you should be aware of ateji, which is based on the same principle.
「綿」(Zhengzhang OC: /*men/, thin, fine threads) is comprised of semantic「糸」(silk threads) and semantic「帛」(silk/fabric).
「蠻」(/*mroːn/, Southern ethnic tribes) is comprised of semantic「虫」(poisonous snake, now written「虺」, the cultural symbol of some ancient Southeastern Tribes) and phonetic「䜌」(/*b·roːn/).「閩」( /*mrɯn/, /*mɯn/, Min ethnicity), with the same semantic「虫」, is almost certainly cognate to「蠻」.
Stringing the above together, you get a reading which is approximately /*men mroːn/, where「蠻」is almost definitely only used to represent sound.
綿蠻 /*men mroːn/ come to mean “(the chirping of) a small bird”?
Two interpretations, which may be related to each other:
綿蠻 (/*men mroːn/) is short for 綿綿蠻蠻, where
綿綿 means fine, delicate.
蠻蠻 (/*mroːn mroːn/) is the topolectical name of a bird.
Together, this forms delicate > small + bird, i.e. small bird, extended to mean chirps of a small bird.
綿蠻 (/*men mroːn/) is a phonologically-shifted reduplication of「微」(微微, /*mɯl mɯl/).「微」means tiny, slight.
王國維《觀堂集林・爾雅草木蟲魚鳥獸名釋例下》：“蟲之小者曰蠛蠓 (/*meːd moːŋ|moːŋʔ/)，鳥之小者亦曰綿蠻，殆皆微字之音轉。”
Small insects are called 蠛蠓 (/*meːd moːŋ|moːŋʔ/), small birds are called 綿蠻 (/*men mroːn/), they are both phonological shifts from「微」(/*mɯl/).
Contextually, the reduplicated word meaning small was only used to describe birds, hence it became associated with birds to get small birds > chirping of small birds.
in the book of odes (詩經), there’s a poem named 綿蠻:
“綿蠻黃鳥” is translated to “little oriole” in ctext.org
however, the scholar 朱熹 suggested that “綿蠻” is the sound of birds (鳥聲)
if you have difficulty, trust him at this moment :)
Chinese poetry is exceptionally hard to read (for me at least), but fortunately Baidu Baiku has a 白话译文 "Spoken language translation":
So the bird is 小黄雀 (see Baidu Image Search) which I think is a type of finch. And each line begins:
That beautiful small finch, ...
(Here I use "finch" to mean something like "black-headed yellow Eurasian siskin" which makes the poem too clunky.)
緜蛮: bird feathers [鸟羽] rich and bright in color [文采] numerous and close together [繁密] appearance [样子]
However, there appears to be multiple interpretations of how to interpret 緜蛮 (zdic.net): small birds appearance (小鸟貌; 文貌), bird sounds (鸟声) and smallest birds (鸟之小者亦曰绵蛮).
Judging from Baidu
绵蛮, this term is normally only mentioned in the context of poetry.