My Chinese friend says 下流 is very, very bad and out of proportion and recommended 輕佻, but dictionaries say this means 'skittish, flippant, giddy, frivolous'. These words seem much too cute to me, almost like I am not scolding the person.
How [does one] grasp the difference between the two words?
You're not misunderstanding your vocabulary; you're misunderstanding the beliefs of the Chinese. Having talked this over with my Chinese friends, their understanding is that
Xiàliú ('indecent, base') or biàntài ('deviant, perverted') de rén are objectionable men.
Qīngtiāo ('flirty') de rén are women who take their social responsibilities lightly. Your photo can't be too objectionable because the woman is attractive; she's displaying herself openly and not being careful with her virtue, but she's nonetheless a prima facie object of beauty and grace.
The exact. same. outfit. worn by an overweight, aged, or otherwise unattractive woman would not be understood as xiàliú or biàntài and doesn't get the benefit of being considered qīngtiāo. They're just 噁心 (ěxin): 'nauseating, revolting, disgusting'. This is where Tang Ho's 不堪入目 (bùkānrùmù) comes in, which is why your teacher and friend didn't think of it. Your ultraminiskirt model is objectively not 'unsightly' nor 'hard to look at'.
''Bùsānbùsì'' (lit. 'neither three nor four' meaning 'neither one thing nor the other, shady') is the closest you can get to being judgmental towards attractive women (also men!) without slipping into full-on animalistic slurs and swears.
The dress itself is tài lùgǔ de ('too open; blunt; lewd'). SXZM's 情趣用品 (qíngqù yòngpǐn)—lit. 'funtime product(s)'—is the Chinese for anything you'd find in a sex shop, incl. those cheap overly skimpy costumes. There's no strong tradition of strippers in China so, to describe the women who wear qíngqù yòngpǐn, you have to just say 'she looks like a whore' or—what they do have a tradition of—'she's xǐtóufáng de' (洗頭房的). She looks like she works at a 'shampoo place'.