Excerpt from 《汉书》卷二十六《天文志》：
从 == 纵 == upright. As for who 纵目人 really was or were, there are a few hypotheses.
The most prominent reference of 纵目 is related to 蜀人. It's said that either their first emperor, or the people themselves, were in possess of 纵目:
有蜀侯蚕丛,其目纵,始称王。——晋 常璩 《阳国志·蜀志》
蜀之先称王者,有蚕丛、柏濩、鱼凫(蒲泽)、开明。是时人萌,椎髻左衽,不晓文字,未有礼乐” ——西汉 杨雄 《蜀王本纪》
Then strange bronze masks unearthed from 蜀 area, current 四川 province. And they do possess extraordinary eyes. First, they are slightly larger to the east Asian standard, second, they are eerily protruded like that of a crab or shrimp.
It's probably that 蜀 people, being in a different ethnicity group, did have somewhat different eye shapes, or they do strange makeups of tattoos around their eyes. But I don't think 汉书 is referring to 蜀人 here, because 蜀 was already part of Han dynasty.
Since 西王母 was mentioned, it is possible that it was a reference to some evil deity, who would come and wreak havoc among the people, or some good deity who serves 西王母, and would come and save the people from suffering. We cannot know for sure.
There is a Taoism deity called 杨戬, who is said to have 纵目:
杨戬 belongs to the good side, and he is also said to be 西王母's nephew. These were later stories, we really don't know originally what it was.
Tales about 杨戬 was first prominent in 蜀 area, then spread all over the country. Late depictions of 杨戬 shows him with one extra eye placed vertically between eyebrows. Early references doesn't seem to mention anything specific about his eye.
I found the two theories utterly implausible. If you see a man with an extra vertical eye in the middle, you would immediately think "three eyes!" instead of "vertical eye!" If it's a rumor about an invasion you would call them by race. If it's about deities then they ought to have names instead of a nickname.
What I observe is an alien invasion. They might be scary monsters that look like this: