It is uncertain if the character 「后」 existed in Shāng oracle bones. The word itself meaning royal heir > king and royal heir > afterwards was written in Shāng oracle bones with the character 「毓」, and in this time period, it is not confirmed whether this was semantically extended to mean queen.
商
甲

甲818
合集27145
「毓」 depicts a woman 「每」 or 「母」 giving birth to a child 「子」. The child is drawn upside-down as 「𠫓」 and the dots are similar to 「氵」, indicating amniotic fluid / water breakage when giving birth. A proposal of how this may be related to 「后」 is given below:
Explicitly described, 「尸」 may have been corrupted into the shape of 「𠂆」, and 「𠫓」 corrupted into 「⿱一口」. There is no clear trace of this happening, however.
In contrast, 「司」 is generally traced like this:
Please pay attention to the time span of where these characters occur. The shape of 「后」 we currently use is only uncontroversially found in the Spring and Autumn period, and by then, it would never have been confused with 「司」.
The debate here is whether characters which look like
商
甲

三集下46.9
鄴中片羽
(or its mirror image) in the Shāng to Western Zhōu periods should be read as 「后」 (queen). As mentioned earlier, it is uncertain whether hòu (Baxter-Sagart OC: /*ɢˁ(r)oʔ/) meant queen in this period.
For an opinion by Qiú Xīguī on reading these oracle bone characters as 「司」 and not 「后」, see 《裘錫圭》說"㚸"(提綱). In summary, the thesis 朱鳳瀚《論卜辭與商周金文中的“后” 》 proposes multiple occurrences in oracle bones of characters which look like



should be read as "Queen ⿱立勹"

just like how 「珷」、「玟」 should be read as "King Wǔ" and "King Wén" in bronze inscriptions, respectively. Qiú Xīguī thinks this proposal is far-fetched and lacks evidence, and instead suggests that all of these characters have 「司」 as a phonetic component, pointing to evidence of interchangeability of 「司」 with 「㠯」 (related: Why is there a 女字旁 in the character 始?).
As a final note regarding this specific dǐng vessel, the article points out that
銘文於"司辛"字加"女"旁,朱鳳瀚等先生指出,其字仍應讀"司辛"而不應讀"司母辛",所謂"司母戊"大鼎之"司戊"亦同。
In bronze inscription texts, 「司辛」 (temple name of Fù Hǎo) is combined with a 「女」 (or 「母」) component, and several academics have pointed out that the inscription is still read as 「司辛」 and not 「司母辛」, so the same goes with reading the inscription as 「司戊」 when it appears to be 「司母戊」.
References: