I probably shouldn't get involved in this discussion, because (1) I haven't really done my research recently and (2) I'm at work and on deadline. But I have some thoughts on this subject.
Feudal Japan may not have used women as regular army, but it was reasonably common for female castle residents to be trained in weapons like the naginata and the bow, as well as siegecraft and logistics, to aid in defense when necessary. Some of this was undoubtedly "vanity training," more a show of dedication than a real commitment to fighting, but the example is there.
Middle Eastern women didn't fight directly, but they did occasionally follow behind the troops to finish off the wounded -- or so the story goes.
Nordic women did follow the example mentioned in the Conan RPG description of Nordheimr, in that they were expected to be able to fight and defend the home, and it wasn't unheard of for them to go to war.
According to a couple of TV documentaries I've seen (not the Ridley Scott movie), Roman-era women gladiators were not unheard of, and were as likely to be free as slaves -- not a great life, but probably more interesting than prostitution.
Women leading troops into battle, whether as a symbol or as a warrior, seems to have more precedent. In addition to Boudicca/Bodicea and Agrippina, two other very well-known examples are Elizabeth I of England, who famously rallied her troops from horseback wearing full (though probably lightweight) armor; and Joan d'Arc, whom I expect needs no description.
Lastly, I have to say the obvious -- the characterizations in REH's stories are supposed to evoke the spirit of legend, not the facts of history. If popular imagination says that women fought in such-and-such culture, I'd rather use those stories and make allowances to improve my game. It's too easy to over-apply gender stereotypes and make female PCs seem freakish.