The funny thing about RPGs versus reality is PCs are exactly the kinds of people who would alarm and terrify townsfolk in the real world. They self-professed mercenaries, looters, reavers, thieves, vandals and are generally always looking for a fight. Gangsters. They are actually the last people you’d admit armed through a city gate... or would hire for any decent or heroic purpose whatsoever.
While that is often true, some people actually like to play as noble Tolkien elves or righteous paladins. And when other players want to play as necromancers, assassins and thieves then you have to remind your group that it's a good idea to create characters which won't kill each other in first session or two.
Personally I allow some leeway to PCs, unless I'm hosting a strictly historical setting, because no one in their right mind would allow such people like Elric of Melniboné, Conan the Cimmerian or Fafhrd and Gray Mouser into the city either. In my dark fantasy games it's all right to have sword fights against rich merchant's guards or thieves who haunt the dark alleys (although thieves most likely have just daggers and clubs etc. rather than actual swords). And it's certainly all right to have some weapons with you when you break into temple where priests, with their long daggers, sacrifice people to city's demonic gods.