Somewhere I have a poster of the "34 Tauri" system, showing all the stars planets and moons of that non-existent system ... it might fit within WBH rules, I'd have to get it out and check. Whedon's Firefly concept, as I understand it, was gravity-drive (close enough to m-drive) but no FTL, so these are all worlds, moons, etc., around a complex multi-star system. Bonus points for knowing where the 34 Tauri reference comes from without using a search engine. Or using AI.
I would say that in general, to still be Traveller mapped in subsectors, it should be one 'system' per hex maximum - though I allowed myself an out for star clusters. But there isn't a good mechanism to support multiple systems in a hex without a new mapping paradigm. And while I like your hex approach, it is a method to stay within in confines of the established mapping system. If I was building a new one, I'd just go down the path of the X, Y, Z 3D grid used in 2300.
Although I love 2300 (I have every copy of it from the first print) I always hated the way the maps looked.
What I did to keep with the traveller mapping style, but allowing for multiple systems, I had the primary system be the one with the highest population. If there where no other systems within the cpc (cubic parsec) then I would not bother further detailing the system (unless I was using it in a database and then, why not).
IF there was more than one system within the cpc, I have a little table I use for placement (3x3x3 grid) and I further notate the system ie
0408-212 Sample System
ie in hex 0408 of the sector, in grid coordinate x2, Y1, Z2, each grid coordinate is about a cubic ly.
This keeps the simplicity of the miller projection, adds at most 4 characters to the detail line (3 digits plus a dash) and gives enough detail to figure out distances within the cpc.
I refer to the xyz of the system detail as the local barycenter.
The tables I use take into consideration the local stellar density, the number of systems within the cpc and if any of those systems have gravitationally bound far companions.
I also allow for rogue worlds in my tables ie no central star (the current numbers coming back have the number of rogue planets being significantly higher than what was previously thought) as well as having codes for non-planetary, non-belt based cultures - I happen to use Asterix for the SIZ but use regular codes for atmosphere and water reflecting the local situation within the space colony.
This expands upon the existing notation, while allowing for some very different alien worlds.
Almost forgot to add, one of the ideas I had was inspired by early Hiver background details about them having inferior jump drives initially, so a series of sub-jump ships ie 1-3 ly at time, using other systems within a cpc as stepping stones until their technology got better.