If the referee decides to allow you to skip basic training (as the organisation will recognise prior service) then it would make sense that your previous rank was reactivated.I would personally ignore that as the GM, but it does seem like it is the actual rule.
If you re-join one of those careers where the basic training is assignment specific and you re-join a previous assignment then it would seem logical that you also skip basic training and again retain rank.
If you re-join one of those careers where the basic training is assignment specific and you re-join in a previously untaken assignment then it would seem logical that you have to do basic training (even through you might already have all of the skills at zero or higher) and lose any rank.
With any of this though it is a story decision. Regardless of what might realistically happen, we have plenty of story examples where a disgraced or retired officer was reinstated in their previous rank for "reasons" - Ripley in Aliens, Deckard etc.
There are also real world circumstances where courts martial or other involuntary separations (i.e. failed survival roll) from service have been reversed and servicepersons reinstated at their former rank. Retired officers can often return at their former rank if there is a need for officers at that rank. Even premature voluntary resignation might not preclude reinstatement of rank if the need is pressing.
Outside the military ranks is less rigid and people can often transfer from one organisation to another in the field and return at an equivalent position as they formerly held (and often in a higher rank). If they left voluntarily it could be seen as a sabbatical with their value considered increased rather than diminished by wider experience. If they were sacked then it might be more difficult but not all employers work to the same guidelines and standards, bullying in one organisation might be seen as banter or robust management in another.
For those rare cases where the dice throw you into a previous career you could presume by default it is one of those "You know the score, pal. You're not cop you're little people!" scenarios.