Swift and Immediate actions were first published in the Complete Adventurer for D&D 3.5, and "any time" is correct, except that you only get to do it once. Free actions are unlimited at any time, Immediate actions are limited to one (1) at any time, and Swift actions are (?) limited to one (1) on your turn. I think that's right. We didn't use them much, but now a Swift action would be the equivalent of a Minor Action in D&D4, so I think my recollection is correct.
So, yes: Anytime, but only once per turn. You would still have to hold a DB until the best INIT moment, but you could do it during your own turn or anyone else's during a single INIT sequence.
8)