I personally bought only the 2nd edition book for starters.
Instead of buying anything else, I borrowed my friend's Conan novels (Conan the Cimmerian, Bloody crown of Conan and ...Savage Sword of Conan, I believe?). I read those back-to-back-to-back.
THAT helped me a whole lot in getting the "Hyborian" feel. I'm an old-fashionned AD&D player, and have been one for about 15 years now. I've gotten used to wizards, sorcerors, orcs, elves, dragons flying in the sky...
To actually read the books really puts things in perspective. Magic is DANGEROUS. It's like handling a poisonous snake with one arm tied behind your back. It MIGHT work, but don't make a single mistake.
Monsters are just that ; mosntrous. What few times Conan actually faces down a creature that isn't human, it's a match for the ages. He dosen't just chop through rows of kobolds; he barely manages to escape with his life against most of those nasties.
Then, after the first few months in my game, I bought Return to the Road of Kings. It is a magnificent piece of work that gives a bucketload of info about each specific country ; religion, standing army, system of government, sights and special things.
I'm planning on getting the Bestiary afterwards.