I had read the story Hand of Nergal (I read the whole 13 volume Conan series as edited by De Camp, so my "memory" source is somewhat fouled up), and a scholar type character is in that. He uses several artifacts that Conan just figures is magic use, although the scholar claims otherwise. These objects function like magic, but may have another source that is not. The writers seem to keep that unclear (I can't say it is REH, maybe someone can clear up the source). That's the view of magic I use in my games, but the players only see the "effects" not the source. I have had no one play a magic user, but have set it up so the "source" of magic can be a variety of things, including raw energy.
A lot of magical use in Conan seems to be restricted to Priests or secret orders that function similarly, such as the wizards of the Black Circle. Scholars seem to know about magic, but do not seem to have the abitlity or desire (it may just be fear) to use major magic. Mostly, an information magic, seems to be what they use. It is the Priests that seem to have much of the secrets of magic under lock and key, and guard their secrets of magic jealously. Priests of Set actually have an inner circle of high order priests that are members of the Black Ring, some claim that some of the Black Circle were members of the Black Ring, I don't know. I'm only saying that I think magic in Hyboria, seems restricted to secretive orders of one type or another. Mainly, they are Priests IMO.