Hmm - interesting thread. But there were several factors that I didn't see taken into account in some of the arguments for or against.
RE: Space is big - First of all, many of the counter-arguments to the "space is big" crowd point out the 100d limit as narrowing the possible travel lanes without taking some other factors into account, most particularly in-system travel, and the 100d limit of the star.
Take our solar system - the Sun's 100d limit is 87 million miles - meaning Earth sits just a mere 6 million miles outside the Sun's 100d limit. Any starship wishing to travel to Venus or Mercury must do some significant in-system travel time. If a system's primary planet is inside it's star's 100d limit, travel time increases, possibly significantly.
And some systems will have multiple inhabited worlds or trade routes between primary planets and asteroid belts/gas-giants, meaning there will be frequent in-system travel. Both factors significantly increase the number of tavel lanes that have to be patrolled, thus increasing piracy* potential.
RE: Risk vs reward
It's been pointed out that one good haul can pay a pirate's expenses for a year or more. That means two good hauls a year can make a pirate very profitable - even if their success rate is 1 in 50 or 1 in 100 attempts.
Therefore piracy attacks that are fought off or otherwise discouraged will be far more frequent than ones that succeed - less than 2% of pirate attacks will result in the loss of cargo or ship. But that's really all that's needed for a pirate to be "successful" - like current day scam artists, all it takes is the right victim for a big payday.
RE: Imperial ships are "everywhere":
Sure they are. But how many are in jump space at any given time? Take the 20,000 scout ships previously mentioned. If each one of those ships jumps only once a month, then at any given time 5,000 of them are in jump space, where they can't patrol anything. So instead of having 50 per system, you have 38 per system. The more frequent they jump, the fewer are available. That doesn't even begin to take into account maintence and shore leave schedules.
And space is still big - think about how many police cars it takes to patrol the average sized US city - even if you consider the fact that an individual space ship can patrol significantly more volume than a single police car, when you consider how much more volume there is to be covered, and the covereage is pretty pathetic.
Like local police, how many of those Imperial ships are right where they need to be at the right time? Even if there is an Imperial presence within a system doesn't mean that piracy can't happen there - it just means it's more unlikely. Do a google search for "robbed across street from police station" - amazing how many stories come up.
RE: Pirate intelligence networks
It really doesn't take much for a single pirate ship to develop it's own "intelligence" network. A few greased palms among cargo brokers and bar tenders at most starports, and the pirate captain has his info. Most of these guys are going to be giving the info to just about anyone who pays up or possibly just to anyone who wants to listen, meaning many pirates - and adventurers like the PCs - are probably using the same "network", probably not even knowing who else is receiving the info.
It doesn't take an organized network of spies and intelligence gathering to get the info desired.
RE: Pirate bases
IMO, 99% of pirates (and pirate attacks) will be ones of opportunity - a less than ethical starship operator who happens upon a situation where he can prey on another merchant and reasonably expect to get away with it.
However unlikely they are, pirate bases are still a possibility - particularly in those empty reaches of space where no one goes. It does't take a lot of resources to cobble together a workable base - you don't have to hire crew and haul too many resources out to build something from scratch - all you really have to do is weld together a few star ships (derelict or not) that you bring out. These type of bases will not be pretty, and they surely won't be safe, but they will have something that is very desirable to the criminal element - they aren't easy to find unless you know they are there.
I would surmise though, that the majority of these type of bases would serve smuggler's and fences primarily, while supporting actual pirates would be merely incidental.
Other Thoughts:
A common view of piracy seems to be boarding and looting another ship, akin to the way it was in the 16-18th centuries. However, a look at modern day piracy compared to that shows piracy can evolve - today it's about seizing and holding a ship for ransom. Pirates of the Third Imperium could, and probably do have even more evolved methods, most of them probably more akin to hijacking and train robbery than good ol' fashioned piracy.
Misc thoughts along this vein are things like launch a nuke out to proximity of the victim, and threaten to set it off unless they dump their cargo, or the Firefly plot of putting an inside man on the ship to take it where you want it to go.
Like I mentioned above, IMO 99% of pirate attacks will be ones of opportunity - the merchant who's most likely a smuggler as well coming across an opportunity he can't pass up - either by chance, or because one of his contacts passed him some info that gives him an advantage. Most pirate attacks will be on outbound trips, just inside the 100d limit, where the pirate has learned not only what the victim is carrying, but their likely jump trajectories, etc, so they can be "lying" in wait.
The "pro" pirates will in more likelyhood operate more along the lines of hijackers - some how arrange for the ship to "misjump" to coordinates of their choice, and deal with the ship at their own leisure. They will be very few and far between, and most will be more in the vien of megacorp "black ops" engaging in trade war tactics.
*Piracy - for the sake of brevity, I'm using the generic term of piracy to refer to a ship being forcibly taken and/or looted. While it technically may be "highway robbery" or even "disrupting interstellar commerce" in the legal definitions of the Third Imperium, as far as fans of space opera are concerned, it's piracy, and piracy is what I will call it.