@Twisted:
Ultimately even if money is no option most of us have limited storage space so we do need to apply some constraints to our collecting:
1. Franchise - pick your favourite franchises and only collect them.
2. Edition - specifically collect the best edition of that franchise as opposed to every edition.
3. Platform - e.g. only the PC compatible version, 360 or PS3 only.
4. Price e.g. no collector's edition costing more than X.
5. Size e.g. no ridiculously big collector's editions e.g. Crysis II Nano
In practice what I do is pick my franchises based on my interests/preferences, in my case Assassin's Creed, Elder Scrolls, Final Fantasy and Mass Effect make the cut, however Call of Duty, Crysis, Dead Island, Halo and Uncharted do not. Occasionally, I'll splash out on the odd edition which interests me if the price is right but on the basis that it's a one off and not intended to be a complete collection.
In terms of platform constraints, I generally prefer the PC version if it's available, sometimes it's cheaper but mainly because it's more likely to stand the test of time. However if that's not available I'll pick the 360 or PS3 based on it's merits. Additionally, I tend to pick up standard versions of the game for each of the other platforms when they reduce in price and/or come in Game of the Year format with added content.
I always try to buy the collector's edition strategy guide for each of the franchises I'm collecting as these tend to be nice collectible pieces with concept art and screen-shots as well as being a generally handy reference. For something like Final Fantasy where the 'best' collectors edition (Crystal Edition) isn't very compelling I'll happily settle for the 'Limited Collector's Edition' in the knowledge that the collector's edition strategy guide is a far better addition to the collection than the questionable value of what they pack in.
Hope that helps in some way!
PS. Just a word on money, if you're borrowing money through credit cards to pay for this stuff, my advice is to stop or cut it down dramatically. Collector's editions just do not hold their value and should you need to sell them to recoup the money you're likely to get far less than you spent. You may see some of these things listed on Amazon / Ebay for exorbitant prices but they don't sell at those prices, that's just wishful thinking on the part of the sellers.