No doubt most have some sort of financial constraints and (hopefully) buy/sell within those constraints. But let's be honest: collecting anything but the absolute lowest quality Japanese swords requires at a minimum a disposable income of some sort. It's just not a hobby you can actively participate in forming a collection at anything but the very lowest level unless you have the capability to drop several thousand dollars at a pop. This is not elitism, it is reality.
Once one has seen quality, it becomes hard to be satisfied with less.
I knew several sword enthusiasts in Japan (notice, I didn't call them collectors) who attended every kantei meeting, exhibition, etc. yet owned not 1 sword. Shibui! When I asked them about this, they said they simply couldn't afford the type of sword they would want to own and rather than live hand to mouth to own a nice sword, or own something of poor quality, they simply studied great swords every weekend at a kantei meeting and let the owners worry about the care and feeding of these treasures. Makes a lot of sense. I often wonder why we don't see this attitude more in the US... Instead, we see people spend years collecting all sorts of very poor quality items rather than studying and focusing their limited resources on a few quality items.
Back when swords were $20 a piece, you could collect treasures for next to nothing and those at any financial level could assemble a wonderful collection. Unfortunately, those days have passed and collecting quality swords, like it or not,, has become what it has always been in Japan- a rich man's hobby. Just like most types of art collecting. Some continue to live in the past and instead of developing an eye for quality, knowledge, patience, and awareness of a good deal, insist on treasure hunting and bottom feeding. When they ask for opinions and are told their items are poor, they get huffy and cry elitism...
Speaking personally, I became interesting in collecting gendai-to when in college and I couldn't afford older swords of quality. I recognized that there were those of very good workmanship and that the market wasn't valuing them (in my opinion)) properly. Now, the better ones are no longer $400-$500 but $10,000 and more. I rarely buy anything anymore for my collection-partly because really good gendai are scarce but also because they are expensive. But I haven't lowered my standards and started buying poor quality, cheap items simply because I can. I would rather not buy anything than buy poor quality items.
For those on very limited budgets, I would suggest finding something undervalued, saving for that great deal that shows up (like this one), and make careful buying decisions with a long time horizon, rather than buying low end items willy nilly and hoping for a miracle. One should also keep in mind that there are always other hobbies that are less expensive but can be equally interesting and fulfilling. I have found these very alluring, and much less expensive than a good sword! As a bonus, they are a good excuse to drink sake: