Michael is correct in his advice. I also struggle to let go of swords that have come into my care, especially because I believe the sword chooses it's caretaker.
My most important criteria is the sword is sold through an experienced nihonto dealer that will place it in a good home and in the hands of an experienced collector so I have the assurance it will continue to be well cared for. It's the same for a direct sale.
Keep in mind that even though we have cared for this great sword the fact remains that there is and always will be a better sword (even from the same maker) out there and you will probably see those examples and have a chance to aquire better as you wait patiently. No sword is a pinnacle really because smiths worked to improve their art and skill so on most cases produced better and better swords. Have faith you'll find better.
Remember that every time you sell a sword (kept in good condition) you're doing your bit to further educate a collector and maintain enthusiasm for this art. If seasoned or journeyman collectors did not pass on good swords the others would not learn, so rest assured that is a big contribution to our society of collectors.
Every time you let go of a sword you spark new vigour to save and buy yourself a new one
You learn the art of collecting through patience.
If you sell a sword in shirasaya and the next buyer mounts it in koshirae you have helped maintain the skills of an entire industry that is vital to our society. Same for polishing.
...just a few thoughts on why you should sleep well if your sale was to a worthy successor.