When your "castle" is on the 10th floor and has 8 or 9 other people's castles below it, your strict concerns for total privacy and denying everyone entry without your prior consent are mitigated by the other people's rights to not have to wait for you to come home in the event of a water leak or other disaster...
Evan, you are right. But there's this: I don't have to trust the management of the building. They are human beings just like the rest of us.
I do agree that some sort of secondary access should be achieved, for no one having to take your door down, but I find that just the existence of a master key in the hands of someone I wouldn't invite to my daughter's birthday is just pushing it. Maybe the existence of said master key but stores somewhere where all access are logged?
Not to mention that having a masterkeyed building opens a lot of security issues. I would IMMEDIATELY disassemble the lock and make a master key for the whole building!
And I'm an honest guy, think of what I would do if I wasn't

My (hopefully future to be) parents in law live and work in Switzerland. On their building it works the same way. Their key (wich is a KABA something) opens the building's front door, laundry and their home door. Their neighbours' keys are the same, each for their own homes. And the owner has a masterkey that opens them all.
What prevents him to enter someone's home and steal whatever he want's when people are out?
Go ahead, keep plugging away, picking on me! You will end up on bypass or with rigor mortise.
- GWiens2001