The key point you’re missing, I think, is that the tax would increase exponentially for each additional house owned. The first one could be, say, a 0.5% tax increase, and it could go up from there.
If you’re in a position where paying 0.5% extra tax on your hunting cabin split 5 ways will bankrupt you, then I’d argue that it isn’t how you’re supposed to spend your money. That’s “Skip eating out once a year” territory.
True equality!