If you were allowed to buy property because you had black skin and your neighbor wasn't because he had white skin, Would you both be living in a free nation?
Never been a problem, all you needed was green money. We're less free because of more government, try to figure it out.