Sowell's A Conflict of Visions: Ideological Origins of Political Struggles now gets to the core of the disapprobation shown by so many with the unconstrained vision:
Many of those with an unconstrained vision and a passionate opposition to inequality of results assume that those who oppose them must be in favor of inequality of results, either on philosophic grounds or as a matter of narrow self-interest. [pg. 144]
It's never been clear to me why this argument, which is so silly when couched in terms of footraces or exams, is treated axiomatically when it comes to income or wealth.