Caste system is a creation of Resource rich people to safeguard their interests. This can be well understood in light of Gramsci’s Cultural Hegemony which means
cultural hegemony is the domination of a culturally diverse society by the ruling class who manipulate the culture of that society—the beliefs, explanations, perceptions, values, and mores—so that their imposed, ruling-class worldview becomes the accepted cultural norm; the universally valid dominant ideology, which justifies the social, political, and economic status quo as natural and inevitable, perpetual and beneficial for everyone, rather than as artificial social constructs that benefit only the ruling class.(Source Wikipedia)
So Caste system is a cultural norm created by Resource rich who project it as a inevitable, perpetual and beneficial for everyone.
People don’t necessarily challenge a cultural norm unless it is negating their interests. Although Lower sects have high population , their opposition to caste system cannot be heard because they don’t have enough means like political power. Moreover they organize into groups on same caste lines to express their depravity as they too believe caste system as inevitable.
One must note that reservation shall only be provided in case of social and educational backwardness. The main idea of reservation is to deal with social injustice and discrimination where economic background has not been provided as a ground in the Constitution which means no reservation can be provided on that ground. It must be noted that Constitution itself doesn’t provide for the reservation rather it empowers the legislatures to provide for the same
Caste opposition is mainly done by resource deficit Upper class (but not resource rich upper class who wield political power but never fiercely remove caste based reservation)
If you miss an opportunity to secure a job or seat you would like to blame it on external factors and one such external factor is Reservation. The one who succeeds in securing a seat or job just offers ideological support to their fellow upper caste brothers but do not actively participate against it.
The supporting argument to above assertion is the fact that those people who oppose caste based reservations blame Ambedkar or fellow lower caste people but never oppose the upper caste people in power. Here too limited rationality exists as people go for emotional preference rather that thinking about the case where in dissent of lower castes to status quo increases if caste based reservation is removed.
And thus people hate caste based reservations rather than caste system it self.