It is uncontroversial that nonhuman animals can suffer. Anyone who has ever stepped on a pet's tail knows that animals can be hurt. It should be equally uncontroversial that we ought not cause unnecessary suffering to any animal, human or otherwise.
We can all admit this, carnivore and herbivore alike, without specifying what constitutes necessary suffering. Even many committed vegans will say that there are necessary forms of suffering, such as that caused through self-defense. But the default position must obviously be not to cause suffering unless it is necessary.
The burden of proof clearly falls on those who are causing suffering. If you eat eggs, the burden of proof falls on you to justify the necessity of eating eggs that outweighs the suffering inflicted on egg-laying chickens. If you enjoy horse racing, the burden of proof falls on you to justify the necessity of horse racing that outweighs the suffering inflicted on racing horses.
Note that this argument is utterly independent of any claims for animal rights, though it is certainly compatible with them. This is merely a basic consequence of the commonsense notion that animals have a welfare that ought not be ignored for any but the most necessary reasons. Those reasons may exist, but they certainly do not in the case of any customary use of animals for food, clothing, or entertainment.