All of civilisation is about chasing trivial pleasures
The principle is not that trivial pleasure can never justify any harm. It is that it cannot justify extreme suffering. Most of what civilisation offers carries small, indirect costs. Factory farming makes intense, deliberate suffering part of the product, for the small difference in taste between one meal and another.
Almost everything people do beyond bare survival is a trivial pleasure: music, sport, travel, coffee, a nicer house, a favourite meal. Nearly all of it harms someone or something along the way, through emissions, cleared habitat, animals killed on roads and in fields, or workers in poor conditions. If enjoying something trivial can never justify harm, most of human life stands condemned, and since nobody accepts that, singling out animal products for this charge is either inconsistent or leans on the word 'trivial' far harder than it can bear.
The strongest version of this objection says the vegan principle proves too much. Almost everything beyond bare survival is trivial by that standard: music, sport, travel, fashion, coffee, a nicer house. Nearly all of it causes harm somewhere, through emissions, habitat loss, wildlife killed on roads or in fields, or poor labour conditions. If trivial pleasure can never justify harm, most of human life stands condemned. Nobody accepts that, so the principle must be too strong, or the word 'trivial' is doing work it hasn't earned.
The answer is that the principle was never 'no harm for pleasure'. It is a claim about proportion: extreme suffering for trivial pleasure. Both halves matter. Most of civilisation's pleasures carry diffuse, indirect and usually small costs, spread across many people and often unintended. Factory farming is different in kind. The suffering is intense, falls on each individual animal, and is deliberately built into how the product is made.
The pleasure being weighed is also narrower than it looks. The choice is not between food and no food. It is between an animal product and a plant alternative, so what is at stake is the difference in taste or habit. That is about as small a benefit as anyone could weigh against a lifetime of confinement and injury.
Where other pursuits do cause disproportionate harm, that is a reason to look at them too. It is not a licence for this one. And the speaker would not accept the general version. Few people think extreme human suffering is acceptable as long as the purpose is entertainment. So 'it's all trivial' is not a principle anyone actually holds. It is being used to blur a distinction that most people draw without difficulty.
Using a single word in two different senses within one argument, so that a conclusion appears to follow when it does not.
How it applies here: The original claim used 'trivial pleasure' to mean a benefit that is slight compared with the suffering it costs. The reply switches to 'trivial' in the sense of anything beyond survival, such as art, travel or comfort. It then treats the two as the same, so that condemning one appears to condemn the other.