This is a very human-centred argument. You can make the world a lot better without making the experience of any single human any better. Things like saving the sea floor from trawling, protecting habitat in remote places, there are a lot of important things we need to do that won't help any individual humans very much, or even groups of humans. I want to live in a better world, that doesn't necessarily mean better (or worse) for any particular humans. I can pick up some litter in a forest, that is an inconvenience for me and doesn't help anyone, but it does make the world a better place.