The word should be payoff rather than inspiration. I take pictures of wildlife. My motive would be "Did you see that critter?" and "What is that critter doing?". That is hardly inspirational. :)

I do not go out thinking that I am making art but just pop out to look at wildlife. Squirrel.

As why fellow wildlife photographers quit, the answer is simple. They got that apex predator shot or action shot. Or they got tired of taking the same critter. Or they got very "helpful" advice from other photographers.

You said it, diminishing returns. It also help to experiment to keep things interesting. Infra red photography (Fun) or post production (Dull).