Yeah a lot of those lists of toxic plants suck because they don't really distinguish between "turns the litterbox into a superfund site when ingested in large quantities" and "deadly to be around".

If your cat isn't one to monch on random plants you can keep a pretty large variety of flowers and plants.

My favorite go-to safe flower is zinnias. Sunflowers are nice but drop a lot of pollen.