Books about the Apartheid, North Korea, Native American culture, pretty much any non-white group in the United States (but try to find books that are written by people from that culture).

Some ideas: Farewell to Manzanar (this is a kids book though) by Jeanne Wakatsuki Houston (Japanese internment camps in the US)

Things Fall Apart by Chinua Achebe (European colonization)

The House on Mango Street by Sandra Cisneros (Mexican American culture)

The Kite Runner by Khaled Hosseini (I really recommend this one) (military intervention in Afhanistan)

Pachinko by Min Jin Lee (a Korean family living in Japan)

The Hate U Give by Angie Thomas (police violence against Black men in the US)

There's a lot out there if excellent novels that provide fresh perspectives on other systems of oppression.

I dont think you'd be an asshole but I do think the Holocaust tends to be a bit overdone in literature (speaking as an Eastern European Jew who reads a LOT of Holocaust literature). I like to think book clubs can be used as a vehicle to expand thinking and knowledge... most "regular" people don't deny the Holocaust but there are a lot of other historical events that happened to other marginalized people that don't get nearly the same amount of attention. Maybe pick one of those?

r/whatsthatbook