Anecdote here, but based on my life experience and thinking back on my experience in college, I was a working class first generation college student. Being in the college environment was so different from the world I knew through the 12th grade. My new classmates were so different. I remember overhearing a conversation between classmates where a guy was complaining about being too poor, but then talking about how he can't wait for spring break cause he just wants to go somewhere with a beach to lie down and tan the whole time. Meanwhile I was planning to pick up extra hours at my minimum wage part time job.

Or the dorm experience. I couldn't afford dorming. I commuted 90 minutes each way to class, and would go him, try to take a 40 minute nap before heading out to my 6-10 shift at my job. I had no time for clubs or events, and didn't have a lot of social interactions outside of classes and class work. Hell, I didn't even know my supervisor advisor till my senior year. As a first generation student I had no clue about that stuff. This and so many other experiences all tied together to make college very hard at times. And yes, I am a minority. But it's not my being a minority that triggered this. It was my socioeconomic status, my family history, and the access to experiences I had or didn't have before getting to college.

So when someone sees the correlation as black = high drop out rate. The immediate reaction may be to be offended or object, but while on the data side that correlation is the easiest identifier, it doesn't actually identify what may be the real issues. And let's face it, with the history of this country, lots of black people have gone through life with obstacles intentionally erected to make sure this is their reality and the reality of their children. For example, while white wwii veterans were coming back home to buy homes with their GI bill and pay for education to build a foundation that roots their family and future generations solidly in the middle class, black veterans where not allowed those same privileges in many areas, rooting them and their future generations in the working class.

I wouldn't say that's the correct conclusion. Techies and academics tend to be weak at understanding optics and cultural/racial issues. supposedly claiming "data is king". Academic probation is a negative term, automatic dumping people into that bucket is the most head slapping PR decision. 20 years ago there were equal opportunity programs at colleges which basically used income level and minority as a filter to quality applicants for additional college aid, work employment, and mentoring all without statistical modelling. The correct takeaway in your advisors case is to bring findings to a holistic cross discipline cross culture committee to examine the root cause such as minorities coming from underfunded school districts that poorly prepared people for college. Descions done in secret and especially without racial representative input continues blindness.

I guess this is inherently unknowable, but I am itching to know if the dropout stats were meaningfully different for black people who choose not to self identify as black on the form. For that matter, what fraction of black people pick "prefer not to say" on these sorts of forms, and is that fraction higher or lower than any other racial demographic?

I agree wholeheartedly with your last sentence, but it goes way beyond “bias” in models. Models are only one piece of an ever more complex system.

In terms of the impressions you’ve inferred, we could talk in good conscience about that for hours. Maybe five or six years ago, it came to light that visual recognition models performed inherently worse on people of dark skin. The tech companies (I was there at the time at a big prominent one) decided to jump ahead of the bad press by condemning themselves and promising to do better. The media fallout was negligible.

It was bunk. Did AI models perform generally worse on black/people of African descent photos? In some cases yes. Was the training data cribbed from the US? Yes. Where black people made up, what 13% of the population? Of course they performed worse: there was 1/10 the data available to train them! It wasn’t racist; it wasn’t some bias built into the models by the human trainers - there was simply less data. But nobody bothered to elaborate on what should have been a nuanced conversation and the prevailing opinion jumped to the wrong perception and the wrong remediation. It kicked off an idiotic path upon which we still find ourselves. Or watch others traversing.

The real problem is nobody understands what’s behind these models. We understand the approaches they take generally, the “convolutions” applied at various training layers - but nobody understands the logic behind the output models any better than we understand the models behind human reasoning. We can infer things but there’s nothing known; not in a binary or truly understood way.

Yet everybody keeps racing ahead to apply these models in ever more profound and - if you’re in the space - unnerving ways. It’s getting scary, and it’s way worse than the stuff that’s being discussed here, which is also a bad idea.

I guess what I’m saying is it’s so much worse than these idiot politicians realize. They’re fighting a battle that was lost ten years ago.