A bit of information about how the Chinese team got their images:
The images used in the research were standard ID photographs of Chinese males between the ages of 18 and 55, with no facial hair, scars, or other markings. Wu and Zhang stress that the ID photos used were not police mugshots, and that out of 730 criminals, 235 committed violent crimes “including murder, rape, assault, kidnap, and robbery.”
How the classification works:
“All four classifiers perform consistently well and produce evidence for the validity of automated face-induced inference on criminality, despite the historical controversy surrounding the topic,” the researchers write. “Also, we find some discriminating structural features for predicting criminality, such as lip curvature, eye inner corner distance, and the so-called nose-mouth angle.” The best classifier, known as the Convolutional Neural Network, achieved 89.51 percent accuracy in the tests.
90% accuracy is really insane and imo precludes the sort of human-bias explanation you’re talking about, as I don’t think it’s arguable that 90% of whether people are convicted of crimes or not can be explained by other people not liking their faces.
Edit: I’m assuming their train and test sets were different, but I can’t tell because they pulled their paper as a result of the controversy around it.
Re-Edit: OK, I found an article saying they used 90% of their dataset to train and the other 10% to test.
Here’s a bit from of the first articles I came across when googling “Bias in AI”
COMPAS
The canonical example of biased, untrustworthy AI is the COMPAS system, used in Florida and other states in the US. The COMPAS system used a regression model to predict whether or not a perpetrator was likely to recidivate. Though optimized for overall accuracy, the model predicted double the number of false positives for recidivism for African American ethnicities than for Caucasian ethnicities.
The COMPAS example shows how unwanted bias can creep into our models no matter how comfortable our methodology. From a technical perspective, the approach taken to COMPAS data was extremely ordinary, though the underlying survey data contained questions with questionable relevance. A small supervised model was trained on a dataset with a small number of features. (In my practice, I have followed a similar technical procedure dozens of times, as is likely the case for any data scientist or ML engineer.) Yet, ordinary design choices produced a model that contained unwanted, racially discriminatory bias.
The biggest issue in the COMPAS case was not with the simple model choice, or even that the data was flawed. Rather, the COMPAS team failed to consider that the domain (sentencing), the question (detecting recidivism), and the answers (recidivism scores) are known to involve disparities on racial, sexual, and other axes even when algorithms are not involved. Had the team looked for bias, they would have found it. With that awareness, the COMPAS team might have been able to test different approaches and recreate the model while adjusting for bias. This would have then worked to reduce unfair incarceration of African Americans, rather than exacerbating it.
You should read some of the articles on it before you believe our faces can predict our behavior.
I’m familiar with COMPAS, they simply didn’t even attempt to control for human bias. Showing that an AI can be racially biased if given racially biased data to look at isn’t an indictment of AI methods in general. It’s just a non sequitur to bring up an entirely different system. It wasn’t even a machine learning model, it was just regression.
BTW, a 2011 Cornell study also demonstrated that people were able to predict criminality (in that case violent criminality) just from facial pictures, even when careful efforts were made to control for attractiveness, photo quality, race and so forth. The p-value was less than 0.0001.
Well the Chinese one was like that, they just used government ID photos. If I remember, I’ll post the full text of the Harrisburg study when that comes out.
In the case of the Cornell study, they asked people whether they thought they could identify mugshots:
A series of mixed effects ANOVAs, with Subject entered as a random effect, and Photo Category (Criminal/Non-Criminal) and Attractiveness entered as fixed effects, revealed that criminals were rated as significantly more likely than non-criminals to have committed murder, rape, theft (p’s < .0001), and forgery (p = .04). There were zero interactions between ratings given to each category and yes/no responses to the question, “Was it obvious that some photos were mug shots?.” Thus, there were no differences in the ratings of those claiming to notice extraneous cues.
They tried to control for facial expression etc as well. Like you can argue that the controls were insufficient, but it’s not like they didn’t think about this stuff.
Yeah, without further evidence I’m going to go with the assumption that it’s micro-cues or whatever detecting people who have been photographed as criminals in an institutional setting rather than soul reading inherent criminality. That’s still a pretty interesting finding, but beyond useless as a law enforcement tool.
I feel like this may end up being an elaborate process to determine that attractive people get away with stuff that ugly people do not get away with. But sure, let’s go with “ugly people are inherently criminal”.
True story. My dad had a slightly lopsided mouth and was pulled over and strip searched at an airport once a very long time ago because they thought he looked guilty.
My favorite related anecdote that I learned while reading about introductory machine learning is the “wolves on snow” problem. They tried to get a machine learning program to learn how to differentiate wolves from dogs based on pictures. The tool performed well in aggregate but had some inexplicable errors, flagging obvious dogs as wolves for example. Eventually the researchers figured out that the vast majority of pictures of wolves show them standing on snow. So the machine got really good at guessing which animal was a wolf by ignoring the animal and looking at the ground. It’s easier to differentiate grass from snow than it is to differentiate Huskies from wolves, so in a sense the machine actually did something clever. On the other hand, it would call a chihuahua a wolf if it was standing on snow.
Intellectually that’s all very interesting but politically it should be a blinking warning light that using AI to categorize people as “criminalish so watch out!” is a monumentally bad idea. I actually wonder how they even got funding for that research. Typically if your research project is also the plot to a dystopian sci fi horror movie someone puts a stop to it!
I’m not a fan of limiting research because of political of ideological reasons. Those things shift a lot. That being said, harm needs to be in the decision matrix, to be sure.
It’s not limiting research, it’s advocating more basic research until we understand how the AI is doing what it’s doing. Creating black boxes that do fantastic inexplicable things leaves open the door for all sorts of edge cases that if allowed to be used for the wrong thing will end up in people dying.
Of course. As was so well illustrated in that book I cited most of the harm comes from rolling out these things at first sign of success without follow up, contextual thought, or re testing in various scenarios.
My comment was in reference to the idea we wouldn’t want to allow research that felt dystopian.
Facial Recognition =/= dystopian, lots of valid reasons for machines to be able to recognize people
Phrenology = dystopian, no valid reasons to pre-judge someone’s character based on physical appearance.
The idea of the research itself is unethical and shouldn’t be funded with public money.
The whole thing is ridiculous. If I think about committing a lot of crimes, does my appearance reflect that? If I play a lot of video games or D&D where I commit crimes, does my appearance reflect that? Do detective novel authors show up as criminals? How long after I commit a crime does it appear in my face, or is this a pre-crime thing? What level of crime is required? What if something is legal in the US and illegal in China, will I show up as a criminal in one country or both?
I don’t think it’s nuts to hypothesize that anti-social behaviours could, theoretically, show phenotypic expression.
It’s obviously super fraught terrain as we are dancing very close to phrenology and other obviously racist things from science’s past.
I just get squeamish at the notion some types of research are automatically off limits due to political or social norms. This same logic made LGBTQ research taboo for generations. It still makes female sexual health research extremely rare.
Worked as an RN at the Atlanta-area hospital last night.
Last night was rougher than usual, but I was on my usual floor, which isn’t the Covid floor.
We don’t need to go into how my night sucked, but the cap on the downer came at shift change as I was leaving.
They called a code blue on a patient that was on the Covid floor (right at shift change, of course). She was a do-not-resuscitate 39-year old (which is rare), who already had metastatic liver cancer.
Anyway, I found out she died. I don’t know if she gets counted in the Official Georgia Covid Stats… but it was a shitty way to end a shitty night for me.
…Currently drinking and pouring one out for all the homeys we’ve lost and will continue to lose, thanks to the brilliant decisions of our local elected leaders.
Stay home if you can, wear masks and wash your hands if you can’t… that’s all I got for now.
It’s not “dancing very close” to Phrenology, it IS PHRENOLOGY. The idea that your behavior can be decided by your appearance and not your actions has zero basis in anything but race or class based pseudo science intent on keeping the others in their place.
Get over your kinky self, this isn’t a political or social norm, this is pure classism/racism.
Who is helped by this research? If our appearance predetermines our behavior how can we be guilty of a crime? “I” didn’t commit the crime, it was this pretty face. Are you ready to allow people to go through conversion therapy if their face determines they’re a criminal?
