An artificial intelligence program that impressed the internet with its ability to generate original images from user prompts has also sparked concern and criticism for what is now a familiar issue with AI: racial bias. and sexist.
And while OpenAI, the company behind the program, called DALL·E 2, has sought to fix the problems, the efforts have also come under scrutiny for what some technologists have called average. superficial to solving the underlying systemic problems of AI systems.
“It’s not just a technical problem. It’s a problem that involves the social sciences,” said Kai-Wei Chang, an associate professor at the UCLA Samueli School of Engineering who studies artificial intelligence. There will be a future in which systems better guard against certain biased notions, but as long as society has biases, AI will reflect that, Chang said.
OpenAI released the second version of its DALL·E image generator in April to rave reviews. The program asks users to enter a series of words related to each other, for example: “an astronaut playing basketball with cats in space in a minimalist style”. And with spatial awareness and objects, DALL·E creates four original images meant to reflect the words, according to the website.
As with many AI programs, it didn’t take long for some users to start reporting what they saw as signs of bias. OpenAI used the sample caption “a builder” which produced images of only men, while the caption “a flight attendant” produced only images of women. In anticipation of these biases, OpenAI released a “Risks and Limitations” document with the limited version of the program before the bias claims were published, noting that “DALL E 2 additionally inherits various biases from its training data , and its releases sometimes reinforce societal stereotypes.
DALL E 2 is based on another AI technology created by OpenAI called GPT-3, a natural language processing program that draws on hundreds of billions of language examples from books, Wikipedia and of the open Internet to create a system that can approach human writing.
Last week, OpenAI announced that it was implementing new mitigation techniques that helped DALL E generate more diverse and thoughtful images of the world’s population – and it claimed that internal users were 12 times more likely to say the images included people from diverse backgrounds.
That same day, Max Woolf, a data scientist at BuzzFeed who was one of several thousand people allowed to test the updated DALL E model, began a Twitter feed pointing out that the updated technology was less accurate than before at creating images based on its written prompt.
Other Twitter users who tested DALL·E 2 responded to Woolf’s thread sharing the same issue, particularly regarding racial and gender bias. They suspected that OpenAI’s diversity solution was as simple as the AI adding words identifying gender or race to unknowingly user-written prompts to inorganically produce diverse sets of images.
“The way this supposed implementation works is that it randomly adds a male or female or Black, Asian or Caucasian to the prompt,” Woolf said in a phone interview.
OpenAI published a blog post last month about its attempt to correct bias by reweighting certain data; it didn’t mention anything about adding gender or race flags to the prompts.
“We believe it is important to address bias and security at all levels of the system, which is why we are pursuing a range of approaches,” an OpenAI spokesperson said in an email. . “We are looking at other ways to correct for bias, including best practices for adjusting training data.”
Concerns about bias in AI systems have grown in recent years as examples around automated hiring, healthcare, and algorithmic moderation have been shown to discriminate against various groups. The issue has sparked discussions about government regulation. New York passed a law in December banning the use of AI in screening job candidates unless the AI passes a “bias audit.”
Much of the problem with AI bias stems from the data that trains AI models to make the right decisions and produce the desired results. The extracted data often contains biases and stereotypes due to societal biases or human error, such as photographic datasets that depict men as executives and women as assistants.
Artificial intelligence companies, including OpenAI, then use data filters to prevent the appearance of graphic, explicit or unwanted results and, in this case, images. When the training data is put through the data filter, what OpenAI calls “bias amplification” produces results that are more biased or skewed than the original training data.
This makes AI bias particularly difficult to correct after a model has been built.
“The only way to really solve this problem is to retrain the whole model on the biased data, and that wouldn’t be in the short term,” Woolf said.
Chirag Shah, an associate professor at the University of Washington’s School of Information, said AI bias is a common problem, and the fix OpenAI appears to have found hasn’t fixed the issues under underlying its program.
“The common thread is that all of these systems are trying to learn from existing data,” Shah said. “They solve superficially and, on the surface, the problem without solving the underlying problem.”
Jacob Metcalf, a research fellow at Data & Society, a nonprofit research institute, said a step forward would be for companies to be open about how they build and train their AI systems.
“For me, the issue is transparency,” he said. “I think it’s great that DALL·E exists, but the only way these systems will be safe and fair is with maximalist transparency about how they’re governed.”
nbcnews Gt