ChatGPT, Lensa, Stable Diffusion, and DALL-E: Generative AI, explained

Artificial intelligence is suddenly everywhere — or at least, that’s what it seems like to me: A few weeks ago, a friend mentioned in passing that his law professor had warned students not to cheat with AI on an upcoming exam. At the same time, I couldn’t escape the uncanny portraits people were generating with the image-editing app Lensa AI’s new Magic Avatar feature and then sharing on social media. A guy on Twitter even used OpenAI’s new machine learning-powered chatbot, ChatGPT, to imitate what I said on a recent podcast (which, coincidentally, was also about ChatGPT) and posted it online.

Welcome to the age of generative AI, when it’s now possible for anyone to create new, original illustrations and text by simply sending a few instructions to a computer program. Several generative AI models, including ChatGPT and an image generator called Stable Diffusion, can now be accessed online for free or for a low-cost subscription, which means people across the world can do everything from assemble a children’s book to produce computer code in just a few clicks. This tech is impressive, and it can get pretty close to writing and illustrating how a human might. Don’t believe me? Here’s a Magic School Bus short story ChatGPT wrote about Ms. Frizzle’s class trip to the Fyre Festival. And below is an illustration I asked Stable Diffusion to create about a family celebrating Hanukkah on the moon.

Stable Diffusion

Generative AI’s results aren’t always perfect, and we’re certainly not dealing with an all-powerful, super AI — at least for now. Sometimes its creations are flawed, inappropriate, or don’t totally make sense. If you were going to celebrate Hanukkah on the moon, after all, you probably wouldn’t depict giant Christmas ornaments strewn across the lunar surface. And you might find the original Magic School Bus stories more entertaining than my AI-generated one.

Still, even in its current form and with its current limitations, generative AI could automate some tasks humans do daily — like writing form emails or drafting simple legal contracts — and possibly make some kinds of jobs obsolete. This technology presents plenty of opportunities, but plenty of complex new challenges, too. Writing emails may suddenly have gotten a lot easier, for example, but catching cheating students has definitely gotten a lot harder.

It’s only the beginning of this tech, so it can be hard to make sense of what exactly it is capable of or how it could impact our lives. So we tried to answer a few of the biggest questions surrounding generative AI right now.

Wait, how does this AI work?

Very simply, a generative AI system is designed to produce something new based on its previous experience. Usually, this technology is developed with a technique called machine learning, which involves teaching an artificial intelligence to perform tasks by exposing it to lots and lots of data, which it “trains” on and eventually learns to mimic. ChatGPT, for example, was trained on an enormous quantity of text available on the internet, along with scripts of dialogue, so that it could imitate human conversations. Stable Diffusion is an image generator created by the startup Stability.AI that will produce an image for you based on text instructions, and was designed by feeding the AI images and their associated captions collected from the web, which allowed the AI to learn what it should “illustrate” based on the verbal commands it received.

While the particular approaches used to build generative AI models can differ, this technology is ultimately trying to reproduce human behavior, creating new content based on the content that humans have already created. In some ways, it’s like the smart compose features you see on your iPhone when you’re texting or your Gmail account when you’re typing out an email. “It learns to detect patterns in this content, which in turn allows it to generate similar but distinct content,” explains Vincent Conitzer, a computer science professor at Carnegie Mellon.

This method of building AI can be extremely powerful, but it also has real flaws. In one test, for example, an AI model called Galactica that Meta built to help write scientific papers suggested that the Soviet Union was the first country to put a bear in space, among several other errors and falsehoods. (The company pulled the system offline in November, after just a few days.) Lensa AI’s Magic Avatar feature, the AI portrait generator, sometimes illustrates people with additional limbs. It also has the concerning tendency to depict women without any clothing.

It’s easy to find other biases and stereotypes built into this technology, too. When the Intercept asked ChatGPT to come up with an airline passenger screening system, the AI suggested higher risk scores for people from — or who had visited — Syria and Afghanistan, among other countries. Stable Diffusion also reproduces racial and gender stereotypes, like only depicting firefighters as white men. These are not particularly new problems with this kind of AI, as Abeba Birhane and Deborah Raji recently wrote in Wired. “People get hurt from the very practical ways such models fall short in deployment, and these failures are the result of their builders’ choices — decisions we must hold them accountable for,” they wrote.

Who is creating this AI, and why?

Generative AI isn’t free out of the goodness of tech companies’ hearts. These systems are free because the companies building them want to improve their models and technology, and people playing around with trial versions of the software give these companies, in turn, even more training data. Operating the computing systems to build artificial intelligence models can be extremely expensive, and while companies aren’t always upfront about their own expenses, costs can stretch into the tens of millions of dollars. AI developers want to eventually sell and license their technology for a profit.

There are already hints about what this new generative AI industry could look like. OpenAI, which developed the DALL-E and ChatGPT systems, operates under a capped-profit model, and plans to receive $1 billion in revenue by 2024, primarily through selling access to its tech (outside developers can already pay to use some of OpenAI’s tech in their apps). Microsoft has already started to use the system to assist with some aspects of computer programming in its code development app. Stability AI, the Stable Diffusion creator, wants to build specialized versions of the technology that it could sell to individual companies. The startup raised more than $100 million this past October.

Some think ChatGPT could ultimately replace Google’s search engine, which powers one of the biggest digital ad businesses in the world. ChatGPT is also pretty good at some basic aspects of coding, and technologies like it could eventually lower the overall costs of developing software. At the same time, OpenAI already has a pricing program available for DALL-E, and it’s easy to imagine how the system could be turned into a way of generating advertisements, visuals, and other graphics at a relatively low cost.

Is this the end of homework?

AI tools are already being used for one obvious thing: schoolwork, especially essays and online exams. These AI-produced assignments wouldn’t necessarily earn an A, but teachers seem to agree that ChatGPT can create at least B-worthy work. While tools for detecting whether a piece of text is AI generated are emerging, the popular plagiarism detection software, Turnitin, won’t catch this kind of cheating.

The arrival of this tech has driven some to declare the end of high school English, and even homework itself. While those predictions are hyperbolic, it’s certainly possible that homework will need to adapt. Some teachers may reverse course on the use of technology in the classroom and return to in-person, paper-based exams. Other instructors might turn to lockdown browsers, which would prevent people from visiting websites during a computer-based test. The use of AI itself may become part of the assignment, which is an idea some teachers are already exploring.

“The sorts of professionals our students want to be when they graduate already use these tools,” Phillip Dawson, the associate director of the Centre for Research in Assessment and Digital Learning, told Recode in December. “We can’t ban them, nor should we.”

Is AI going to take my job?

It’s hard to predict which jobs will or won’t be eradicated by generative AI. Greg Brockman, one of OpenAI’s co-founders, said in a December tweet that ChatGPT is “not yet ready to be relied on for anything important.” Still, this technology can already do all sorts of things that companies currently need humans to do. Even if this tech doesn’t take over your entire job, it might very well change it.

Take journalism: ChatGPT can already write a pretty compelling blog post. No, the post might not be particularly accurate — which is why there’s concern that ChatGPT could be quickly exploited to produce fake news — but it can certainly get the ball rolling, coming up with basic ideas for an article and even drafting letters to sources. The same bot can also earn a good score on a college-level coding exam, and it’s not bad at writing about legal concepts, either. A photo editor at New York magazine pointed out that while DALL-E doesn’t quite understand how to make illustrations dealing with complex political or conceptual concepts, it can be helpful when given repeated prodding and explicit instructions.

While there are limits on what ChatGPT could be used for, even automating just a few tasks in someone’s workflow, like writing basic code or copy editing, could radically change a person’s workday and reduce the total number of workers needed in a given field. As an example, Conitzer, the computer science professor, pointed to the impact of services like Google Flights on travel agencies.

“Online travel sites, even today, do not offer the full services of a human travel agent, which is why human travel agents are still around, in larger numbers than many people expect,” he told Recode. “That said, clearly their numbers have gone down significantly because the alternative process of just booking flights and a place to stay yourself online — a process that didn’t exist some decades ago — is a fine alternative in many cases.”

Should I be worried?

Generative AI is going mainstream rapidly, and companies aim to sell this technology as soon as possible. At the same time, the regulators who might try to rein in this tech, if they find a compelling reason, are still learning how it works.

The stakes are high. Like other breakthrough technologies — things like the computer and the smartphone, but also earlier inventions, like the air conditioner and the car — generative AI could change much of how our world operates. And like other revolutionary tech, the arrival of this kind of AI will create complicated trade-offs. Air conditioners, for example, have made some of the hottest days of the year more bearable, but they’re also exacerbating the world’s climate change problem. Cars made it possible to travel extremely long distances without the need for a train or horse-drawn carriage, but motor vehicle crashes now kill tens of thousands of people, at least in the United States, every year.

In the same way, decisions we make about AI now could have ripple effects. Legal cases about who deserves the profit and credit — but also the liability — for work created by AI are being decided now, but could shape who profits from this technology for years to come. Schools and teachers will determine whether to incorporate AI into their curriculums, or discard it as a form of cheating, inevitably influencing how kids will relate to these technologies in their professional lives. The rapid expansion of AI image generators could center Eurocentric art forms at the expense of other artistic traditions, which are already underrepresented by the technology.

If and when this AI goes fully mainstream, it could be incredibly difficult to unravel. In this way, the biggest threat of this technology may be that it stands to change the world before we’ve had a chance to truly understand it.

This story was first published in the Recode newsletter. Sign up here so you don’t miss the next one!