Introduction to Generative AI

A quick scan of the headlines makes it seem like generative artificial intelligence is everywhere these days. In fact, some of those headlines may actually have been written by generative AI, like OpenAI’s ChatGPT, a chatbot that has demonstrated an uncanny ability to produce text that seems to have been written by a human. But what do people really mean when they say “generative AI?”

What is Generative AI?

Generative AI can be thought of as a machine-learning model that is trained to create new data, rather than making a prediction about a specific dataset. A generative AI system is one that learns to generate more objects that look like the data it was trained on. Phillip Isola, an associate professor of electrical engineering and computer science at MIT, notes that the distinctions between generative AI and other types of AI can be a little bit blurry, as the same algorithms can be used for both.

History of Generative AI

Despite the hype that came with the release of ChatGPT and its counterparts, the technology itself isn’t brand new. These powerful machine-learning models draw on research and computational advances that go back more than 50 years. An early example of generative AI is a much simpler model known as a Markov chain, which was introduced by Russian mathematician Andrey Markov in 1906. In machine learning, Markov models have long been used for next-word prediction tasks, like the autocomplete function in an email program.

Increase in Complexity

In recent years, researchers have shifted their focus from finding a machine-learning algorithm that makes the best use of a specific dataset to using larger datasets to train models that can achieve impressive results. The base models underlying ChatGPT and similar systems work in much the same way as a Markov model, but with one big difference: they are far larger and more complex, with billions of parameters, and have been trained on an enormous amount of data.

More Powerful Architectures

Several major research advances have led to more complex deep-learning architectures, including generative adversarial networks (GANs), diffusion models, and transformer architectures. These models have been used to create realistic-looking images, generate text, and even design novel protein structures.

Applications of Generative AI

What all of these approaches have in common is that they convert inputs into a set of tokens, which are numerical representations of chunks of data. As long as your data can be converted into this standard, token format, then in theory, you could apply these methods to generate new data that look similar. This opens up a huge array of applications for generative AI, including creating synthetic image data, designing novel protein structures, and even generating text.

Potential Uses

For instance, Isola’s group is using generative AI to create synthetic image data that could be used to train another intelligent system. Jaakkola’s group is using generative AI to design novel protein structures or valid crystal structures that specify new materials. Devavrat Shah, the Andrew and Erna Viterbi Professor in Electrical Engineering and Computer Science at MIT, notes that generative AI models can become a terrific interface to machines that are human-friendly.

Raising Red Flags

While generative AI has the potential to revolutionize many fields, it also raises some concerns. Generative AI chatbots are now being used in call centers to field questions from human customers, but this application underscores one potential red flag of implementing these models — worker displacement. Additionally, generative AI can inherit and proliferate biases that exist in training data, or amplify hate speech and false statements.

Conclusion

In conclusion, generative AI is a powerful technology that has the potential to revolutionize many fields, from art and design to science and engineering. While it raises some concerns, such as worker displacement and bias, it also has the potential to empower artists, designers, and scientists to create new and innovative things. As research continues to advance, we can expect to see even more exciting applications of generative AI in the future. With its ability to generate new data, images, and text, generative AI is an exciting and rapidly evolving field that is worth keeping an eye on.