Some very smart scientists have found a cool way to help artificial intelligence (AI) models understand languages better. They’ve developed a new type of machine learning that lets AI pick up languages quicker and more effectively. The trick? They actually teach the machines to forget things on purpose. It might sound strange, but it’s a pretty clever idea.
Jea Kwon, an AI engineer who works for the Institute for Basic Science in South Korea, believes this is a huge step forward. Normally, AI that can understand languages uses a system called artificial neural networks. Picture a massive network of neurons, similar to our brains, but instead of neurons, there are tiny calculators all talking to each other. Initially, their conversation is all over the place, but as they’re fed more and more text in various languages, they start to get the hang of how these languages work.
However, training these AI “brains” takes a ton of computer power. And if you want to teach them a new language or update something, it’s pretty tricky. Mikel Artetxe, one of the scientists behind this research, mentioned how tough it is to add a new language even to an AI that already knows 100 languages. Starting from zero is a big pain.
To get around this, Artetxe and his team came up with a smart solution. They trained their AI in one language, then made it forget the basic stuff about that language while keeping its overall knowledge. Next, they introduced a new language to the AI. And guess what? It worked! The AI could grasp the new language because it understood the concepts behind the words, not just the words themselves. Yihong Chen, who led this study, pointed out that this works because, across languages, we might use different words, but the ideas are often the same.
But this method of teaching an AI to forget and then learn a new language still required a lot of data and computing power. So Chen suggested letting the AI forget things here and there from the very beginning. This way, the AI gets good at forgetting and relearning, which makes it easier to teach it new languages down the line.
When they tested this idea with a well-known AI model, they found that their “forgetful” model, although not perfect, was quite adept at picking up new languages with less data and computing effort. Mikel Artetxe is excited that this method might help AI understand less common languages, like his own native Basque, more effectively.
The researchers believe this technique of forgetting and relearning helps the AI grasp languages more deeply, kind of like how we humans don’t remember every little detail but understand the main ideas. Benjamin Levy, a neuroscientist, mentioned that this is similar to how our memory works, focusing on the broader picture instead of every tiny fact.
Their ultimate goal is to see a future filled with diverse AI models that can quickly adapt to new languages or tasks, breaking away from the few large models that currently dominate. Chen dreams of a future where AI can easily switch to understanding new languages and concepts, making technology more accessible and helpful for everyone, no matter what language they speak.
This article is based on the following article:
Background Information
Understanding these concepts provides a solid foundation for comprehending how AI researchers are tackling the challenge of improving language understanding in AI systems. Their innovative approach of teaching AI models to forget certain aspects in order to learn new languages more efficiently could revolutionize the way AI systems understand and interact using human languages.
1. Artificial Intelligence (AI)
AI is a branch of computer science focused on creating systems that can perform tasks that normally require human intelligence. These tasks include learning, reasoning, problem-solving, understanding language, and recognizing patterns.
2. Machine Learning
A subset of AI, machine learning is the method by which computers improve at a task with experience. The more data a machine learning system is exposed to, the more it learns and the better it becomes at making decisions or predictions.
3. Neural Networks and Artificial Neural Networks
Neural networks are inspired by the human brain’s structure. Just like our brain is made up of neurons interconnected by synapses, artificial neural networks (ANNs) are made up of artificial neurons or nodes. These networks can learn to perform tasks by considering examples, generally without being programmed with any task-specific rules.
4. Language Understanding in AI
Understanding language is a complex task for AI. It involves not just recognizing words, but also understanding grammar, context, idioms, and the nuances of language. This is typically achieved through natural language processing (NLP), a field of AI focused on the interaction between computers and humans using natural language.
5. The Importance of Forgetting
The concept of intentionally making an AI forget certain information might seem counterintuitive. However, in human learning, forgetting is a natural process that helps our brains prioritize important information over less relevant details. By mimicking this process, AI systems can become more efficient in learning new information, such as languages, by focusing on the underlying concepts rather than holding onto every detail.
Computational Power
AI and machine learning models, especially those dealing with multiple languages, require significant computational power. This is the capacity of computer systems to perform the vast number of calculations needed for AI tasks. High computational power is crucial for processing and analyzing the large datasets used in training AI models.
6. The Challenge of Adding New Languages
Adding a new language to an AI model that already understands multiple languages is challenging. This is because the model must learn from scratch the new language’s unique structures, vocabulary, and rules, which requires a significant amount of data and computational resources.
Please subscribe to Insight Fortnight, our biweekly newsletter!