AI Learns to Predict Without Ever Seeing the Real World
Translated & summarized from NEWSru Israel by baba
Researchers from Tel Aviv University and Stanford University have trained an AI model to predict texts, sounds, and DNA without seeing real-world data. The model learned by playing a game where one AI generated tasks and another predicted outcomes, improving prediction quality across various data types. This method could make learning from real-world data more efficient, challenging the long-held assumption that extensive real data is always necessary for AI training.
The story in 6 lines · by baba
- An AI model was trained by generating its own data, learning to predict texts, sounds, and DNA.
- The training involved a 'generator' AI creating tasks and a 'student' AI predicting outcomes.
- The model demonstrated improved prediction quality across texts, images, sounds, and DNA sequences.
- Mathematical patterns like Fibonacci numbers emerged organically during the AI's training.
- This method could significantly enhance the efficiency of AI training on real-world data.
- Researchers claim to have disproven the assumption that AI needs vast amounts of real data to learn.
Researchers from Tel Aviv University and Stanford University have developed a novel artificial intelligence model capable of predicting texts, sounds, and DNA sequences without direct exposure to real-world data. Typically, AI models are trained on vast datasets like books, websites, and images to identify patterns. However, this new research, published on the arXiv preprint server, explored whether a model could achieve general learning abilities solely from data generated internally during the training process.
The training method involves a game between two models starting from scratch. A 'generator' model creates short programs that produce symbol sequences, while a 'student' model learns to predict the next symbol. The generator receives feedback on the usefulness of its tasks and gradually designs exercises at the edge of the student's current capabilities. The key test was transferability: although neither model had seen natural data, the trained student model was tested on texts, images, sounds, music, software code, and DNA sequences, demonstrating stable improvements in prediction quality across all domains as computational costs increased.
The model also learned to derive rules from examples during testing. Familiar mathematical patterns, including Fibonacci numbers, emerged organically within the tasks the generator created for the student. Furthermore, pre-training with this new method accelerated subsequent training on real-world data. The researchers emphasize that while this method cannot teach factual knowledge about the world, which requires experiencing it directly, it can significantly enhance the efficiency of learning from real data.
"For many years, it was assumed that to learn something useful, a model must first be 'fed' a huge amount of real data. We have disproven this fundamental assumption," stated Yoav Levin, the head of the work. The model does not acquire new information about the world; instead, it internalizes necessary skills through the game's rules, akin to how theorems are embedded within axioms. This process, requiring extensive computation, allows the model to develop skills for complex calculations.
