-
Research
Improving language models by retrieving from trillions of tokens
We explore an alternate path for improving language models: we augment transformers with retrieval over a database of text passages including web pages, books, news and code. We call our method...
-
Research
Exploring the beauty of pure mathematics in novel ways
More than a century ago, Srinivasa Ramanujan shocked the mathematical world with his extraordinary ability to see remarkable patterns in numbers that no one else could see. The self-taught...
-
Research
On the Expressivity of Markov Reward
Our main results prove that while reward can express many tasks, there exist instances of each task type that no Markov reward function can capture. We then provide a set of polynomial-time...
-
Research
Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons
Our brain has an amazing ability to process visual information. We can take one glance at a complex scene, and within milliseconds be able to parse it into objects and their attributes, like...
-
Company
Real-world challenges for AGI
When people picture a world with artificial general intelligence (AGI), robots are more likely to come to mind than enabling solutions to society’s most intractable problems. But I believe the...
-
Company
Opening up a physics simulator for robotics
When you walk, your feet make contact with the ground. When you write, your fingers make contact with the pen. Physical contacts are what makes interaction with the world possible. Yet, for such a...
-
Research
Stacking our way to more general robots
Picking up a stick and balancing it atop a log or stacking a pebble on a stone may seem like simple — and quite similar — actions for a person. However, most robots struggle with handling more...
-
Research
Predicting gene expression with AI
When the Human Genome Project succeeded in mapping the DNA sequence of the human genome, the international research community were excited by the opportunity to better understand the genetic...
-
Research
Nowcasting the next hour of rain
Our lives are dependent on the weather. At any moment in the UK, according to one study, one third of the country has talked about the weather in the past hour, reflecting the importance of...
-
Research
Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration
We argue that merely using curiosity for fast environment exploration or as a bonus reward for a specific task does not harness the full potential of this technique and misses useful skills....
-
Research
Challenges in Detoxifying Language Models
In our paper, we focus on LMs and their propensity to generate toxic language. We study the effectiveness of different methods to mitigate LM toxicity, and their side-effects, and we investigate...
-
Research
Building architectures that can handle the world’s data
Most architectures used by AI systems today are specialists. A 2D residual network may be a good choice for processing images, but at best it’s a loose fit for other kinds of data — such as the...
