Physical World Models

Research Theme

Sensor Signal Processing 

Autonomous Sensing Platforms 

Aim

This project aims to understand the physical dynamics in video through world models.

Objectives

1. Develop World Models that work in real-world data, not just in simulation, for Video Understanding

2. Develop World Models that are physically plausible

3. Develop methods that can understand cause and effect in video

Description

World Models are models that learn to link actions and consequences. They have shown excellent abilities to capture temporal dynamics for applications in robotics and video generation. However, they suffer from several issues: 1) they are trained to predict the “appearance” of the world, and therefore they may “look” accurate but be physically inaccurate: a table might bend, an object might disappear, etc. 2) they can be fairly short-term, as they are auto-regressive 3) they work mostly on synthetic scenes .

The goals of these projects are to address these three issues, with the final objective of having models that can understand temporal patterns in video at different timescales. This technology has a wide range of applications: from anomaly detection, to anticipating actions and events, or detecting previously unknown patterns in activity. 

Minimum entry qualification - an Honours degree at 2:1 or above (or International equivalent) in a relevant science or engineering discipline, possibly supported by an MSc Degree. 

Further information on English language requirements for EU/Overseas applicants.

This position is fully funded.