Today we are sharing Continuo, a project exploring a question that guides our work on physical intelligence: what can an intelligent system learn when it stays with people, observes everyday life as it unfolds, and connects those experiences over time?
Continuo currently runs on smart glasses as an always-on AI companion, building a lifelog from what the wearer sees, hears, and does. It connects new observations with past experience to maintain a continuously updated understanding of what is happening, what matters to the wearer, and what may be worth doing next.
Through Continuo, we are exploring how continuous observation and memory can support decisions and actions. This is a step toward our long-term goal at Action Intelligence: building a robotic system that can understand the physical world and act within it.
At ECCV, we are putting this idea into practice in a conference setting. Continuo accompanies participants through posters, talks, and conversations, helping them keep track of papers and people, connect ideas across encounters, remember unfinished threads, and leave with a personal ECCV Memory.
The physical world does not reset between prompts
Most current AI systems begin from context supplied at the moment of interaction. A user asks a question, provides files or images, and the system reasons within that temporary context. Physical environments are different. What matters now may depend on something that happened hours earlier. A plan that was sensible in the morning may become irrelevant in the afternoon. Something initially unimportant may later become central because of a new encounter.
This changes the technical problem. A system cannot simply keep accumulating observations. It has to maintain and update an internal state of the world: what happened, what remains relevant, what changed, and which open threads still matter. Long-term intelligence is therefore not only a matter of increasing context length. It requires memory selection, temporal reasoning, state updating, and continual replanning.
Continuo is built around this idea. At ECCV, rather than treating each poster or conversation as an independent interaction, it continuously updates structured memory connecting people, papers, topics, interests, intentions, and previous encounters. New observations can reinforce a connection, create one, or make an earlier assumption obsolete.
Understanding continuous video
A video-language model operating continuously must track actions, interactions, and state changes across an open-ended stream. Adjacent frames often contain little new information, while interpreting a brief event may require context from much earlier. Our modeling work focuses on preserving these temporal dependencies while reducing computation on redundant observations.
In Continuo, this calls for low-latency streaming inference that updates the model’s internal state incrementally. Perception and response generation must also be decoupled: the system should continue interpreting new observations and updating memory while generating a response only when needed.
Memory for reasoning and action
We represent experience as compact captions enriched with visual, auditory, and temporal information. These memory entries capture entities, events, and their semantic and temporal relationships, connecting observations to the wearer’s interests, prior experiences, and unresolved intentions, questions, or tasks.
As experience accumulates, the system must retrieve relevant memories, incorporate new evidence, and revise earlier assumptions or plans. Following an ongoing task, for example, requires remembering its goal, what has already happened, and what remains to be done. We therefore treat video understanding, memory, and reasoning as a coupled problem: current goals guide what the system retains and retrieves, while accumulated experience informs its next decisions.
Privacy by design
We are developing a local processing layer that uses small models to detect events, filter redundant input, and flag sensitive situations—initially on a paired phone, with more computation moving onto the glasses as hardware improves. The design calls for masking faces and identifying details before encoding selected inputs into tokens, which are encrypted for transmission to the cloud. Raw audio and video should remain in temporary memory without being written to persistent storage; transmission must stop if required privacy checks fail.
For ECCV, deployment requirements include explicit consent, separate opt-in for transcription, visible sensing indicators, immediate pause controls, and EU-based processing. Tokens and generated memories remain subject to access controls and retention limits. Participants choose whether to keep their memories after the session and can request deletion.
Staying with humans first
There is a practical reason to develop these capabilities alongside humans before asking robots to operate autonomously in open environments. Humans provide a rich stream of perception, intention, correction, and action. Over the course of an ordinary activity, a person continuously reveals what they care about, which observations matter, when a plan should change, and how ambiguous situations are resolved.
A system that stays with a human for an extended period can therefore be evaluated against more than recognition accuracy:
- Does it remember the right things?
- Does it understand which connections are useful?
- Does it adapt as the participant’s goals change?
- Does it make a suggestion at the right time?
- Does that suggestion actually help?
This creates an iteration loop that is especially valuable at the current stage of development. The first objective is not maximum autonomy, but sustained usefulness. Long-running interaction exposes errors in memory, reasoning, relevance estimation, and planning that are difficult to find in static benchmarks. With explicit participant feedback, those failures can guide the next round of system development.
The same underlying capabilities become increasingly important as intelligence moves from assisting humans to controlling physical agents. A household robot also needs to remember earlier states, understand how a task is progressing, detect when assumptions no longer hold, and replan from new observations.
ECCV as a real-world test
A research conference is a compact but demanding environment for this experiment. Over several days, a participant may see dozens of papers, meet many researchers, change plans repeatedly, and accumulate more information than they can reasonably organize. Each individual interaction is relatively simple. The difficulty lies in preserving the right relationships across them.
At the end of the experience, Continuo turns its accumulated state into a personal ECCV Memory: papers that remained relevant, people and conversations worth following up on, topics that emerged repeatedly, connections discovered across sessions, and unresolved questions or actions. The goal is not a transcript of the conference. It is to reconstruct the part of the conference that mattered to that participant.
For us, the more interesting output is what happens before the final report. Can the system maintain coherent state for hours? Can it forget appropriately? Can it recover an earlier piece of context at the right moment? Can it update a plan when circumstances change? Can it remain useful without becoming intrusive?
These are small versions of much larger problems in physical intelligence. Continuo does not solve them. It gives us a place to work on them in the real world. A series of research are conducting within Action Intelligence.
If you are attending ECCV 2026, visit Action Intelligence at Booth #44 or book a live tour to experience Continuo.
Stay with humans first. Learn how the world unfolds. Then act in it.