PDF document

[2310.16029] Finetuning Offline World Models in the Real World

In this work, we seek to get the best of both worlds: we consider the problem of pretraining a world model with offline data collected on a real robot, and then finetuning the model on online data collected by planning with the learned model.

Source: arxiv.org

Open / Download at source Report this link

Related files to “2310-16029-finetuning-offline-world-models-in-the-real-world”

PDF

Unifying Unsupervised and Offline RL for Fast Adaptation ...

In this letter we propose to consider both of these approaches together, resulting in a setting where different types of data streams are available and fast online adaptation to new tasks is required.

PDF

Efficient Reinforcement Learning by Guiding World Models with ...

During pre-training, rather than training separate models per task as in previous work [54–56], we train one world model per benchmark and demonstrate that a single multi-task & embodiment world model can effectively leverage non-curated data.

PDF

ICLR Combining Unsupervised and Offline RL via World Models

In this paper we propose to consider both of these approaches together and argue that this results in a more realistic setting where different types of data are available, and fast online adaptation to new tasks is required.

PDF

Efficient Reinforcement Learning by Guiding World Models with ...

The work addresses the challenge of improving sample efficiency in reinforcement learning (RL) by leveraging non-curated offline datasets which are reward-free, of mixed quality, and collected from multiple embodiments during online reinforcement learning.

PDF

Investigating Online RL in World Models - OpenReview

We propose an algorithm and a data curation method that addresses both of these concerns by demonstrating that effective full-length rollout training is possible without hand-crafted penalties by treating each member of the world model ensemble as a level in the Unsupervised Environment Design (UED) framework.