Skip to content
Latent state

Dreamer

Dreamer learns a compact model of its world, then learns to act by practising inside its own imagination.

simulusion.create_model("dreamer")

"dreamer" is an alias for "dreamer-v1".

Get startedRead the paper
Return 75 → 493cartpole · 60k steps
Family
Latent state
Versions
dreamer-v1 (default), dreamer-v2
Environment backends
8, from DeepMind Control to Brax
Default task
walker-walk on DeepMind Control
Observations
64 × 64 pixel images
Imagination horizon
15 steps ahead

How Dreamer imagines. It never needs the real world to practise.

  1. 1 · OBSERVE

    Compress what it sees

    RSSM: 200 + 30 latent
  2. 2 · IMAGINE

    Dream ahead in its head

    imagine_horizon=15
  3. 3 · ACT

    Learn from imagined rewards

    actor-critic, λ=0.95
recording · dreamer · cartpole swing-up
Real environmentImagined, 28 steps aheadImagined by Dreamer, 28 steps ahead

See Dreamer imagine. Five real frames, then its own dream.

On the left, the cartpole simulator. On the right, Dreamer's world model. After five observed frames its eyes are closed: everything it shows comes from its own imagination, driven by the policy's actions.

Trained for
60k steps · 4 GB laptop GPU
Imagines
Open loop, from frame 6
Reproduce
python demos/dreamer_demo.py record
See all demos

How it compares. Same cartpole task, same starting score.

  • Dreamer60k steps493
  • PlaNet75 minutes435
  • DreamerV260k steps250

Evaluation return on DeepMind Control cartpole swing-up; higher is better. Dreamer and DreamerV2 trained for 60k environment steps on a 4 GB laptop GPU, PlaNet for 75 minutes.

Train it. Pass any setting straight to create_model.

train.py
import simulusion as simagent = sim.create_model(  "dreamer",  env="Pendulum-v1",  env_backend="gym",  total_steps=5_000,)agent.train()
Terminal
$ pip install "simulusion[gym]"$ simulusion models list$ python train.py

Common settings and their defaults

env_backendWhere the world comes from
"dmc"
envWhich task to learn
"walker-walk"
total_stepsHow long to train
5_000_000
imagine_horizonSteps imagined per update
15
batch_sizeSequences per update
50
action_repeatFrames per chosen action
2
Read the docsRead the Dreamer docsAll settingsBrowse all settings

Related models. Other models that imagine with a compact latent state.

DreamerV2Symlog two-hot heads and balanced KL, built for sharper memories.create_model("dreamer-v2")
PlaNetPlans each move by imagining many possible futures.create_model("planet")
ModularRSSMThe recurrent state-space model behind them, as building blocks.create_model("modular-rssm")

The research behind it.

ICLR 2020 · arXiv:1912.01603

Dream to Control: Learning Behaviors by Latent Imagination

Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi

Read on arXiv

Train your first world model today.

Open source, MIT licensed, and runs on a laptop GPU.

pip install simulusion
Get startedStar on GitHub