Applying control optimization directly to a physical system is both slow and risky. Instead controllers are usually optimized in simulation, and then fine-tuned on the physical system. This requires an accurate simulation model that runs orders-of-magnitude faster than real time. Contact dynamics are particularly hard to simulate accurately and efficiently. We are developing new algorithms to make this possible , that go beyond the linear complementarity approach used in existing engines such as ODE and PhysX. We are also implementing a new physics engine (MuJoCo) which is designed from the ground-up for the purpose of control optimization, and exploits the latest advances in parallel processing hardware. It combines our new algorithms for contact simulation with the fastest recursive methods for multi-joint dynamics. The above controllers for running, walking, and swimming were optimized using MuJoCo. Here is a movie of dancing-like behavior arising from an experimental modification of the equations of motion, without any control. MuJoCo will soon be made publicly availabe.