How to get free ~2x improvement in single env mentioned in the README?

[LucasAlegre]

Hi,

I tested running a single "Hopper-v4" environment using envpool, but the running time is actually slower.

env = envpool.make_gym('Hopper-v4', num_envs=1) -> 200k steps in 1min29s

env = gym.make('Hopper-v4') -> 200k steps in 43s

Am I missing something?

[Trinkle23897]

Could you please share your hardware configuration and the way that you test? Thanks!

[LucasAlegre]

My hardware is a GeForce GTX 1050, Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz and 32GB RAM. Ubuntu 22

I used the following code to test it:

import gym
import envpool
import numpy as np

#env = envpool.make_gym('Hopper-v4', num_envs=1)
env = gym.make('Hopper-v4')

env.reset()
for _ in range(200000):
    action = env.action_space.sample()
    obs, reward, done, info = env.step(action)
    # for envpool: obs, reward, done, info = env.step(np.array([action]))
    if done:
        env.reset()

[Trinkle23897]

In [18]: import gym
    ...: import envpool
    ...: import numpy as np
    ...: import time
    ...: env = envpool.make_gym('Hopper-v4', num_envs=1)
    ...: #env = gym.make('Hopper-v4')
    ...: env.action_space.seed(0)
    ...: n = 200000
    ...: env.reset()
    ...: t = time.time()
    ...: actions = np.array([env.action_space.sample() for _ in range(n)])
    ...: print(time.time() - t)
    ...: print(actions.shape)
    ...: t = time.time()
    ...: for i in range(n):
    ...:     #action = env.action_space.sample()
    ...:     #obs, reward, done, info = env.step(actions[i])
    ...:     #for envpool: 
    ...:     obs, reward, done, info = env.step(actions[i:i+1])
    ...:     if done:
    ...:         env.reset()
    ...: print(time.time() - t)

I'd recommend moving the random action part ahead of env.step. In the above example, my computer's result is:

gym

4.456811189651489
(200000, 3)
20.11348295211792

envpool

20.41367816925049
(200000, 3)
17.36430072784424

So the slowest part is the action generation. I tested with some v4 environments but unfortunately didn't see a significant speedup for a single env. That may be because deepmind's mujoco binary and python binding is highly optimized. However, if you test with dm_control environments, you'll definitely see a huge speedup because dm_control's python binding with mujoco is not optimized.