hive.replays.recurrent_replay module
- class hive.replays.recurrent_replay.RecurrentReplayBuffer(capacity=10000, max_seq_len=1, n_step=1, gamma=0.99, observation_shape=(), observation_dtype=<class 'numpy.uint8'>, action_shape=(), action_dtype=<class 'numpy.int8'>, reward_shape=(), reward_dtype=<class 'numpy.float32'>, extra_storage_types=None, hidden_spec=None, num_players_sharing_buffer=None)[source]
Bases:
CircularReplayBufferFirst implementation of recurrent buffer without storing hidden states
Constructor for RecurrentReplayBuffer.
- Parameters
capacity (int) – Total number of observations that can be stored in the buffer. Note, this is not the same as the number of transitions that can be stored in the buffer.
max_seq_len (int) – The number of consecutive transitions in a sequence sampled from an episode.
n_step (int) – Horizon used to compute n-step return reward
gamma (float) – Discounting factor used to compute n-step return reward
observation_shape – Shape of observations that will be stored in the buffer.
observation_dtype – Type of observations that will be stored in the buffer. This can either be the type itself or string representation of the type. The type can be either a native python type or a numpy type. If a numpy type, a string of the form np.uint8 or numpy.uint8 is acceptable.
action_shape – Shape of actions that will be stored in the buffer.
action_dtype – Type of actions that will be stored in the buffer. Format is described in the description of observation_dtype.
reward_shape – Shape of rewards that will be stored in the buffer.
reward_dtype – Type of rewards that will be stored in the buffer. Format is described in the description of observation_dtype.
extra_storage_types (dict) – A dictionary describing extra items to store in the buffer. The mapping should be from the name of the item to a (type, shape) tuple.
num_players_sharing_buffer (int) – Number of agents that share their buffers. It is used for self-play.
- add(observation, next_observation, action, reward, terminated, truncated, **kwargs)[source]
Adds a transition to the buffer. The required components of a transition are given as positional arguments. The user can pass additional components to store in the buffer as kwargs as long as they were defined in the specification in the constructor.