Storage

Storage utility class for agents.

class conformer_rl.agents.storage.Storage(rollout: int, workers: int)

Saves and stores experiences from the agent.

Parameters

• rollout (int) – The rollout length of the agent in each sampling iteration (per worker).

• workers (int) – The number of workers used for sampling by the agent.

__getitem__(key: str) → list

Returns all batched items in storage with the given key.

Parameters

key (str) – The key of the items to be accessed.

Returns

A list of all the items with the associated key previously appended to the storage.

Return type

list

append(data: dict) → None

Appends item(s) to the storage

Parameters

data (dict) – The items to be appended.

order(key: str) → list

Splits each batch of items associated with key by worker, and then orders the items into a list sorted firstly by worker and secondly by the order the items were appended to storage.

Parameters

key (str) – The key associated with the item to be retrieved.

Returns

The list of the items, split by batch and ordered.

Return type

list

Notes

An example of how the ordering is performed: Suppose the storage is initialized with rollout=3 and workers=2. Then in each sampling iteration of the agent, a batch of 2 items (one for each worker) will be appended to storage for a total of 3 iterations. Thus, at the end of the iterations, the storage value for any given item will look similar to this:

[[worker 1 sample 1, worker 2 sample 1],
 [worker 1 sample 2, worker 2 sample 2],
 [worker 1 sample 3, worker 2 sample 3]].

The list returned by order() would then be:

[worker 1 sample 1, worker 1 sample 2, worker 1 sample 3,
worker 2 sample 1, worker 2 sample 2, worker 3 sample 3].

The purpose of this ordering is to maintain chronologically consecutive blocks of rollout samples, which is useful for agents with recursive components.

reset() → None

Empties the storage.