Base_ac_agent

class conformer_rl.agents.base_ac_agent.BaseACAgent(config: conformer_rl.config.agent_config.Config)

Bases: conformer_rl.agents.base_agent.BaseAgent

Base interface for building reinforcement learning agents that use actor-critic algorithms.

Parameters

config (Config) – Configuration object for the agent. See notes for a list of config parameters used by specific pre-built agents.

step() → None

Performs one iteration of acquiring samples on the environment and then trains on the acquired samples.

_sample() → None

Collects samples from the training environment.

_calculate_advantages() → None

Performs advantage estimation.

Uses either SARSA or generalized advantage estimation (GAE) for estimating advantages, depending on the config.

_eval_episode() → dict

Evalutes the agent on a single episode of the evaluation environment.

Returns

Information from the evaluation environment to be logged by the eval_logger.

Return type

dict

evaluate() → None

Evaluates the agent on the evaluation environment.

Information dict returned by the environment’s conformer_rl.environments.conformer_env.ConformerEnv.step() method is logged by the eval_logger and saved.

load(filename: str) → None

Loads the neural network with weights.

Parameters

filename (str) – The path where the neural network weights are saved.

run_steps() → None

Trains the agent.

Trains the agent until the maximum number of steps (specified by config) is reached. Also periodically saves neural network parameters and performs evaluations on the agent, if specified in the config.

save(filename: str) → None

Saves the neural network weights to a file.

Parameters

filename (str) – The path where the neural network weights are to be saved.