sharpie.runner.manage

Client using the asyncio API.

Functions

get_agent_data(…)	Safely extracts agent-specific data from potentially nested structures.
get_policy_actions(…)	Get actions from policies for all agents that have policies.
load_environment(…)	Create a new environment
load_episode(…)	Create a new episode
load_policies(…)	Create or load policies for every RL agent
main()	Start a runner and connect to the webserver
override_actions(…)	Applies human overrides to policy-suggested actions.
override_rewards(…)	Replaces environment rewards with human feedback where configured.
receive_message(…)	Receive actions from all participant-controlled agents.
run_episode(…)	Step through a whole episode, potentially with learning
sanitize_data(…)	Convert numpy types to JSON-serializable Python types.
send_message(…)	Send a message to the webserver.
start_experiment(…)	Connect to a room and run an episode
train_policies(…)	Coordinates training for all agents.
update_single_agent(…)	Executes the update call for a single policy module.

sharpie.runner.manage.sanitize_data(data)

Convert numpy types to JSON-serializable Python types.

Large numpy arrays (>= BASE64_THRESHOLD elements) are compressed with LZMA, pickled, and base64 encoded. Small arrays are converted to lists.

sharpie.runner.manage.send_message(websocket, env, step_count, terminated, truncated, obs, actions, reward)

Send a message to the webserver.

Contains: - An image of the environment - The current step - Gymnasium data (terminated, truncated, observations, rewards, actions)

sharpie.runner.manage.receive_message(websocket, agents_settings)

Receive actions from all participant-controlled agents.

Server batches all participant actions into a single message with ‘batch_actions’ key, which significantly reduces message overhead for large numbers of participants.

sharpie.runner.manage.get_policy_actions(obs, policy_modules, participant_inputs=None, agents_settings=None)

Get actions from policies for all agents that have policies.

Args:

obs: Current observation(s) from the environment policy_modules: Dictionary mapping agent names to their policy modules participant_inputs: Dictionary mapping agent names to participant input values agents_settings: Dictionary mapping agent names to their settings (including inputs_type)

Returns:

Dictionary mapping agent names to actions

sharpie.runner.manage.load_episode(websocket)

Create a new episode

Todo

Rename to create_episode or clarify usage

sharpie.runner.manage.load_environment(env_config)

Create a new environment

Todo

Rename to create_environment or clarify usage

sharpie.runner.manage.load_policies(agents_settings)

Create or load policies for every RL agent

Todo

Rename to create_policies or clarify usage

sharpie.runner.manage.override_actions(agents_settings, participant_inputs, policy_actions)

Applies human overrides to policy-suggested actions.

sharpie.runner.manage.override_rewards(agents_settings, participant_inputs, env_reward)

Replaces environment rewards with human feedback where configured.

sharpie.runner.manage.get_agent_data(agent_name, data, default=0)

Safely extracts agent-specific data from potentially nested structures.

sharpie.runner.manage.update_single_agent(agent_name, policy_module, s, a, r, s_prime, done)

Executes the update call for a single policy module.

sharpie.runner.manage.train_policies(policy_modules, intervals, step, s, a, r, term, trunc, settings)

Coordinates training for all agents. Determines eligibility based on intervals and delegates the update.

sharpie.runner.manage.run_episode(websocket, environment_settings, agents_settings, experiment_settings)

Step through a whole episode, potentially with learning

sharpie.runner.manage.start_experiment(hostname, port, connection_key, link, room)

Connect to a room and run an episode

Todo

Not sure

sharpie.runner.manage.main()

Start a runner and connect to the webserver