Running NRP-Core experiments from Python

NRP-Core experiments can be run from the provided application, NRPCoreSim. In this way, the experiment is loaded from a specified configuration file and run until the specified timeout. Additionally, we provide a Python module, nrp_client.NrpCore, which can be used to start experiments from Python and which enable more control over the experiment execution.

The main class used to launch and control NRP-Core experiment is nrp_client.NrpCore. When instantiated, it internally launches an NRPCoreSim process in server mode and forwards requests made through its methods to the NRPCoreSim GRPC server. The NRPCoreSim process can be launched either as a forked process or inside of a docker container. More details on these options are given below.

nrp_client.NrpCore provides methods which match the available services in NRPCoreSim server: initialize(), run_loop(int number_of_iterations), run_until_timeout(), stop(), reset() and shutdown(). Each of these methods calls the corresponding GRPC service as described here.

Below there is an example on how to use NrpCore for launching the tf_exchange experiment and running it:

# Import NrpCore
from nrp_client import NrpCore
# Start NRPCoreSim server with tf_exchange experiment and connects to it
n = NrpCore('localhost:5345',"nrp-core/examples/tf_exchange","simulation_config.json")
# Initialize the experiment
n.initialize()
# Run the simulation loop for 10 iterations
n.run_loop(10)
# Check the current state of the experiment
n.current_state()
# Resets the experiment
n.reset()
# Run the experiment until timeout
n.run_until_timeout()
# Shut down the experiment
n.shutdown()

NrpCore takes as arguments:

  • address (str): the address that will be used by NRPCoreSim server

  • experiment_folder (str): path to the folder containing all files needed to execute the experiment. By default is an empty string, which is interpreted as the current folder

  • config_file (str): path to the experiment configuration file. It can be an absolute path or relative to experiment_folder. By default, “simulation_config.json”

  • args (str): additional NRPCoreSim command line parameters

  • log_output (str): if not empty, console output from NRPCoreSim process will be logged into a file with the given name, relative to the experiment directory

  • server_timeout (int): time in seconds for waiting for connecting to the NRPCoreSim grpc server. 10 seconds by default.

  • docker_daemon_address (str): IP address of the docker daemon used to launch the experiment in a docker container (more detail below). ‘unix:///var/run/docker.sock’ by default, which is the value expected when docker daemon is running locally with its default configuration.

  • image_name (str): name of the docker image which will be used to run the experiment (more detail below). Empty by default.

  • get_archives (list): list of archives (files or folders) that should be retrieved from the docker container when shutting down the simulation (eg. folder containing logged simulation data from a data transfer engine). Empty list by default.

Each of the methods provided to control the experiment returns a boolean value which is True if the request was processed correctly and False otherwise. They can return False either because the request was not possible from the current state of the experiment (see here for more information on the experiment lifecycle FSM) or because there was an error while executing the request. In the latter case the experiment is transitioned to Failed state and only shutdown() is possible.

Finally, all the request methods are blocking, i.e. the return only after the request has been processed and the corresponding GRPC service returns. With the exception of run_loop() and run_until_timeout(), which accepts a parameter run_async (bool) which allows to execute the request in a separate thread, and therefore non-blockingly. When called with run_async=True they return a Thread object which can be used to monitor the state of the request. In combination with the stop() request and the datatransfer engine, this option could be used to monitor the simulation and stop it after some condition fulfills or visualize or process experiment data in realtime.

Passing data between client and engines

It is possible to send data from the Python client directly to the Engines that take part in the simulation. The data can be in one of the following formats:

  • a Python dictionary that can be serialized into a JSON string

  • any of the protobuf messages available for the engines

The data should be passed to the NRP Core client object using the following methods:

nrp_core = NrpCore('localhost:5345',"nrp-core/examples/tf_exchange","simulation_config.json")

# JSON-serializable objects
data = {}
data["test_data"] = 888
nrp_core.set_json_datapack("actions", "python_1", data)

# protobuf objects
data = TestPayload()
data.integer = 555
nrp_core.set_proto_datapack("actions", "python_1", data)

# Pass the data to the engines
nrp_core.run_loop(1)

The data will be passed to NRP Core on the first subsequent call to the run_loop() service. NRP Core will convert the data into DataPacks using the datapack name and engine name specified as arguments to the set_x_datapack methods. These DataPacks will be then forwarded to the proper engines.

The same mechanism can be used to pass DataPacks to the reset() engine function:

# JSON-serializable objects
data = {}
data["test_data"] = 888
nrp_core.set_json_datapack("actions", "python_1", data)

# protobuf objects
data = TestPayload()
data.integer = 555
nrp_core.set_proto_datapack("actions", "python_1", data)

# Pass the data to the engines
nrp_core.reset()

The data (observations) may also be passed from the engines to the client, using the so-called Status Function. The Status Function is capable of producing observations continuously when the simulation is executed step-by-step with the run_loop() service. It can also return trajectories (observations accumulated over multiple simulation steps) in case of episode-based simulations that are executed using run_until_timeout() service.

The run_loop() method returns a tuple that consists of the done_flag and the trajectory (a list of observations). Both are generated by the Status Function. If the done_flag is False, the trajectory will be empty.

The run_until_timeout() returns a tuple with the following:

  • done_flag - when set to True, the simulation episode has ended. Set by the Status Function.

  • timeout_flag - when set to True, the simulation has reached the timeout define in the simulation config. Set internally by NRP Core.

  • trajectory - a list of observations accumulated during the episode. The trajectory will be returned no matter which of the two status flags (done_flag, timeout_flag) is set. The order of observations returned from the Status Function is preserved.

More information about how to write the Status Function can be found on this page.

# Step-by-step execution
done_flag, trajectory = self.nrp_core.run_loop(1)
if done_flag:
    for observation in trajectory:
        # Process the observation

# Episode based execution
done_flag, timeout_flag, datapacks = self.nrp_core.run_until_timeout()
if timeout_flag:
    # The simulation timed out, but we may still want to process the trajectory
    for observation in trajectory:
        # Process the observation
if done_flag:
    # The episode has ended
    for observation in trajectory:
        # Process the observation

Additionally, DataPacks can be returned from initialize() and reset() service calls:

# Main script

observations = self.nrp_core.initialize()
for observation in observations:
    # Process the observation

observations = self.nrp_core.reset()
for observation in observations:
    # Process the observation

Vectorization of NRP Core clients using gym interface

An easy way to vectorize NRP Core environments is to create a simple wrapper with gym interface around the NRP Core Python client, and to use this wrapped environment with e.g. SubprocVecEnv provided with stable baselines. An example of such interface, together with the vectorization process, is present in examples/nrp_vectorization.

Launching Experiments in Docker Containers

By default, when instantiating a NrpCore object, NRPCoreSim is started in a forked process. In case of willing to run NRPCoreSim in a docker container instead, set the parameters image_name, and optionally docker_daemon_address, as described above.

When running NRPCoreSim in a container from the NrpCore Python client, the behavior of the latter remains unchanged, i.e. as explained above in this page. The only noticeable difference / constraint, is that the path specified in the config_file parameter of NrpCore client must be relative to experiment_folder, and the configuration file should be itself placed inside of the experiment folder. This is the normal case anyways, therefore this constraint will go unnoticed most of the times.

As explained above, if log_output is set to True when instantiating the NrpCore client, the experiment console output is logged to a file named .console_output.log. When running the experiment in a docker container this file is fetched from the container and saved to experiment_folder when the experiment is shutdown.

In the same way, it is possible to specify in the NrpCore constructor an additional list of archives (files or folders) that should be fetched from the container and saved to experiment_folder when the experiment is shutdown. These archives are passed to the constructor using the get_archives parameter.

The latter is useful for example when including a Data Transfer Engine in the experiment. In this case datapacks are either streamed to an MQTT topic (which can be subscribed to from outside the container) or logged into a file. These files are stored into a folder (data by default) which can be added to the get_archives list parameter so the logs are retrieved from the container before stopping it.

NOTE:

In order to launch experiments in docker containers, a docker daemon must be accessible at the IP specified in the parameter docker_daemon_address. See this guide for more information on how to do this.

Also keep in mind that the address of the NRPCoreSim server (‘address’ parameter in the NrpCore constructor) will be an address in the host where docker daemon is running, and must be accessible from the host where NrpCore is instantiated.