Running the example experiments¶
In this section, we show how to run some of the example experiments. Different experiments utilize different features of the NRP, the use of which will be highlighted.
Experiments are run using the provided application NRPCoreSim. It takes as argument the simulation configuration file used to configure the experiment. Optionally a list of engine plugins to be loaded at run time can be specified. This can be done using the “-p” parameter. If “-p” is not used, all engine plugins built with nrp-core will be loaded.
Husky braitenberg¶
This experiment replicates the Husky Braitenberg experiment often used in previous versions of the NRP to exemplify its functionality. More details about the experiment can be found here. It displays a virtual robot connected to a brain reacting to color stimuli.
The experiment engages two simulators, Gazebo as robot simulator, and NEST as neural network simulator.
Both Gazebo and NEST are wrapped into so-called engines, which allow the NRP to manage simulation execution and synchronization. Data exchange between the simulators is done with the help of transceiver functions and datapacks. In short, the transceiver functions allow to translate output of one simulator into input of another one, while the datapacks serve as generic containers for the data.
The Gazebo engine starts Gazebo server (gzserver), so you should be able to peek into the simulation by connecting with Gazebo client (gzclient) from a separate terminal.
To run the example, if you haven’t done it yet, you have to install first some additional Gazebo models. Instructions on how to do it can be found in this page: Additional Models for Braitenberg Husky experiments. Once this is done, switch to the proper directory and run the NRP simulation executable:
cd examples/husky_braitenberg NRPCoreSim -c simulation_config.json
Please note that the NRPCoreSim must be executed from the example directory!
Here is a short description of the files participating in the experiment after running the command above:
simulation_config.json - simulation configuration file. An explanation of the simulation configuration can be found here.
braitenberg.py - contains the setup of the neuronal network running the experiment. Used by the NEST engine.
husky_world.sdf - simulation world file in Simulation Description Format. Used by the Gazebo engine.
husky.sdf - husky robot file in Simulation Description Format. Used by the Gazebo engine.
cam_pf.py - preprocessing function which takes images acquired by the robot and detect levels of red. The result is stored in a JsonDataPack JsonDataPack which can be accessed by other transceiver functions in the experiment.
brain_stimulation_tf.py - takes the JsonDataPack produced by the preprocessing function described above and estimulates the NEST neural network in function of the detected red values.
mot_tf.py - transceiver function that converts output from NEST into movement commands.
Experiment Variants¶
In the same experiment folder husky_braitenberg are provided two other configuration files which runs slightly different versions of the same experiment but exemplifying other engines:
simulation_config_nest_server.json - replaces the NEST JSON Engine used in the original experiment with NEST server. In this variant the neural network used in the experiment is defined in the file braitenberg_nest_server.py.
simulation_config_data_transfer.json - in this configuration file a DataTransferEngine is added to the experiment. This engine is used by an additional trasceiver function defined in data_transfer_tf.py for logging and streaming several datapacks.
Each of these variants can be run as the original one but changing simulation_config.json with one of the configuration files listed above.
DataPack Exchange using the Python JSON Engine¶
This simple example shows two instances of Python JSON Engine exchanging data as datapacks. Data exchange between the simulators is done with the help of Transceiver Functions (TFs) and datapacks.
The PythonJSONEngine implements a Python class, EngineScript, the methods initialize, runLoop and shutdown of which can be overriden to execute any arbitrary piece of Python code synchronously in a NRP experiment. This opens the possibility of easily integrating any simulator with a Python API in an experiment (e.g. OpenSim, OpenAI Gym, etc.). Under the hood, EngineScript manages DataPack I/O operations with the Simulation Loop.
In the example, which can be found in examples/tf_exchange folder, an engine defined in engine_1.py file registers a datapack of type JsonDataPack with id “datapack1”. The datapack stores a dictionary with the current simulation time in the engine and the number of simulation steps the engine has already advanced. Then a TF defined in tf_1.py gets this datapack and relays it to a second engine defined in engine_2.py, which simply prints out its data.
Python JSON Engine only supports JsonDataPack datapack type, which stores a wraps a JSON object in an attribute data. Therefore it allows to send any kind of data between Engines and TFs with the only constraint of it being JSON serializable. JsonDataPack is defined in the nrp_core.nrp_json module from which it can be imported. Eg:
from nrp_core.data.nrp_json import JsonDataPack
To launch the example, just execute:
cd examples/tf_exchange NRPCoreSim -c simulation_config.json
In this case, no additional plugin needs to be specified.
Pysim experiments¶
These three experiments show the use of the Pysim engine. In the three of them a Pysim engine is used to simulate the environment (in Mujoco, Opensim and OpenAI respectively) which is controlled by another Engine (either NRPNestJSONEngine or NRPPythonJSONEngine) through NRP-Core. The communication between the simulated environment and the controller is done with the help of transceiver functions and engine datapacks. In short, as usual the transceiver functions allow translating the I/O data between different platforms, while the datapacks serve as generic containers for the data.
To run the experiments:
cd examples/pysim_examples/<experiment_folder> NRPCoreSim -c simulation_config.json
This will start the simulation including visualization of the environment.
Below are described each of the experiments in more detail.
Example for OpenAI simulation¶
This experiment implements a NEST controller for a simple OpenAI model of a car between two mountain (MountainCar-v0, see https://gym.openai.com/envs/MountainCar-v0/) in an OpenAI scene.
Here is a short description of all files that are located in the example directory “openAI_nest_py”:
gym_simulator.py - contains the setup of the OpenAI simulation, including initialize, runLoop, reset, and shutdown
nest_controller.py - contains the setup of the NEST controller
from_nest.py - transceiver function that converts output from the NEST controller into simulation commands
from_gym.py - transceiver function which converts feedback observation data from OpenAI
simulation_config.json - simulation configuration file. An explanation of the simulation configuration can be found here
Example for Bullet simulation¶
This experiment implements a dummy python controller for a model of a car in Bullet.
Here is a short description of all files that are located in the example directory “bullet_control”:
bullet_simulator.py - contains the setup of the Bullet simulation, including initialize, runLoop, reset, and shutdown
controller.py - transceiver function which receives car position and feedbacks response action
simulation_config.json - simulation configuration file. An explanation of the simulation configuration can be found here
Example for Mujoco simulation¶
This experiment implements a dummy python controller for an easy Mujoco model of a swing pole.
Here is a short description of all files that are located in the example directory “mujoco_control”:
Folder “model” - include the simulation file of the Mujoco scene
simple_mujoco.py - contains the setup of the Mujoco simulation, including initialize, runLoop, reset, and shutdown
controller.py - contains the setup of the python controller
send_cmd.py - transceiver function that converts output from the Python controller into simulation commands
rec_pos.py - transceiver function which converts feedback position data from Mujoco
simulation_config.json - simulation configuration file. An explanation of the simulation configuration can be found here
Example for OpenSim simulation¶
This experiment implements a dummy python controller for a simple OpenSim model of the human arm (arm26, see https://simtk-confluence.stanford.edu:8443/display/OpenSim/Musculoskeletal+Models) in an OpenSim scene.
Here is a short description of all files that are located in the example directory “opensim_control”:
Folder “arm26” - include the simulation file of the OpenSim scene
server.py - contains the setup of the OpenSim simulation, including initialize, runLoop, reset, and shutdown
client.py - contains the setup of the python controller
send_cmd.py - transceiver function that converts output from the Python controller into simulation commands
rec_joints.py - transceiver function which converts feedback muscle data from OpenSim
simulation_config.json - simulation configuration file. An explanation of the simulation configuration can be found here
Multi robot experiment¶
This experiment shows how multiple robots simulated in a single Gazebo Engine can be controlled in an NRPCore experiment.
The experiment is a vairant of the Husky braitenberg experiment described above in this page. In this case two husky robots are spawn in the Gazebo simulated world. Each of them is controlled by a NEST JSON Engine with names “husky_brain_1” and “husky_brain_2”, each of them running the same Spiking Neural Network.
As in the single robot version of the experiment, TFs are used to process and rely camera data from Gazebo to NEST, and to process NEST model “output” and to translate into robot joint commands.
cd examples/husky_braitenberg_multi_robot NRPCoreSim -c simulation_config.json