.. index:: pair: page; Python JSON Engine .. _doxid-python_json_engine: Python JSON Engine ================== This versatile engine enables users to execute a user-defined python script as an engine server, thus ensuring synchronization and enabling datapack data transfer with the Simulation Loop process. It can be used to integrate any simulator with a Python API in a NRP-core experiment. Engines based on PythonJSONEngine can be implemented as Python classes based on **EngineScript** (as in the example listed below). .. _doxid-python_json_engine_1python_json_engine_script: EngineScript ~~~~~~~~~~~~ ``EngineScript`` provides a base class from which custom Engines can inherit. The derived class must implement methods: * ``:ref:`initialize() ``` : executed when the engine is initialized * ``runLoop(timestep_ns)`` : executed when the engine is requested to advance its simulation (from EngineClient::runLoopStep) * ``:ref:`shutdown() ``` : executed when the engine is requested to shutdown Optionally, the derived class can implement a ``:ref:`reset() ``` function. If it is implemented, it will be used for resetting the Engine. Otherwise the Engine is reset by calling ``:ref:`shutdown() ``` and ``:ref:`initialize() ``` sequentially. Besides, ``EngineScript`` provides the following ready-to-use methods to handle datapack communication: * ``_time_ns`` : the internal simulation time of the Engine. * ``_getDataPack(datapack_name)`` : returns the latest value available of a datapack with name ``datapack_name`` * ``_setDataPack(datapack_name, data)`` : sets a new value for a datapack with name ``datapack_name``. ``data`` is always a Python dictionary. * ``_registerDataPack(datapack_name)`` : registers a datapack with the engine. Just registered datapacks can be ``set`` and ``get``. Under the hood, registered datapacks are sent to the corresponding :ref:`EngineClient ` upon request and their values updated when the :ref:`EngineClient ` send them. * ``_config`` : the engine configuration as a JSON object Below is an example of a class inheriting from ``EngineScript``. The example is taken from the ``examples/tf_exchange`` experiment. .. ref-code-block:: cpp """Python Engine 1. Will get current engine time and make it accessible as a datapack""" from nrp_core.engines.python_json import EngineScript class Script(EngineScript): def :ref:`initialize `(self): """Initialize datapack1 with time""" print("Engine 1 is initializing. Registering datapack...") self._registerDataPack("datapack1") self._setDataPack("datapack1", {"time": self._time_ns, "timestep": 0 }) def runLoop(self, timestep_ns): """Update datapack1 at every timestep""" self._setDataPack("datapack1", {"time": self._time_ns, "timestep": timestep_ns }) print("DataPack 1 data is " + :ref:`str `(self._getDataPack("datapack1"))) def :ref:`shutdown `(self): print("Engine 1 is shutting down") def :ref:`reset `(self): print("Engine 1 is resetting") .. _doxid-python_json_engine_1python_json_datapacks: DataPacks ~~~~~~~~~ The Python JSON engine supports a unique datapack type: *JsonDataPack*, which can be used to transfer information between the engine and TFs. The data contained in this datapack can be any JSON-serializable Python object; that is, any object that can be decoded/encoded by `JSONDecoder/JSONEncoder `__. This data can be accessed in TransceiverFunctions from the datapack *data* attribute, as shown in this example TF (also taken from ``examples/tf_exchange``): .. ref-code-block:: cpp from nrp_core import * from nrp_core.data.nrp_json import * @:ref:`EngineDataPack `(keyword='datapack_python', id=:ref:`DataPackIdentifier `('datapack1', 'python_1')) @:ref:`TransceiverFunction `("python_2") def transceiver_function(datapack_python): rec_datapack1 = :ref:`JsonDataPack `("rec_datapack2", "python_2") for k in datapack_python.data.keys(): rec_datapack1.data[k] = datapack_python.data[k] return [rec_datapack1] ========= ======================================================= ============ ============== Attribute Description Python Type C type ========= ======================================================= ============ ============== data data contained in the datapack as a NlohmannJson object NlohmannJson nlohmann::json ========= ======================================================= ============ ============== .. _doxid-python_json_engine_1python_json_configuration: Engine Configuration Parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This Engine type parameters are defined in the PythonJSONEngine schema (listed :ref:`here `), which in turn is based on :ref:`EngineBase ` and :ref:`EngineJSON ` schemas and thus inherits all parameters from them. To use the Python JSON engine in an experiment, set ``EngineType`` to **"python_json"**. * Parameters inherited from :ref:`EngineBase ` schema: ===================== =================================================================================================================================================== ====== ========================== ======== ===== Name Description Type Default Required Array ===================== =================================================================================================================================================== ====== ========================== ======== ===== EngineName Name of the engine string X EngineType Engine type. Used by string X EngineProcCmd Engine Process Launch command string EngineProcStartParams Engine Process Start Parameters string [] X EngineEnvParams Engine Process Environment Parameters string [] X EngineLaunchCommand object {"LaunchType":"BasicFork"} EngineTimestep Engine Timestep in seconds number 0.01 EngineCommandTimeout Engine Timeout (in seconds). It tells how long to wait for the completion of the engine runStep. 0 or negative values are interpreted as no timeout number 0.0 ===================== =================================================================================================================================================== ====== ========================== ======== ===== * Parameters inherited from :ref:`EngineJSON ` schema: ========================= =========== ====== ============== ======== ===== Name Description Type Default Required Array ========================= =========== ====== ============== ======== ===== ServerAddress string localhost:9002 RegistrationServerAddress Address string localhost:9001 ========================= =========== ====== ============== ======== ===== * Parameters specific to this engine type: ============== ================================================================================================================================================================================================================================== ====== ======= ======== ===== Name Description Type Default Required Array ============== ================================================================================================================================================================================================================================== ====== ======= ======== ===== PythonFileName Path to the Python script containing the engine definition string X ServerOptions Additional options that will be used by the server (gunicorn) on startup. The string should contain a Python dictionary in the following format - "{'key1': value, 'key2': 'value_str'}". The full list of options can be found at string ============== ================================================================================================================================================================================================================================== ====== ======= ======== ===== .. _doxid-python_json_engine_1python_json_schema: Schema ~~~~~~ As explained above, the schema used by the PythonJSON engine inherits from :ref:`EngineBase ` and :ref:`EngineJSON ` schemas. A complete schema for the configuration of this engine is given below: .. ref-code-block:: cpp {"python_base" : { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Python Engine Base", "description": "Python Engine Base Configuration", "$id": "#PythonEngineBase", "allOf": [ { "$ref": "json://nrp-core/engines/engine_comm_protocols.json#/engine_json" }, { "properties": { "PythonFileName" : { "type": "string", "description": "Path to the python script containing the engine definition" } }, "required": ["PythonFileName"] } ] }, "python_json" : { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Python Json Engine", "description": "Python Json Engine Configuration", "$id": "#PythonJSONEngine", "allOf": [ { "$ref": "#/python_base" }, { "properties": { "EngineType": { "enum": ["python_json"] }, "ServerOptions" : { "type": "string", "default": "", "description": "Additional options that will be used by the server (gunicorn) on startup. The string should contain a Python dictionary in the following format - \"{'key1': value, 'key2': 'value_str'}\". The full list of options can be found at the official page - https://docs.gunicorn.org/en/stable/settings.html." } } } ] }, "python_grpc" : { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Python Grpc Engine", "description": "Python Grpc Engine Configuration", "$id": "#PythonGRPCEngine", "allOf": [ { "$ref": "#/python_base" }, { "properties": { "EngineType": { "enum": ["python_grpc"] } } } ] }, "py_sim" : { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Python Simulation Engine", "description": "A simulation engine for simulators offering a Python API.", "$id": "#PySim", "allOf": [ { "$ref": "#/python_base" }, { "properties": { "EngineType": { "enum": ["py_sim"] }, "ServerOptions" : { "type": "string", "default": "", "description": "Additional options that will be used by the server (gunicorn) on startup. The string should contain a Python dictionary in the following format - \"{'key1': value, 'key2': 'value_str'}\". The full list of options can be found at the official page - https://docs.gunicorn.org/en/stable/settings.html." }, "Simulator": { "enum": ["Opensim","OpenAI","Mujoco","Bullet"], "description": "The simulators that are supported" }, "WorldFileName": { "type": "string", "description": "Path to the file of simulation world" }, "Visualizer": { "type": "boolean", "default": false, "description": "To show the simulation in visualizer or not" } }, "required": ["Simulator", "WorldFileName"] } ] } }