Python GRPC Engine¶

This versatile Python-based engine is well suited for integrating components with a Python API, and in particular, in the cases where efficient data transmission is required. To this end, this engine employs protobuf messages over GRPC as a communication mechanism. The engine server is defined as a Python class based on GrpcEngineScript (as in the example listed below).

For a complete example of an nrp-core experiment using the Python GRPC Engine see examples/tf_exchange/simulation_config_grpc.json.

GrpcEngineScript¶

GrpcEngineScript provides a base class from which custom Engines must inherit. The derived class must implement methods:

initialize() : initialize the engine and registers required DataPacks with a protobuf message type.
runLoop(timestep_ns) : advance required simulation and communicate with other engines via TransceiverFunctions.
shutdown() : executed when the engine is requested to shutdown.

Besides, GrpcEngineScript provides the following ready-to-use methods and properties to handle datapack communication:

_time_ns : returns the internal simulation time of the Engine in nanoseconds.
_registerDataPack(datapack_name, protobuf_type) : registers a datapack to the engine with the name datapack_name. Registered datapacks are stored in a python dictionary and can be accessed with _setDataPack and _getDataPack. protobuf_type must a Python Protobuf message type and is the expected type of the data stored in this datapack. The stored data will be updated, under the hood, from datapacks returned by TransceiverFunctions at every loop.
_getDataPack(datapack_name) : returns “datapack_name” datapack data, as a protobuf message.
_setDataPack(datapack_name, data) : sets “datapack_name” datapack data. “data” is always protobuf a message of type the registered for “datapack_name”.

Below is an example of a class inheriting from GrpcEngineScript. The example is taken from the examples/tf_exchange experiment.

"""Python Engine 1. Will get current engine time and make it accessible as a datapack"""

from nrp_core.engines.python_grpc import GrpcEngineScript
from nrp_protobuf import dump_pb2

class Script(GrpcEngineScript):
    def initialize(self):
        """Initialize datapack1 with time"""
        print("Engine 1 is initializing. Registering datapack...")
        self._registerDataPack("datapack1", dump_pb2.String)
        d = dump_pb2.String()
        d.string_stream = str(self._time_ns)
        self._setDataPack("datapack1", d)

    def runLoop(self, timestep_ns):
        """Update datapack1 at every timestep"""
        self._getDataPack("datapack1").string_stream = str(self._time_ns)
        print("DataPack 1 data is ", self._getDataPack("datapack1").string_stream)

    def shutdown(self):
        print("Engine 1 is shutting down")

    def reset(self):
        print("Engine 1 is resetting")

DataPacks¶

The Python GRPC engine supports Protobuf DataPacks, which can be used to transfer information between the engine and TFs.

Any of the Protobuf message definitions compiled with nrp-core can be used in the Python GRPC Engine. These can be found in the folder nrp-core-msgs/protobuf/engine_proto_defs. Also it is possible to compile your own Protobuf message definitions from .proto files and use them with this Engine. See here for a full example showing how to compile .proto files and using the new message types in an NRPCore experiment including a Python GRPC Engine.

Engine Configuration Parameters¶

This engined is based on EngineBase and EngineGRPC schemas.

Parameters inherited from EngineBase schema:

Name	Description	Type	Default	Required	Array
EngineName	Name of the engine	string		X
EngineType	Engine type. Used by	string		X
EngineProcCmd	Engine Process Launch command	string
EngineProcStartParams	Engine Process Start Parameters	string	[]		X
EngineEnvParams	Engine Process Environment Parameters	string	[]		X
EngineLaunchCommand		object	{“LaunchType”:”BasicFork”}
EngineTimestep	Engine Timestep in seconds	number	0.01
EngineCommandTimeout	Engine Timeout (in seconds). It tells how long to wait for the completion of the engine runStep. 0 or negative values are interpreted as no timeout	number	0.0

Parameters inherited from the EngineGRPC schema:

Name	Description	Type	Default	Array
ServerAddress	gRPC Server address. Should this address already be in use, simulation initialization will fail	string	localhost:9004
ProtobufPluginsPath	Path were to search for specified ProtobufPlugin libraries	string
ProtobufPackages	Protobuf Packages containing protobuf msg types that will be exchanged by this Engine. It is assumed that these packages have been compiled with NRPCore	string	[]	X

Parameters specific to this engine type:

Name	Description	Type	Default	Required	Array
PythonFileName	Path to the Python script containing the engine definition	string		X

Schema¶

As explained above, the schema used by the PythonGrpc engine inherits from EngineBase and EngineGRPC schemas. A complete schema for the configuration of this engine is given below:

{"python_base" : {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Python Engine Base",
    "description": "Python Engine Base Configuration",
    "$id": "#PythonEngineBase",
    "allOf": [
      { "$ref": "json://nrp-core/engines/engine_comm_protocols.json#/engine_json" },
      {
        "properties": {
          "PythonFileName" : {
            "type": "string",
            "description": "Path to the python script containing the engine definition"
          }
        },
        "required": ["PythonFileName"]
      }
    ]
  },
  "python_json" : {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Python Json Engine",
    "description": "Python Json Engine Configuration",
    "$id": "#PythonJSONEngine",
    "allOf": [
      { "$ref": "#/python_base" },
      {
        "properties": {
          "EngineType": { "enum": ["python_json"] },
          "ServerOptions" : {
            "type": "string",
            "default": "",
            "description": "Additional options that will be used by the server (gunicorn) on startup. The string should contain a Python dictionary in the following format - \"{'key1': value, 'key2': 'value_str'}\". The full list of options can be found at the official page - https://docs.gunicorn.org/en/stable/settings.html."
          }
        }
      }
    ]
  },
  "python_grpc" : {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Python Grpc Engine",
    "description": "Python Grpc Engine Configuration",
    "$id": "#PythonGRPCEngine",
    "allOf": [
      { "$ref": "#/python_base" },
      {
        "properties": {
          "EngineType": { "enum": ["python_grpc"] }
        }
      }
    ]
  },
  "py_sim" : {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Python Simulation Engine",
    "description": "A simulation engine for simulators offering a Python API.",
    "$id": "#PySim",
    "allOf": [
      { "$ref": "#/python_base" },
      {
        "properties": {
          "EngineType": {
            "enum": ["py_sim"]
          },
          "ServerOptions" : {
            "type": "string",
            "default": "",
            "description": "Additional options that will be used by the server (gunicorn) on startup. The string should contain a Python dictionary in the following format - \"{'key1': value, 'key2': 'value_str'}\". The full list of options can be found at the official page - https://docs.gunicorn.org/en/stable/settings.html."
          },
          "Simulator": {
            "enum": ["Opensim","OpenAI","Mujoco","Bullet"],
            "description": "The simulators that are supported"
          },
          "WorldFileName": {
            "type": "string",
            "description": "Path to the file of simulation world"
          },
          "Visualizer": {
            "type": "boolean",
            "default": false,
            "description": "To show the simulation in visualizer or not"
          }
        },
        "required": ["Simulator", "WorldFileName"]
      }
    ]
  }
}