Python GRPC Engine

This versatile Python-based engine is well suited for integrating components with a Python API, and in particular, in the cases where efficient data transmission is required. To this end, this engine employs protobuf messages over GRPC as a communication mechanism. The engine server is defined as a Python class based on GrpcEngineScript (as in the example listed below).

For a complete example of an nrp-core experiment using the Python GRPC Engine see examples/tf_exchange/simulation_config_grpc.json.

GrpcEngineScript

GrpcEngineScript provides a base class from which custom Engines must inherit. The derived class must implement methods:

  • initialize() : initialize the engine and registers required DataPacks with a protobuf message type.

  • runLoop(timestep_ns) : advance required simulation and communicate with other engines via TransceiverFunctions.

  • shutdown() : executed when the engine is requested to shutdown.

Besides, GrpcEngineScript provides the following ready-to-use methods and properties to handle datapack communication:

  • _time_ns : returns the internal simulation time of the Engine in nanoseconds.

  • _registerDataPack(datapack_name, protobuf_type) : registers a datapack to the engine with the name datapack_name. Registered datapacks are stored in a python dictionary and can be accessed with _setDataPack and _getDataPack. protobuf_type must a Python Protobuf message type and is the expected type of the data stored in this datapack. The stored data will be updated, under the hood, from datapacks returned by TransceiverFunctions at every loop.

  • _getDataPack(datapack_name) : returns “datapack_name” datapack data, as a protobuf message.

  • _setDataPack(datapack_name, data) : sets “datapack_name” datapack data. “data” is always protobuf a message of type the registered for “datapack_name”.

Below is an example of a class inheriting from GrpcEngineScript. The example is taken from the examples/tf_exchange experiment.

"""Python Engine 1. Will get current engine time and make it accessible as a datapack"""

from nrp_core.engines.python_grpc import GrpcEngineScript
from nrp_protobuf import dump_pb2

class Script(GrpcEngineScript):
    def initialize(self):
        """Initialize datapack1 with time"""
        print("Engine 1 is initializing. Registering datapack...")
        self._registerDataPack("datapack1", dump_pb2.String)
        d = dump_pb2.String()
        d.string_stream = str(self._time_ns)
        self._setDataPack("datapack1", d)

    def runLoop(self, timestep_ns):
        """Update datapack1 at every timestep"""
        self._getDataPack("datapack1").string_stream = str(self._time_ns)
        print("DataPack 1 data is ", self._getDataPack("datapack1").string_stream)

    def shutdown(self):
        print("Engine 1 is shutting down")

    def reset(self):
        print("Engine 1 is resetting")

DataPacks

The Python GRPC engine supports Protobuf DataPacks, which can be used to transfer information between the engine and TFs.

Any of the Protobuf message definitions compiled with nrp-core can be used in the Python GRPC Engine. These can be found in the folder nrp-core-msgs/protobuf/engine_proto_defs. Also it is possible to compile your own Protobuf message definitions from .proto files and use them with this Engine. See here for a full example showing how to compile .proto files and using the new message types in an NRPCore experiment including a Python GRPC Engine.

Engine Configuration Parameters

This engined is based on EngineBase and EngineGRPC schemas.

Name

Description

Type

Default

Required

Array

EngineName

Name of the engine

string

X

EngineType

Engine type. Used by

string

X

EngineProcCmd

Engine Process Launch command

string

EngineProcStartParams

Engine Process Start Parameters

string

[]

X

EngineEnvParams

Engine Process Environment Parameters

string

[]

X

EngineLaunchCommand

object

{“LaunchType”:”BasicFork”}

EngineTimestep

Engine Timestep in seconds

number

0.01

EngineCommandTimeout

Engine Timeout (in seconds). It tells how long to wait for the completion of the engine runStep. 0 or negative values are interpreted as no timeout

number

0.0

  • Parameters inherited from the EngineGRPC schema:

Name

Description

Type

Default

Required

Array

ServerAddress

gRPC Server address. Should this address already be in use, simulation initialization will fail

string

localhost:9004

ProtobufPluginsPath

Path were to search for specified ProtobufPlugin libraries

string

ProtobufPackages

Protobuf Packages containing protobuf msg types that will be exchanged by this Engine. It is assumed that these packages have been compiled with NRPCore

string

[]

X

  • Parameters specific to this engine type:

Name

Description

Type

Default

Required

Array

PythonFileName

Path to the Python script containing the engine definition

string

X

Schema

As explained above, the schema used by the PythonGrpc engine inherits from EngineBase and EngineGRPC schemas. A complete schema for the configuration of this engine is given below:

{"python_base" : {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Python Engine Base",
    "description": "Python Engine Base Configuration",
    "$id": "#PythonEngineBase",
    "allOf": [
      { "$ref": "json://nrp-core/engines/engine_comm_protocols.json#/engine_json" },
      {
        "properties": {
          "PythonFileName" : {
            "type": "string",
            "description": "Path to the python script containing the engine definition"
          }
        },
        "required": ["PythonFileName"]
      }
    ]
  },
  "python_json" : {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Python Json Engine",
    "description": "Python Json Engine Configuration",
    "$id": "#PythonJSONEngine",
    "allOf": [
      { "$ref": "#/python_base" },
      {
        "properties": {
          "EngineType": { "enum": ["python_json"] },
          "ServerOptions" : {
            "type": "string",
            "default": "",
            "description": "Additional options that will be used by the server (gunicorn) on startup. The string should contain a Python dictionary in the following format - \"{'key1': value, 'key2': 'value_str'}\". The full list of options can be found at the official page - https://docs.gunicorn.org/en/stable/settings.html."
          }
        }
      }
    ]
  },
  "python_grpc" : {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Python Grpc Engine",
    "description": "Python Grpc Engine Configuration",
    "$id": "#PythonGRPCEngine",
    "allOf": [
      { "$ref": "#/python_base" },
      {
        "properties": {
          "EngineType": { "enum": ["python_grpc"] }
        }
      }
    ]
  },
  "py_sim" : {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Python Simulation Engine",
    "description": "A simulation engine for simulators offering a Python API.",
    "$id": "#PySim",
    "allOf": [
      { "$ref": "#/python_base" },
      {
        "properties": {
          "EngineType": {
            "enum": ["py_sim"]
          },
          "ServerOptions" : {
            "type": "string",
            "default": "",
            "description": "Additional options that will be used by the server (gunicorn) on startup. The string should contain a Python dictionary in the following format - \"{'key1': value, 'key2': 'value_str'}\". The full list of options can be found at the official page - https://docs.gunicorn.org/en/stable/settings.html."
          },
          "Simulator": {
            "enum": ["Opensim","OpenAI","Mujoco","Bullet"],
            "description": "The simulators that are supported"
          },
          "WorldFileName": {
            "type": "string",
            "description": "Path to the file of simulation world"
          },
          "Visualizer": {
            "type": "boolean",
            "default": false,
            "description": "To show the simulation in visualizer or not"
          }
        },
        "required": ["Simulator", "WorldFileName"]
      }
    ]
  }
}