Engine Communication Protocols¶

Final engine implementations can be based on one of the engine communication protocol (ECP) provided with NRP-core. These are intermediate engine implementations (ie. can’t be instantiated, nor directly used in an experiment) that implement engine client / server communication and datapack de-/serialization for a given communication protocol or middleware. Therefore, engine implementations based thereon can focus on the aspects specific to the functionality that the engine is meant to provide.

A tutorial on how to create new engine implementations based on the provided ECPs can be found here.

The ECPs provided with NRP-core are described below.

JSON over REST¶

This ECP uses an HTTP REST Server as the base for the engine server. The client can then use REST calls to communicate. All communications will be de-/serialized using JSON.

The server side of the engine is implemented in EngineJSONServer and the client in EngineJSONNRPClient. Final engine implementations can inherit from these two classes. In order to handle datapack I/O operations in the engine server, concrete datapack controllers can be based on the DataPackController class.

To help create servers, EngineJSONOptsParser can be used to parse start parameters and extract relevant information. Additionally, NRPCoreSim will launch an instance of EngineJSONRegistrationServer which can be used by EngineJSONServers to communicate its address to clients.

Engine Configuration Parameters¶

This engine type parameters are defined in EngineJSON schema (listed here), which in turn is based on EngineBase schema and thus inherits all parameters from the latter.

Parameters inherited from EngineBase schema:

Name	Description	Type	Default	Required	Array
EngineName	Name of the engine	string		X
EngineType	Engine type. Used by	string		X
EngineProcCmd	Command (executable) that will be run in the engine process	string
EngineProcStartParams	Engine Process Start Parameters	string	[]		X
EngineEnvParams	Engine Process Environment Parameters	string	[]		X
EngineLaunchCommand		object	{“LaunchType”:”BasicFork”}
EngineTimestep	Engine Timestep in seconds	number	0.01
EngineCommandTimeout	Engine Timeout (in seconds). It tells how long to wait for the completion of the engine runStep. 0 or negative values are interpreted as no timeout	number	0.0

Parameters specific to this engine type:

Name	Description	Type	Default	Required	Array
ServerAddress		string	localhost:9002
RegistrationServerAddress	Address	string	localhost:9001

Protobuf over gRPC¶

In this case a gRPC server is used for communication. DataPacks are serialized into protobuf format.

Protobuf libraries in NRPCore¶

Implementations of the Protobuf over gRPC Engine interface will use some Protobuf message types to exchange data between client and server.

In order to make the set of Protobuf message types that an Engine can use both extensible and modular, NRPCore compiles .proto files into independent libraries that can be loaded dynamically by Engines at runtime. Concretely, for each .proto file compiled with NRPCore, the next components are created:

a <package>.pb.h header file generated by protoc.
a <package>_pb2.py python module generated by protoc which can be used in Python GRPC Engine.
a libProto<Package>.so library, containing the cpp code generated by protoc.
a libNRPProto<Package>Ops.so linking to the former one and containing Protobuf conversion functions needed by GRPC Engine clients and servers
a <package>.so library, also linking to libProto<Package>.so and containing Protobuf Python bindings to use the compiled msgs in Transceiver Functions

In the file and library names listed above, <Package> is the “package” specifier of the .proto file. All compiled .proto files must have a package specifier, otherwise there is a configuration error

During the build process of NRPCore, .proto files required by GRPC-based Engines shipped with NRPCore are compiled and installed. These .proto files can be found in the folder nrp-core-msgs/protobuf/engine_proto_defs.

Messages compiled from .proto files can be used at runtime by GRPC-based Engines by listing them in the “ProtobufPackages” configuration parameter described below. It contains a list of strings with all the Protobuf Packages which messages can be used by the Engine to exchange data with the client. All the specified packages must have been compiled and installed previously, otherwise a runtime error will follow. See this guide to know how to compile additional .proto files so they become afterwards available to Engines and TFs.

Engine Configuration Parameters¶

This engine type parameters are defined in EngineGRPC schema (listed here), which in turn is based on EngineBase schema and thus inherits all parameters from the latter.

Parameters inherited from EngineBase schema:

Name	Description	Type	Default	Required	Array
EngineName	Name of the engine	string		X
EngineType	Engine type. Used by	string		X
EngineProcCmd	Command (executable) that will be run in the engine process	string
EngineProcStartParams	Engine Process Start Parameters	string	[]		X
EngineEnvParams	Engine Process Environment Parameters	string	[]		X
EngineLaunchCommand		object	{“LaunchType”:”BasicFork”}
EngineTimestep	Engine Timestep in seconds	number	0.01
EngineCommandTimeout	Engine Timeout (in seconds). It tells how long to wait for the completion of the engine runStep. 0 or negative values are interpreted as no timeout	number	0.0

Parameters specific to this engine type:

Name	Description	Type	Default	Array
ServerAddress	gRPC Server address. Should this address already be in use, simulation initialization will fail	string	localhost:9004
ProtobufPluginsPath	Path were to search for specified ProtobufPlugin libraries	string
ProtobufPackages	Protobuf Packages containing protobuf msg types that will be exchanged by this Engine. It is assumed that these packages have been compiled with NRPCore	string	[]	X

Schema¶

Inherits from EngineBase schema

{"engine_json" : {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Engine Json",
    "description": "Base Json Engine configuration schema",
    "$id": "#EngineJSON",
    "allOf": [
      { "$ref": "json://nrp-core/engines/engine_base.json#EngineBase" },
      {
        "properties" : {
          "ServerAddress": {
            "type": "string",
            "default": "localhost:9002",
            "description": "Address from which the engine server sends/receives data"
          },
          "RegistrationServerAddress": {
            "type": "string",
            "default": "localhost:9001",
            "description": "Address to which servers should register to"
          }
        }
      }
    ]
  },
  "engine_grpc" : {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Engine Grpc",
    "description": "Base Grpc Engine configuration schema",
    "$id": "#EngineGRPC",
    "allOf": [
      { "$ref": "json://nrp-core/engines/engine_base.json#EngineBase" },
      {
        "properties" : {
          "ServerAddress": {
            "type": "string",
            "default": "localhost:9004",
            "description": "Address from which the engine server sends/receives data"
          },
          "ProtobufPackages": {
            "type": "array",
            "items": {"type": "string"},
            "description": "Protobuf Packages containing protobuf msg types that will be exchanged by this Engine. It is assumed that these packages have been compiled with NRPCore",
            "default": ["EngineTest"]
          },
          "ProtobufPluginsPath": {
            "type": "string",
            "description": "Path were to search for specified ProtobufPlugin libraries"
          }
        }
      }
    ]
  }
}