.. index:: pair: page; Engine Communication Protocols .. _doxid-engine_comm: Engine Communication Protocols ============================== Final engine implementations can be based on one of the engine communication protocol (ECP) provided with NRP-core. These are intermediate engine implementations (ie. can't be instantiated, nor directly used in an experiment) that implement engine client / server communication and datapack de-/serialization for a given communication protocol or middleware. Therefore, engine implementations based thereon can focus on the aspects specific to the functionality that the engine is meant to provide. A tutorial on how to create new engine implementations based on the provided ECPs can be found :ref:`here `. The ECPs provided with NRP-core are described below. .. _doxid-engine_comm_1engine_json: JSON over REST ~~~~~~~~~~~~~~ This ECP uses an HTTP REST Server as the base for the engine server. The client can then use REST calls to communicate. All communications will be de-/serialized using JSON. The server side of the engine is implemented in :ref:`EngineJSONServer ` and the client in :ref:`EngineJSONNRPClient `. Final engine implementations can inherit from these two classes. In order to handle datapack I/O operations in the engine server, concrete datapack controllers can be based on the :ref:`DataPackController ` class. To help create servers, :ref:`EngineJSONOptsParser ` can be used to parse start parameters and extract relevant information. Additionally, NRPCoreSim will launch an instance of :ref:`EngineJSONRegistrationServer ` which can be used by EngineJSONServers to communicate its address to clients. .. _doxid-engine_comm_1engine_json_config_section: Engine Configuration Parameters ------------------------------- This engine type parameters are defined in EngineJSON schema (listed :ref:`here `), which in turn is based on :ref:`EngineBase ` schema and thus inherits all parameters from the latter. * Parameters inherited from EngineBase schema: ===================== =================================================================================================================================================== ====== ========================== ======== ===== Name Description Type Default Required Array ===================== =================================================================================================================================================== ====== ========================== ======== ===== EngineName Name of the engine string X EngineType Engine type. Used by string X EngineProcCmd Command (executable) that will be run in the engine process string EngineProcStartParams Engine Process Start Parameters string [] X EngineEnvParams Engine Process Environment Parameters string [] X EngineLaunchCommand object {"LaunchType":"BasicFork"} EngineTimestep Engine Timestep in seconds number 0.01 EngineCommandTimeout Engine Timeout (in seconds). It tells how long to wait for the completion of the engine runStep. 0 or negative values are interpreted as no timeout number 0.0 ===================== =================================================================================================================================================== ====== ========================== ======== ===== * Parameters specific to this engine type: ========================= =========== ====== ============== ======== ===== Name Description Type Default Required Array ========================= =========== ====== ============== ======== ===== ServerAddress string localhost:9002 RegistrationServerAddress Address string localhost:9001 ========================= =========== ====== ============== ======== ===== .. _doxid-engine_comm_1engine_grpc: Protobuf over gRPC ~~~~~~~~~~~~~~~~~~ In this case a gRPC server is used for communication. DataPacks are serialized into protobuf format. .. _doxid-engine_comm_1engine_grpc_protobuf_libraries: Protobuf libraries in NRPCore ----------------------------- Implementations of the Protobuf over gRPC Engine interface will use some Protobuf message types to exchange data between client and server. In order to make the set of Protobuf message types that an Engine can use both extensible and modular, NRPCore compiles ``.proto`` files into independent libraries that can be loaded dynamically by Engines at runtime. Concretely, for each ``.proto`` file compiled with NRPCore, the next components are created: * a ``.pb.h`` header file generated by ``protoc``. * a ``_pb2.py`` python module generated by ``protoc`` which can be used in :ref:`Python GRPC Engine `. * a ``libProto.so`` library, containing the cpp code generated by ``protoc``. * a ``libNRPProtoOps.so`` linking to the former one and containing Protobuf conversion functions needed by GRPC Engine clients and servers * a ``.so`` library, also linking to ``libProto.so`` and containing Protobuf Python bindings to use the compiled msgs in Transceiver Functions In the file and library names listed above, is the "package" specifier of the ``.proto`` file. All compiled ``.proto`` files must have a package specifier, otherwise there is a configuration error During the build process of NRPCore, ``.proto`` files required by GRPC-based Engines shipped with NRPCore are compiled and installed. These ``.proto`` files can be found in the folder *nrp-core-msgs/protobuf/engine_proto_defs*. Messages compiled from ``.proto`` files can be used at runtime by GRPC-based Engines by listing them in the "ProtobufPackages" configuration parameter described below. It contains a list of strings with all the Protobuf Packages which messages can be used by the Engine to exchange data with the client. All the specified packages must have been compiled and installed previously, otherwise a runtime error will follow. See this :ref:`guide ` to know how to compile additional ``.proto`` files so they become afterwards available to Engines and TFs. .. _doxid-engine_comm_1engine_grpc_config_section: Engine Configuration Parameters ------------------------------- This engine type parameters are defined in EngineGRPC schema (listed :ref:`here `), which in turn is based on :ref:`EngineBase ` schema and thus inherits all parameters from the latter. * Parameters inherited from EngineBase schema: ===================== =================================================================================================================================================== ====== ========================== ======== ===== Name Description Type Default Required Array ===================== =================================================================================================================================================== ====== ========================== ======== ===== EngineName Name of the engine string X EngineType Engine type. Used by string X EngineProcCmd Command (executable) that will be run in the engine process string EngineProcStartParams Engine Process Start Parameters string [] X EngineEnvParams Engine Process Environment Parameters string [] X EngineLaunchCommand object {"LaunchType":"BasicFork"} EngineTimestep Engine Timestep in seconds number 0.01 EngineCommandTimeout Engine Timeout (in seconds). It tells how long to wait for the completion of the engine runStep. 0 or negative values are interpreted as no timeout number 0.0 ===================== =================================================================================================================================================== ====== ========================== ======== ===== * Parameters specific to this engine type: =================== ======================================================================================================================================================== ====== ============== ======== ===== Name Description Type Default Required Array =================== ======================================================================================================================================================== ====== ============== ======== ===== ServerAddress gRPC Server address. Should this address already be in use, simulation initialization will fail string localhost:9004 ProtobufPluginsPath Path were to search for specified ProtobufPlugin libraries string ProtobufPackages Protobuf Packages containing protobuf msg types that will be exchanged by this Engine. It is assumed that these packages have been compiled with NRPCore string [] X =================== ======================================================================================================================================================== ====== ============== ======== ===== .. _doxid-engine_comm_1engine_comm_protocols_schema: Schema ~~~~~~ Inherits from :ref:`EngineBase ` schema .. ref-code-block:: cpp {"engine_json" : { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Engine Json", "description": "Base Json Engine configuration schema", "$id": "#EngineJSON", "allOf": [ { "$ref": "json://nrp-core/engines/engine_base.json#EngineBase" }, { "properties" : { "ServerAddress": { "type": "string", "default": "localhost:9002", "description": "Address from which the engine server sends/receives data" }, "RegistrationServerAddress": { "type": "string", "default": "localhost:9001", "description": "Address to which servers should register to" } } } ] }, "engine_grpc" : { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Engine Grpc", "description": "Base Grpc Engine configuration schema", "$id": "#EngineGRPC", "allOf": [ { "$ref": "json://nrp-core/engines/engine_base.json#EngineBase" }, { "properties" : { "ServerAddress": { "type": "string", "default": "localhost:9004", "description": "Address from which the engine server sends/receives data" }, "ProtobufPackages": { "type": "array", "items": {"type": "string"}, "description": "Protobuf Packages containing protobuf msg types that will be exchanged by this Engine. It is assumed that these packages have been compiled with NRPCore", "default": ["EngineTest"] }, "ProtobufPluginsPath": { "type": "string", "description": "Path were to search for specified ProtobufPlugin libraries" } } } ] } }