Nerve Data Services Gateway
The Gateway is the central application of the Data Services. Its purpose is receiving data from a source via a certain protocol on an input interface and forwarding it to a destination using a different protocol on an output interface. This behavior is reflected in the user configuration as well. In general, the configuration file consists of inputs, outputs and connections between them:
- Inputs are collection interfaces where data is received.
- Outputs are providing interfaces where data can be received from.
- Connections are logical links between inputs and outputs.
Each input and output provides connectors where a connection can be attached to. A connector is a subset of the data available at an input. This subset is forwarded to the connector of an output where it is distributed further to the data receiver. On an abstract level, this concept is illustrated in the figure below.
Inputs and outputs hold general configuration data for the protocol they implement. This can be a URL to a server, a port number or something related to timing. A connector of an input or output holds more specific information on how to get or where to send data, for example the node IDs of an OPC UA Server an OPC UA Client input connects to, or the name of a topic for an MQTT Publisher. Each input and output can provide multiple connectors at a time. Data from an input connector can also be connected to multiple output connectors at the same time to distribute data to multiple data consumers. The figure below shows a more specific example of that concept.
Data Gateway UI
The Data Gateway UI is the landing page of the Data Services and will be displayed by default after selecting Data > CONFIGURATION in the navigation on the left in either Local UI or Management System. The editor in the middle is used to apply Gateway configurations in the JSON format.
Item | Description |
---|---|
Gateway status and controls (1) | The status of the Gateway is displayed in the upper-right corner with controls next to it that allow starting, stopping or restarting of the Gateway. Running indicates that the configuration has been accepted and the Gateway is running. When the status is stopped, there is either a problem in the configuration or the Gateway has been stopped manually. |
Editor (2) | Enter a configuration in the JSON format into the editor to configure the Gateway. The configuration can be written directly in the editor or uploaded from the workstation.
|
Configuration and Logs tabs (3) | The Configuration tab is displayed by default. The latest logs of the Gateway can be viewed within the Logs tab. The entirety of the logs can be exported into a file using the Download button in this tab. |
Applying a configuration to the Gateway
The configuration of the Gateway is applied in the editor in the Gateway section of the Data Services UI. The current configuration of the Gateway is loaded into the JSON editor if there is a configuration present in the Gateway. However, the editor will be empty on initial startup. A new configuration can be entered into the editor directly or imported from an existing JSON file.
- Upload existing JSON files using the Upload button.
- Apply the configuration in the editor by selecting Apply. When the configuration is successful, the status in the upper-right corner will display running.
- Check the Logs tab for any warnings or errors.
The basic structure of the configuration consist of a list of inputs such as an OPC UA server or an MQTT subscriber, outputs such as a Timescale database or an MQTT publisher, and connections between these inputs and outputs. Refer to Gateway configuration file syntax below for the exact syntax.
The examples in the Examples section show step by step applications in common use cases.
Gateway configuration file syntax
The following list gives additional clarification about how to set parameters with the JSON editor. The Gateway configuration JSON file includes sections, which are defined as main sections (inputs, outputs, connections). The main sections may contain one set of subsections or an array of items, which in this case are defined as an array of subsection sets.
Subsections can also include additional subsections. A section is defined as an array if the section contains two or more subsections. Subsection keys are defined as parameters. Every parameter has its field for a value. The main sections can contain the same and different parameters as other sections.
The syntax of the various inputs and outputs of the Gateway available in its configuration file is illustrated below for the available protocols.
Inputs | Outputs |
---|---|
MQTT subscriber | MQTT publisher |
OPC UA PubSub over MQTT | OPC UA PubSub over MQTT |
OPC UA PubSub over UDP | OPC UA PubSub over UDP |
OPC UA Client | OPC UA server |
Modbus server | ZeroMQ publisher |
S7 server | Azure IoT Hub Device |
TimescaleDB | |
Influx DB |
Inputs
In the tables below, fields in bold are mandatory and fields in italics are optional when writing a configuration for the corresponding protocol.
OPC UA Client
Item | Description |
---|---|
type | Type of input (OPC_UA_CLIENT). |
name | Name of OPC_UA_CLIENT instance used in log messages. |
serverUrl | URL of the server to connect to, i.e. opc.tcp://myserver.com:4840 . |
login | Authentication parameters. username Username for client authentication. password Password for client authentication. |
certificate | Secure connection parameters. certFilePath Certificate file path for client authentication. keyFilePath Key file path for client authentication. |
trustList | Array of certificates of trusted servers. |
security | Connection security mode and policy. securityMode Security mode (None, Sign, SignAndEncrypt). securityPolicy Security policy (None, Basic128Rsa15, Basic256, Basic256Sha256). |
description | Description of the client. applicationUri Application URI of the client, used in secure connection certificate. |
pollingInterval_ms | Interval at which to poll values from the server in milliseconds. |
connectors | Array of connectors for OPC_UA_CLIENT. name Connector name. accessType Access mode, either polling or subscription .publishingIntervalAtServer_ms Interval at which the server is publishing values in milliseconds. samplingIntervalAtServer_ms Sampling intervall at which the server updates its values in milliseconds. nodes Array of nodeID strings, e.g. ns=4;i=1 , ns=mynamespace;s=myvar.int0 . If the nodeID of an object is provided, all variables of that object are used. |
MQTT Subscriber
Objects | Values and descriptions |
---|---|
type | Type of input (MQTT_SUBSCRIBER). |
name | Name of the MQTT_SUBSCRIBER input used in log messages. |
clientId | Client ID of the MQTT_SUBSCRIBER. |
serverUrl | Protocol and URL of the server to connect to, i.e. tcp://myserver.com:4840 . Possible protocols are: tcp , mqtt , mqtts , ssl , ws or wss . |
username | MQTT broker authentication username. |
password | MQTT broker authentication password. |
keepAliveInterval_s | Maximum time allowed of no communication between client and server. |
cleanSession | Defines whether the server should remember state for the client across reconnect. |
qos | Quality of Service value (0-2). |
sslOptions | Secure connection options: ca_certsFilePath Path to CA certificate file. certFilePath Path to certificate file. keyFilePath Path to key file. keyPassword Password for the given key. serverAuthRequired Defines whether server authentication is required. |
connectors | Array of connectors for the MQTT_SUBSCRIBER: name Connector name. topic Name of the MQTT topic to use. timestamp Timestamp related options
Array of variables expected to be received:
|
OPC UA PubSub Subscriber
Item | Description |
---|---|
type | Type of input (OPC_UA_PS_SUBSCRIBER). |
name | Name of the OPC_UA_PS_SUBSCRIBER input used in log messages. |
mqttClientId | Client ID of the OPC_UA_PS_SUBSCRIBER when MQTT transport is used. |
networkAddressUrl | URL of the OPC UA PubSub connection to subscribe to. |
networkInterface | Name of the network interface to use. |
encoding | Type of message encoding (JSON/UADP). |
connectors | Array of connectors for the OPC_UA_PS_SUBSCRIBER: name Connector name. publisherId Publisher ID for the writer group. writerGroupId Writer group ID. dataSetWriterId Data set writer ID. fieldsReversePublished Some publishers send fields within a data set message in a reversed order compared to how the fields were added to that publisher. If this field is set to true, the user can use field indexes in the config file in the same order as the fields were added to these publishers. Must be set to true when connecting to a publisher using the TTTech PubSub library (i.e. CODESYS).mqttTopic Name of the MQTT topic if transport protocol is MQTT, ignored otherwise. variables Array of variables expected to be received:
|
S7 Client
Item | Description |
---|---|
type | Type of output (S7_CLIENT). |
name | Name of the S7_CLIENT instance used in log messages. |
serverUrl | URL of the S7 server to connect to. |
port | Port at which to connect to. |
connectionType | Connection type (PG, OP, S7_BASIC). |
localTsap | Local tsap, mandatory if s7 basic connection type. |
remoteTsap | Remote tsap, mandatory if s7 basic connection type. |
pollingInterval_ms | Interval for polling values from server in miliseconds. |
rack | S7 device rack. |
slot | S7 device slot. |
connectors | Array of connectors for S7_CLIENT: name Connector name. merkers Array of S7 merkers:
Array of S7 inputs:
Array of S7 outputs:
Array of S7 timers:
Array of S7 counters:
Array of S7 datablocks:
|
Modbus Client
Item | Description |
---|---|
type | Type of output (MODBUS_CLIENT). |
name | Name of the MODBUS_CLIENT instance used in log messages. |
serverUrl | URL of the Modbus server to connect to. |
port | Port at which to connect to. |
pollingInterval_ms | Interval at which to poll values from the server in milliseconds. |
connectors | Array of connectors for the MODBUS_CLIENT: name Connector name. coils Array of Modbus coil type:
Array of Modbus discrete input type:
Array of Modbus input register type:
Array of Modbus holding register type:
|
Outputs
In the tables below, fields in bold are mandatory and fields in italics are optional when writing a configuration for the corresponding protocol.
TimescaleDB
Item | Description |
---|---|
type | Type of output (DB_TIMESCALE). |
name | Name of the DB_TIMESCALE instance used in log messages. |
url | URL of the TimescaleDB server to connect to. |
port | Port at which to connect to. |
dataRate_MBps | Hypertable data rate in MegaBytes per second. |
chunkTimeInterval_s | Hypertable chunk time interval in seconds. |
connectors | Array of connectors for DB_TIMESCALE: name Connector name. dbName Database name to connect to. tableName Table name to write into. user Username for authentication. password Password for authentication. booleanAsSmallint Whether boolean should be represented as PostgreSQL small int data type. |
InfluxDB
Item | Description |
---|---|
type | Type of output (DB_INFLUX). |
name | Name of the DB_INFLUX instance used in log messages. |
url | URL of the InfluxDB server to connect to. |
port | Port at which to connect to. |
connectors | Array of connectors for DB_INFLUX: name Connector name. dbName Database name to connect to. user Username for authentication. password Password for authentication. |
MQTT Publisher
Item | Description |
---|---|
type | Type of output (MQTT_PUBLISHER). |
name | Name of the MQTT_PUBLISHER instance used in log messages. |
clientId | Client ID of the MQTT_PUBLISHER. |
serverUrl | Protocol and URL of the server to connect to, i.e. tcp://myserver.com:4840 . Possible protocols are: tcp , mqtt , mqtts , ssl , ws or wss . |
username | MQTT broker authentication username. |
password | MQTT broker authentication password. |
keepAliveInterval_s | Maximum time allowed of no communication between client and server. |
cleanSessions | Defines whether the server should remember state for the client across reconnect. |
qos | Quality of Service value (0-2). |
sslOptions | Secure connection options: ca_certsFilePath Path to CA certificate file. certFilePath Path to certificate file. keyFilePath Path to key file. keyPassword Password for the given key. serverAuthRequired Defines whether server authentication is required. |
connectors | Array of connectors for MQTT_PUBLISHER: name Connector name. topic Name of the MQTT topic to use. timestampRequired Defines whether a timestamp is added to each message or not. timestampFormat Format of the timestamp in a message, either iso (ISO 8601) or unix_ns (UNIX time in nanoseconds since Jan 01 1970 (UTC)).maxListSize Maximum array size of variables, if sent as arrays. If size is larger than the max data will be sent in chunks of maxListSize. |
OPC UA PubSub Publisher
Items | Description |
---|---|
type | Type of output (OPC_UA_PS_PUBLISHER). |
name | Name of OPC_UA_PS_PUBLISHER instance used in log messages. |
networkAddressUrl | URL of the OPC UA PubSub connection to publish to. |
networkInterface | Name of the network interface to use. |
encoding | Type of message encoding (JSON/UADP). |
connectors | Array of connectors for OPC_UA_PS_PUBLISHER: name Connector name. publisherId Publisher ID of OPC UA PubSub connection. writerGroupId Writer group ID in the OPC UA PubSub connection. dataSetWriterId Data set writer ID in the writer group. publishInterval_ms Interval at which publisher publishes messages. keyFrameCount Key frame count for messages. mqttTopic Topic to publish to in case of MQTT connection. mqttClientId MQTT client ID for PubSub connection in case of MQTT connection. This must be unique for each connector. |
OPC UA Server
Item | Description |
---|---|
type | Type of output (OPC_UA_SERVER). |
name | Name of OPC_UA_SERVER instance used in log messages. |
customHostname | Custom hostname of the server, used for discovery URL. |
port | Port to listen at (defaults to 4840). |
logins | Array of username/password pairs available for authentication: username Username for authentication. password Password for authentication. |
securities | Array of available security modes and policies: securityMode Security Mode ("all" keyword is supported to create all available endpoints for chosen Security Policy). securityPolicy Security Policy ("all" keyword is supported to create all available endpoints for chosen Security Mode). |
certificate | Server's certificate: certFilePath Path to Server Instance Certificate. keyFilePath Path to private key. |
description | Server Description: applicationUri Server Application URI. |
trustList | Array of trusted certificates. |
addressSpaceFolders | OPC_UA_SERVER address space tree: browseName OPC UA Folder browse name. displayName OPC UA Folder display name. description OPC UA Folder description. namespaceURI OPC UA Folder namespace URI. identifier OPC UA Folder nodeID identifier. children Array of OPC UA childFolders, same format as addressSpaceFolders field. connectorIndicies Array of indicies of connectors, placed in the OPC UA Folder. |
connectors | Array of connectors for OPC_UA_SERVER: name Connector name. browseName OPC UA node browse name. displayName OPC UA node display name. description OPC UA node description. namespaceURI OPC UA node namespace URI. identifier OPC UA node nodeID identifier. |
ZeroMQ Publisher
Item | Description |
---|---|
type | Type of output (ZEROMQ_PUBLISHER). |
name | Name of the ZEROMQ_PUBLISHER instance used in log messages. |
serverUrl | Server endpoint to publish to. Subscribers connect to it. |
connectors | Array of connectors for ZEROMQ_PUBLISHER: name Connector name. topic Name of the ZeroMQ topic to publish to. timestampRequired Defines whether a timestamp is added to each message or not. timestampFormat Format of the timestamp in a message, either iso (ISO 8601) or unix_ns (UNIX time in nanoseconds since Jan 01 1970 (UTC)).maxListSize Maximum array size of variables, if sent as arrays. If size is larger than the max data will be sent in chunks of maxListSize. |
Azure IoT Hub Device
Item | Description |
---|---|
type | Type of output (AZURE_IOT_HUB_DEVICE). |
name | Name of the AZURE_IOT_HUB_DEVICE instance used in log messages. |
deviceConnectionString | Connection string of the device created in the MS Azure IoT Hub. |
mqttOverWss | Determines if the AZURE_IOT_HUB_DEVICE instance should use MQTT over WSS (port 433) or regular MQTT over TLS/SSL (port 8883) to connect to the Azure IoT Hub broker. |
sslOptions | Secure connection options for MQTT transport: certFilePath Path to certificate file. keyFilePath Path to key file. keyPassword Password for the given key. |
connectors | Array of connectors for AZURE_IOT_HUB_DEVICE: name Connector name. timestampRequired Defines whether a timestamp is added to each message or not. timestampFormat Format of the timestamp in a message, either iso (ISO 8601) or unix_ns (UNIX time in nanoseconds since Jan 01 1970 (UTC)).maxListSize Maximum array size of variables if sent as arrays. If size is larger than the maximum, data will be sent in chunks of maxListSize .customProperties Custom properties array that is included with every published message:
|
Connections
Item | Description |
---|---|
name | Name of the connection, will define the name of the TABLE within TimescaleDB if tableName field is omitted. |
input | Input of the connection. index Index in the array of INPUTs. connector Index in the array of connectors at the input. |
output | Output of the connection. index Index in the array of OUTPUTs. connector Index in the array of connectors at the output. |
Gateway configuration file keywords
Parts of the Gateway configuration are eased by the use of keywords. Keywords can be used on both Gateway instances on the node and in the Management System. However, the keyowrds that can be used differ. All keywords must be put in angled brackets, for example <SN>
. To be able to use keywords. the node must be registered in the Management System.
Refer to the list below for keywords that can be used with usage examples right after.
Supported on the node
Keyword | Description |
---|---|
SN |
|
SID |
|
LOCAL |
|
MS‑MQTT‑BROKER | The MQTT broker URL is fetched from the node registration data on the node.
|
Supported in the Management System
Keyword | Description |
---|---|
SN:<nodename> |
|
MS‑MQTT‑BROKER | The MQTT broker URL is fetched from the Gateway system configuration in the Management System.
|
Examples of keyword usage
SN:<nodename>
{ "name": "cloud_timescale", "type": "DB_TIMESCALE", "url": "localhost", "port": 5432, "connectors": [ { "name": "EXAMPLE-NODE", "dbName": "<SN:EXAMPLE-NODE>" } ] }
SN, SID, MS-MQTT-BROKER
{ "name": "ms_mqtt_broker", "type": "MQTT_SUBSCRIBER", "serverUrl": "<MS-MQTT-BROKER>", "username": "<SN>", "password": "<SID>", "keepAliveInterval_s": 20, "qos": 1, "cleanSession": true, "sslOptions": { "serverAuthRequired": false }, "connectors": [ { "name": "demo_sensor_connector_0", "topic": "<SN>/demo_sensor_topic_0", "variables": [ { "name": "temperature", "type": "int64" } ] } ] }
LOCAL
{ "type": "DB_TIMESCALE", "name": "timescaledb_0", "url": "<LOCAL>" }
SN:<nodename>, MS-MQTT-BROKER
{ "name": "ms_mqtt_broker", "type": "MQTT_SUBSCRIBER", "serverUrl": "<MS-MQTT-BROKER>", "keepAliveInterval_s": 20, "qos": 1, "cleanSession": true, "connectors": [ { "name": "demo_sensor_connector_1", "topic": "<SN:EXAMPLE-NODE>/demo_sensor_topic_1", "variables": [ { "name": "humidity", "type": "uint64" } ] } ] }