Skip to content
PDF

Nerve Data Services Gateway

The Gateway is the central application of the Data Services. Its purpose is receiving data from a source via a certain protocol on an input interface and forwarding it to a destination using a different protocol on an output interface. This behavior is reflected in the user configuration as well. In general, the configuration file consists of inputs, outputs and connections between them:

  • Inputs are collection interfaces where data is received.
  • Outputs are providing interfaces where data can be received from.
  • Connections are logical links between inputs and outputs.

Each input and output provides connectors where a connection can be attached to. A connector is a subset of the data available at an input. This subset is forwarded to the connector of an output where it is distributed further to the data receiver. On an abstract level, this concept is illustrated in the figure below.

Gateway Concept

Inputs and outputs hold general configuration data for the protocol they implement. This can be a URL to a server, a port number or something related to timing. A connector of an input or output holds more specific information on how to get or where to send data, for example the node IDs of an OPC UA Server an OPC UA Client input connects to, or the name of a topic for an MQTT Publisher. Each input and output can provide multiple connectors at a time. Data from an input connector can also be connected to multiple output connectors at the same time to distribute data to multiple data consumers. The figure below shows a more specific example of that concept.

Gateway Concept Example

Data Gateway UI

The Data Gateway UI is the landing page of the Data Services and will be displayed by default after selecting Data > CONFIGURATION in the navigation on the left in either Local UI or Management System. The editor in the middle is used to apply Gateway configurations in the JSON format.

Gateway UI

Item Description
Gateway status and controls (1) The status of the Gateway is displayed in the upper-right corner with controls next to it that allow starting, stopping or restarting of the Gateway. Running indicates that the configuration has been accepted and the Gateway is running. When the status is stopped, there is either a problem in the configuration or the Gateway has been stopped manually.
Editor (2) Enter a configuration in the JSON format into the editor to configure the Gateway. The configuration can be written directly in the editor or uploaded from the workstation.
  • Upload
    Select this to upload the contents of an existing JSON file from the workstation into the editor.
  • Reload
    Select this to load the currently applied configuration of the Gateway into the editor.
  • Apply
    Click here to set the current content of the editor as the Gateway configuration. The Gateway will be restarted.
Configuration and Logs tabs (3) The Configuration tab is displayed by default. The latest logs of the Gateway can be viewed within the Logs tab. The entirety of the logs can be exported into a file using the Download button in this tab.

Applying a configuration to the Gateway

The configuration of the Gateway is applied in the editor in the Gateway section of the Data Services UI. The current configuration of the Gateway is loaded into the JSON editor if there is a configuration present in the Gateway. However, the editor will be empty on initial startup. A new configuration can be entered into the editor directly or imported from an existing JSON file.

  • Upload existing JSON files using the Upload button.
  • Apply the configuration in the editor by selecting Apply. When the configuration is successful, the status in the upper-right corner will display running.
  • Check the Logs tab for any warnings or errors.

The basic structure of the configuration consist of a list of inputs such as an OPC UA server or an MQTT subscriber, outputs such as a Timescale database or an MQTT publisher, and connections between these inputs and outputs. Refer to Gateway configuration file syntax below for the exact syntax.

The examples in the Examples section show step by step applications in common use cases.

Gateway configuration file syntax

The following list gives additional clarification about how to set parameters with the JSON editor. The Gateway configuration JSON file includes sections, which are defined as main sections (inputs, outputs, connections). The main sections may contain one set of subsections or an array of items, which in this case are defined as an array of subsection sets.

Subsections can also include additional subsections. A section is defined as an array if the section contains two or more subsections. Subsection keys are defined as parameters. Every parameter has its field for a value. The main sections can contain the same and different parameters as other sections.

The syntax of the various inputs and outputs of the Gateway available in its configuration file is illustrated below for the available protocols.

Inputs Outputs
MQTT subscriber MQTT publisher
OPC UA PubSub over MQTT OPC UA PubSub over MQTT
OPC UA PubSub over UDP OPC UA PubSub over UDP
OPC UA Client OPC UA server
Modbus server ZeroMQ publisher
S7 server Timescale DB
Influx DB

Inputs

In the tables below, fields in bold are mandatory and fields in italics are optional when writing a configuration for the corresponding protocol.

OPC UA Client

Item Description
type Type of input (OPC_UA_CLIENT).
name Name of OPC_UA_CLIENT instance used in log messages.
serverUrl URL of the server to connect to, i.e. opc.tcp://myserver.com:4840.
login Authentication parameters.

username
Username for client authentication.

password
Password for client authentication.
certificate Secure connection parameters.

certFilePath
Certificate file path for client authentication.

keyFilePath
Key file path for client authentication.
trustList Array of certificates of trusted servers.
security Connection security mode and policy.

securityMode
Security mode (None, Sign, SignAndEncrypt).

securityPolicy
Security policy (None, Basic128Rsa15, Basic256, Basic256Sha256).
description Description of the client.

applicationUri
Application URI of the client, used in secure connection certificate.
pollingInterval_ms Interval at which to poll values from the server in milliseconds.
connectors Array of connectors for OPC_UA_CLIENT.

name
Connector name.

accessType
Access mode, either polling or subscription.

publishingIntervalAtServer_ms
Interval at which the server is publishing values in milliseconds.

samplingIntervalAtServer_ms
Sampling intervall at which the server updates its values in milliseconds.

nodes
Array of nodeID strings, e.g. ns=4;i=1, ns=mynamespace;s=myvar.int0. If the nodeID of an object is provided, all variables of that object are used.

MQTT Subscriber

Objects Values and descriptions
type Type of input (MQTT_SUBSCRIBER).
name Name of the MQTT_SUBSCRIBER input used in log messages.
clientId Client ID of the MQTT_SUBSCRIBER.
serverUrl Protocol and URL of the server to connect to, i.e. tcp://myserver.com:4840. Possible protocols are: tcp, mqtt, mqtts, ssl, ws or wss.
username MQTT broker authentication username.
password MQTT broker authentication password.
keepAliveInterval_s Maximum time allowed of no communication between client and server.
cleanSession Defines whether the server should remember state for the client across reconnect.
qos Quality of Service value (0-2).
sslOptions Secure connection options:

ca_certsFilePath
Path to CA certificate file.

certFilePath
Path to certificate file.

keyFilePath
Path to key file.

keyPassword
Password for the given key.

serverAuthRequired
Defines whether server authentication is required.
connectors Array of connectors for the MQTT_SUBSCRIBER:

name
Connector name.

topic
Name of the MQTT topic to use.

variables
Array of variables expected to be received:
  • name
    Variable name as found in the received message.
  • type
    Data type Gateway uses for further processing, must match receiving variable data type. Possible values are bool, sbyte, byte, int16, uint16, int32, uint32, int64, uint64, double, float, string, bytestring, datetime or guid.

OPC UA PubSub Subscriber

Item Description
type Type of input (OPC_UA_PS_SUBSCRIBER).
name Name of the OPC_UA_PS_SUBSCRIBER input used in log messages.
mqttClientId Client ID of the OPC_UA_PS_SUBSCRIBER when MQTT transport is used.
networkAddressUrl URL of the OPC UA PubSub connection to subscribe to.
networkInterface Name of the network interface to use.
secureConnectionOptions Secure connection options for MQTT transport:

ca_certsFilePath
Path to CA certificate file.

certFilePath
Path to certificate file.

keyFilePath
Path to key file.

keyPassword
Password for the given key.

serverAuthRequired
Defines whether server authentication is required.

username
Username for authentication.

password
Password for authentication.
writerGroups Array of writer groups at this connection:

publisherId
Publisher ID for the writer group.

writerGroupId
Writer group ID.

fieldsReversePublished
Some publishers send fields within a data set message in a reversed order compared to how the fields were added to that publisher. If this field is set to true, the user can use field indexes in the config file in the same order as the fields were added to these publishers. Must be set to true when connecting to a publisher using the TTTech PubSub library (i.e. CODESYS).

mqttTopic
Name of the MQTT topic if transport protocol is MQTT, ignored otherwise.
connectors Array of connectors for the OPC_UA_PS_SUBSCRIBER:

name
Connector name.

writerGroupIdx
Index of the writer group from the array of writer groups above.

dataSetWriterId
Data set writer ID.

variables
Array of variables expected to be received:
  • fieldIndex
    Field index of the variable in the data set.
  • name
    Name of the variable as found in the received message.
  • type
    Data type of the variable. Possible values are bool, int8, uint8, int16, uint16, int32, uint32, int64, uint64, double, float, string, bytestring, datetime or guid.

S7 Client

Item Description
type Type of output (S7_CLIENT).
name Name of the S7_CLIENT instance used in log messages.
serverUrl URL of the S7 server to connect to.
port Port at which to connect to.
connectionType Connection type (PG, OP, S7_BASIC).
localTsap Local tsap, mandatory if s7 basic connection type.
remoteTsap Remote tsap, mandatory if s7 basic connection type.
pollingInterval_ms Interval for polling values from server in miliseconds.
rack S7 device rack.
slot S7 device slot.
connectors Array of connectors for S7_CLIENT:

name
Connector name.

merkers
Array of S7 merkers:
  • name
    S7 merker name.
  • offset
    Address of first data.
  • quantity
    Number of merkers to be read.
  • type
    S7 merker data type.
inputs
Array of S7 inputs:
  • name
    S7 input name.
  • offset
    Address of first data.
  • quantity
    Number of inputs to be read.
  • type
    S7 input data type.
outputs
Array of S7 outputs:
  • name
    S7 output name.
  • offset
    Address of first data.
  • quantity
    Number of outputs to be read.
  • type
    S7 output data type.
timers
Array of S7 timers:
  • name
    S7 timer name.
  • offset
    Address of first data.
  • quantity
    Number of timers to be read.
counters
Array of S7 counters:
  • name
    S7 counter name.
  • offset
    Address of first data.
  • quantity
    Number of counters to be read.
datablocks
Array of S7 datablocks:
  • name
    S7 datablock name.
  • datablock
    S7 datablock number.
  • offset
    Address of first data.
  • quantity
    Number of datablocks to be read.
  • type
    S7 datablock data type.

Modbus Client

Item Description
type Type of output (MODBUS_CLIENT).
name Name of the MODBUS_CLIENT instance used in log messages.
serverUrl URL of the Modbus server to connect to.
port Port at which to connect to.
pollingInterval_ms Interval at which to poll values from the server in milliseconds.
connectors Array of connectors for the MODBUS_CLIENT:

name
Connector name.

coils
Array of Modbus coil type:
  • name
    Coil name, single string or an array of string.
  • address
    Starting address of first instance.
  • quantity
    Number of coils to be read.
discreteInputs
Array of Modbus discrete input type:
  • name
    Discrete input name, single string or an array of string.
  • address
    Starting address of first instance.
  • quantity
    Number of discrete inputs to be read.
inputRegisters
Array of Modbus input register type:
  • name
    Input register name, single string or an array of string.
  • address
    Starting address of first instance.
  • quantity
    Number of input registers to be read.
  • type
    Input register data type.
holdingRegisters
Array of Modbus holding register type:
  • name
    Holding register name, single string or an array of string.
  • address
    Starting address of first instance.
  • quantity
    Number of holding registers to be read.
  • type
    Holding register data type.

Outputs

In the tables below, fields in bold are mandatory and fields in italics are optional when writing a configuration for the corresponding protocol.

TimescaleDB

Item Description
type Type of output (DB_TIMESCALE).
name Name of the DB_TIMESCALE instance used in log messages.
url URL of the TimescaleDB server to connect to.
port Port at which to connect to.
dataRate_MBps Hypertable data rate in MegaBytes per second.
chunkTimeInterval_s Hypertable chunk time interval in seconds.
connectors Array of connectors for DB_TIMESCALE:

name
Connector name.

dbName
Database name to connect to.

tableName
Table name to write into.

user
Username for authentication.

password
Password for authentication.

booleanAsSmallint
Whether boolean should be represented as PostgreSQL small int data type.

InfluxDB

Item Description
type Type of output (DB_INFLUX).
name Name of the DB_INFLUX instance used in log messages.
url URL of the InfluxDB server to connect to.
port Port at which to connect to.
connectors Array of connectors for DB_INFLUX:

name
Connector name.

dbName
Database name to connect to.

user
Username for authentication.

password
Password for authentication.

MQTT Publisher

Item Description
type Type of output (MQTT_PUBLISHER).
name Name of the MQTT_PUBLISHER instance used in log messages.
clientId Client ID of the MQTT_PUBLISHER.
serverUrl Protocol and URL of the server to connect to, i.e. tcp://myserver.com:4840. Possible protocols are: tcp, mqtt, mqtts, ssl, ws or wss.
username MQTT broker authentication username.
password MQTT broker authentication password.
keepAliveInterval_s Maximum time allowed of no communication between client and server.
cleanSessions Defines whether the server should remember state for the client across reconnect.
qos Quality of Service value (0-2).
sslOptions Secure connection options:

ca_certsFilePath
Path to CA certificate file.

certFilePath
Path to certificate file.

keyFilePath
Path to key file.

keyPassword
Password for the given key.

serverAuthRequired
Defines whether server authentication is required.
connectors Array of connectors for MQTT_PUBLISHER:

name
Connector name.

topic
Name of the MQTT topic to use.

timestampRequired
Defines whether a timestamp is added to each message or not.

timestampFormat
Format of the timestamp in a message, either iso (ISO 8601) or unix_ns (UNIX time in nanoseconds since Jan 01 1970 (UTC)).

maxListSize
Maximum array size of variables, if sent as arrays. If size is larger than the max data will be sent in chunks of maxListSize.

OPC UA PubSub Publisher

Items Description
type Type of output (OPC_UA_PS_PUBLISHER).
name Name of OPC_UA_PS_PUBLISHER instance used in log messages.
mqttClientId Client ID of OPC_UA_PS_PUBLISHER when MQTT transport is used.
networkAddressUrl URL of the OPC UA PubSub connection to publish to.
networkInterface Name of the network interface to use.
keepAlive_s Maximum time allowed of no MQTT communication between client and server.
secureConnectionOptions Secure connection options for MQTT transport:

ca_certsFilePath
Path to CA certificate file.

certFilePath
Path to certificate file.

keyFilePath
Path to key file.

keyPassword
Password for the given key.

serverAuthRequired
Defines whether server authentication is required.

username
Username for authentication.

password
Password for authentication.
pubsubConnections Array of OPC UA PubSub connections

publisherId
Publisher ID of the connection.

writerGroups
List of writer groups for this Publisher ID:
  • writerGroupId
    Writer group ID.
  • publishInterval_ms
    Interval at which client publishes data in milliseconds.
  • mqttTopic
    MQTT topic name.
  • qos
    Quality of Service value (0-2).
  • dataSetWriters
    Array of data set writers:
      • dataSetWriterId
        Data set writer ID.
      • keyFrameCount
        Number of delta frames allowed to be sent before sending a key frame.
      • publishedDataSetIdxi
        Index in the list of published data set to be used by this dat set writer.
publishedDataSets
Array of published data sets:
  • fields
    Array of fields (variables) of the published data sets:
      • name
        Name of the field (variable) used within the Gateway.
      • type
        Data type of the field (variable). Possible values are bool, int8, uint8, int16, uint16, int32, uint32, int64, uint64, double, float, string, bytestring, datetime or guid.
connectors Array of connectors for OPC_UA_PS_PUBLISHER:

name
Connector name.

publisherIdx
Index of the publisher in the array of OPC UA PubSub connections.

writerGroupIdx
Index in the list of writer groups at the OPC UA PubSub connection.

dataSetWriterId
Index in the list of data set writers in the writer group at the OPC UA PubSub connection.

OPC UA Server

Item Description
type Type of output (OPC_UA_SERVER).
name Name of OPC_UA_SERVER instance used in log messages.
customHostname Custom hostname of the server, used for discovery URL.
port Port to listen at (defaults to 4840).
addressSpaceFolders OPC_UA_SERVER address space tree:

browseName
OPC UA Folder browse name.

displayName
OPC UA Folder display name.

description
OPC UA Folder description.

namespaceURI
OPC UA Folder namespace URI.

identifier
OPC UA Folder nodeID identifier.

children
Array of OPC UA childFolders, same format as addressSpaceFolders field.

connectorIndicies
Array of indicies of connectors, placed in the OPC UA Folder.
connectors Array of connectors for OPC_UA_SERVER:

name
Connector name.

browseName
OPC UA node browse name.

displayName
OPC UA node display name.

description
OPC UA node description.

namespaceURI
OPC UA node namespace URI.

identifier
OPC UA node nodeID identifier.

ZeroMQ Publisher

Item Description
type Type of output (ZEROMQ_PUBLISHER).
name Name of the ZEROMQ_PUBLISHER instance used in log messages.
serverUrl Server endpoint to publish to. Subscribers connect to it.
connectors Array of connectors for ZEROMQ_PUBLISHER:

name
Connector name.

topic
Name of the ZeroMQ topic to publish to.

timestampRequired
Defines whether a timestamp is added to each message or not.

timestampFormat
Format of the timestamp in a message, either iso (ISO 8601) or unix_ns (UNIX time in nanoseconds since Jan 01 1970 (UTC)).

maxListSize
Maximum array size of variables, if sent as arrays. If size is larger than the max data will be sent in chunks of maxListSize.

Connections

Item Description
name Name of the connection, will define the name of the TABLE within TimescaledDB if tableName field is omitted.
input Input of the connection.

index
Index in the array of INPUTs.

connector
Index in the array of connectors at the input.
output Output of the connection.

index
Index in the array of OUTPUTs.

connector
Index in the array of connectors at the output.

Gateway configuration file keywords

Parts of the Gateway configuration are eased by the use of keywords. Keywords can be used on both Gateway instances on the node and in the Management System. However, the keyowrds that can be used differ. All keywords must be put in angled brackets, for example <SN>. To be able to use keywords. the node must be registered in the Management System.

Refer to the list below for keywords that can be used with usage examples right after.

Supported on the node

Keyword Description
SN
  • Replaces the keyword with the serial number of the node.
  • Used as a username for MQTT Subscriber/Publisher when connecting to the Management System broker.
  • Can also be used as a root MQTT topic.
SID
  • Replaces the keyword with the secure ID of the node.
  • Used as a password for MQTT Subscriber/Publisher when connecting to the Management System broker.
MGMT-SYS
  • Replaces the keyword with the URL of the Management System the node is registered in.
  • Used in serverUrl field of the MQTT Subscriber/Publisher when connecting to the Management System broker.
  • Adds unique clientId field to the input or output the keyword is found in. If clientId already exists, it is replaced.
LOCAL
  • Replaces the keyword with localhost.
  • Used in url field of TimescaleDB output.
  • Adds port field to the TimescaleDB output.
  • Adds connectors array with a single connector to the TimescaleDB output. If connectors already exists, replaces it. Adds name, dbName, user and password fields to that connector.

Supported in the Management System

Keyword Description
SN:<nodename>
  • Replaces the keyword with the serial number of the node that was specified in the keyword.
  • Used in dbName field of TimescaleDB outputs connectors.
  • Can also be used as a root MQTT topic.
  • When used in dbName field of TimescaleDB outputs connector, adds user and password fields to it.
MS‑MQTT‑BROKER
  • Replaces the keyword with the URL of the MQTT broker in the Management System.
  • Used in serverUrl field of the MQTT Subscriber/Publisher when connecting to the Management System broker.
  • Adds unique clientId field to the input or output the keyword is found in. If clientId already exists, it is replaced. Also adds username and password fields to the same input or output.

Examples of keyword usage

SN:<nodename>

{
    "name": "cloud_timescale",
    "type": "DB_TIMESCALE",
    "url": "localhost",
    "port": 5432,
    "connectors": [
        {
            "name": "EXAMPLE-NODE",
            "dbName": "<SN:EXAMPLE-NODE>"
        }
    ]
}

SN, SID, MGMT-SYS

{
    "name": "ms_mqtt_broker",
    "type": "MQTT_SUBSCRIBER",
    "serverUrl": "wss://<MGMT-SYS>:443/dp/mqtt/wss",
    "username": "<SN>",
    "password": "<SID>",
    "keepAliveInterval_s": 20,
    "qos": 1,
    "cleanSession": true,
    "sslOptions": {
        "serverAuthRequired": false
    },
    "connectors": [
        {
            "name": "demo_sensor_connector_0",
            "topic": "<SN>/demo_sensor_topic_0",
            "variables": [
                {
                    "name": "temperature",
                    "type": "int64"
                }
            ]
        }
    ]
}

LOCAL

{
    "type": "DB_TIMESCALE",
    "name": "timescaledb_0",
    "url": "<LOCAL>"
}

SN:<nodename>, MS-MQTT-BROKER

{
    "name": "ms_mqtt_broker",
    "type": "MQTT_SUBSCRIBER",
    "serverUrl": "<MS-MQTT-BROKER>",
    "keepAliveInterval_s": 20,
    "qos": 1,
    "cleanSession": true,
    "connectors": [
        {
            "name": "demo_sensor_connector_1",
            "topic": "<SN:EXAMPLE-NODE>/demo_sensor_topic_1",
            "variables": [
                {
                    "name": "humidity",
                    "type": "uint64"
                }
            ]
        }
    ]
}