> Data Gateway - Nerve Blue 2.2.1 Documentation Skip to content
PDF

Nerve Data Services Gateway

The Gateway is the central application of the Data Services. Its purpose is receiving data from a source via a certain protocol on an input interface and forwarding it to a destination using a different protocol on an output interface. This behavior is reflected in the user configuration as well. In general, the configuration file consists of inputs, outputs and connections between them:

  • Inputs are collection interfaces where data is received.
  • Outputs are providing interfaces where data can be received from.
  • Connections are logical links between inputs and outputs.

Each input and output provides connectors where a connection can be attached to. A connector is a subset of the data available at an input. This subset is forwarded to the connector of an output where it is distributed further to the data receiver. On an abstract level, this concept is illustrated in the figure below.

!Gateway Concept

Inputs and outputs hold general configuration data for the protocol they implement. This can be a URL to a server, a port number or something related to timing. A connector of an input or output holds more specific information on how to get or where to send data, for example the node IDs of an OPC UA Server an OPC UA Client input connects to, or the name of a topic for an MQTT Publisher. Each input and output can provide multiple connectors at a time. Data from an input connector can also be connected to multiple output connectors at the same time to distribute data to multiple data consumers. The figure below shows a more specific example of that concept.

!Gateway Concept Example

Data Gateway UI

The Data Gateway UI is the landing page of the Data Services and will be displayed by default after selecting Data > CONFIGURATION in the navigation on the left in either Local UI or Management System. The editor in the middle is used to apply Gateway configurations in the JSON format.

!Gateway UI

Item Description
Gateway status and controls (1) The status of the Gateway is displayed in the upper-right corner with controls next to it that allow starting, stopping or restarting of the Gateway. Running indicates that the configuration has been accepted and the Gateway is running. When the status is stopped, there is either a problem in the configuration or the Gateway has been stopped manually.
Editor (2) Enter a configuration in the JSON format into the editor to configure the Gateway. The configuration can be written directly in the editor or uploaded from the workstation.
  • Upload
    Select this to upload the contents of an existing JSON file from the workstation into the editor.
  • Reload
    Select this to load the currently applied configuration of the Gateway into the editor.
  • Apply
    Click here to set the current content of the editor as the Gateway configuration. The Gateway will be restarted.
Configuration and Logs tabs (3) The Configuration tab is displayed by default. The latest logs of the Gateway can be viewed within the Logs tab. The entirety of the logs can be exported into a file using the Download button in this tab.

Applying a configuration to the Gateway

The configuration of the Gateway is applied in the editor in the Gateway section of the Data Services UI. The current configuration of the Gateway is loaded into the JSON editor if there is a configuration present in the Gateway. However, the editor will be empty on initial startup. A new configuration can be entered into the editor directly or imported from an existing JSON file.

  • Upload existing JSON files using the Upload button.
  • Apply the configuration in the editor by selecting Apply. When the configuration is successful, the status in the upper-right corner will display running.
  • Check the Logs tab for any warnings or errors.

The basic structure of the configuration consist of a list of inputs such as an OPC UA server or an MQTT subscriber, outputs such as a Timescale database or an MQTT publisher, and connections between these inputs and outputs. Refer to Gateway configuration file syntax below for the exact syntax.

The examples in the Examples section show step by step applications in common use cases.

Gateway configuration file syntax

The following list gives additional clarification about how to set parameters with the JSON editor. The Gateway configuration JSON file includes sections, which are defined as main sections (inputs, outputs, connections). The main sections may contain one set of subsections or an array of items, which in this case are defined as an array of subsection sets.

Subsections can also include additional subsections. A section is defined as an array if the section contains two or more subsections. Subsection keys are defined as parameters. Every parameter has its field for a value. The main sections can contain the same and different parameters as other sections.

The syntax of the various inputs and outputs of the Gateway available in its configuration file is illustrated below for the available protocols.

Inputs Outputs
MQTT subscriber MQTT publisher
OPC UA PubSub over MQTT OPC UA PubSub over MQTT
OPC UA PubSub over UDP OPC UA PubSub over UDP
OPC UA Client OPC UA server
Modbus server ZeroMQ publisher
S7 server Azure IoT Hub Device
TimescaleDB
Influx DB

Inputs

In the tables below, fields in bold are mandatory and fields in italics are optional when writing a configuration for the corresponding protocol.

OPC UA Client

Item Description
type Type of input (OPC_UA_CLIENT).
name Name of OPC_UA_CLIENT instance used in log messages.
serverUrl URL of the server to connect to, i.e. opc.tcp://myserver.com:4840.
login Authentication parameters.

username
Username for client authentication.

password
Password for client authentication.
certificate Secure connection parameters.

certFilePath
Certificate file path for client authentication.

keyFilePath
Key file path for client authentication.
trustList Array of certificates of trusted servers.
security Connection security mode and policy.

securityMode
Security mode (None, Sign, SignAndEncrypt).

securityPolicy
Security policy (None, Basic128Rsa15, Basic256, Basic256Sha256).
description Description of the client.

applicationUri
Application URI of the client, used in secure connection certificate.
pollingInterval_ms Interval at which to poll values from the server in milliseconds.
connectors Array of connectors for OPC_UA_CLIENT.

name
Connector name.

accessType
Access mode, either polling or subscription.

publishingIntervalAtServer_ms
Interval at which the server is publishing values in milliseconds.

samplingIntervalAtServer_ms
Sampling intervall at which the server updates its values in milliseconds.

nodes
Array of nodeID strings, e.g. ns=4;i=1, ns=mynamespace;s=myvar.int0. If the nodeID of an object is provided, all variables of that object are used.

MQTT Subscriber

Objects Values and descriptions
type Type of input (MQTT_SUBSCRIBER).
name Name of the MQTT_SUBSCRIBER input used in log messages.
clientId Client ID of the MQTT_SUBSCRIBER.
serverUrl Protocol and URL of the server to connect to, i.e. tcp://myserver.com:4840. Possible protocols are: tcp, mqtt, mqtts, ssl, ws or wss.
username MQTT broker authentication username.
password MQTT broker authentication password.
keepAliveInterval_s Maximum time allowed of no communication between client and server.
cleanSession Defines whether the server should remember state for the client across reconnect.
qos Quality of Service value (0-2).
sslOptions Secure connection options:

ca_certsFilePath
Path to CA certificate file.

certFilePath
Path to certificate file.

keyFilePath
Path to key file.

keyPassword
Password for the given key.

serverAuthRequired
Defines whether server authentication is required.
connectors Array of connectors for the MQTT_SUBSCRIBER:

name
Connector name.

topic
Name of the MQTT topic to use.

timestamp
Timestamp related options
  • path
    JSON path pointing to the timestamp value(s).
variables
Array of variables expected to be received:
  • name
    Variable name as found in the received message. If omitted, path is required and will be used as the name.
  • type
    Data type Gateway uses for further processing, must match receiving variable data type. Possible values are bool, sbyte, byte, int16, uint16, int32, uint32, int64, uint64, double, float, string, bytestring, datetime or guid.
  • path
    JSON path pointing to the value(s). If both name and path are provided, the path will be path + name.
  • maxValues
    Maximum values expected when using expansion operator [*] in the path. Otherwise, omitted.

OPC UA PubSub Subscriber

Item Description
type Type of input (OPC_UA_PS_SUBSCRIBER).
name Name of the OPC_UA_PS_SUBSCRIBER input used in log messages.
mqttClientId Client ID of the OPC_UA_PS_SUBSCRIBER when MQTT transport is used.
networkAddressUrl URL of the OPC UA PubSub connection to subscribe to.
networkInterface Name of the network interface to use.
encoding Type of message encoding (JSON/UADP).
connectors Array of connectors for the OPC_UA_PS_SUBSCRIBER:

name
Connector name.

publisherId
Publisher ID for the writer group.

writerGroupId
Writer group ID.

dataSetWriterId
Data set writer ID.

fieldsReversePublished
Some publishers send fields within a data set message in a reversed order compared to how the fields were added to that publisher. If this field is set to true, the user can use field indexes in the config file in the same order as the fields were added to these publishers. Must be set to true when connecting to a publisher using the TTTech PubSub library (i.e. CODESYS).

mqttTopic
Name of the MQTT topic if transport protocol is MQTT, ignored otherwise.

variables
Array of variables expected to be received:
  • fieldIndex
    Field index of the variable in the data set.
  • name
    Name of the variable as found in the received message.
  • type
    Data type of the variable. Possible values are bool, int8, uint8, int16, uint16, int32, uint32, int64, uint64, double, float, string, bytestring, datetime or guid.

S7 Client

Item Description
type Type of output (S7_CLIENT).
name Name of the S7_CLIENT instance used in log messages.
serverUrl URL of the S7 server to connect to.
port Port at which to connect to.
connectionType Connection type (PG, OP, S7_BASIC).
localTsap Local tsap, mandatory if s7 basic connection type.
remoteTsap Remote tsap, mandatory if s7 basic connection type.
pollingInterval_ms Interval for polling values from server in miliseconds.
rack S7 device rack.
slot S7 device slot.
connectors Array of connectors for S7_CLIENT:

name
Connector name.

merkers
Array of S7 merkers:
  • name
    S7 merker name.
  • offset
    Address of first data.
  • quantity
    Number of merkers to be read.
  • type
    S7 merker data type.
inputs
Array of S7 inputs:
  • name
    S7 input name.
  • offset
    Address of first data.
  • quantity
    Number of inputs to be read.
  • type
    S7 input data type.
outputs
Array of S7 outputs:
  • name
    S7 output name.
  • offset
    Address of first data.
  • quantity
    Number of outputs to be read.
  • type
    S7 output data type.
timers
Array of S7 timers:
  • name
    S7 timer name.
  • offset
    Address of first data.
  • quantity
    Number of timers to be read.
counters
Array of S7 counters:
  • name
    S7 counter name.
  • offset
    Address of first data.
  • quantity
    Number of counters to be read.
datablocks
Array of S7 datablocks:
  • name
    S7 datablock name.
  • datablock
    S7 datablock number.
  • offset
    Address of first data.
  • quantity
    Number of datablocks to be read.
  • type
    S7 datablock data type.

Modbus Client

Item Description
type Type of output (MODBUS_CLIENT).
name Name of the MODBUS_CLIENT instance used in log messages.
serverUrl URL of the Modbus server to connect to.
port Port at which to connect to.
pollingInterval_ms Interval at which to poll values from the server in milliseconds.
connectors Array of connectors for the MODBUS_CLIENT:

name
Connector name.

coils
Array of Modbus coil type:
  • name
    Coil name, single string or an array of string.
  • address
    Starting address of first instance.
  • quantity
    Number of coils to be read.
discreteInputs
Array of Modbus discrete input type:
  • name
    Discrete input name, single string or an array of string.
  • address
    Starting address of first instance.
  • quantity
    Number of discrete inputs to be read.
inputRegisters
Array of Modbus input register type:
  • name
    Input register name, single string or an array of string.
  • address
    Starting address of first instance.
  • quantity
    Number of input registers to be read.
  • type
    Input register data type.
holdingRegisters
Array of Modbus holding register type:
  • name
    Holding register name, single string or an array of string.
  • address
    Starting address of first instance.
  • quantity
    Number of holding registers to be read.
  • type
    Holding register data type.

Outputs

In the tables below, fields in bold are mandatory and fields in italics are optional when writing a configuration for the corresponding protocol.

TimescaleDB

Item Description
type Type of output (DB_TIMESCALE).
name Name of the DB_TIMESCALE instance used in log messages.
url URL of the TimescaleDB server to connect to.
port Port at which to connect to.
dataRate_MBps Hypertable data rate in MegaBytes per second.
chunkTimeInterval_s Hypertable chunk time interval in seconds.
connectors Array of connectors for DB_TIMESCALE:

name
Connector name.

dbName
Database name to connect to.

tableName
Table name to write into.

user
Username for authentication.

password
Password for authentication.

booleanAsSmallint
Whether boolean should be represented as PostgreSQL small int data type.

InfluxDB

Item Description
type Type of output (DB_INFLUX).
name Name of the DB_INFLUX instance used in log messages.
url URL of the InfluxDB server to connect to.
port Port at which to connect to.
connectors Array of connectors for DB_INFLUX:

name
Connector name.

dbName
Database name to connect to.

user
Username for authentication.

password
Password for authentication.

MQTT Publisher

Item Description
type Type of output (MQTT_PUBLISHER).
name Name of the MQTT_PUBLISHER instance used in log messages.
clientId Client ID of the MQTT_PUBLISHER.
serverUrl Protocol and URL of the server to connect to, i.e. tcp://myserver.com:4840. Possible protocols are: tcp, mqtt, mqtts, ssl, ws or wss.
username MQTT broker authentication username.
password MQTT broker authentication password.
keepAliveInterval_s Maximum time allowed of no communication between client and server.
cleanSessions Defines whether the server should remember state for the client across reconnect.
qos Quality of Service value (0-2).
sslOptions Secure connection options:

ca_certsFilePath
Path to CA certificate file.

certFilePath
Path to certificate file.

keyFilePath
Path to key file.

keyPassword
Password for the given key.

serverAuthRequired
Defines whether server authentication is required.
connectors Array of connectors for MQTT_PUBLISHER:

name
Connector name.

topic
Name of the MQTT topic to use.

timestampRequired
Defines whether a timestamp is added to each message or not.

timestampFormat
Format of the timestamp in a message, either iso (ISO 8601) or unix_ns (UNIX time in nanoseconds since Jan 01 1970 (UTC)).

maxListSize
Maximum array size of variables, if sent as arrays. If size is larger than the max data will be sent in chunks of maxListSize.

OPC UA PubSub Publisher

Items Description
type Type of output (OPC_UA_PS_PUBLISHER).
name Name of OPC_UA_PS_PUBLISHER instance used in log messages.
networkAddressUrl URL of the OPC UA PubSub connection to publish to.
networkInterface Name of the network interface to use.
encoding Type of message encoding (JSON/UADP).
connectors Array of connectors for OPC_UA_PS_PUBLISHER:

name
Connector name.

publisherId
Publisher ID of OPC UA PubSub connection.

writerGroupId
Writer group ID in the OPC UA PubSub connection.

dataSetWriterId
Data set writer ID in the writer group.

publishInterval_ms
Interval at which publisher publishes messages.

keyFrameCount
Key frame count for messages.

mqttTopic
Topic to publish to in case of MQTT connection.

mqttClientId
MQTT client ID for PubSub connection in case of MQTT connection. This must be unique for each connector.

OPC UA Server

Item Description
type Type of output (OPC_UA_SERVER).
name Name of OPC_UA_SERVER instance used in log messages.
customHostname Custom hostname of the server, used for discovery URL.
port Port to listen at (defaults to 4840).
logins Array of username/password pairs available for authentication:

username
Username for authentication.

password
Password for authentication.
securities Array of available security modes and policies:

securityMode
Security Mode ("all" keyword is supported to create all available endpoints for chosen Security Policy).

securityPolicy
Security Policy ("all" keyword is supported to create all available endpoints for chosen Security Mode).
certificate Server's certificate:

certFilePath
Path to Server Instance Certificate.

keyFilePath
Path to private key.
description Server Description:

applicationUri
Server Application URI.
trustList Array of trusted certificates.
addressSpaceFolders OPC_UA_SERVER address space tree:

browseName
OPC UA Folder browse name.

displayName
OPC UA Folder display name.

description
OPC UA Folder description.

namespaceURI
OPC UA Folder namespace URI.

identifier
OPC UA Folder nodeID identifier.

children
Array of OPC UA childFolders, same format as addressSpaceFolders field.

connectorIndicies
Array of indicies of connectors, placed in the OPC UA Folder.
connectors Array of connectors for OPC_UA_SERVER:

name
Connector name.

browseName
OPC UA node browse name.

displayName
OPC UA node display name.

description
OPC UA node description.

namespaceURI
OPC UA node namespace URI.

identifier
OPC UA node nodeID identifier.

ZeroMQ Publisher

Item Description
type Type of output (ZEROMQ_PUBLISHER).
name Name of the ZEROMQ_PUBLISHER instance used in log messages.
serverUrl Server endpoint to publish to. Subscribers connect to it.
connectors Array of connectors for ZEROMQ_PUBLISHER:

name
Connector name.

topic
Name of the ZeroMQ topic to publish to.

timestampRequired
Defines whether a timestamp is added to each message or not.

timestampFormat
Format of the timestamp in a message, either iso (ISO 8601) or unix_ns (UNIX time in nanoseconds since Jan 01 1970 (UTC)).

maxListSize
Maximum array size of variables, if sent as arrays. If size is larger than the max data will be sent in chunks of maxListSize.

Azure IoT Hub Device

Item Description
type Type of output (AZURE_IOT_HUB_DEVICE).
name Name of the AZURE_IOT_HUB_DEVICE instance used in log messages.
deviceConnectionString Connection string of the device created in the MS Azure IoT Hub.
mqttOverWss Determines if the AZURE_IOT_HUB_DEVICE instance should use MQTT over WSS (port 433) or regular MQTT over TLS/SSL (port 8883) to connect to the Azure IoT Hub broker.
sslOptions Secure connection options for MQTT transport:

certFilePath
Path to certificate file.

keyFilePath
Path to key file.

keyPassword
Password for the given key.
connectors Array of connectors for AZURE_IOT_HUB_DEVICE:

name
Connector name.

timestampRequired
Defines whether a timestamp is added to each message or not.

timestampFormat
Format of the timestamp in a message, either iso (ISO 8601) or unix_ns (UNIX time in nanoseconds since Jan 01 1970 (UTC)).

maxListSize
Maximum array size of variables if sent as arrays. If size is larger than the maximum, data will be sent in chunks of maxListSize.

customProperties
Custom properties array that is included with every published message:
  • name
    Name of the custom property.
  • value
    Value of the custom property. May be a constant value, or the value from one of the input variables of the message, in which case it should be the variable name enclosed in { }.

Connections

Item Description
name Name of the connection, will define the name of the TABLE within TimescaleDB if tableName field is omitted.
input Input of the connection.

index
Index in the array of INPUTs.

connector
Index in the array of connectors at the input.
output Output of the connection.

index
Index in the array of OUTPUTs.

connector
Index in the array of connectors at the output.

Gateway configuration file keywords

Parts of the Gateway configuration are eased by the use of keywords. Keywords can be used on both Gateway instances on the node and in the Management System. However, the keyowrds that can be used differ. All keywords must be put in angled brackets, for example <SN>. To be able to use keywords. the node must be registered in the Management System.

Refer to the list below for keywords that can be used with usage examples right after.

Supported on the node

Keyword Description
SN
  • Replaces the keyword with the serial number of the node.
  • Used as a username for MQTT Subscriber/Publisher when connecting to the Management System broker.
  • Can also be used as a root MQTT topic.
SID
  • Replaces the keyword with the secure ID of the node.
  • Used as a password for MQTT Subscriber/Publisher when connecting to the Management System broker.
LOCAL
  • Replaces the keyword with localhost.
  • Used in url field of TimescaleDB output.
  • Adds port field to the TimescaleDB output.
  • Adds connectors array with a single connector to the TimescaleDB output if none exist. Adds or replaces name, dbName, user and password fields for all connectors.
MS‑MQTT‑BROKER The MQTT broker URL is fetched from the node registration data on the node.
  • Replaces the keyword with the URL of the MQTT broker.
  • Used in the serverUrl field of the MQTT Subscriber/Publisher when connecting to the Management System broker.
  • Adds unique clientId field to the input or output the keyword is found in. If clientId already exists, it is replaced.

Supported in the Management System

Keyword Description
SN:<nodename>
  • Replaces the keyword with the serial number of the node that was specified in the keyword.
  • Used in dbName field of TimescaleDB outputs connectors.
  • Can also be used as a root MQTT topic.
  • When used in dbName field of TimescaleDB outputs connector, adds user and password fields to it.
MS‑MQTT‑BROKER The MQTT broker URL is fetched from the Gateway system configuration in the Management System.
  • Replaces the keyword with the URL of the MQTT broker.
  • Used in the serverUrl field of the MQTT Subscriber/Publisher when connecting to the Management System broker.
  • Adds unique clientId field to the input or output the keyword is found in. If clientId already exists, it is replaced. Also adds username and password fields that are read from the Management System YAML configuration file to the same input/output.

Examples of keyword usage

SN:<nodename>

{
    "name": "cloud_timescale",
    "type": "DB_TIMESCALE",
    "url": "localhost",
    "port": 5432,
    "connectors": [
        {
            "name": "EXAMPLE-NODE",
            "dbName": "<SN:EXAMPLE-NODE>"
        }
    ]
}

SN, SID, MS-MQTT-BROKER

{
    "name": "ms_mqtt_broker",
    "type": "MQTT_SUBSCRIBER",
    "serverUrl": "<MS-MQTT-BROKER>",
    "username": "<SN>",
    "password": "<SID>",
    "keepAliveInterval_s": 20,
    "qos": 1,
    "cleanSession": true,
    "sslOptions": {
        "serverAuthRequired": false
    },
    "connectors": [
        {
            "name": "demo_sensor_connector_0",
            "topic": "<SN>/demo_sensor_topic_0",
            "variables": [
                {
                    "name": "temperature",
                    "type": "int64"
                }
            ]
        }
    ]
}

LOCAL

{
    "type": "DB_TIMESCALE",
    "name": "timescaledb_0",
    "url": "<LOCAL>"
}

SN:<nodename>, MS-MQTT-BROKER

{
    "name": "ms_mqtt_broker",
    "type": "MQTT_SUBSCRIBER",
    "serverUrl": "<MS-MQTT-BROKER>",
    "keepAliveInterval_s": 20,
    "qos": 1,
    "cleanSession": true,
    "connectors": [
        {
            "name": "demo_sensor_connector_1",
            "topic": "<SN:EXAMPLE-NODE>/demo_sensor_topic_1",
            "variables": [
                {
                    "name": "humidity",
                    "type": "uint64"
                }
            ]
        }
    ]
}