1. Home
  2. TC10.9 - IOT Directory and Multiple Brokers

TC10.9 - IOT Directory and Multiple Brokers

×

Warning message

You can't delete this newsletter because it has not been sent to all its subscribers.

Test Case Title

TC10.9 - IOT Directory and Multiple Brokers

Goal

To provide evidence about the fact that the solution:

  • Support multiple brokers
  • Support brokers of different kind
  • Support security end-to-end from device to dashboard

Prerequisites

Access to the platform.

The following functionalities are available only for specific Snap4city users with specific privileges.

Expected successful result

Evidence about the fact that the platform supports multiple IOT devices, IOT brokers and that the solution is secure.

Steps

  

 

overview on IOT directory and management: Snap4City IOT Directory and Devices

The Snap4City platform supports multiple communication patterns without limitations, among them: discovery; telemetry; inquiries; commands; notifications.

For covering the different approaches/protocols, the Snap4City platform adopts: multiple IOT Brokers (IOT Orion Broker, RabbitMQ, ActiveMQ, ..), Wrappers, ETL Processes for direct connections, NodeRED blocks.

In details, Snap4City is multiprotocol integrating different solutions for connecting and interacting with IOT devices (on protocols such as: COAP, MQTT, STOMP, AMQP, TLS, DTLS, etc.) and other level protocols (SOAP, HTTP, OneM2M, REST, HTTPS, SFTP, ODBC, WebSocket, etc.).  

In Snap4City it is possible to handle any kind of brokers without the need to modify their code. This is an important feature in order to enhance the acceptance of Snap4City in different cities, regions. Each IOT Broker should offer basic facilities for the registration of sensors/actuators and checking the presence of new sensors/actuators. A monitor is made available for checking the presence of a new sensors/actuators in a given context broker. When this happens, the presence of the sensor/actuator is registered in the IoT Directory.  

Depending on the characteristics of the context broker, details on the sensor/actuator can be automatically generated and stored in the knowledge base or marked for being annotated by hand from the user. The IOT Directory is a Web application to also show the registered sensors, those that need to be manually annotated, and those that have been automatically annotated. The basic annotations are related to the kind of sensor/actuator, its geographical position, the schema of the observations produced by the sensor (or the kinds of actions that can be invoked in case of actuators). Moreover, geographical and dynamic information about the IOT devices is stored in the Knowledge Base like the minimal, maximal values that they produce, the usual frequency of production of observations, the last generated value, etc. This dynamic information is used for monitoring the status of the sensor (active and working properly, active but producing unusual data, and/or for detecting inactivity). All the IOT events generated by the registered IOT Devices are stored in HBase/Phoenix storage. In this way, backups of all generated events are maintained for further analysis, for audit, for performance assessment, etc. In the context of the Snap4City Application Builder, specific NodeRED blocks is developed for providing MicroServices. These blocks is able to exploit the information maintained in the IOT Directory for discovering sensors/actuators needed for a given analysis or to apply intelligent behaviours.  

The following figure depicts the relationships among the IOT Brokers and the internal components of the Snap4City Solution. In the context of IOT the main components are:

  • IOT Directory: a tool for:
    • registering IOT Devices via a number of IOT Brokers. At the registration the IOT Directory:
      • index them,
      • perform the registration on the Knowledge base,
      • generate a couple of keys, if  they are not provided by the IOT broker or user
    • Searching IOT Devices by using keywords and features via textual web interface;
    • Searching IOT Devices by using geographical and map based interface which is based on ServiceMap. So that, it is possible to search for IOT devices, close to a point, along a line, in an area, etc.
    • providing a unique point for searching IOT Devices, while subscriptions are performed directly.
    • addressing multiple brokers and protocols: Orion Broker, Mosquitto, AMQT, etc.
    • Working with the Knowledge base for the IOT Discovery, around a point, along a path, in an area, etc.
  • IOT Discovery: a tool based on the Knowledge Base and Advanced Smart City API (that is ServiceMap) to visually search IOT devices any kind, exploiting text and geo search. IOT devices along a path, in an area, closer to a point, of a given type/subtype, etc. The result is a set of Identification ID, ServiceURI of Km4City ontology, corresponding to the Device IDs into the IOT Directory.
     

     
  • IOT Brokers: a collection of IOT Brokers of different kind, addressing one or multiple protocols, supporting or not the protected protocols. Among the brokers:  Orion Context Broker which is mainly based on NGSI protocols; RabbitMQ, Mosquitto.
  • IOT Event Logger (Broker of brokers, also called Multiprotocol Subscriber): a tool and process to log all the activities performed by IOT devices on the different Brokers in a unique storage. The IOT Event Logger storage can be indexed with SOLR and browsed with Banana, etc.
  • IOT Wrappers: are modules transcoding IOT protocols so as to make possible the connection of certain brokers with: (i) upper level brokers, (ii) IOT directory. On the other hand, some of the IOT Wrappers can be also located from the Device to the broker. So as to make possible the registration and work with a Brokers with devices having also unaddressed protocols. 
  • IOT Authentication and Authorization: connection to reuse the Authentication and Authorization in the context of the IOT communications.

 
Technical Registration
The following registration is technical and it is not describing the action to be performed by the Final User for registering a device, but only the technical details on HOW it works.
The registration of an IOT Device to the IOT Broker and from the broker to the IOT Directory.

 

  • (1) Registering the device on an IOT Broker (e.g., IOT Orion Fi-Ware) to access or communicate any event on it. In some cases the IOT Broker provides only manual registration. Thus
    • The Broker has to be automatically registered to any event of that IOT device,
    • Any change appears and it is stored into the Broker
  • (2) The Broker performs the registration on the IOT Directory, which supports multiple protocols.
    • (2a) eventual manual registration would be possible
  • (3) save on Knowledge Base in terms of triples as an IOT device on the Map, now it can be discovered on the map; The Knowledge Base/ServiceMap has all information about the new IOT Device including a link to:
    • (i) see its story of changes on IOT Logger, broker of brokers,
    • (ii) the identifier to pass it at the NodeRED when IOT Discovery on Map or by text is performed.
  • (4) return back to the IOT Directory the unique ServiceURI
  • (4a) the Device is now available on Snap4City Application Builder (based on NodeRED) as MicroService.
    • A block is available for registering a new IOT device on the Context Broker
    • A service is available for retrieving the list of registered IOT devices on the context Broker
    • For any new IOT device a new instance of that Block (connected to that IOT Device) has to be automatically generated for subscribing to the events on that IOT device when a NodeRED process is put in execution/play/debug.
    • The subscription is directly performed on the IOT Brokers .
  • (5) (6) NIFI based MultiProtocol Subscriber is capable to perform a subscription to all the IOT Brokers in the architecture. This allow to get data streams and index them for monitoring and historical access on Solr Index.
    Once registered the IOT Device/ETL data become ready for subscription and thus accessible on the IOT Directory for Discovery on it and on the KB. 


Technical Telemetry
In the following figure the process of execution with telemetry and application usage is provided.
The Telemetry protocol is based on the fact that any changes in the IOT Device are communicated to the IOT Broker.



 
In the above figure,

  1. a IOT device communicates  data change and/or value to a broker. The same approach can be followed by some ETL processes reading Real-time data from some server (1bis).
  2. the broker communicates with all subscribers
    1. the Multiprotocol Subscriber (Broker of Brokers) The Multiprotocol Subscriber logs all the changes into the Hbase (directly or via Phoenix). Making them accessible for the Dashboard Builder / Engine as referral/historical data. Thus the log provokes the indexing of the new data into the SOLR index (via Hbase Indexer (4)), making them accessible on the Developer Dashboard.
    2. The IOT Applications via (2bis). They can also read ETL and IOT devices providing data in real-time. The Snap4City Application:
  • can exploit event driven data, and also the access to referral data requesting them from Hbase/Phoenix in SQL or from SOLR (2ter).
  • may also have some output block Dashboard as depicted in the figure and from them to the Dashboards of the Dashboard Builder.
    Please note that:
  • Snap4City Applications can also write on the Hbase for creating new historical data with results. They could be also Data Analytics or ETL processes as well.
  • City Dashboards may have actuators widgets. They are registered on some IOT Brokers and can produce (6) messages to them, and from them to change the input of some Application.


Security Aspects (see for more details the previous TC)
The Snap4City provides an end-to-end secure solution from IOT Devices to Dashboard.


The main points are;

  • The IOT brokers which does not provide authentication have been wrapped by using a Proxy Filter Security (Orion Broker Filter, NGSI Security Wrapper) module that is capable to communicate with module “MyPersonalData and Ownership” (the user Profiler) which store key and other info in an encrypted database, via HTTPS SSO (secure channel based on TLS and KeyCloak authentication based on Token).
  • Each request to the IOT Broker has to pass from the Proxy Filter Security (Orion Broker Filter, NGSI Security Wrapper), which verify if the user performing  the request is authorized, for example for: Subscription, Telemetry from the IOT device, etc.
  • Each Message produced by the IOT Broker to other devices and IOT applications is on HTTPS.
  • user performs the registration of the IOT Device on the IOT directory. During the registration the IOT Directory produces a couple of Keys (K2, K2), the user provides the DEVICE NAME ID, location, etc. IOT Directory provides/assigns at the device an IOT broker according to the IOT Device Model. This information is accessible for the user, that can insert those values into the IOT device. This approach is viable for devices connected with IOT Orion broker and other Broker.
    • While for SigFOX the corresponding service portal provide at the user the name (on the sensor box) and the KEYS, while  the URL of the broker is obviously SigFOX
  • During the registration, the couple Keys, DeviceNameID, and Broker are saved in the MyPersonalData. Thus, this information is read by the IOT Application via HTTPS SSO.
  • Thus, the IOT Applications can access to the Keys, DeviceNameID, and thus may perform:

 
This mean that the Snap4city solution provides:

 

  • Share