TC2.30 - Bulk Load for IoT Devices Registration

Test Case Title

TC2.30 - Bulk Load for IoT Devices Registration

Goal

To permits an user to register a set of IoT Devices via a bulk load (via a configuration file, without human interaction)

Prerequisites

Using a PC or Mobile with a web browser, acceding to the IoT Directory

Expected successful result

The IoT Devices listed in the configuration file has to be registered to the logged user

Steps

 

see the Tutorial on the Bulk Load Approach for IOT Devices on Snap4City

The bulk device load feature allows the user to upload thousands of devices at once. The user should prepare a csv file containing the information of the devices he wishes to add,  and select the context broker to be used for his devices. Further optional parameters can be specified (e.g. the corresponding model, edge gateway type and edge gateway uri).

The workflow of this feature is formed of three main steps: (i) the user uploads a csv file containing the information about the devices; and a validation process is carried out over every device to guarantees its “validity” with respect the adopted Knowledge Base; (ii) the user can check and modify the devices in order to make them valid; (iii) all the valid devices are permanently stored in the Knowledge base, their ownership is reported in the user’s personal data, and the devices are registered in the corresponding context broker (when this is required). A device is valid when its registration passes the constraints imposed for the registration on the Knowledge Base. A validity tag (valid/invalid) along with a validity message (that explains the invalid errors if exist) are added to the properties of each device. Then all the devices, with their added properties, are sent to the server to be saved in a temporary device table. These devices will be made available to the user in a specific interface. He is then able to update the information of every device, modify the details of invalid ones in order to become valid, delete the unwanted devices. Finally, the user can do the final insertion of the valid devices to the main devices (the ones that appear in devices page). The use of a temporary table allows the user to complete the load of devices in different instant of time and preserve their work in different sessions.

In this document we explain how this feature works, we start by describing the interface, the structure of the file we expect the user to upload, and explain the strict rules and the tolerant rules that this file must obey. Afterwards we point out how the validation process works and how the user is informed about the mistakes he has to correct in order to make his devices valid. Also the editing of each device’s properties and attributes is explained. Finally, we show the insertion of the valid devices.

 

1- The interface:

As shown in the Figure 1, the interface is composed by the following features:

  1. The statistic bar: to show how many valid devices and invalid devices the user has in the table (6).
  2. The button to upload a csv file: when clicked a dialogue box will appear allowing the user to select his file. Only csv files are accepted.
  3. The context broker selector: the selected context broker from this drop down menu will be adopted for all the devices of the file.
  4. The model and edge gateway type can be selected from drop down menus, and the edge gateway uri value can be entered by the user. These values will be adopted also for all the devices of the file. They are optional. In case the model is not selected (remain empty), the default “custom” model is selected.
  5. Upload button: when clicked, the verification process is executed over all the devices of the file, and they are sent to the server, then-if successfully sent and received- will be filled-in the table (6).
  6. The table shows the uploaded devices. The data in this table are retrieved from the database on the server each time this page is opened or refreshed. Each row represents a device, more details are shown if the user clicks on the (+) buttons, (Figure 2). The status field can be either “Valid” or “Invalid”. When clicked, a message will appear as alert reporting if the device is valid (Figure 3), or explaining the error otherwise (Figure 4).
  • Insert valid devices button: when clicked the valid devices in the table will be inserted with the official devices (this registration included also the registration in the Knowledge Base, the registration of the ownership and the registration in the context broker). The inserted devices will be deleted from this temporary table (6) and the its corresponding table in the database.

Figure 1: user Interface

 

Figure 2: Device details

 

Figure 3: validity message in case the device is valid, appears when the status field is clicked

 

Figure 4: validity message showing the errors in case the device is invalid, appears when clicked on the status field

 

2- CSV file constraints:

This file should be of “csv” type, and having AT LEAST 19 fields:

"name", "devicetype", "macaddress", "frequency", "kind", "protocol", "format", "producer", "latitude", "longitude", "value_name", "data_type", "value_type", "editable", "value_unit", "healthiness_criteria", "healthiness_value", "k1", "k2".

 

Figure 5: example of csv file

 

The headers (titles of each column) should be written in a way that it can be analyzed to match one of the fields names mentioned above, example:

  • The “devicetype” is detected if the header contains “dev” and “type”
  • The “macaddress” is detected if the header contains “mac”
  • The “frequency” is detected if the header contains “freq”
  • The “protocol” is detected if the header contains “proto”
  • The “value_name” is detected if the header contains “value” and “name”
  • The “healthiness_criteria” is detected if the header contains “health” and “criteria”
  • … and so on

Since a device can contain more that one sensor/actuator, the presence of more than one row with the same device name and context broker is interpreted as the introduction of an another sensor/actuator for the same device.

An example of the csv is provided in attachment (https://www.snap4city.org/drupal/sites/default/files/files/BulkLoadCSVEx...) please rename with csv extension.

 

3-  The validation of each device:

The validation process aims to check if the details of the devices are valid, so it can be inserted into the devices page.

The validation works as follow:

Its properties should be obeying their constraints:

  • The device name is mandatory, and at least 5 characters.
  • The device type, frequency, kind, format, protocol, k1, k2 are mandatory.
  • The mac address format should be Letter (A-F) and number (eg. 3D:F2:C9:A6:B3:4F).
  • The latitude and longitude are valid values

The attributes of a  device should be valid too, every device should have at least one attribute.

And according to the model of this device, the attributes are checked. If it is a “custom” model, the details of each attribute is checked, and should obey the constraints of these details:

  • Value name: mandatory
  • Data type: mandatory, and is one of the values we find in the “data type” drop down menu
  • Value type: mandatory, and is one of the values we find in the “value type” drop down menu
  • Healthiness criteria : mandatory, and is one of the values we find in the “healthiness criteria” drop down menu
  • Editable: true or false
  • Healthiness value: mandatory

In the case where the model is specified, (it is not custom), the number of the attributes of the device should be exactly as the one corresponding to the model, also every detail to every attribute should match its corresponding one for the model, otherwise it is considered as invalid.

 

4- Edit and delete:

Figure 6: Edit device

 

 

Editing the device properties and its attributes is possible by clicking the “edit” field. The window in the Figure 6 appears. After confirming the edit, the verification process of the device is executed again before sending the new data to the server.

Deleting the device is possible by clicking on the “Del” button.

 

5- Inserting the valid devices:

The valid devices in the temporary device table will be included in the user devices when clicking on the button “Insert valid device”. Note that only at this stage the ownership of the device is registered. Moreover, only at this stage, the device is registered in the Knowledge Base and in the corresponding Context Broker (when a registration is required e.g. Orion Context Broker). In this way, we are sure to register only devices that adhere to the constraints imposed by the KB and that the consistency of the IoT directory is preserved. Finally, the inserted devices are removed from the temporary table and removed from the visualization.

In more details:

When the button “insert valid Devices” is clicked. A request will be sent in order to add all the valid devices to this user’s devices.

The addition process of these valid devices is done asynchronously, in the background. Which means that the user can work on other pages, or even closes the website while the addition process is being continued.

The user can check the status of the addition operation by visiting the page, and he can stop this process anytime.

An alert message will appear to notify the user that his valid devices are being added. Beside of waiting the process to finish, or closing the website, this alert provides the user with two more options: to be redirected to other page of the same website by clicking on “Ok”, or to stop the process by clicking on “Stop the Process” (Figure 7).

Figure 7: Alert shown to the user when "insert Valid Devices" is clicked

 

The message in the alert will inform the user about the status of the addition progress (show the percentage of the valid devices that have been processed, Figure 8).

Figure 8: Progress of the valid devices addition

In case where the user chooses “ok”, he will be redirected to the main page. in case he selects “stop the process”, the addition process will be stopped, the page will be refreshed.

If the user returns to this page, while the uploading is still processing this alert with the corresponding message will re-appear.

When the process is over, the user will be notified (Figure 9).

Figure 9: Valid devices addition is finished

 

 

Public File Upload 
Plain text icon BulkLoadCSVExample.csv_.txt