1. Home
  2. TC3.3 - DevDash, Development Dashboard, monitoring data ingested in the platform

TC3.3 - DevDash, Development Dashboard, monitoring data ingested in the platform

×

Warning message

You can't delete this newsletter because it has not been sent to all its subscribers.

Test Case Title

TC3.3 - DevDash, Development Dashboard, monitoring data ingested in the platform

Goal
  • Access to city data on their time line, on historical large data storage in fast manner
  • use multiple devices of different kind to access at the dashboards without installation
  • see the real-time data, and data H24/7 with automated update for each widget
  • Monitor the status via different views, graphs and maps
  • Perform drill down on timeline
  • Applying multiple filtering, observing the other views updating
  • Sort data dynamically, by text, by facet (by SOLR), etc.
  • Search by text string, keywords
  • Navigate on the status of a city element with real-time data (such as traffic flow, parking, etc.) to see real-time data and trends.
  • Comparing trends of different kind of data/sensors (traffic flow, environment, pollution, IOT, referral data, etc. etc.) on the basis of time, enabling visual exploration for new relationships

Prerequisites

Using a PC or Mobile with a web browser. All the data arriving in the platform are collected into a noSQL storage and indexed in real-time. Thus, they are accessible in the Developer Dashboard for drill down and browsing according to different aspects.

Expected successful result

See the dashboard and play with them. The user can access to the dashboard and perform a number of action to drill down on data, on time, facet, kind of data, etc. The resulting combination of filter is a new dashboard/view that can be saved locally or on cloud (ProcessLoader, in the future) and shared with other colleagues, also via ProcessLoader.

Steps

 

 

Please note that some of the following links could be accessible only for registered users.

The visual exploration on Developer Dashboard data is possible according to different aspects: geo, text, tabular, relationships. The general idea of Snap4City is to allow performing drill down to arrive at extracting data from different data stores: IOT, mobile, social media, historical, etc. and in any way to identify queries that can be executed automatically by some ETL or DataAnalytics script.

This feature is implemented by starting from the Developer Dashboard, ServiceMap, and developing ETL or Data Analytics processes in R, Java, Python, ETL, .., which in turn, can activate algorithms and tools for machine learning tools, clustering, kriging, etc. According to Snap4City, the scheduling of the Data Analytics can be performed by using DISCES tool, thus making them available in different contexts and scenarios looking for the identification of unexpected correlations as well as anomalies.

 

Example 1: Using the Developer Dashboard for data understating

  1. Access and explore the Developer Dashboard
  • TIMEPICKER (1): from to date/time. The user can identify FROM TO date/time;
  • QUERY (2): text search. The user can pose a query;
  • FILTERING (3): reporting the applied filters. The user can see the applied filters and can remove them returning to a different config, can edit them;
  • HITS (4): reporting some metrics on the selection performed by filtering. The user can ask to have more data;
  • FACET (5): a selection of facet fields, in this case: Classname, MeasureType, SubMeasureType, deviceName, SRC (ETL or IOT), kind (sensor/actuator), unit (of measure). The user can make a selection for each Facet Field, thus filtering and drilling down on kind;
  • HISTOGRAM (6): with a counting of events along the timeline filtered, limited to 100.000;
  • TERMS (7): a pie in which several kinds of distribution can be shown on the basis of the Facet fields;
  • SMARTCITYMAP panel (8): reporting on SmartCity map the measures identified, limited to 10.000;
  • TABLE (9): a table with data coming from the data store SOLR index, with a selection of columns.

 

  1. Open the above link/dashboard on different terminals
  2. See / Set the Automated update activated for this dashboard.
  3. Using a number of panel/widgets for Developer dashboard
    Developer Dashboards is a tool for accessing to data collected in an interactive and fast manner. It is based on SOLR to index the data, to enable the drill down. In addition, the model and tools have been (and are going to be) customized to make possible the browsing via cross links back and forward among data, IOT, etc. and the Knowledge Base tools as ServiceMap. The Developer Dashboard has been developed by Banana and/or HUE.
     

List of Widget for the Developer Dashboard:
 

  • TimeLine: 

  • Valuetrend: 

 SmartCityMap:

 

  • Facet:

 

  • Terms Pie chart: 

 

  • Counting:

 

 

  • Table:

 

Example 2: Using the DeveloperDash-V3 Dashboard 

  1. Access and explore the DeveloperDash-V3 Dashboard
    • The DeveloperDash-V3 Dashboard is an improved version of the previously described Developer Dashboard. It shows real IOT/ETL data, in real-time.
    • Log in the Snap4City portal (https://www.snap4city.org/ ) and click on the “Management” menu item on the left-side bar. A sub-menu opens, in which you can find a list of management tools. Click on the “Data Analyzer: DevDash”:
       

       
    • In order to load the DeveloperDash-V3 Dashboard, click on the load-folder icon on the top-right corner of the screen, and select the DeveloperDash-V3 dashboard from the drop-down menu.
       

       
  • Now you have access to the DeveloperDash-V3 Dashboard, which appears like in the figure below: 

 

This dashboard includes Widgets of the following Kind with their usage (see the next figure):

  • TIMEPICKER (1): from to date/time. The user can identify FROM TO date/time;
  • QUERY (2): text search. The user can pose a query;
  • HITS (3): reporting some metrics on the selection performed by filtering. The user can ask to have more data;
  • FILTERING (4): reporting the applied filters. The user can see the applied filters and can remove them returning to a different config, can edit them;
  • HISTOGRAM (5) showing event counts on time trend;
  • FACET (6): a selection of facet fields, in this case: data_type (the type of data measured by the device, e.g.: string, integer, float, Boolean etc.), value_name, value_str, value_unit (measure unit), SRC (ETL or IOT), kind (sensor/actuator), sensorID, deviceName. The user can make a selection for each Facet Field, thus filtering and drilling down on kind;
  • HISTOGRAM (6): with a counting of events along the timeline filtered, limited to 100.000;
  • TERMS (7), (8), (9): a pie in which several kinds of distribution can be shown on the basis of the Facet fields (in the example the facet field are the deviceName, value_type and data_type);
  • SUNBURST (10), (11), (12), (13): the Sunburst representation is a multi-level ring/pie chart which allow to easily visualize multi-level faceting diagrams, depending on the faceting input order.
  • SMARTCITYMAP panel (14): reporting on SmartCity map the measures identified, limited to 10.000;
  • TABLE (15): a table with data coming from the data store SOLR index, with a selection of columns. 

 

  1. Exploring the panels/widgets in DeveloperDash-V3 dashboard
     
  • TimeLine

 

  • Histrogram, showing events count time-trend, according to the TimeLine panel: 

 

  • Facet: 

 

  • Terms: 

 

  • Sunburst: in this case, the four sunburst panels show the following multi facet views:
    • a) 2-levels faceting data by: value_type and data_type;
    • b) 3-levels faceting data by: deviceName, value_name and value_type;
    • c) 2-levels faceting data by: src and deviceName;
    • d) 3-levels faceting data by: value_type, value_unit and src; 

 

  • SmartCityMap: this widget has been developed specifically for the Snap4City project. The SmartCityMap panel allows the user to geographically filter the visualization of services. To this end, the user can simply adjust with the zoom and the visualization bounds of the map, and finally click on the new Geo-Facet button in order to filter the desired services to be viewed on the SmartCity Map.  

This is not only a visual filtering, but a full facet functionality. Actually, with this operation, all data displayed by the other panels is filtered, displaying data related to the services which are geolocated within the chosen geographical selection. The next figures illustrate this simple process: 

 

As a first step, the user visualizes the services of a certain geographical area. When the user has chosen some services of interest and want to filter all the other panels data to drill down and visualize only the data related to the services of interest, he has simply to adjust the zoom and the position of the map to include only the desired services. Then, by simply clicking on the Geo-Facet button, all the panels is filtered accordingly. In the example, we have chosen the Weather sensor represented by the green icon on the previous figure. The result of the geo-faceting is shown in next figure: 

 

As a result of the geo-faceting, we can see that a geographic filter has been added also in the filtering panel:

 

 

  • Share