How To Dev - Concept of Time Series


Warning message

  • You can't delete this newsletter because it has not been sent to all its subscribers.
  • You can't delete this newsletter because it has not been sent to all its subscribers.


The Time Series are sequences of data values with an associated time stamp. In Snap4City, any Entity and MyKPI can be used to model a time series. MyKPIs provides only a single variable changing over time, and thus are limited, also since, so far, they are not modeled into the KB. They started as personal information to be put in safe.

In Snap4City, a more diffusely used and powerful solution for Time Series is based on Entity Instances which may have a number of variables, each of which can change over time, thus creating a Time Series. Therefore, an Entity Instance (IoT Device) may send a new measure with its related time stamp (date and time) (Red Lines) in the next figure. This means that regular or unregular time slots can be valid as well.

For each Entity Instance, new measures (Entity Messages) are conventionally time ordered according to variable «dateObserved», with value type = timestamp, which has to be Unique into the Entity Model definition.
If other variables/values need to keep a timestamp, they have to be called in other manner and have to be of kind “DateTime”.

The example of the next image reports a device with two time series, humidity and temperature. The advantage of having the data variables organized as time series is that they can be represented in a multiple and single time trend graph diagrams. The example of this Figure (click to open) reports a device user45driveanalysis with multiple timeseries: location, doctor, status, etc. This Figure (click to open) reports the evolution of an Entity Instance via its Entity Messages.

Figure: Entity Instance (IoT Device) and Time Series from its variables.

Please note that, in all the Snap4City Entity Models (IoT Device Models):

  • Nature and Subnature (classification), Lat/Lon (GPS positioning), and Entity/Device in Mobility (yes or not) attributes do not need to be specified by the model designer since they are defined for default for all the Entity Models (IoT Device Models) and for all the Smart Data Models as well.

On the contrary, if you would create an Entity Model producing Time Series of the Entity Instance (thus accepting Entity Messages over time, you have to:

If the dateObserved, with value type = timestamp, is not specified, the versioning or the state evolution are not time variables, so that the device is substantially static, and its variable cannot be taken as time series.
At each timestamp, a new Entity Message can be posted for a given Entity Instance. Thus, the Message will provide a different value for the dateObserved.
Please note that, values assigned to dateObserved in Entity Messages can be in past for loading historical data, and in the future for loading predictions, future data over the time series.

Moreover, at each new timestamp/Message of a given Entity Instance (IoT Device), the other variables (e.g., V1, V2, V3, etc., of the next figure) can be updated/provided or not. If they are not updated/received, there are two options:
(a) omitting the new value,
(b) putting a null as a value.
The two cases are producing different effects on rendering tables of those data on Dashboards and on time trends (multiseries graphs) as reported in the image below here and explained in the following.  

Figure: tricks and tips on time series.

The counter indication of omitting the variable value can be very relevant.

(a) omitting the new value

If the data values are not provided, the broker is keeping its last value and thus this last value is saved into the platform storage as well (see the figure above here, cases of V1 and V5). Thus, the values shown on graph (in this case in dashed line, while on dashboard will be continuous), is the last value. On the other hand, it can be very old, it come from the last measure, may be of 1 months ago. In some cases, this approach is viable. For the rendering on table (the value is present in the storage), so that the data replicated will be shown and thus the orange cells will not be distinguishable from the others since they have been feed into the storage.
If the above approach is not acceptable, one has to write “null” in variable value thus going for case (b) as follows.

(b) putting a null as a value

Putting in the Entity Message a null value for certain variable will imply to write it into the broker and the broker will not post any value on the storage, leaving a hole, that will be interpolated in the graphs if a new value will come later, otherwise not. The graph can be inspected with mouse to pop up the actual values and where they are, using the dots, where not dots are there means no data. Moreover, the Tables are going to show empty cells where there are the “null”s since the data is missing. The same effect of null in requesting last values may happen in rendering data on barseries, spidernet, single content, donut, etc. It may be corrected to send them values via some Business Logic which could enlarge the request to the Recent Last value, for example with a time interval comparable to the rate of the device or two time of the device rate, etc. This logic has to be implemented by case, by the developer on Client or Server Sides Business Logics.

These approaches are also impacting the queries requesting the last value of an Entity Instance (IoT Device) from the Smart City API and from the broker as well. An alternative approach may be to split the Entity Model in smaller Models having variables which are sampled at the same time, sent to the broker and thus written at the same time, at the same dateObserved. The smaller models can be also referred to an aggregating Model which links them via their ServiceURIs, and vice versa if needed, etc.