1. Home
  2. Data Analytics developed by using Snap4City solutions and tools

Data Analytics developed by using Snap4City solutions and tools

×

Warning message

You can't delete this newsletter because it has not been sent to all its subscribers.

  

CLICK on the above thumbnail to get a booklet on data analytics of Snap4City, Snap4Industry cases

For the development of data analytics, the data scientist and developers can use Python and/or RStudio from the online platform and on their premises. Python and RStudio platforms may exploit any kind of libraries such as Keras, Pandas, and hardware accelerator as NVIDIA to use Tensor Flow, etc. The developers can access the KB and Big Data store respecting the privacy and the data licensing by using authenticated Smart City APIs. The access has to permit the reading of historical and real-time data, and saving the resulting data provided by the algorithms, for example, heatmap-related predictions, the assessment of data quality, and labels of detected anomalies. Data scientist’ work may be finished once they develop the algorithm they should be aware of. On the other hand, the same algorithm (e.g., for computing heatmaps, parking prediction), should allow being:

•          Used on different services of the same kind located in different places and based on several parameters (e.g., target precision and list of data sources). This means that data analytics itself has to be designed with the needed flexibility and generality;

•          put in execution from IoT App by passing a set of parameters and collecting the results on the Data Storage or as a result of the invocation. The executions can be periodic or event-driven — e.g., the arrival of a request or the arrival of the new set of data values;

•          controlled for collecting eventual errors and mistakes, in debug and at run time for logging. This may be for informing the developer and/or the administrator of eventual mistakes and problems by sending notifications; and

•          dynamically allocated on the cloud in one or multiple instances to plan a massive computation of the same data analytic process on several data sets and services at the same time.

In Python and/or RStudio cases, the script code has to include a library for creating a REST Call, namely: Plumber for RStudio and Flask for Python. In this manner, each process presents a specific API, which is accessible from an IoT Application as a MicroService, that is, a node of the above-mentioned Node-RED visual programming tool for data flow. Data scientists can develop and debug/test the data analytic processes on the Snap4City cloud environment since it is the best way to access at the Smart City API with the needed permissions. The source code can be shared among developers with the tool “Resource Manager”, which also allows the developers to perform queries and retrieve source code made available by other developers.

Read more on: https://www.snap4city.org/download/video/course/da/

Snap4City Analytics, the data analytics at your disposal on Snap4City

The data analytics in Snap4City are focussed on providing support for Decision Makers to improve quality of life, match the Sustainable Development Goals, specific KPIs; and assess the conditions for 15 Min City Indexes.

The following examples and those reported in the training course can give you an idea of the capability of the platform. We recommend that you browse the training course: https://www.snap4city.org/download/video/course/da/

A selection of our Data Analytics are:

  • Mobility and Transport
    • What if analysis: routing, traffic flow, demand vs offer, pollutant, etc. (Simulation + ML)
    • Traffic flow reconstruction from sensors and other sources (simulation + ML)
    • Predictions for: traffic flow, smart parking, smart bike sharing, people flows, etc. (ML, DL)
    • Public Transportation: Ingestion and modelling of GTFS and Transmodel
      • Analysis of transport/mobility supply and demand mobility based on public transportation and multiple data sources (Simulation)
      • Assessing quality of public transportation (analysis)
    • Accidents heatmaps, anomaly detection (analysis, ML)
    • Tracking fleets, and people, via devices: OBU, OBD2, mobile apps, etc.
    • Routing and multimodal routing (multi-stop travel planning), constrained routing, dynamic routing
    • Computing Origin-Destination Matrices from different kinds of data (analysis)
    • Computing typical trajectories based on tracks (analysis, ML)
    • Computing Messages for Connected drive
    • Slow and Fast Mobility 15 Minute City Indexes (analysis, ML)
    • Computing and comparing traffic flow on devices and at the city border (analysis)
    • Typical time trends for traffic flow and IoT Time series. (analysis, ML)
    • Impact of COVID-19 on mobility and transport
  • City Users and Social
    • People detection and classification: persona, carts, bikes, etc. (ML, DL)
    • People counting and tracking (via thermal cameras, ML, DL)
    • People counting via head counting (via thermal cameras, ML, DL)
    • People flows prediction and reconstruction, (ML, DL)
    • Computing User engagement and suggestions for sustainable mobility (Rule Based, ML)
    • User’s behaviour analysis,
      • Origin-Destination matrices, hot places, schedule, Recency and frequency, permanence, typical trajectory, etc.
      • People flow analysis from PAX Counters and heterogenous data sources
    • Social media analysis on a specific channel, specific keywords: see Twitter Vigilance,
      • Reputation, service assessment: MultiLingual NLP and Sentiment Analysis, SA
      • Tweet proneness, retweet-ability of tweets, impact guessing
      • Audience predictions on TV channels and physical events, locations
    • SDG, 15 Minute City Index , etc. (modelling and computability)
  • Environment and Weather
    • Predictions of pollution as: NOX based on traffic flow, PM10, etc., for the next 48 hours, or longer term.
    • Long-term predictions of European Commission KPIs on
      • NO2 average value over the year, PM10, …….
    • Prediction of landslides, 24 hours in advance
    • Computation of CO2 based on traffic flows
      • each road for each time slot of the day
    • Prediction of MicroClimate conditions for the diffusion of
      • NO2, PM10, PM2.5, etc.
    • Heatmaps production, dense data interpolation for
      • Weather conditions: temperature, humidity, wind, DEW
      • Pollutants and Aerosol: NO, NO2, CO2, PM10, PM2.5, etc.
    • Impact of COVID-19 on Environmental aspects
  • Management and Strategies
    • What-if analysis, dynamic routing, origin-destination matrices production from a large range of sources
    • Early warning computation
    • Estimation of KPI and local indexes for: quality of life (15MinCityIndex)
    • Production Optimization
    • Planning and Monitoring renovation works via objective KPIs
    • Managing Maintenance and teams
    • Predictive Maintenance and costs predictions: chemical plant, vehicles, boats
  • Resilience and Risks Analysis
    • Resilience analysis according to European Guidelines on Resilience of critical infrastructure, and transport systems
    • Risk analysis: natural and nonnatural disaster
  • Time Series
    • Time Series Anomaly detection
    • Data quality assessment and control
    • short and long-term prediction
    • Interpolation of Data on the regular grid for calibrated heatmaps
  • Semantic Reasoning
    • Ontology Modelling and integration, expert system construction
    • Knowledge modelling and reasoning on RDF stores: spatial, temporal, relational
    • Virtual Assistant construction
  • Matrices, Images, Maps, and 3D Digital Models
    • Conversion of Satellite data images into regular ground images
    • Extraction information from Orthomaps, LIDAR, etc., regarding city structures
    • 3D Digital Twin of Cities and Objects: pattern extraction, 3D model reconstruction
  • Etc.

They are developed by using a large range of statistics, operatig research, ML, AI, XAI techniques such as:

  • RF, XGBoost, BRNN, RNN, SVR, MLP,
  • DNN, LSTM, CNN-LSTM, Autoencoders, …, YOLO, etc.
  • Clustering: K-means, K-Medoid, etc.,
  • Simulated Annealing, Genetic Search, Taboo Search, etc.
  • XAI: Shap, variations

------------------------------

  1. First step to reduce the fossil emission is to monitor the present emission, identify the most critical points and factors. The emissions from fossil combustion are mainly CO2 and NO1, NO2. And they are due to vehicles, house heating and industries… but where they are now precisely in the city and lands
  1. As first step the present traffic and housing conditions have to be assed, then the progresses can be kept under control. The assessment can be performed by computing indicators. For example the 15 Min City index for serviceability at 15 minute distance by walking in the city and with different parameters in peripheral and rural areas, which includes an index for housing and slow and fast mobility. Relevant aspects are: (i) to assess the capability of  the public transport to satisfy the demand, (ii) predicting traffic flow and reconstruction, (iii) making the what if analysis to be capable to assess how the traffic conditions would change due to changes in the road network, for example for restructuring, etc.

--- former version---------

This page reports what Snap4City has done for Helsinki and Antwerp in terms of Data Analytic.

List of all scenarious: https://www.snap4city.org/4

Please note that Snap4City has also other capabilities and tools on this matter so that see also:

 

All the algorithms have worked continuously providing services to the users and also information to US: What they requested, What they visited, POI; where they passes: trajectories and OD matrix, Language, Routing, Clicking on services and icons, Engaging: responding, Searching.

 

Description of the main categories of Data Analysis provided.

  • Share