TC8.6 - Snap4City Living Lab Life Cycle

Test Case Title

TC8.6 - Snap4City Living Lab Life Cycle

Goal

To provide support for the Living Lab in terms of life cycle.

The living lab is and environment in which stakeholdes can create their own applications, and share them with others.

Prerequisites

Knowing the goals of the Select4Cities.

The following functionalities are available only for specific Snap4city users with specific privileges.

Expected successful result

Understand the mechanisms to set up and start the snap4city platform in third phase.

Steps

 

 

In order to create an efficient Living Lab, the following key principles have to be taken into account: i) Value (create value for users and customers as a key aspect for business success: involve SMEs, mitigate the competition, open new markets); ii)  Influence (users seen as active, competent partners and domain experts, concretize new services coming from the citizens ideas); iii) Sustainability (e.g., choose right materials, implementing user-friendly approaches, considering the social and economic impact of the innovation), iv) Openess (open collaboration between people with different expertise and backgrounds. Different perspectives could lead to a successful innovation process), v) Realism (the innovation actions are carried out in the real-life, realistic and natural settings, this to increase understanding on how innovation can bring advantages valid in the real market) [8]. Another fundamental aspect is also connected to territorial needs, this is the reason why sometimes it is stated about the Urban Living Labs, that “are environments in which innovation is spatialized, i.e., it is generated within a specific spatial environment. These are environments in which the openness of innovation manages to transcend the organizational infrastructures that are traditionally operating in the city and to invent new institutional figures for, or ways of, dialoguing between citizens and institutions” [29], [30], [31].

In the context of Smart City, Living Labs have to provide a set of key features for an infrastructure capable to manage all the complex aspects identified. A Smart City Living Lab infrastructure have to be: i) able to activate practice-based knowledge production in collective and private environments (e.g., taking into account both Private and Open Data and contexts); ii) able to learn internally but also externally, by experimenting with other cities that share the same problems; iii) aware of new civic engagement models being experimented all over the world; iv) aware of the growing demand for new citizenship models; v) capable of directing investments toward opportunity creation (i.e., experiments) rather than pre-developed solutions, [32]; vi) stimulating the participation of all stakeholders in the activities of data collection and process/solution production.  Those types of infrastructures manage a massive set of data (Big Data context) that can be shared, processed, used to generate new knowledge; moreover, are capable to ingest both Open and Personal/Private Data (e.g. the user’s profiles and actions done in the city). All the processes realized by this kind of infrastructures imply the necessity to pose attention on a relevant aspect: the treatment of the privacy rights on data (especially the sensitive ones), underlined also by the General Data Protection Regulation (GDPR), [33], [34], [35], [36].  

Snap4City solution is a Living Lab Smart City infrastructure, responding to the key features/requirements enabling the Living Lab paradigm especially in the acquisition of City Data and developing smart city solutions. 

Architecture 

Smart Cities need to set up a flexible Living Lab to cope with the city evolution in terms of services, city users’ needs and capabilities. To this end, the Snap4City solution provides a set of tools and a flexible method and solution to quickly create a large range of smart city applications exploiting heterogeneous data and stakeholder services also enabled by Internet of Thing (IoT)/Internet of Everting (IoE) technologies and Big Data analytic. The Snap4City solution and all the innovation activities carried out for its development, have been realized involving different kind of Organizations (Universities, SME and Large Industries, Public Administrations) and users (City Operators,  Resource Operators, Inhouse companies, Tech providers, Category Associations, Corporations, Research groups, Strat-ups, Early Adopters, large industries, advertisers, City users, Community builders, etc.), thus reflecting the features, described in Quadruple Helix (QH), [7], to facilitate the Living Lab approach in a Smart City, Fig. 1. The innovative aspects of the solution proposed are related to semantic computing of entities for discovering and search information, resources management, parallel and distributed computing and cloud management, applications based on microservices and external services, dashboard and development tool kits, etc. Snap4City flexible enough to support extensions at distinct levels of granularity: data, analytics, tools and applications.

snap4city_flow
Snap4City Life Cycle.

 

One of the first activities for creating a Living Lab in a city is the process of setting up the technical infrastructure which in turn is grounded on many valuable enabling tools. They must support the city in: the modeling of data; the upload of context data and open data; the connection of IoT/IoE sources and external services; the creation of Extract Transform and Load (ETL) processes and data analytics algorithms, to arrive at producing smart city dashboards and at starting the production of Snap4City Applications based on Microservices. All these phases must be accompanied and supported by the availability of a set of development tools, easy to use, accessible and open. To this aim, the Snap4City solution has been designed to create a collaborative environment in which different kinds of stakeholders can mutually collaborate. At the same time in which the setup is created, the collaboration among stakeholders can start by creating: agreements, collaborations, networking, producing tutorials, workshops, hackathon, etc. Fig. 1, so at to arrive at involving the stakeholder around case studies, and finally to sign contracts of partnership, licensing, etc. Thus, the delivering of specific solutions to city users, operators, etc., is becoming possible. This process must be driven by the municipality and, on the other hand, the municipality needs support for technical aspects if it is not very large and technological oriented. Typically, the single companies even if participated by the city or the city operators, do not have the view and the mission to put in common a so large multi-domain multiservice framework and environment.

From the technological point of view, the above process is released by means of a set of tools, to provide a support for collaboration and sharing at distinct levels. We propose the Snap4City architecture (see Fig. 2) as capable to solve all the problems described above and which consists in:

  • A layer to ingest all the different kind of data coming from a smart city that can be classified in:  Open Data, Personal Data, IoT and IoE, Social Media, static and real time. The set of data regard many distinct categories such as: transport systems, Mobility, Car park; Public Services, Security, Museums; Sensors, Cameras, IoT, IoE; events; Environment, Water, Energy; Shops, Services, operators; Social Media, Wi-Fi, Networks, etc. The flow of data coming from the city data can be: static, slow, real time.

Architettura
Snap4City Architecture.

 

  • Collecting Data tools. These tools are both for developers (ETL processes) and for users with no technical skills, and were developed to exploit the possibility of bringing together people with different skills, experiences, roles, expertise, motivations, etc. and many kinds of organizations such as universities, public administrations, SMEs, stakeholders, industries, etc.
    • ETL processes, for developers and based on the Pentaho Kettle Tool, [38]. The Extract Transform and Load (ETL) processes can be personalized to manage many different kinds of data: open and personal, static and real time or periodic, geo-localized. The ETLs consider that data can come from: any kind of sources and providers, IoT/IoE or Sensor Networks, city users’ devices, social media, open street map and in different protocols and formats. Their final aim is transform data so that it conforms to km4city multi-ontology and load them in the Sna4City Knowledge Base (in the form of RDF Store for e the static data, and in an HBase -NoSQL- store for the real time data, [39]).
    • ProcessLoader/ResourceManager and DISCES Scheduler have been developed to manage processes to be executed in specific scheduling applications (for example ETL processes analyzing real time or periodic data which need to be launched every day/Hour/minute, as well as  Data Analytics processes in R Studio, Java, Python, etc.).
    • DataGate: is a web-based open source management system for the storage, distribution, qualification, reconciliation and aggregation of Open Data. It is an extension of the very diffused Open Data platform CKAN (Comprehensive Knowledge Archive Network), [12].
  • Services for:
    • executing data analytics and computations that can exploit data to provide advanced smart services one demand, early warning, both periodically and in real time modality.
    • creating applications that can be: data driven and/or periodic, based on Micro Services. These Services can be applications running on the platform itself (for example by using NodeRED or Pentaho [40], [38]), such as Dashboards and Mobile or Web applications.
  • Data storage layer, collecting data in a Knowledge Base (KB) connected to the Km4City Multi-Ontology and making data indexing to prepare services on the data themselves, such as: data retrieval with the capability of inference and reasoning, search and retrieval, etc. [25]. While the real time Data are collected in a NoSQL database (HBase, [39]).
  • Advanced Smart City API, capable to provide access to Snap4City data and services. The APIs can be exploited by web and mobile applications, as well as by many tools and cities, [11], [41].
  • A set of Tools for all the Living Lab Actors, useful to test the effectiveness utility of the Snap4City solution in different contexts:
    • Tools for Living Lab developers, such as the web applications Service Map and Linked Open Graph for navigating on data results considering both the geographical metadata and the semantics aspects [42].
    • Tools for Living Lab testers and final users (community builders, city users, advertisers, Category Associations, etc. Fig. 1) showing and via dashboards and in turn make easy the production of specific dashboards for decision makers, city operators, etc. [41], [42]
      In the following paragraphs, Datagate and ProcessLoader, as tools relevant to enable the Living Lab activities in a Smart City, is described.
       

Datagate

Datagate (https://datagate.snap4city.org/ , Fig. 3) has been designed with the aim to offer a collaborative environment enabling the data providers to archive, manage, share datasets. It primarily manages static (or periodic) data. Moreover, it allows the data providers to upload their content, with the following features and advantages, which go behind the simple i) data storage on the portal:  ii) data enrichment and geo-localization (with the city street graph or directly with Open Street Map); iii) data aggregation, with all the other data contained in the Datagate portal and with those archived in the Snap4City Knowledge Base, thanks to the semantic aggregation made through the KM4City multi-ontology; iv)  data sharing (with the IPR license chosen); v) data visualization using different tools (Service Map, Dashboards, web and Mobile Apps, etc., Fig. 2), Fig. 3. These advantages can be attained in an automatic or semi-automatic modality and are described here after.

 


DataGate: Organizations. 

 

Data storage and sharing. These features are directly derived from the CKAN standard Open Data portal. Each data provider can be registered on the web portal and upload its datasets. A set of instruments to visualize the data are offered by CKAN (views, statistics on downloads, etc.), [12]. It is what can be viewed in Fig. 3: a set of datasets have been uploaded by their providers and then published on the web and visible to all the public Datagate users.

Data enrichment and geo-localization. This, and the following features have been added by the ‘DataEnhancer’ plugin developed in the Snap4City context. If a dataset is uploaded following the specific template, the ‘DataEnhancer’ features is available on it, in addition to all default services offered by CKAN. The template is written in the form of a csv file and provides a set of fields (both mandatory and optional) to be filled to have the most possible advantages: i) mandatory fields: data name, geometry following the Well-known text (WKT) geometric objects: points (POINT), lines (LINESTRING) and areas (POLYGON) or address (city, street, civic number, etc.); ii) optional fields (description, web page, phone number, links, e-mail, etc.). Some fields are automatically enriched by Datagate (e.g. it can automatically calculate: postal code from address, latitude and longitude from address with civic number, moreover it reports incorrectly formed fields such as: e-mail, web portals, links, etc.), Fig. 4.  

 

FromD_toKB
Datagate: autocompletion and error reporting. From Datagate to ServiceMap.

 

Data aggregation and visualization. Once the datasets are uploaded in the Datagate Portal, their responsible can connect the data to the KM4City multi-ontology by selecting one of the categories and subcategories present in the ontology and accessible from Datagate thanks to the possibility of selecting ‘KM4City Categories’ and the ‘KM4City Sub-Categories’ from a drop-down menu, Fig. 4. Click the publish on Snap4City button and upload the data also in the Snap4City Knowledge Base. In this way the data is visible

 

Process Loader/Resource Manager

Algorithm/Process Loader is a web application, developed for allowing the creation and management of processes to be executed in specific scheduling applications with a user interface that receives input data in the form of compressed files that are analyzed, archived and finally transmitted to the desired scheduler. The main application’s activities are focused on uploading many compressed zip archives containing files and directories required to create and execute a process on an external scheduling application through a series of API requests sent to the application. The processes cab be realized by developers coming from different context and smart cities and realized using different kind of technologies (ETL processes, R Studio, Dashboards, nodeRED, R, etc.).

The user interface provides the following services:

  • Ingestion of processes (in the form of compressed files) by authorized users. Each process is analyzed, archived and finally transmitted to the desired scheduler and properly launched.
  • Process Execution:  once the authorized users have uploaded their processes, they can launch and execute them, thanks to the presence of a scheduler, Fig. 5. The Process Loader users have only to insert some mandatory metadata such as the frequency with which a process has to be launched and other parameters that can depend on each process features (e.g. the web server from which the data are taken, the smart city related to the data, etc.).  For example, an ETL process that links to an Open Data portal providing busses’ timetables in a certain city, updated once a month. The user can create an ETL capable of collecting data and transform them as he or she sees fit (for example, to insert them in the Knowledge Base of Snap4city in order to take advantage of all the services offered by this solution). Then he/she can upload the ETL process in the process Loader, set the ETL to run periodically (e.g. once a month). In this way he/she can always have (for example on one of the services offered by Snap4City such as ServiceMap) the updated data as results without having to do any further work.

Archiving and indexing of all the processes, that can be shared and easily retrieved, downloaded, re-used (basing on the license associated to each of them), ranked. All the metadata related to the processes are indexed via Apache Solr [44]. Each process publisher can make public its processes so that they can be shared. A web page (Fig. 6) offers the list of the public processes and an easy form to search the ETLs basing on the metadata associated to them.  


CKAN_ETL_sce
Process Loader/ResourceManager: processes running in the DISCES scheduler

 


Process Loader/ResourceManager faceted search.
 

Conclusions

Methodologies and tools for Smart City Living Lab start up, management and life cycling are becoming relevant. In this document, the solution developed for Snap4City project has been described. Snap4City has been developed on the basis of Km4City ontology and tools with the aim of adding solution for supporting Living Lab in response to the competitive call of Select4Cities European commission project directed by three major cities in Europe: Helsinki, Antwerp and Copenhagen. The proposed Snap4City solution included: (i) development model suitable for IOT/IOE applications; (ii) life cycle presented in Figure 1; (ii) a set of tools for data collection, data sharing, processes and analytics development, collection and management, and sharing as presented in this paper as DataGate and ProcessLoader. The paper also reported the experience of using these tools.  As a conclusion, in order to create an efficient Living Lab, in the following table a summary of the relations among the key principles described in [10] and the Snap4City services and tool are presented.

Key principles for a Living Lab

Snap4City solution

Value

The data managed, and the services available in the Snap4City solution, come from many different providers: municipalities, SMEs, research centers, etc. enabling the connection among users and stakeholders considering the business aspects as playing a fundamental role for the solution success.

Influence

Many of the services proposed put the users as key actors to: give suggestions, upload data in the system, share data and opinions, etc. Moreover thanks to the Datagate and to the Process Loader tools, it is possible to: i) see the activities realized in other city; ii) to apply a set of a results in a city a starting point to realize the same services in other smart cities (e.g. use data coming from a city as a proof of concept in another one); iii) take inspiration from has already be done, to concretize new services directly coming from the citizens ideas. 

Sustainability

Snap4City has been produced as an Open Architecture (also open source) that is applied in some Italian (e.g., Tuscany, Sardinia, Veneto, Emilia Romagna) and European Region and smart cities (e.g., Helsinki, Antwerp, Copenhagen). It can be reused without additional costs in every smart city context, thanks to its versatility and openess.

Openness

Thanks to several types of services offered, the work done on the Snap4City solution has involved, and continues to involve, many people with different skills and who play a different role in society enabling the interaction among different perspectives and needs.

Realism

All the data managed, and the services offered (previsions, dashboards, etc.) are made in a real context and are directly used by citizens, stakeholders, public administrations. This increase the possibility to study both innovations or advantages than cab be useful in the real market.

Tab. 1. Key principles for a Living Lab & Snap4City solution.

 

References

  1. N. Villanueva-Rosales, L. Garnica-Chavira, V. M. Larios, L. Gómez and E. Aceves, "Semantic-enhanced living labs for better interoperability of smart cities solutions," 2016 IEEE International Smart Cities Conference (ISC2), Trento, 2016, pp. 1-2. doi: 10.1109/ISC2.2016.7580775
  2. Coenen, Tanguy & van der Graaf, Shenja & Walravens, Nils. (2014). Firing Up the City – A Smart City Living Lab Methodology. Interdisciplinary studies journal. Vol.3. January 2014.
  3. Ellie Cosgrave, Kate Arbuthnot, Theo Tryfonas. Living Labs, Innovation Districts and Information Marketplaces: A Systems Approach for Smart Cities. Procedia Computer Science, Volume 16, 2013, Pages 668-677.
  4. Majeed A., Bhana R., Haq A.U., Shah H., Williams ML., Till A. (2017) Living Labs (LILA): An Innovative Paradigm for Community Development - Project of “XploR” Cane for the Blind. In: Benlamri R., Sparer M. (eds) Leadership, Innovation and Entrepreneurship as Driving Forces of the Global Economy. Springer Proceedings in Business and Economics. Springer, Cham.
  5. Concilio G. (2016) Urban Living Labs: Opportunities in and for Planning. In: Concilio G., Rizzo F. (eds) Human Smart Cities. Urban and Landscape Perspectives. Springer, Cham. DOI https://doi.org/10.1007/978-3-319-33024-2_2.
  6. Maya Alba, Manuel Avalos, Carlos Guzmán, Victor M. Larios. Synergy Between Smart Cities’ Hackathons and Living Labs as a Vehicle for Accelerating Tangible Innovations on Cities. 2016 IEEE International Smart Cities Conference (ISC2). 12-15 Spet. 2016.
  7. Arnkil, R., Järvensivu, A., Koski, P., & Piirainen, T. (2010). Exploring the Quadruple Helix. Report of Quadruple Helix Research for the CLIQ Project. Tampere.
  8. Anna Ståhlbröst and Marita Holst, Social Informatics at Luleå University of Technology and CDT – Centre for Distance-spanning Technology, Sweden. The Living Lab Methodology Hand book.
  9. Paskaleva, Krassimira & Cooper, Ian & Linde, Per & Peterson, Bo & Gotz, Christina. (2015). Stakeholder Engagement in the Smart City: Making Living Labs Work. 115-145. 10.1007/978-3-319-03167-5.
  10. EnoLL: http://www.openlivinglabs.eu/node/1429
  11. C.Badii, P. Bellini, D. Cenni, A. Difino, P. Nesi, M. Paolucci. Analysis and assessment of a knowledge based smart city architecture providing service APIs. Future Generation Computer Systems 75 (2017) 14–29.
  12. CKAN: http://ckan.org.
  13. OpenDataSoft: https://www.opendatasoft.com
  14. ArcGIS OpenData: http://opendata.arcgis.com
  15. 5 Stars Open Data from Tim Barneers Lee. http://www.slideshare.net/TheODINC/tim-bernerslees-5star-open-data-scheme
  16. RDF https://www.w3.org/RDF/
  17. SPARQL: https://www.w3.org/TR/rdf-sparql-query
  18. P. Bellini, P. Nesi, A. Venturi, Linked Open Graph: browsing multiple SPARQL entry points to build your own LOD views, Int. J. Visual Lang. Comput. (2014) http://dx.doi.org/10.1016/j.jvlc.2014.10.003, http://log.disit.org
  19. C. Badii, P. Bellini, D. Cenni, G. Martelli, P. Nesi, M. Paolucci, Km4City Smart City API: an integrated support for mobility services, in: (SMARTCOMP) IEEE International Conference on Smart Computing, IEEE, 2016.
  20. DATEXII: http://www.datex2.eu/sites/www.datex2.eu/files/Datex_Brochure_2011.pdf.
  21. IETF: https://www.ietf.org.
  22. F.J. Lin, Y. Ren, E. Cerritos, A feasibility study on developing IoT/M2M applications over ETSI M2M architecture, in: 2013 International Conference on Parallel and Distributed Systems, ICPADS, IEEE, 2013.
  23. J. Swetina, et al., Toward a standardized common M2M service layer platform: Introduction to oneM2M, IEEE Wirel. Commun. 21 (3) (2014) 20–26.
  24. Green Button Connect: http://www.greenbuttonconnect.com
  25. P. Bellini, M. Benigni, R. Billero, P. Nesi and N. Rauch, "Km4City Ontology Building vs Data Harvesting and Cleaning for Smart-city Services", International Journal of Visual Language and Computing, Elsevier, 2014, http://dx.doi.org/10.1016/j.jvlc.2014.10.023,  http://www.sciencedirect.com/science
  26. N. Korn, C. Oppenheim, Licensing Open Data: A Practical Guide. In: Discovery [online]. June 2011 [cit. 2012-02-20]. Retrieved from http://discovery.ac.uk/files/pdf/Licensing_Open_Data_A_Practical_Guide.pdf
  27. S. Villata, N. Delaforge, F. Gandon, A. Gyrard, An Access Control Model for Linked Data, in: OTM Workshops, in: LNCS, vol. 7046, Springer, Heraklion, Greece, 2011, pp. 454–463. Oct.
  28. P. Bellini, L. Bertocci, F. Betti, P. Nesi, Rights enforcement and licensing understanding for RDF stores aggregating open and private data sets, in: second IEEE International Smart Cities Conference, ISC2 2016, Trento, Italy, SLIDES, 12 to 15 September 2016. http://events.unitn.it/en/isc2-2016.
  29. Luciano De Bonis, Grazia Concilio, Eugenio Leanza, Jesse Marsh, Ferdinando Trapani. Co-Creative,     Re-Generative      Smart        Cities. Smart         Cities      and Planning        in a Living             Lab Perspective. TeMA, Journal of Land Use, Mobility and Environment. 2014
  30. Luciano De Bonis and Ferdinando Trapani.  “For a “Living (Lab)” Approach to Smart Cities”, Smart Cities Atlas- Western and Eastern Intelligent Communities. Pp 143-158- November 2016.
  31. Bastiaan Baccarne, Dimitri Schuurman, Peter Mechant, Lieven De Marez . The role of Urban Living Labs in a Smart City -  XX.V ISPIM Conference – Innovation for Sustainable Economy & Society, Dublin, Ireland
  32. Grazia Concilio. Urban Living Labs: Opportunities in and for Planning. Human Smart Cities, Rethinking the Interplay between Design and Planning - Springer International Publishing Switzerland 2016
  33. P. Bellini, I. Bruno, P. Nesi, N. Paolucci, "IPR centered Institutional Services and Tools for Content and Metadata Management", International Journal on Software Engineering and Knowledge Engineering, World Scientific Publishing Company, Volume 25, Issue 08, October 2015.
  34. GDPR, General Data Protection Regulation,  https://www.eugdpr.org
  35. Melbourne Networked Society Institute. Cities as Living Labs Creating Innovative, Connected Cities -  Discussion Paper 01/2015. http://networkedsociety.unimelb.edu.au/__data/assets/pdf_file/0007/1663756/MNSI-D01-15-Cities-as-Living-Labs.pdf
  36. Sinta Dewi Rosadi; Suhardi, Samuel Andi Krystian. Privacy Challenges in The Application of Smart City in indonesia. 2017 International Conference on Technology Sistems and Innovation. Bandung, October 23-24, 2017
  37. Nesti G. (2017) Living Labs: A New Tool for Co-production? In: Bisello A., Vettorato D., Stephens R., Elisei P. (eds) Smart and Sustainable Planning for Cities and Regions. SSPCR 2015. Green Energy and Technology. Springer, Cham.  DOI https://doi.org/10.1007/978-3-319-44899-2_16.
  38. Pentaho Kettle tools, http://www.pentaho.com/
  39. HBase, Apache HBase, https://hbase.apache.org/
  40. Nodered, IoT programming tools: https://nodered.org
  41. C. Badii, P. Bellini, D. Cenni, A. Difino, P. Nesi, M. Paolucci, Analysis and Assessment of a Knowledge Based Smart City Architecture Providing Service APIs, Future Generation Computer Systems, Elsevier, 2017, http://dx.doi.org/10.1016/j.future.2017.05.001
  42. C. Garau, P. Zamperlin, M. Azzari, P. Nesi, G. Balletto, M. Paolucci, THE ROLE OF KM4CITY DASHBOARD IN URBAN POLICIES: GOVERNANCE STRATEGIES FOR DYNAMIC URBAN SYSTEMS from 2nd International Conference on Smart and Sustainable Planning for Cities and Regions 2017, Bolzano/Bozen (Italy), 22-24 March 2017.
  43. C. Badii, P. Bellini, D. Cenni, G. Martelli, P. Nesi, M. Paolucci, "Km4City Smart City API: an integrated support for mobility services", 2nd IEEE International Conference on Smart Computing (SMARTCOMP 2016), St. Louis, Missouri, USA, 18-20 May 2016.
  44. Apache Solr, http://lucene.apache.org/solr/