Installing Snap4City: Appliances, Virtual Machines/Containers ready to use on your premise for Smart City and IOT Applications

(10 September 2020) Snap4City DataCity-Small (aligned to version 1.5 of VM plus some new features) has been released as DOCKER Compose, see in the following.

(15 August 2020) Snap4CityMAIN version 1.5 has been released with included: synoptics, more widgets, real time buttons, IOT App updated, plus other VMs have been updated for composing a number of configurations: IOTOBSF,  IOTDSES, KBSSM. See in the following.

Snap4City platform can be freely downloaded from this portal, and thus you can produce a number of services with these tools in the view of Smart City IOT as a Service, SCIaaS. Therefore, if you are interested in using Snap4City.org for trial from this portal you do not need to perform any installation on your computers.

On the contrary, this page is dedicated to who would like to perform a local installation on premise or cloud. In fact, in this page you can find the instructions for the installation and a number of Snap4City configurations. Snap4City is 100% open source, license free solution, and the Appliances that you can download from this page are license free. You can download and install Snap4City tools on premise or on cloud as you like, by starting from the following Appliances and/or from the GITHUB/DISIT sources (Source Code and licenses), and docker compose provided. In the following figures, the deploy of Snap4City solution from small business to large scalable cases are presented. The configurations are listed and coded with a letter to simplify the identification of the configuration. While other mixed solutions and intermedium configurations are also possible. In addition, some of the tools we are mentioning/providing in the following configurations can be substituted with your legacy tools. For example your CKAN, your GIS, ArcGIS server, etc., etc. For shortening your deploy and/or installation activities we have created a number of Virtual Machines, VM, Appliances and Docker containers/compose. The VMs can be put in execution on VMplayer or VMware workstation or Vcenter Vsphere VMware infrastructure or you can convert them for any kind of Cloud infrastructures, while Containers on Docker. The configurations are presented below from the smaller and simpler solutions to the most capable and wide deploy for large scale smart city with living lab support. For each of them an incremental description and user manual for installation. The same tools can also be downloaded from GITHUB/DISIT.

How to proceed:
Revise the 7 Configurations models proposed from A:Alone to F:FullPlatform+LivingLab, but they can be customized as you like.
From this page you are guided to identify the most suitable configuration according to your needs;
Once the most suitable Configuration has been identified, you can:
i.Download and deploy the Appliances provided as Virtual Machines or Containers for the desired configuration, or
ii.Download and install modules from GITHUB according to the recipes for  VM/Container and the user manual of the single tools, or
iii.Mixt of the above (i) and (ii) approaches.
IOT Edge can be created by (1) installing Node-RED, (2) adding Snap4City library of Nodes from the Palette Manager of Node-RED, (3) registering device or messages on Snap4City (cloud or your local installation): https://nodered.org/docs/user-guide/editor/palette/manager

Configuration Kind

Min #VM / Docker

IOT Broker int (any# and kind)

IOT Broker Ext (any# and kind)

MAP WMS web

KBSM semantic engine reasoner

IOT APP, MicroServices #Containers

Storage & IOT Data Shadow

SSO, Roles LDAP

IOT Security full stack end2end

Dash + Wizard, widgets, Data Inspector

GIS input WFS WMS

GIS output WFS WMS

Heatmaps WMS

Mob. App & Micro Apps.

ASCAPI Smart City API, Federated

Living Lab
Support

Optional services supported by the configuration

A: Alone/MAIN

1

 

X

X

 

1 mf

Small

X

X

X

X

 

 

 

(x)

 

ETL, DG, CM

B: Basic

2

X

X

X

 

1 mf

Small

X

X

X

X

 

 

 

(x)

 

ETL, DG, CM

C: CityStart

3-4

X

X

X

X

1 mf

Small

X

X

X

X

X

X

X

X

 

ETL, DG, R, CM, Eng

D: DataCity

5-6

X

X

X

X

70

Medium

X

X

X

X

X

X

X

X

 

ETL, DG, R, CM, Eng, LL

E: ExtensiveCity

6-8+

X

X

X

X

70

Scalable

X

X

X

X

X

X

X

X

 

ETL, DG, R, CM, Eng, LL

F: FullPlatform

12-14+

X

X

X

X

Scalable

Scalable

X

X

X

X

X

X

X

X

 

ETL, DG, R, RS, CM, Eng, LL

F: FullPlatform + LivingLab

13-15+

X

X

X

X

Scalable

Scalable

X

X

X

X

X

X

X

X

X

ETL, DG, R, RS, CM, Eng

mf: multiple flows for each IOT App; + means that may depend on the volume you would  like to manage in flow and storage.
In the following configurations you:
have: Multiple IOT Brokers; unlimited number of data sources; interoperability support; KB
may have: HA, DRS, FT, Balancing, cloning and configuring additional VM/Containers;
may pass at the next configuration without reinstalling the VM/Containers;
may have VM or Containers, please ask to Snap4City@disit.org;
--may add one or more Optional Services as VM/Containers: ETL, DataGate, Routing, Engager, Data Analytics, etc. etc.
From Configuration E:ExtensiveCity and upper, the number of VMs/Containers depends on the volume of DataStorage and on the volume of Data Stream in input.
These two aspects are managed by two independent clusters of VMs/containers and thus they scale independently each other. This allows to satisfy any different combination of volumes in streams and data storage.
We suggest using Living Lab support only on FullPlatform, while it can be installed on Smaller Configurations with some limitations in terms of features

Legenda of the above table for Optional Services

Eeach VM is provided configured with large footprint as 16/24 cores 2.2 Ghz, 16-24 GB Ram, 500 GB HD in thin provisioning, with 25-40 GB HD space used at the first start, in most cases Debian, this initial size is only cautelative to get maximum performances, while the minimum footprint is much smaller (please note that configurations A and B can be executed on 4 cores, 4 GB Ram, 44Gbyte HD free space).
to estimate the needed CPU, RAM, Storage for each configuration it is easy from the #of VM in the configuration.
-- to estimate the size of the platform on the basis of the data that you need to ingest is also very simple, ask for a quotation describing your data (send email to snap4city@disit.org)
-- all the VMs in download are in files RAR of 1-12 Gbytes size, so that you can download from browser. If you have problem we can arrange a different modality.
In solutions, with a very large number of users on the Front End: Smart City API (mobile App users, Dashboard users, a frontend balancer and more FrontEnd servers would be needed). Please note that the Smart City API are also used by DataAnalytics processes and by MicroServices in the IOT Applications.
Typically a thousand of simultaneous users on the front-end can be sustained for each VM/Container
More precise estimations can be performed by knowing the actual demanded workload

A: Alone/MAIN) For Small Size Business, for example an industry 4.0 application you can need to install only one VM: Snap4CityMAIN, ContainerCompose "Alone".

it may be used to create dashboards exploting data coming from external third party IOT brokers, from external data stores / databases of several kinds (ODBC, JDBC, Mongo, RDF, etc.), etc. Please note that the solution released as VM allows you to test also Smart IOT Applications, since Snap4CityMAIN includes one IOT App on which you can load/design multiple flows.

Therefore, 1 (one) VM, one Container Compose with multiple Containers and you can start using.

Snap4CityMAIN (also called dashboard) includes the Snap4City Dashboard Builder and a full starting Snap4City environment for small size businesses as well as for large solutions. Snap4CityMAIN includes tools configured for supporting features in a Snap4City local environment on your premise, so that with limitations only on configuration aspects to make the VM consistent and capable to be executed and provide Snap4City services in your network. it can be the seeding of a larger and scalable solution on your premise. Snap4CityMAIN  can be replicated to create a cluster in balance. The database can be shared from the frist Snap4CityMAIN while it cam be also extracted and put in some network storage.

What you can do with ALONE configuration you can:

  • create a small smart city business, (or industry 4.0 solution) acquiring data by using this VM only, for example to get data with Node-RED and presenting them on Dashboards;
  • create and manage a number of users into your on premise Snap4City platform, according to GDPR;
  • directly connect your Dashboard Builder environment to get data from any your database in your network;
  • register External Services for showing them into your dashboards;
  • register MIcro Applications;
  • Communicate with Dashboard by using Web Socket streams.
  • define you myKPI and POI and manager them on dashboards, they can be time series, and time series with GPS locations;
  • connect the SNAP4CITYMAIN with (a) the IOTOBSF: for ingesting data from the IOT Brokers in push, (bETL for ingesting data from batch processes in PULL; This means that with two VM you can create you Industry 4.0 and Smart City solutions. The only limit is the volume of data, the number of IOT Applications, and the lack of massive IOT device management. 
  • add more IOT or other protocols using the IOT Applications as described into https://www.snap4city.org/65  
  • add Data Analytics as described in: https://www.snap4city.org/501
  • please note that:
    • you can start with Snap4City MAIN anche step by step creating more complete and high performance solutions without the need to restart since any solution include also the MAIN
    • MAIN does not include any knowledge base map management. You can use your GIS or direcly the maps from open sources. for the KB you need to instal KBSSM
  • etc

Alone/MAIN Configuration: Snap4CityMAIN Virtual Machine

Snap4CityMAIN VM can be found on:

The suggested VM player is: https://www.snap4city.org/download/video/vm/VMware-player-12.1.0-3272444.exe

Documentation for installation can be found  on HOW TO: configure/install StartSNAP4CITYVM: starting appliance of Snap4City  and the VM appliance Snap4CityMAIN includes tools as:

  • Dashboard Builder for creating dashboards of several kinds, and distributing them on a large number of users, also in connection with IOT Applications, including Dashboard Wizards and Native Widgets. A large number of widgets updated and/or added in version 1.5 and most of them are dynamic: time series, bar series, kiviat, multiseries, single content, buttons, tables, event driven with synoptics, etc., most of them allow you to browse on time, etc.)
  • Data Inspector: for exploring the data accessible on the solution. A large set of new features have been added on Ver 1.5, for managing the collection of data, links to processes, links to IOT Apps, licensing models, owners, etc.)
  • Dashboard engine, for collecting data from several data sources with a Pull approach and save them on local database of Dashboard Builder. The Dashboard engine is compatible with several protocols
  • Authentication tool, Single Sign On, SSO: for registering users and managing keys (keycloak based), LDAP support for managing roles and organizations.
  • Notificator for programming notifications to be sent when some event occurs on Dashboard metrics.
  • IOT Application, IOT App: one Node-RED Docker for implementing IOT Application flows, a number of them, also supporting Snap4City MicroServices, mutual authentications, etc., working with data in Push and Pull and several protocols. The library of Snap4City MicroServices presents some limitations since the Knowledge Base as Servicemap is missing in the configuration.  With Version 1.5, stronlgy update of nodes for node-red for managin dynamic dashboard widgets, synoptics directly, barseries, timeseries, etc., in addition the compatibility with a number of new protocols
  • MySQL database, local instance for configuration and data management. Which can use also Elastic Search even if not configured yet.
  • External Service manager: to collect external services, legacy servers and tools and exploit them on Dashboards. Please remind that External Services as web pages can be also embedded into Dashboard widgets.
  • MenuManager also called Main Snap4City: to manager different tools connected on a Snap4City environment, also external services, legacy servers, etc. On version 1.5, added the possibility of having menu attached to the single organization and single dashboards.
  • OwnershipsManager: to manage users' ownerships and delegations to access at different kinds of data and resources, delegation to access, revokations, etc. Added in Ver. 1.5 the possibility of controlling ownership and delegation via IOT Applications.
  • MyKPI: for collecting and managing MyKPI, personal MyPOI, etc. It includes the Personal Data Manager, based on GDPR. (updated on ver 1.5)
  • WS Server: WebSocket Server for secure communications among IOT Applications and Dashboard, Synoptics, widgets, etc. (strongly updated on ver. 1.5, there is also Kafka for managing event driven data, plus WS server)
  • MicroApplications: a set of MicroApplications which can be tuned on the content accessible on your area. The MicroApplications may have sense when the Main Snap4City is configured together with KBSSM providing knowledge base of city entities on the maps. Please remind that MicroApplications can be also embedded into Dashboard widgets
  • IOT device management with IOT DirectoryIOT Directory: is a tool for managing a large number of IOT Brokers of different kinds, and supporting different IOT protocols. The IOT Directory needs at least a ServiceMap and to perform IOT Discovery. The IOT Directory allows at the users to add IOT Devices, IOT devices in bulk, to manage delegation on devices and IOT Devices values, etc. 
  • Resource Manager: to publish, exchange and access to a large set of resources developed for other smart cities and Snap4City solutions; The Resource Manager includes market place for IOT Applications and flows, ETL processes, Data Analytic processes, etc. It is a very powerful tool for information exchange and for the publication of smart applications.  The Resource Manager also allows the user to create MicroServices for the IOT Applications, when they are based on rest call of external services. Added on ver 1.5 the Data Dictionary, to define nature, subnature, value types, value units.
  • Synoptic support since version 1.5: a set of tools for creating synoptics and custom Widgets for the dashboards using MyKPI, sensors, costants, variables on WS.
  • User Stats since version 1.6 (coming soon): a set of metrics and statistics on resources consumed by each user for self assessment;
  • we postponed other fetures to  the next versions. On the other hand, a number of features that you see on Snap4City.org can be added on the main taking the source code from GITHUB and make from yourself the configuration needed.

Alone/MAIN Configuration: Snap4CityMAIN Docker Containers

The Docker version for this "Alone Configuration" and is available

The configuration in terms of user names and password is the same of the VM described above: HOW TO: configure/install StartSNAP4CITYVM: starting appliance of Snap4City,  On  the other hand, the Container Compose includes the details of the ports used that should not conflict with the host or server in which you deploy it, so t hat you can change in the file in case of conflict. In addition, you have to insert into the hosts file of your operating system: <your-host-ip> dashboard keycloak wsserver iotapp personaldata
This will allow all processes to refer to compoments: dashboard, keycloak, wsserver, iotapp, personaldata, as localhost by using their names and internal mapping. The same if you have other computers in the same network. Please note that, if you are going to make the solution public you have to provide a domain name, etc., and change the configuration according to your setup.

The cerate group of contaiers are put in execution by the Container Compose (version 1.3) and in particular each of them are accessible from Docker HUB and include:

B: Basic) Addressing  a relevant number of IOT devices/ IOT brokers in your smart city or industry 4.0, a few smart Applications with multiple  flows

If you have a relevant number of IOT Devices you need to have several IOT brokers. They can be registered with their devices into the Snap4City IOT Directory in the Snap4CityMAIN. Moreover, you may need to implement and thus provide some of those IOT Brokers. You can do that by using one or more IOTOBSF, which includes an IOT Orion Broker supporting NGSI and other protocols see :https://www.snap4city.org/467 for Mutual Authentication. In the context of IOT for Industry applications, you may need to have some intelligence and IOT Applications located on the field, as well as on premise. For example in factory in the case of Industry 4.0, on the farm in the case of smart farm, etc. This smart tools can be IOT Applications working on IOT Edge. This configuration alloow have them connected to the Smart4CityMAINVM, thus sending data to the solution at MyKPI, via IOT brokers, etc. see below the list of  the IOT Edge solutions on Snap4City with Security support. You may have IOT brokers and/or IOT Edge, both solutions or  only one, depending on your data and preferences. Please note that the solution allows you to test also Smart IOT Application, since Snap4CityMAIN includes one IOT App on which you can load/design multiple flows.

Therefore, 1 (one) VM + N IOT broker VM and you can start using.

C: CityStart) Addressing  a relevant number of GeoLocated Points of Interest, PIN on maps, and eventually heatmaps, a Small Smart City with a few smart applications on cloud and many on edge

If you need to manage a relevant number of city entities located on maps, we suggest you the following minimal configuration which includes KBSSM modeling a knowledge base for street graphs which can be feed with OSM graph or with your GIS as well (also included into it). You can exploit maps from GIS solutions (see https://www.snap4city.org/368) or you can exploit the Knowledge Base approach which is more powerful in terms of semantic relationships among city entities. To this end you need to have KBSSM appliance into your platform. If the workload on Smart City API become very high the  KBSSM  can be cloned to create a cluster and a balancer. The first KBSSM may keep the storage and share with partners. Please note that the solution allows you to test also Smart IOT Application, since Snap4CityMAIN includes one IOT App on which you can load/design multiple flows.

Therefore, at the minimum, 2 (two) VM and you can start using. If you have also Geoserver 3 VM. if you have a huge amount of mobile apps exploiting the Smart City data or you have to provide dashboard towards a large number of users, you may need to create a balanced workload, etc.