2024 HOW TO: Deploy/Install your Snap4City Solution on private or public Clouds, VM with Containers
-
First version of Snap4City installation based on Kubernetes (03/01/2022): https://github.com/disit/snap4city-kubernetes
-
Platform Overview: PDF FILE https://www.snap4city.org/download/video/Snap4City-PlatformOverview.pdf
-
june 2022 - APPLIANCE single VM will almost all Snap4City MAIN Version 1.6: https://www.snap4city.org/download/video/vm/Appliance-Snap4CityMAIN-Dashboard-1-6.rar
-
Snap4City installations have to use the following logo linked to Https://www.snap4city.org at the bottom right of each dashboard and in the main page of the service.
-
Snap4City & Snap4Industry Registered Instances Installations
(02-02-2021) Notes on Installing DataCity-Large 2020: HOW TO: Deploy & Installation of Snap4City DataCity-Large 2020
(10 September 2020) Snap4City DataCity-Small (aligned to version 1.5 of VM plus some new features) has been released as DOCKER Compose, see in the following.
(15 August 2020) Snap4CityMAIN version 1.5 has been released with included: synoptics, more widgets, real time buttons, IOT App updated, plus other VMs have been updated for composing a number of configurations: IOTOBSF, IOTDSES, KBSSM. See in the following.
Snap4City platform can be freely downloaded from this portal, and thus you can produce a number of services with these tools in the view of Smart City IOT as a Service, SCIaaS. Therefore, if you are interested in using Snap4City.org for trial from this portal you do not need to perform any installation on your computers.
On the contrary, this page is dedicated to who would like to perform a local installation on premise or cloud. In fact, in this page you can find the instructions for the installation and a number of Snap4City configurations. Snap4City is 100% open source, license free solution, and the Appliances that you can download from this page are license free. You can download and install Snap4City tools on premise or on cloud as you like, by starting from the following Appliances and/or from the GITHUB/DISIT sources (Source Code and licenses), and docker compose provided. In the following figures, the deploy of Snap4City solution from small business to large scalable cases are presented. The configurations are listed and coded with a letter to simplify the identification of the configuration. While other mixed solutions and intermedium configurations are also possible. In addition, some of the tools we are mentioning/providing in the following configurations can be substituted with your legacy tools. For example your CKAN, your GIS, ArcGIS server, etc., etc. For shortening your deploy and/or installation activities we have created a number of Virtual Machines, VM, Appliances and Docker containers/compose. The VMs can be put in execution on VMplayer or VMware workstation or Vcenter Vsphere VMware infrastructure or you can convert them for any kind of Cloud infrastructures, while Containers on Docker. The configurations are presented below from the smaller and simpler solutions to the most capable and wide deploy for large scale smart city with living lab support. For each of them an incremental description and user manual for installation. The same tools can also be downloaded from GITHUB/DISIT.
Configuration Kind |
Min #VM / Docker |
IOT Broker int (any# and kind) |
IOT Broker Ext (any# and kind) |
MAP WMS web |
KBSM semantic engine reasoner |
IOT APP, MicroServices #Containers |
Storage & IOT Data Shadow |
SSO, Roles LDAP |
IOT Security full stack end2end |
Dash + Wizard, widgets, Data Inspector |
GIS input WFS WMS |
GIS output WFS WMS |
Heatmaps WMS |
Mob. App & Micro Apps. |
ASCAPI Smart City API, Federated |
Living Lab |
Optional services supported by the configuration |
A: Alone/MAIN |
1 |
|
X |
X |
|
1 mf |
Small |
X |
X |
X |
X |
|
|
|
(x) |
ETL, DG, CM |
|
B: Basic |
2 |
X |
X |
X |
|
1 mf |
Small |
X |
X |
X |
X |
|
|
|
(x) |
ETL, DG, CM |
|
C: CityStart |
3-4 |
X |
X |
X |
X |
1 mf |
Small |
X |
X |
X |
X |
X |
X |
X |
X |
ETL, DG, R, CM, Eng |
|
D: DataCity |
5-6 |
X |
X |
X |
X |
70 |
Medium |
X |
X |
X |
X |
X |
X |
X |
X |
ETL, DG, R, CM, Eng, LL |
|
E: ExtensiveCity |
6-8+ |
X |
X |
X |
X |
70 |
Scalable |
X |
X |
X |
X |
X |
X |
X |
X |
ETL, DG, R, CM, Eng, LL |
|
F: FullPlatform |
12-14+ |
X |
X |
X |
X |
Scalable |
Scalable |
X |
X |
X |
X |
X |
X |
X |
X |
ETL, DG, R, RS, CM, Eng, LL |
|
F: FullPlatform + LivingLab |
13-15+ |
X |
X |
X |
X |
Scalable |
Scalable |
X |
X |
X |
X |
X |
X |
X |
X |
X |
ETL, DG, R, RS, CM, Eng |
Legenda of the above table for Optional Services
- DG: DataGate CKAN, open data portal, harvesting and federation
- R: Routing for mobile App and for What-IF analysis
- ETL: ETL SDK for developing ETL process for data ingestion and transformation in addition to IOT App already in
- RS: R Studio Server, for developing Data Analytics (other languages can be used as well) with different kind of servers
- Eng: Engager for engaging users on mobile App according to their context and behavior
- CM: Chat Manager, for private chat room management attached to the dashboards
- LL: Living Lab, for supporting the set up and management of the living lab, see how to install DRUPAL for living lab below
- etc.
A: Alone/MAIN) For Small Size Business, for example an industry 4.0 application you can need to install only one VM: Snap4CityMAIN, ContainerCompose "Alone".
it may be used to create dashboards exploting data coming from external third party IOT brokers, from external data stores / databases of several kinds (ODBC, JDBC, Mongo, RDF, etc.), etc. Please note that the solution released as VM allows you to test also Smart IOT Applications, since Snap4CityMAIN includes one IOT App on which you can load/design multiple flows.
Therefore, 1 (one) VM, one Container Compose with multiple Containers and you can start using.
Snap4CityMAIN (also called dashboard) includes the Snap4City Dashboard Builder and a full starting Snap4City environment for small size businesses as well as for large solutions. Snap4CityMAIN includes tools configured for supporting features in a Snap4City local environment on your premise, so that with limitations only on configuration aspects to make the VM consistent and capable to be executed and provide Snap4City services in your network. it can be the seeding of a larger and scalable solution on your premise. Snap4CityMAIN can be replicated to create a cluster in balance. The database can be shared from the frist Snap4CityMAIN while it cam be also extracted and put in some network storage.
What you can do with ALONE configuration you can:
- create a small smart city business, (or industry 4.0 solution) acquiring data by using this VM only, for example to get data with Node-RED and presenting them on Dashboards;
- create and manage a number of users into your on premise Snap4City platform, according to GDPR;
- directly connect your Dashboard Builder environment to get data from any your database in your network;
- register External Services for showing them into your dashboards;
- register MIcro Applications;
- Communicate with Dashboard by using Web Socket streams.
- define you myKPI and POI and manager them on dashboards, they can be time series, and time series with GPS locations;
- connect the SNAP4CITYMAIN with (a) the IOTOBSF: for ingesting data from the IOT Brokers in push, (b) ETL for ingesting data from batch processes in PULL; This means that with two VM you can create you Industry 4.0 and Smart City solutions. The only limit is the volume of data, the number of IOT Applications, and the lack of massive IOT device management.
- add more IOT or other protocols using the IOT Applications as described into https://www.snap4city.org/65
- add Data Analytics as described in: https://www.snap4city.org/501
- please note that:
- you can start with Snap4City MAIN anche step by step creating more complete and high performance solutions without the need to restart since any solution include also the MAIN
- MAIN does not include any knowledge base map management. You can use your GIS or direcly the maps from open sources. for the KB you need to instal KBSSM
- etc
Alone/MAIN Configuration: Snap4CityMAIN Virtual Machine
Snap4CityMAIN VM can be found on:
- SNAP4CITY MAIN version 1.6 (June 2022): https://www.snap4city.org/download/video/vm/Appliance-Snap4CityMAIN-Dashboard-1-6.rar
- SNAP4CITY MAIN version 1.5: https://www.snap4city.org/download/video/vm/Appliance-Snap4CityMAIN-Dashboard-1.5.rar
- former version 1.4: https://www.snap4city.org/download/video/vm/Appliance-Snap4CityMAIN-Dashboard-1.4.rar
The suggested VM player is: https://www.snap4city.org/download/video/vm/VMware-player-12.1.0-3272444.exe
Documentation for installation can be found on HOW TO: configure/install StartSNAP4CITYVM: starting appliance of Snap4City and the VM appliance Snap4CityMAIN includes tools as:
- Dashboard Builder for creating dashboards of several kinds, and distributing them on a large number of users, also in connection with IOT Applications, including Dashboard Wizards and Native Widgets. A large number of widgets updated and/or added in version 1.5 and most of them are dynamic: time series, bar series, kiviat, multiseries, single content, buttons, tables, event driven with synoptics, etc., most of them allow you to browse on time, etc.)
- if you decide to go for the direct source code installation see: https://github.com/disit/dashboard-builder/wiki
- Data Inspector: for exploring the data accessible on the solution. A large set of new features have been added on Ver 1.5, for managing the collection of data, links to processes, links to IOT Apps, licensing models, owners, etc.)
- Dashboard engine, for collecting data from several data sources with a Pull approach and save them on local database of Dashboard Builder. The Dashboard engine is compatible with several protocols.
- Authentication tool, Single Sign On, SSO: for registering users and managing keys (keycloak based), LDAP support for managing roles and organizations.
- Notificator for programming notifications to be sent when some event occurs on Dashboard metrics.
- IOT Application, IOT App: one Node-RED Docker for implementing IOT Application flows, a number of them, also supporting Snap4City MicroServices, mutual authentications, etc., working with data in Push and Pull and several protocols. The library of Snap4City MicroServices presents some limitations since the Knowledge Base as Servicemap is missing in the configuration. With Version 1.5, stronlgy update of nodes for node-red for managin dynamic dashboard widgets, synoptics directly, barseries, timeseries, etc., in addition the compatibility with a number of new protocols
- MySQL database, local instance for configuration and data management. Which can use also Elastic Search even if not configured yet.
- External Service manager: to collect external services, legacy servers and tools and exploit them on Dashboards. Please remind that External Services as web pages can be also embedded into Dashboard widgets.
- MenuManager also called Main Snap4City: to manager different tools connected on a Snap4City environment, also external services, legacy servers, etc. On version 1.5, added the possibility of having menu attached to the single organization and single dashboards.
- OwnershipsManager: to manage users' ownerships and delegations to access at different kinds of data and resources, delegation to access, revokations, etc. Added in Ver. 1.5 the possibility of controlling ownership and delegation via IOT Applications.
- MyKPI: for collecting and managing MyKPI, personal MyPOI, etc. It includes the Personal Data Manager, based on GDPR. (updated on ver 1.5)
- WS Server: WebSocket Server for secure communications among IOT Applications and Dashboard, Synoptics, widgets, etc. (strongly updated on ver. 1.5, there is also Kafka for managing event driven data, plus WS server)
- MicroApplications: a set of MicroApplications which can be tuned on the content accessible on your area. The MicroApplications may have sense when the Main Snap4City is configured together with KBSSM providing knowledge base of city entities on the maps. Please remind that MicroApplications can be also embedded into Dashboard widgets.
- IOT device management with IOT Directory: IOT Directory: is a tool for managing a large number of IOT Brokers of different kinds, and supporting different IOT protocols. The IOT Directory needs at least a ServiceMap and to perform IOT Discovery. The IOT Directory allows at the users to add IOT Devices, IOT devices in bulk, to manage delegation on devices and IOT Devices values, etc.
- including the process for ingesting IOT Devices from external IOT Brokers (that are IOT Brokers not managed by IOT Directory, so that they are managed by third party for adding IOT Devices).
- on ver 1.5, the management of IOT Brokers, IOT Devices, sensors, actuators, etc. has been much more simplified and automated by increasing its flexibility in managing internal and external Brokers
- Resource Manager: to publish, exchange and access to a large set of resources developed for other smart cities and Snap4City solutions; The Resource Manager includes market place for IOT Applications and flows, ETL processes, Data Analytic processes, etc. It is a very powerful tool for information exchange and for the publication of smart applications. The Resource Manager also allows the user to create MicroServices for the IOT Applications, when they are based on rest call of external services. Added on ver 1.5 the Data Dictionary, to define nature, subnature, value types, value units.
- Synoptic support since version 1.5: a set of tools for creating synoptics and custom Widgets for the dashboards using MyKPI, sensors, costants, variables on WS.
- User Stats since version 1.6 (coming soon): a set of metrics and statistics on resources consumed by each user for self assessment;
- we postponed other fetures to the next versions. On the other hand, a number of features that you see on Snap4City.org can be added on the main taking the source code from GITHUB and make from yourself the configuration needed.
Alone/MAIN Configuration: Snap4CityMAIN Docker Containers
The Docker version for this "Alone Configuration" and is available
- version 1.4 of docker that will be aligned to MAIN 1.5: is coming..... see DataCity-Small docker compose for Version 1.5 below in this web page
- version 1.3 https://github.com/disit/snap4city-docker/tree/master/Alone
- version 1.2 from: https://github.com/disit/snap4city-docker/tree/master/Alone
The configuration in terms of user names and password is the same of the VM described above: HOW TO: configure/install StartSNAP4CITYVM: starting appliance of Snap4City, On the other hand, the Container Compose includes the details of the ports used that should not conflict with the host or server in which you deploy it, so t hat you can change in the file in case of conflict. In addition, you have to insert into the hosts file of your operating system: <your-host-ip> dashboard keycloak wsserver iotapp personaldata
This will allow all processes to refer to compoments: dashboard, keycloak, wsserver, iotapp, personaldata, as localhost by using their names and internal mapping. The same if you have other computers in the same network. Please note that, if you are going to make the solution public you have to provide a domain name, etc., and change the configuration according to your setup.
The cerate group of contaiers are put in execution by the Container Compose (version 1.3) and in particular each of them are accessible from Docker HUB and include:
- https://hub.docker.com/r/disitlab/dashboard-builder
- Dashboard Builder for creating dashboards of several kinds, and distributing them on a large number of users, also in connection with IOT Applications, including Dashboard Wizards and Native Widgets.
- if you decide to go for the direct source code installation see: https://github.com/disit/dashboard-builder/wiki
- Data Inspector: for exploring the data accessible on the solution.
- Dashboard engine, for collecting data from several data sources with a Pull approach and save them on local database of Dashboard Builder. The Dashboard engine is compatible with several protocols.
- Notificator for programming notifications to be sent when some event occurs on Dashboard metrics.
- External Service manager: to collect external services, legacy servers and tools and exploit them on Dashboards. Please remind that External Services as web pages can be also embedded into Dashboard widgets.
- MenuManager also called Main Snap4City: to manager different tools connected on a Snap4City environment, also external services, legacy servers, etc.
- OwnershipsManager: to manage users' ownerships and delegations to access at different kinds of data and resources, delegation to access, revokations, etc.
- MicroApplications: a set of MicroApplications which can be tuned on the content accessible on your area. The MicroApplications may have sense when the Main Snap4City is configured together with KBSSM providing knowledge base of city entities on the maps. Please remind that MicroApplications can be also embedded into Dashboard widgets.
- IOT device management with IOT Directory: IOT Directory: is a tool for managing a large number of IOT Brokers of different kinds, and supporting different IOT protocols. The IOT Directory needs at least a ServiceMap and to perform IOT Discovery. The IOT Directory allows at the users to add IOT Devices, IOT devices in bulk, to manage delegation on devices and IOT Devices values, etc.
- except for the process for ingesting IOT Devices from external IOT Brokers (that are IOT Brokers not managed by IOT Directory, so that they are managed by third party for adding IOT Devices). that would be available soon (coming soon):
- Resource Manager since version 1.4 (coming soon): to publish, exchange and access to a large set of resources developed for other smart cities and Snap4City solutions; The Resource Manager includes market place for IOT Applications and flows, ETL processes, Data Analytic processes, etc. It is a very powerful tool for information exchange and for the publication of smart applications. The Resource Manager also allows the user to create MicroServices for the IOT Applications, when they are based on rest call of external services.
- User Stats since version 1.4 (coming soon): a set of metrics and statistics on resources consumed by each user for self assessment;
- Synoptic support since version 1.4 (coming soon): a set of tools for creating synoptics and custom Widget for the dashboards.
- Dashboard Builder for creating dashboards of several kinds, and distributing them on a large number of users, also in connection with IOT Applications, including Dashboard Wizards and Native Widgets.
- https://hub.docker.com/r/disitlab/personaldata
- https://hub.docker.com/r/disitlab/snap4demo-keycloak, https://hub.docker.com/r/disitlab/snap4demo-openldap
- Authentication tool, Single Sign On, SSO: for registering users and managing keys (keycloak based), LDAP support for managing roles and organizations.
- https://hub.docker.com/r/disitlab/snap4city-nodered-adv, https://hub.docker.com/r/disitlab/snap4city-nodered-basic
- IOT Application, IOT App1: one Node-RED Docker for implementing IOT Application flows, a number of them, also supporting Snap4City MicroServices, mutual authentications, etc., working with data in Push and Pull and several protocols. The library of Snap4City MicroServices presents some limitations since the Knowledge Base as Servicemap is missing in the configuration.
- A standard Container for MySQL
- MySQL database, local instance for configuration and data management.
- https://hub.docker.com/r/disitlab/websocketserver
- WS Server: WebSocket Server for secure communication among IOT Applications and Dashboard, Synoptics, widgets, etc.
B: Basic) Addressing a relevant number of IOT devices/ IOT brokers in your smart city or industry 4.0, a few smart Applications with multiple flows
If you have a relevant number of IOT Devices you need to have several IOT brokers. They can be registered with their devices into the Snap4City IOT Directory in the Snap4CityMAIN. Moreover, you may need to implement and thus provide some of those IOT Brokers. You can do that by using one or more IOTOBSF, which includes an IOT Orion Broker supporting NGSI and other protocols see :https://www.snap4city.org/467 for Mutual Authentication. In the context of IOT for Industry applications, you may need to have some intelligence and IOT Applications located on the field, as well as on premise. For example in factory in the case of Industry 4.0, on the farm in the case of smart farm, etc. This smart tools can be IOT Applications working on IOT Edge. This configuration alloow have them connected to the Smart4CityMAINVM, thus sending data to the solution at MyKPI, via IOT brokers, etc. see below the list of the IOT Edge solutions on Snap4City with Security support. You may have IOT brokers and/or IOT Edge, both solutions or only one, depending on your data and preferences. Please note that the solution allows you to test also Smart IOT Application, since Snap4CityMAIN includes one IOT App on which you can load/design multiple flows.
Therefore, 1 (one) VM + N IOT broker VM and you can start using.
- IOTOBSF includes (i) the standard IOT Orion Broker FiWare, plus (ii) the Snap4City Proxy Filter Security for enforcing mutual authentication and secure connection from IOT Devices and IOT Broker, from IOT Broker and Node-RED IOT Applications of Snap4City. You CAN:
- A) Download and put in execution IOTOBSF VM
- HOW TO: configure/install IOTOBSF VM appliance of Snap4City
- VM version 1.2 to be used with MAIN version 1.5: https://www.snap4city.org/download/video/vm/Appliance-Snap4CityIOTOBSF-1.2.rar
- configured to use eth0 thus simplifing the intallaton on your local cloud.
- the suggested VM player is: https://www.snap4city.org/download/video/vm/VMware-player-12.1.0-3272444.exe
- VM version 1.1 with MAIN version 1.4: https://www.snap4city.org/download/video/vm/Appliance-Snap4CityIOTOBSF-1.1.rar
- B) Download and put in execution IOTOBSF Docker
- C) create from scratch:
- VM: download and install the official Fi-Ware VM or Docker for IOT Orion Broker that you can find at: https://fiware-orion.readthedocs.io/en/master/ (see also https://fiware-orion.readthedocs.io/en/master/admin/extra/ha/index.html).
- DOCKER: get the official Docker container with Orion Broker from https://hub.docker.com/u/fiware/
- If you need secure connection with Snap4City with mutual authentication you have to install the secure filter into VM or DOCKER that you can find on: https://github.com/disit/snap4city/tree/master/OrionBrokerFilter
- In any cases read: HOW TO: configure/install IOTOBSF VM/Container appliance of Snap4City
- A) Download and put in execution IOTOBSF VM
- IOT Edge you may need to have some intelligence, processes and Applications located on the field, as well as on premise. For example in factory in the case of Industry 4.0, on the farm in the case of smart farm, on the city center for mobility, on some park, etc. etc. This partially authonomous smart tools can be implemented by IOT Application working on IOT Edge, they are IOT Application = Node-RED + Snap4City library of MicroServices Nodes, and can be installed as:
- Linux Ubuntu Appliance VM as IOT Edge: https://www.snap4city.org/298 Snap4City IOT Edge Linux on VM Ubuntu: IOT Application, to be configured and updated at the first installation.
- You Can install them from Docker Hub: https://hub.docker.com/r/disitlab/snap4city-nodered-adv, https://hub.docker.com/r/disitlab/snap4city-nodered-basic
- IOT Application, IOT App1: one Node-RED Docker for implementing IOT Application flows, a number of them, also supporting Snap4City MicroServices, mutual authentications, etc., working with data in Push and Pull and several protocols. The library of Snap4City MicroServices presents some limitations when the Knowledge Base as Servicemap is missing in the configuration.
- You Can install them from Docker Hub: https://hub.docker.com/r/disitlab/snap4city-nodered-adv, https://hub.docker.com/r/disitlab/snap4city-nodered-basic
- Android as IOT Edge https://www.snap4city.org/278 and
- Raspberry Pi as IOT Edge: https://www.snap4city.org/77 You can install Snap4City library in any Node-RED from the official Node-RED library
- Linux Ubuntu Appliance VM as IOT Edge: https://www.snap4city.org/298 Snap4City IOT Edge Linux on VM Ubuntu: IOT Application, to be configured and updated at the first installation.
- IOT Applications = Node-RED + Snap4City library of MicroServices, this means that you can start from any installation of Node-RED and add Snap4City by simply installing Snap4City Library from the official Library of Node-RED, that in Node-RED means to add new Nodes into the Palette as https://nodered.org/docs/user-guide/editor/palette/manager : access for the microservice collection from the JS foundation with full documentation inside
- BASIC: Final User: https://flows.nodered.org/?term=snap4city&num_pages=1
- ADVANCED: DEVELOPER (you have to install both libraries: basic and advanced): https://flows.nodered.org/?term=snap4city&num_pages=1
- Docker Containers of IOT Applications: https://hub.docker.com/r/disitlab/snap4city-nodered-adv, https://hub.docker.com/r/disitlab/snap4city-nodered-basic
- IOT Application, IOT App1: one Node-RED Docker for implementing IOT Application flows, a number of them, also supporting Snap4City MicroServices, mutual authentications, etc., working with data in Push and Pull and several protocols. The library of Snap4City MicroServices presents some limitations since the Knowledge Base as Servicemap is missing in the configuration.
C: CityStart) Addressing a relevant number of GeoLocated Points of Interest, PIN on maps, and eventually heatmaps, a Small Smart City with a few smart applications on cloud and many on edge
If you need to manage a relevant number of city entities located on maps, we suggest you the following minimal configuration which includes KBSSM modeling a knowledge base for street graphs which can be feed with OSM graph or with your GIS as well (also included into it). You can exploit maps from GIS solutions (see https://www.snap4city.org/368) or you can exploit the Knowledge Base approach which is more powerful in terms of semantic relationships among city entities. To this end you need to have KBSSM appliance into your platform. If the workload on Smart City API become very high the KBSSM can be cloned to create a cluster and a balancer. The first KBSSM may keep the storage and share with partners. Please note that the solution allows you to test also Smart IOT Application, since Snap4CityMAIN includes one IOT App on which you can load/design multiple flows.
Therefore, at the minimum, 2 (two) VM and you can start using. If you have also Geoserver 3 VM. if you have a huge amount of mobile apps exploiting the Smart City data or you have to provide dashboard towards a large number of users, you may need to create a balanced workload, etc.