HOW TO: configure/install KBSSM VM appliance of Snap4City

this Snap4City KBSSM VM has been described in https://www.snap4city.org/drupal/node/534 It includes:

This VM is also called "kbssm" internally.

This virtual machine is based on Debian operating system v8.11 (without graphic Desktop Environment) and it has been run on VMware ESXi v6.5, but you can run on VMware player. In particular, this VM contains what is described in https://www.snap4city.org/drupal/node/471
As a prerequisite for using this virtual machine, it is necessary to download and install the free VMware Workstation Player (tested with version 15). With this tool you can launch the VM, which by default was created with the following specifications (which can be modified in the VMware "Settings" configuration panel):

number of processor: 8 (you can increase when needed)

RAM:                                    8 GB (it can be easily increased)

Hard-Disk: 500 GB (thin provisioning) you need at least ..X. Gbyte at the first running on your HD

Access to the Operating System, credentials

IP: static verify into your system (ifconfig from Debian shell)

Operating system: Debian v8.11

username: debian, password: debian, password of super user (su): debian

Access to Services

APACHE TOMCAT http://kbssm:8080/manager

username: tomcat password: tomcat roles: manager-gui

username: admin password: snap4admin roles: admin (used by ServiceMap to access configuration status)

MySQL

username: servicemap password: snap4map

username: root password: password

virtuoso RDF store http://kbssm:8890

username: dba password: dba

Configuration and Deploy

1) The VM is provided in DHCP network configuration, use command ifconfig to see the assigned ip address or set static IP (according to your preference). To ensure correct operation in any local network, check and note the IP assigned when the VM is turned on.

2) It is necessary to reconfigure the /etc/hosts file as superuser ("sudo pico /etc/hosts"), assigning to the hostname "kbssm" the VM current IP and change alle the other IP associated with "dashboard" (the MAIN VM) and "iotdses".

It's important to configure this information also in the Snap4CityMAINVM /etc/hosts file.

You also need to associate the IP of the VM with the host "kbssm" in your work environment, in the computer that you use to work with the Snap4City VM, for example to open dashboard or to create them.
In Linux, please modify the usual /etc/hosts file as above,
in Windows the hosts file to be modified is located in the path ......\Windows\System32\drivers\etc

ServiceMap Configuration

the main configuration file is /var/lib/tomcat8/servicemap/servicemap.properties if you change any property restart tomcat ("sudo service tomcat8 restart")

SuperServiceMap Configuration

the configuration file is /var/lib/tomcat8/settings.xml, it mainly contains the credentials to connect with the DB.

In particular the servicemaps table lists the servicemaps instances it should forward requests to. Each servicemaps is associated with the servicemap base url and a WKT geometry that describes the geographic area the servicemap covers. To add or remove a servicemap you need to update the table manually.

Building a street graph

How to build a street graph in form of RDF triples for a geographical area of your interest
This section describes the key steps for generating a set of RDF triples that conforms to the Km4City Ontology and represents the street graph of a given geographical region of your interest, retrieving the necessary data from the Open Street Map. Scripts that are mentioned in this document, all locate in the /home/Debian/samples/osm folder of your Snap4City Virtual Machine.

Initialization

The first time that you decide to build the street graph for a given area, you will have to:

create a relational (Postgresql) database where you will put the Open Street Map data that relate to the region of your interest, and enable for it the necessary extensions so that you can store and manage with geometric and geographic data;
create a Postgresql user, and let it be the owner of the newly created database;
shape your newly created database in such a manner that it could be suitable for housing Open Street Map data;
download Open Street Map source files for your region of interest from geofabrik, or from the Open Street Map directly if it is not a problem for you to store locally the map of the whole planet
load Open Street Map source files into your newly created database, by using a tool, namely the osmosis, that offers a set of functionalities to manipulate Open Street Map data
perform a set of further operations on your newly created Postgresql database, that are wrapped into the init.sql SQL script, and that will speed up the future generation of RDF triples. The boundary parameter that you must provide when executing the script, is the OSM ID of the OSM Relation that defines the external boundary of your region of interest. It is constrained by the source file that you have downloaded from geofabrik and loaded into your database. Indeed, if you have loaded let’s say, the central Italy, you cannot specify as a boundary the OSM ID of the Relation that corresponds to the Sicily. The smaller it is the boundary that you specify, the faster will be the generation of triples. On the other hand, if you specify for example the OSM ID 41977 as your boundary here, that corresponds to Tuscany, you will not be able to generate RDF triples for Rome (unless you repeat the initialization), despite data for Rome would be available in your local database (central Italy includes Rome indeed).

An example of how all of this can be done for the geographical region of central Italy can be found in the init.sh shell script. The script relies on the init.sql script, that is thought to be left unchanged. This means that when you will first import data for your own region of interest, you will only need manipulating the init.sh script, leaving the other unaltered.

Below here are some Web resources where you can learn more about tools and projects that we have met in this section of the guide:

Update

The Open Street Map evolves very rapidly. Every day, millions of voluntary contributors all around the World submit new features or modify existing ones. Therefore, you are strongly recommended to update regularly the local copy of the Open Street Map that you keep stored in your Postgresql database.
The update.sh script shows you how it can be done. Execute it line by line the first time that you execute it. This way, you will be able to better understand what happens at each step, and it will be easier for you to perform customizations that are necessary to update your own map.

Here are the key operations that the script performs:

Export the local copy of the Open Street Map that is stored in your Postgresql database to a PBF file. The PBF is the most compressed of the file formats in which an OSM map can be represented. Use the osmosis tool to do that;
Download the most up-to-date version of the same map from geofabrik;
Compare the PBF file that you have generated from your local copy of the map, with the PBF file that you have downloaded from geofabrik, and represent differences as an OSC file, that is the file format that Open Street Map uses for representing differences between maps. Use the osmosis tool to do all this;
Apply changes that are described in the OSC file to your local copy of the map. Use the osmosis tool to do that;
Execute the update.sql script as is on your Postgresql database, to refresh indexes and other non-OSM tables that are useful for speeding up the generation of triples, and that could not to be up to date now that the OSM data has been refreshed in your database.

Triplify

Loading maps in a local database and keeping them up to date is functional to the generation of RDF triples, that is what we perform in this third step. See the triplify.sh script to learn how it can be done. Note that it is the only script that you will need to customize to generate your own triples. Indeed, both the triplify.sql and triplify.sml scripts are thought to be left unaltered. It is a three-step process:

Prepare data in your Postgresql database executing the triplify.sql script. In this step you must provide a parameter, again named boundary, that is the OSM ID of the OSM Relation that defines the boundary for the generation of triples. This boundary must be fully contained within the boundary that we have set during initialization;
Generate RDF triples from the data that you have prepared in your Postgresql database. Use the sparqlify tool to do that;
Perform some clean-up operations on the n3 file that the sparqlify produces.