HOW TO (Draft): configure/install MCLSCont VM and NCLSCont VM appliances of Snap4City

×

Warning message

  • You can't delete this newsletter because it has not been sent to all its subscribers.
  • You can't delete this newsletter because it has not been sent to all its subscribers.

-- in progress --

This virtual machine is based on Debian GNU/Linux 9.8 (stretch) (without graphic Desktop Environment) and it runs on VMware ESXi v6.5, but you can run it on VMware Workstation Player. As a prerequisite for using this virtual machine, it is necessary to download and install the free VMware Workstation Player (tested with version 15). With this tool you can launch the VM, which by default was created with the following specifications (which can be modified in the VMware "Settings" configuration panel):

 

number of processor: 8 (you can increase when needed)

RAM:                                    8 GB (it can be easily increased)

Hard-Disk:                           500 GB (thin provisioning) you need at least 34 Gbyte at the first running on your HD

 

Access to the Operating System, credentials

IP: static à verify into your system (ifconfig from Debian shell)

Operating system: Debian v9.11

username: debian,     password: debian,     password of super user (su): debian

 

You must modify the file /etc/hosts and update the local IP of the machine “mclscont”.

The main services installed in the VM are:

Apache Mesos

Apache Marathon

Apache Zookeeper

MySQL Server 5.7.27

 

Each of these services (configured for autostart at boot) can be started/stopped/restarted with

  • sudo service mesos-master [start/stop/restart]
  • sudo service mesos-slave [start/stop/restart]
  • sudo service marathon [start/stop/restart]
  • sudo service zookeeper [start/stop/restart]
  • sudo service mysql [start/stop/restart]

 

This VM contains an installation of DISCES-EM, a python controller script that interacts with the VMware vSphere environment, and provides elastic scaling capabilities for docker containers.

The DISCES-EM folder is located in

  • /home/debian/snap4city/Disces-em

 

You have to modify the file settings.py in that folder, filling it with the your VMware vSphere credentials

# vCenter

config["vCenter_host"] = 'localhost'

config["vCenter_user"] = 'user'

config["vCenter_password"] = 'password'

config["vCenter_port"] = '443'

 

You need to change MySQL credentials

# setup MySQL connection

config["mysql_user"] = 'root'

config["mysql_password"] = 'debian'

config["mysql_host"] = 'localhost'

config["mysql_database"] = 'quartz'

 

Note that before performing a git update of the Snap4City repository you have to backup your settings.py to avoid overwriting.

You can leave any other parameter as it is.

The Apache Mesos web interface is reachable at:

The Apache Marathon web interface is reachable at:

The default MySQL credentials are:

  • Username: root
  • Password: debian

 

In order to put in periodic execution the disces-em script you must change the crontab configuration.

For this purpose, type in the command line (as root):

crontab -e

And then uncomment the following lines

@reboot /usr/local/bin/jupyter notebook > /dev/null 2>&1

@reboot /home/debian/snap4city/Disces-em/disces-em-t.sh > /dev/null 2>&1

 

Apache Web Server configuration

 

You have to change the settings file

  • /var/www/html/settings.php

<?php

$config["mysql_host"] = "localhost";

$config["mysql_username"] = "root";

$config["mysql_password"] = "debian";

$config["mysql_schema"] = "quartz";

?>

 

Remember to backup it before performing a git update of the Snap4City repository.

If you want to scale up the configuration provided by this VM (1 Mesos master/slave, 1 Marathon, 1 Zookeeper)

  • If you want to add one more Mesos master node
    • Clone the VM
    • Disable mesos-slave with:
      • update-rc.d mesos-slave disable
    • Configure ZooKeeper
      • cat <<EOF >> etc/zookeeper/conf/myid1EOF
      • cat <<EOF>> /etc/zookeeper/conf/zoo.cfgserver.1=`hostname`:2888:3888EOF
    • Configure Mesos Mastercat <<EOF >> /etc/mesos/zkzk://`hostname`:2181/mesosEOFcat <<EOF >> /etc/mesos-master/quorum1EOF
    • Update hostname and IP of the master VM by editing the files /etc/mesos-master/hostname and /etc/mesos-master/ip
    • Configure Zookeeper adding the IPs of all master nodes on all master VMs in /etc/zookeeper/conf/zoo.cfg
      • E.g.
        • server.1=192.168.1.1:2888:3888
        • server.2=192.168.1.2:2888:3888
        • server.3=192.168.1.3:2888:3888
    • Configure Zookeeper adding the IPs of all master nodes on all master VMs in /etc/mesos/zk zk://192.168.1.1:2181,192.168.1.2:2181,192.168.1.3:2181/mesos
    • Choose a unique id (integer) for each Zookeeper instance by editing the file /etc/zookeeper/conf/myid on each master VM
    • Configure Marathon IPs on all master VMs by adding their IPs to the file
      • /etc/marathon/conf/master
      • E.g.: zk://192.168.1.1:2181,192.168.1.2:2181,192.168.1.3:2181/mesos
  • If you want to add one more Mesos slave node
    • Clone the VM
    • Disable mesos-master with:
      • update-rc.d mesos-master disable
    • Disable zookeeper with:
      • update-rc.d zookeeper disable
    • Update mesos’ zookeeper configuration in /etc/mesos/zk with the IPs of mesos masters
      • E.g.: zk://192.168.1.1:2181,192.168.1.2:2181,192.168.1.3:2181/mesos
    • Update hostname and IP of the slaveVM by editing the files /etc/mesos-slave/hostname and /etc/mesos-slave/ip