Install and Configure Marathon for Mesos Cluster on Photon Hosts

In my previous post I described how to install and configure Mesos cluster on Photon hosts. In this post I am going to explain how to install and configure Marathon for Mesos cluster. I will use Photon OS. All following steps should be done on each Mesos master.

First of all download marathon:

root@pt-mesos-master1 [ ~ ]# mkdir -p  /opt/mesosphere/marathon/ && cd /opt/mesosphere/marathon/
root@pt-mesos-master1 [ /opt/mesosphere/marathon ]#  curl -O
root@pt-mesos-master1 [ /opt/mesosphere/marathon ]# tar -xf marathon-0.13.0.tgz
root@pt-mesos-master1 [ /opt/mesosphere/marathon ]# mv marathon-0.13.0 marathon

Create configuration for Marathon:

root@pt-mesos-master1 [ /opt/mesosphere/marathon ]# ls -l /etc/marathon/conf/
total 8
-rw-r--r-- 1 root root 68 Dec 24 14:33 master
-rw-r--r-- 1 root root 71 Dec 24 14:33 zk
root@pt-mesos-master1 [ /opt/mesosphere/marathon ]# cat /etc/marathon/conf/*
root@pt-mesos-master1 [ /opt/mesosphere/marathon ]# cat /etc/systemd/system/marathon.service
ExecStart=/opt/mesosphere/marathon/bin/start \
    --master zk://,, \
    --zk zk://,,
root@pt-mesos-master1 [ /opt/mesosphere/marathon ]#

Last thing we need to do is to change marathon startup script, since Photon hosts using not standard JRE. Make sure you add JAVA_HOME to java path:
root@pt-mesos-master1 [ /opt/mesosphere/marathon ]# tail -n3 /opt/mesosphere/marathon/bin/start
# Start Marathon
marathon_jar=$(find "$FRAMEWORK_HOME"/target -name 'marathon-assembly-*.jar' | sort | tail -1)
exec "${JAVA_HOME}/bin/java" "${java_args[@]}" -jar "$marathon_jar" "${app_args[@]}"
root@pt-mesos-master1 [ /opt/mesosphere/marathon ]#

Now we can start Marathon service:

root@pt-mesos-master1 [ /opt/mesosphere/marathon ]# systemctl start marathon
root@pt-mesos-master1 [ /opt/mesosphere/marathon ]# ps -ef | grep marathon
root     15821     1 99 17:14 ?        00:00:08 /opt/OpenJDK- -jar /opt/mesosphere/marathon/bin/../target/scala-2.11/marathon-assembly-0.13.0.jar --master zk://,, --zk 
root     15854 14692  0 17:14 pts/0    00:00:00 grep --color=auto marathon
root@pt-mesos-master1 [ /opt/mesosphere/marathon ]#

Installing DCOS CLI for Mesos

To install the DCOS CLI:

Install virtualenv. The Python tool virtualenv is used to manage the DCOS CLI’s environment.

$ sudo pip install virtualenv

Tip: On some older Python versions, ignore any ‘Insecure Platform’ warnings. For more information, see

From the command line, create a new directory named dcos and navigate into it.

$ mkdir dcos
$ cd dcos
$ curl -O

Run the DCOS CLI install script, where <hosturl> is the hostname of your master node prefixed with http://:

$ bash <install_dir> <mesos-master-host>

For example, if the hostname of your mesos master node is

$ bash .

Follow the on-screen DCOS CLI instructions and enter the Mesosphere verification code. You can ignore any Python ‘Insecure Platform’ warnings.

Confirm whether you want to add DCOS to your system PATH:

$ Modify your bash profile to add DCOS to your PATH? [yes/no]

Since DCOS CLI is used for DCOS cluster, need to reconfigure Marathon and Mesos masters urls with the following commands:

dcos config set core.mesos_master_url http://<mesos-master-host>:5050
dcos config set marathon.url http://<marathon-host>:8080

Configure NTP Client and Server Centos

Configure NTP Client and Server Centos

Install NTP

The ntp package is same for client and server. This package will synchronize your server’s time via NTP protocol. In my example server’s address is and client’s address is

# yum -y install ntp


Configure NTP server

Make sure the following lines exists in /etc/ntp.conf. Stratum is used to synchronize the time with the server based on distance.

furge stratum 10


Start the NTP service

For SysVinit based systems run:

service ntpd start

For Systemd systems run:

systemctl start ntpd.service


Configure NTP client

Make sure the following line exists in /etc/ntp.conf.

It might take about 15 minutes until ntp will synchronize the clock.

Now start the NTP service same as above.


Debug the NTP service

There are several useful commands you can use when working with NTP. Here are some of them.

To sync the clock:

service ntpd stop && ntpd -dgq && service ntpd start

Show ntp sync status:

ntpq -pn

Checking the Memory Usage Linux

Checking Memory in Linux

Sometimes you want to check your memory usage and there are a lot of tools to do so. I”ll try to explain about some most useful tools to check memory usage of your system.


knesenko@knesenko-centos7:~$ free -m
total used free shared buffers cached
Mem: 3778 861 2916 16 84 451
-/+ buffers/cache: 325 3452
Swap: 2047 0 2047

What you can see here, is that total amount of your memory is 3778 MB325 MB are used by processes. 3452 MB are free yo use. Last line shows you the swap usage.


Also you can check /proc/meminfo

knesenko@knesenko-centos7:~$ egrep --color 'Mem|Cache|Swap' /proc/meminfo

MemTotal: 3869004 kB

MemFree: 2986092 kB

MemAvailable: 3392864 kB

Cached: 479408 kB

SwapCached: 0 kB

SwapTotal: 2097148 kB

SwapFree: 2097148 kB


vmstat shows you a bit more info about your memory and other resources usage.

knesenko@knesenko-centos7:~$ vmstat

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----

r b swpd free buff cache si so bi bo in cs us sy id wa st

2 0 0 2986092 87172 462452 0 0 2  8  39 73 0  0  100 0 0

r: How many processes are waiting for CPU time.
b: Wait Queue – Process which are waiting for I/O (disk, network, user input,etc..) 

swpd: shows how many blocks are swapped out to disk (paged). Total Virtual memory usage. 

free: Idle Memory

buff: Memory used as buffers, like before/after I/O operations
cache: Memory used as cache by the OS

si: How many blocks per second the operating system is swapping in. i.e Memory swapped in from the disk (Read from swap area to Memory)
so: How many blocks per second the operating system is swapped Out. i.e Memory swapped to the disk (Written to swap area and cleared from Memory). In Ideal condition, We like to see si and so at 0 most of the time, and we definitely don’t like to see more than 10 blocks per second.

bi: Blocks received from block device – Read (like a hard disk) 
bo: Blocks sent to a block device – Write

in: The number of interrupts per second, including the clock. 
cs: The number of context switches per second. 

us: percentage of cpu used for running non-kernel code. (user time, including nice time) 
sy: percentage of cpu used for running kernel code. (system time – network, IO interrupts, etc) 
id: cpu idle time in percentage.
wa: percentage of time spent by cpu for waiting to IO.

knesenko@knesenko-centos7:~$ vmstat -s
      3869516 K total memory
      1758256 K used memory
       279100 K active memory
      1317776 K inactive memory
      2111260 K free memory
        59584 K buffer memory
      1331872 K swap cache
      2097148 K total swap
            0 K used swap
      2097148 K free swap
        15833 non-nice user cpu ticks
            9 nice user cpu ticks
        26954 system cpu ticks
      6886801 idle cpu ticks
          319 IO-wait cpu ticks
            5 IRQ cpu ticks
         2636 softirq cpu ticks
            0 stolen cpu ticks
       340359 pages paged in
      1229646 pages paged out
            0 pages swapped in
            0 pages swapped out
     10782519 interrupts
     21597473 CPU context switches
   1424333862 boot time
         8042 forks


Also I would recommend to use top, htop commands.