Installing Kale Skyline

London skyline / loveoflondon.tumblr.com

Installing Kale Skyline Link to heading

A more thorough installation Link to heading

At my current job we’re discussing using Skyline to track down anomalies. It’s something that came up frequently in talks as the amount of metrics being collected and the amount of knowledge around each of them is increasing.

After checking the project’s github account I felt the installation instruction were not clear enough and there were a few particular occasions that I got stuck for a while, browsing other forums I noticed more people struggling with its installation so here is a detailed step by step reproducible description that anyone can follow to try out Skyline!

Get a box Link to heading

First thing is to get a working barebones server installation. We use Debian for production servers so I picked Wheezy which is the latest stable release.

Not only that but I also found out that the previous version, Debian Squeeze, struggled with an easy installation of Redis and I’d rather do it all within regular Debian ports.

I also use Vagrant for all my local testing so I found out a helpful already made box here which will allow you to not only be able to bypass building a new Vagrant Debian Wheezy box but also use exactly the same initial setup as I did. vagrant box add wheezy64 http://dl.dropbox.com/u/937870/VMs/wheezy64.box ``mkdir skyline``cd skyline``vagrant init wheezy64

Now you have a ready to use Wheezy box but to save a restart later on you should set up port forwarding on your local Vagrantfile which will bind to Skyline’s web application. vim Vagrantfile

And change: config.vm.network :forwarded_port, guest: 1500, host: 1500

Boot up and login: vagrant up``vagrant ssh

And after login in don't forget to update your box. sudo apt-get update``sudo apt-get upgrade

Skyline requirements Link to heading

Python Link to heading

Once logged in to our new updated box we can start installing all requirements.

First things first we need pip to be able to add some of the requirements for python, as well as python-dev without which some of the packages used with pip will not work! sudo apt-get install python-dev``sudo apt-get install python-pip

After that we can clone the project’s repository git clone [https://github.com/etsy/skyline.git](https://github.com/etsy/skyline.git)

Once we have the project we can change into the folder and start going through all the python dependencies. cd skyline``sudo pip install -r requirements.txt``sudo apt-get install python-numpy python-scipy python-scikits.statsmodels``sudo pip install patsy msgpack_python

Redis Link to heading

Debian is not famous for having packages running on the latest version but for its stability. Unfortunately we do need the latest version which is 2.6 rather than the stable one which is 2.2.

But not all is bad, version 2.6 is available on the Wheezy backport which means it will be available in the next Debian stable version and was back ported to Wheezy.

You can do this easily by adding the backport URL for Wheezy and installing that specific version. sudo vim /etc/apt/sources.list

Change. deb [http://ftp.ch.debian.org/debian/](http://ftp.ch.debian.org/debian/) wheezy-backports main

Update and install Redis-Server, don't forget to kill the process as it will be started with the default config right after the installation. sudo apt-get update``sudo apt-get -t wheezy-backports install redis-server``sudo pkill redis

Configuration setup Link to heading

With all dependencies installed you can now copy an example for the settings file. This will be the place to add your Graphite URL and other details but for now we only need to update the IP bound to the web application so the Vagrant host will display it. cp src/settings.py.example src/settings.py

Edit file for bind. vim src/settings.py

Change. WEBAPP_IP = ‘0.0.0.0'

And one more thing. Add the log folders for all the applications. sudo mkdir /var/log/skyline sudo mkdir /var/run/skyline

And ensure /var/log/redis already exists

Running Skyline Link to heading

Start all components Link to heading

You are ready to start the application, just change into the bin directory and start all components. cd bin``sudo redis-server redis.conf``sudo ./horizon.d start``sudo ./analyzer.d start``sudo ./webapp.d start

Ensure there are no errors in the logs, for example if you forgot to kill the Redis process earlier on you’d run into problems now!

Testing Link to heading

cd ../utils``python seed_data.py

Should show you this output mentioning the installation was successful. Connecting to Redis… Loading data over UDP via Horizon… Congratulations! The data made it in. The Horizon pipeline seems to be working.

Visualising it Link to heading

You have now Skyline ready to go so just check the port you forwarded into your Vagrant host to use the web application!

http://localhost:1500

Skyline UI in Vagrant Host