London skyline / loveoflondon.tumblr.com
Installing Kale Skyline
A more thorough installation
At my current job we’re discussing using Skyline to track down anomalies. It’s something that came up frequently in talks as the amount of metrics being collected and the amount of knowledge around each of them is increasing.
After checking the project’s github account I felt the installation instruction were not clear enough and there were a few particular occasions that I got stuck for a while, browsing other forums I noticed more people struggling with its installation so here is a detailed step by step reproducible description that anyone can follow to try out Skyline!
Get a box
First thing is to get a working barebones server installation. We use Debian for production servers so I picked Wheezy which is the latest stable release.
Not only that but I also found out that the previous version, Debian Squeeze, struggled with an easy installation of Redis and I’d rather do it all within regular Debian ports.
I also use Vagrant for all my local testing so I found out a helpful already made box here which will allow you to not only be able to bypass building a new Vagrant Debian Wheezy box but also use exactly the same initial setup as I did.
vagrant box add wheezy64 http://dl.dropbox.com/u/937870/VMs/wheezy64.box ``mkdir skyline``cd skyline``vagrant init wheezy64
Now you have a ready to use Wheezy box but to save a restart later on you should set up port forwarding on your local Vagrantfile which will bind to Skyline’s web application.
config.vm.network :forwarded_port, guest: 1500, host: 1500
Boot up and login:
vagrant up``vagrant ssh
And after login in don't forget to update your box.
sudo apt-get update``sudo apt-get upgrade
Once logged in to our new updated box we can start installing all requirements.
First things first we need pip to be able to add some of the requirements for python, as well as python-dev without which some of the packages used with pip will not work!
sudo apt-get install python-dev``sudo apt-get install python-pip
After that we can clone the project’s repository
git clone [https://github.com/etsy/skyline.git](https://github.com/etsy/skyline.git)
Once we have the project we can change into the folder and start going through all the python dependencies.
cd skyline``sudo pip install -r requirements.txt``sudo apt-get install python-numpy python-scipy python-scikits.statsmodels``sudo pip install patsy msgpack_python
Debian is not famous for having packages running on the latest version but for its stability. Unfortunately we do need the latest version which is 2.6 rather than the stable one which is 2.2.
But not all is bad, version 2.6 is available on the Wheezy backport which means it will be available in the next Debian stable version and was back ported to Wheezy.
You can do this easily by adding the backport URL for Wheezy and installing that specific version.
sudo vim /etc/apt/sources.list
deb [http://ftp.ch.debian.org/debian/](http://ftp.ch.debian.org/debian/) wheezy-backports main
Update and install Redis-Server, don't forget to kill the process as it will be started with the default config right after the installation.
sudo apt-get update``sudo apt-get -t wheezy-backports install redis-server``sudo pkill redis
With all dependencies installed you can now copy an example for the settings file. This will be the place to add your Graphite URL and other details but for now we only need to update the IP bound to the web application so the Vagrant host will display it.
cp src/settings.py.example src/settings.py
Edit file for bind.
WEBAPP_IP = ‘0.0.0.0'
And one more thing. Add the log folders for all the applications.
sudo mkdir /var/log/skyline sudo mkdir /var/run/skyline
And ensure /var/log/redis already exists
Start all components
You are ready to start the application, just change into the bin directory and start all components.
cd bin``sudo redis-server redis.conf``sudo ./horizon.d start``sudo ./analyzer.d start``sudo ./webapp.d start
Ensure there are no errors in the logs, for example if you forgot to kill the Redis process earlier on you’d run into problems now!
cd ../utils``python seed_data.py
Should show you this output mentioning the installation was successful.
Connecting to Redis… Loading data over UDP via Horizon… Congratulations! The data made it in. The Horizon pipeline seems to be working.
You have now Skyline ready to go so just check the port you forwarded into your Vagrant host to use the web application!
Skyline UI in Vagrant Host