Awesome Dashing dashboards with Icinga 2

We at NETWAYS are using Dashing on our office dashboards already. This blog post solely targets integrating yet another new API providing data – the Icinga 2 REST API introduced in v2.4.
The following instructions were taken from the existing Vagrant boxes and their puppet manifests to allow faster installation. Doing it manually shouldn’t be an issue though 😉

Requirements

Ensure that the following packages are installed, example for RHEL 7 with EPEL enabled:

package { [ 'rubygems', 'rubygem-bundler', 'ruby-devel', 'openssl', 'gcc-c++', 'make', 'nodejs' ]:
  ensure => 'installed',
  require => Class['epel']
}

Furthermore put a specific /etc/gemrc file which disables installing the documentation for gems – this can take fairly long and is not required by default. Especially not when provisioning a Vagrant box or a Docker container.

dashing-icinga2

I’ve created that project as demo for Icinga Camp Portland with the help of the existing Icinga 1.x dashing scripts from Markus, and a new job for fetching the Icinga 2 status data from its REST API.
Clone the git repository somewhere applicable. You don’t need any webserver for it, Dashing uses Thin to run a simple webserver on its own.

vcsrepo { '/usr/share/dashing-icinga2':
  ensure   => 'present',
  path     => '/usr/share/dashing-icinga2',
  provider => 'git',
  revision => 'master',
  source   => 'https://github.com/Icinga/dashing-icinga2.git',
  force    => true,
  require  => Package['git']
}

Install the dashing gem

The installation might take pretty long when it tries to install the gem’s documentation files. Therefore the flags „–no-rdoc“ and „–no-ri“ ensure that this isn’t done and only the dashing gem and its dependencies are installed into the system.

exec { 'dashing-install':
  path => '/bin:/usr/bin:/sbin:/usr/sbin',
  command => "gem install --no-rdoc --no-ri dashing",
  timeout => 1800
}

Install the gems for dashing-icinga2

Next to the dashing application itself the project requires additional gems, such as a rest client for communicating with the Icinga 2 REST API (check the Gemfile for details). Additionally the bundled gems are not installed into the system’s library but locally into the dashing-icinga2 git clone underneath the „binpaths“ directory (this is to prevent conflicts with rubygem packages in the first place).

exec { 'dashing-bundle-install':
  path => '/bin:/usr/bin:/sbin:/usr/sbin',
  command => "cd /usr/share/dashing-icinga2 && bundle install --path binpaths", # use binpaths to prevent 'ruby bundler: command not found: thin'
  timeout => 1800
}

Dashing startup script

Put a small startup script somewhere executable to (re)start the Dashing application.

file { 'restart-dashing':
  name => '/usr/local/bin/restart-dashing',
  owner => root,
  group => root,
  mode => '0755',
  source => "puppet:////vagrant/files/usr/local/bin/restart-dashing",
}

Dashing runs as Thin process which puts its pid into the local tree. It is merely all about killing the process, removing the pid and then starting dashing again. „-d“ puts the process into daemonize mode (not foreground) as well as „-p 8005“ tells the application where to listen for browsers connecting to. Adjust that for your needs 🙂

#!/bin/bash
cd /usr/share/dashing-icinga2
kill -9 $(cat tmp/pids/thin.pid)
rm -f tmp/pids/thin.pid
/usr/local/bin/dashing start -d -p 8005

Now run Dashing.

exec { 'dashing-start':
  path => '/bin:/usr/bin:/sbin:/usr/sbin',
  command => "/usr/local/bin/restart-dashing",
  require => Service['icinga2'],
}

Configure the Icinga 2 API

The dashing job script just requires read-only access to the /v1/status endpoint. Being lazy I’ve just enabled everything but you should consider limited access 🙂

object ApiUser "dashing" {
  password = "icinga2ondashingr0xx"
  client_cn = NodeName
  permissions = [ "*" ]
}

Configure the Dashing job

There’s a bug in Dashing where job scripts ignore the settings from the config.ru file so there is no other way than to put the Icinga 2 REST API credentials and PKI paths directly into the jobs/icinga2.rb file.

$node_name = Socket.gethostbyname(Socket.gethostname).first
if defined? settings.icinga2_api_nodename
  node_name = settings.icinga2_api_nodename
end
#$api_url_base = "https://192.168.99.100:4665"
$api_url_base = "https://localhost:5665"
if defined? settings.icinga2_api_url
  api_url_base = settings.icinga2_api_url
end
$api_username = "dashing"
if defined? settings.icinga2_api_username
  api_username = settings.icinga2_api_username
end
$api_password = "icinga2ondashingr0xx"
if defined? settings.icinga2_api_password
  api_password = settings.icinga2_api_password
end

Modifications?

You really should know your HTML and Ruby foo before starting to modify the dashboards. The main widget used inside the dashboards/icinga2.erb file is „Simplemon“ defined as data-view attribute. It is already provided inside the dashing-icinga2 repository. data-row and data-col define the location on the dashboard matrix.

    <li data-row="2" data-col="2" data-sizex="1" data-sizey="1">
      <div data-id="icinga-host-down" data-view="Simplemon" data-title="Hosts Down"></div>
    </li>

The important part is the data-id attribute – that’s the value coming from the icinga2 job defined in jobs/icinga2.erb.
The job update interval is set to 1 second in jobs/icinga2.erb:

SCHEDULER.every '1s' do

Connecting to the Icinga 2 REST API, fetching the status data as JSON and then iterating over these dictionaries is pretty straight forward. Additional programming examples can be found inside the Icinga 2 documentation.
Take the „hosts down“ example from above:

hosts_down = status["num_hosts_down"].to_int

Now send the event to dashing by calling the send_event function providing the previosuly extracted value and the demanded color.

  send_event('icinga-host-down', {
   value: hosts_down.to_s,
   color: 'red' })

In case you’re wondering which values are fetched, let dashing run in foreground and print the „status“ dictionary to get an idea about possible keys and values. Or query the Icinga 2 REST API with your own client first.

More?

You can play around with an already pre-installed environment inside the icinga2x Vagrant box and if you’re interested in an automated setup, check the puppet provisioner manifest.
I’m fairly certain that I might improve these puppet manifests after joining the NETWAYS Puppet Practitioner & Architect trainings in February 😉 In case you’ll need your own dashboards and custom modifications, just ask 🙂