Running Icinga in NWS with Slack notifications

Slack notifications through Icinga2. This is what we activated last week for our Icinga2 Master apps on our NWS platform! The feature came highly recommended, so we decided to give it a try. And we did. It really is awesome!!

First of all, I want to show you how it will look like on your side, so have a look at the small demo-video!


To work with this feature, all you need is a Slack workspace and a chatroom for your alerts. once you are done with configuration part, just follow the 7 steps below.

  1. Go to Configuration/Commands in your Icinga2 app
  2. Open command-slack-host/command-slack-service and open the drop-down Custom properties
  3. Fill in the slack_channel you want to use
  4. Fill in the slack_webhook_url (You can get your webhook url from your Slack-account settings)
  5. Create a new user in Configuration/User and add user to the two groups for Slack-Message on critical hosts/services. Also give user the user template user-template in Imports
  6. Add the states you want to receive as a notifications for to the Configuration / User/Contacts / User-template / modify / State and transition type filtersfield
  7. Deploy your changes in the Configuration/Deployments

If you have ideas for more features for our apps, just contact us via email, twitter, facebook or our NWS chat!


This slideshow requires JavaScript.

Marius Gebert


Marius ist seit September 2013 bei uns beschäftigt. Er hat im Sommer 2016 seine Ausbildung zum Fachinformatiker für Systemintegration absolviert und kümmert sich nun um den Support unserer Hostingkunden. Seine besonderen Themengebiete erstrecken sich vom Elastic-Stack bis hin zu Puppet. Auch an unserem Lunchshop ist er ständig zu Gange und versorgt die ganze Firma mit kulinarischen Köstlichkeiten. Seine Freizeit verbringt Marius gern an der frischen Luft und wandert dabei durch die fränkische Schweiz

NETWAYS Web Services: WordPress now up and running!

This entry is part of 1 in the series NWS

We are proud to announce a new app hosted in our NWS platform! WordPress is now available!

“Simply for everyone – Perfect for everyone who wants to create individual content. Simple and safe.”

Just as our slogan for this app tells you, we decided to create an app for our users, which can be used by anybody. An app, which is ready to use, easy to configure and practically for almost everyone, without writing code or configuring credentials etc.

We built up an automated migration program, so you can migrate your existing WordPress instance to our platform, no matter which version you are running. We also decided to give the users the opportunity to restore their website by their self and to manage their website without the need of one of our WordPress experts.

The WordPress app includes our S3 compatible replication-based and distributed storage.

All in all it will bring you the following enhanced functions to your cloud:

  • Automated Updates
  • Easy migration
  • Super fast assests delivery via S3
  • Full domain freedom / No domain needed
  • Free CNAME
  • It is available immediately
  • Automated Backups, which are stored for 24 hours
  • Preinstalled plugins and themes
  • Access to DocRoot / Backup via WebDAV

Give it a try! If you are new to our platform, you can use it 30 days for free. The app is of course monthly callable but we think you will like it!


This slideshow requires JavaScript.

Marius Gebert


Marius ist seit September 2013 bei uns beschäftigt. Er hat im Sommer 2016 seine Ausbildung zum Fachinformatiker für Systemintegration absolviert und kümmert sich nun um den Support unserer Hostingkunden. Seine besonderen Themengebiete erstrecken sich vom Elastic-Stack bis hin zu Puppet. Auch an unserem Lunchshop ist er ständig zu Gange und versorgt die ganze Firma mit kulinarischen Köstlichkeiten. Seine Freizeit verbringt Marius gern an der frischen Luft und wandert dabei durch die fränkische Schweiz

Gitlab now supports Let’s Encrypt

Since last week our Gitlab-ce and Gitlab-ee instances are able to use Let’s Encrypt for SSL encryption. As an owner of one of our instances, you are able to use Let’s Encrypt simply by activating it in your product view on our NWS platform.

With this, you can now use your own domain, without the need of an existing SSL-certificate. If you already have a SSL-certificate active and want to test Let’s Encrypt, you can do so. Your active certificate will be stored and will be activated again, as soon as you deactivate the SSL encryption with Let’s Encrypt.

In the screenshot below, you can see an example of how it looks in the product view. Activation and deactivation will always require a restart of your instances, since these are major configurations changes to your container.

If you are interested in one of our instances, just have a look on! We have many more open-source apps available, such as Rocket.Chat, RT or Nextcloud and are currently working on some new features/apps.

Marius Gebert


Marius ist seit September 2013 bei uns beschäftigt. Er hat im Sommer 2016 seine Ausbildung zum Fachinformatiker für Systemintegration absolviert und kümmert sich nun um den Support unserer Hostingkunden. Seine besonderen Themengebiete erstrecken sich vom Elastic-Stack bis hin zu Puppet. Auch an unserem Lunchshop ist er ständig zu Gange und versorgt die ganze Firma mit kulinarischen Köstlichkeiten. Seine Freizeit verbringt Marius gern an der frischen Luft und wandert dabei durch die fränkische Schweiz

Obstacles when setting up Mesos/Marathon

Obstacles when setting up Mesos/Marathon

Sebastian has already mentioned Mesos some time ago, now it’s time to have a more practical look into this framework.

We’re currently running our NWS Platform under Mesos/Marathon and are quite happy with it. Sebastians talk at last years OSDC can give you a deeper insight into our setup. We started migrating our internal coreos/etcd/fleetctl setup to Mesos with Docker and also could provide some of our customers with a new setup.

Before I will give you a short description about snares I ran into during the migration, let’s have a quick overview on how Mesos works. We will have a look at Zookeeper, Mesos, Marathon and Docker.

Zookeeper acts as centralized key-value store for the Mesos cluster and as such has to be installed on both the Mesos-Master and -Slaves

Mesos is a distributed system kernel and runs on the Mesos-Masters and Slaves. The Masters distribute jobs and workload to the slaves and therefore need to know about their available ressources, e.g. RAM and CPU

Marathon is used for orchestration of docker containers and can access on information provided by Mesos.

Docker is one way to run containerized applications and used in our setup.

As we can see, there are several programs running simultaneously which creates needs for seamless integration.

What are obstacles you might run into when setting up your own cluster?

1. Connectivity:

When you set up e.g. different VMs to run your cluster, please make sure they are connected to each other. Which might look simple, can become frustrating when the Zookeeper nodes can’t find each other due to “wrong” etc/hosts settings, such as  localhost

This should be altered to $hostname, e.g. mesos-slave1

2. Configuration

Whenever you make changes to your configuration, it has to be communicated through your complete cluster. Sometimes it doesn’t even a need a service restart. Sometimes you may need to reboot. In desperate times you might want to purge packages and reinstall them. In the end it will work and you will happily run into


3. Bugs

While Marathon provides you with an easy to use Web-UI to interact with your containers, it has one great flaw in the current version. As the behaviour is so random, you could tend to search for issues in your setup.  You might or might not be able to make live changes to your configured containers. Worry not, the “solution” may be simply using an older version of Marathon.

Version 1.4.8 may help.


Have fun setting up your own cluster and avoiding annoying obstacles!

Edit 20180131 TA: fixed minor typo

Tim Albert

Autor: Tim Albert

Tim kommt aus einem kleinen Ort zwischen Nürnberg und Ansbach, an der malerischen B14 gelegen. Er hat in Erlangen Lehramt und in Koblenz Informationsmanagement studiert, wobei seine Tätigkeit als Werkstudent bei IDS Scheer seinen Schwenk von Lehramt zur IT erheblich beeinflusst hat. Neben dem Studium hat Tim sich außerdem noch bei einer Werkskundendienstfirma im User-Support verdingt. Blerim und Sebastian haben ihn Anfang 2016 zu uns ins Managed Services Team geholt, wo er sich nun insbesondere um Infrastrukturthemen kümmert. In seiner Freizeit engagiert sich Tim in der Freiwilligen Feuerwehr - als Maschinist und Atemschutzgeräteträger -, spielt im Laientheater Bauernschwänke und ist auch handwerklich ein absolutes Allroundtalent. Angefangen von Mauern hochziehen bis hin zur KNX-Verkabelung ist er jederzeit einsatzbereit. Ansonsten kocht er sehr gerne – alles außer Hase!

Modern open source community platforms with Discourse

Investing into open source communities is key here at NETWAYS. We do a lot of things in the open, encourage users with open source trainings and also be part of many communities with help and code, be it Icinga, Puppet, Elastic, Graylog, etc.

Open source with additional business services as we love and do only works if the community is strong, and pushes your project to the next level. Then it is totally ok to say “I don’t have the time to investigate on your problem now, how about some remote support from professionals?”. Still, this requires a civil discussion platform where such conversations can evolve.

One key factor of an open source community is to encourage users to learn from you. Show them your appreciation and they will like it and start helping others as you do. Be a role model and help others on a technical level, that’s my definition of a community manager. Add ideas and propose changes and new things. Invest time and make things easier for your community.

I’ve been building a new platform for based on Discourse in the last couple of days. The old platform based on Woltlab was old-fashioned, hard to maintain, and it wasn’t easy to help everyone. It also was closed source with an extra license, so feature requests were hard for an open source guy like me.

Discourse on the other hand is 100% open source, has ~24k Github stars and a helping community. It has been created by the inventors of StackOverflow, building a conversation platform for the next decade. Is is fast, modern, beautiful and both easy to install and use.


Setup as Container

Discourse only supports running inside Docker. The simplest approach is to build everything into one container, but one can split this up too. Since I am just a beginner, I decided to go for the simple all-in-one solution. Last week I was already using the 1.9.0beta17, running really stable. Today they released 1.9.0, I’ll share some of the fancy things below already 🙂

Start on a fresh VM where no applications are listening on port 80/443. You’ll also need to have a mail server around which accepts mails via SMTP. Docker must be installed in the latest version from the Docker repos, don’t use what the distribution provides (Ubuntu 14.04 LTS here).

mkdir /var/discourse
git clone /var/discourse

cd /var/discourse

The setup wizard ask several questions to configure the basic setup. I’ve chosen to use as hostname, and provided my SMTP server host and credentials. I’ve also set my personal mail address as contact. Once everything succeeds, the configuration is stored in /var/discourse/container/app.yml.


Nginx Proxy

My requirement was to not only serve Discourse at /, but also have redirects for other web applications (the old Woltlab based forum for example). Furthermore I want to configure the SSL certificates in a central place. Therefore I’ve been following the instructions to connect Discourse to a unix socket via Nginx.

apt-get install nginx

rm /etc/nginx/sites-enabled/default

vim /etc/nginx/sites-available/proxy.conf
server {
    listen 443 ssl;  listen [::]:443 ssl;

    ssl on;
    ssl_certificate      /etc/nginx/ssl/;
    ssl_certificate_key  /etc/nginx/ssl/;

    ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
    ssl_prefer_server_ciphers on;

    # openssl dhparam -out /etc/nginx/ssl/dhparam.pem 4096
    ssl_dhparam /etc/nginx/ssl/dhparam.pem;

    # OCSP Stapling ---
    # fetch OCSP records from URL in ssl_certificate and cache them
    ssl_stapling on;
    ssl_stapling_verify on;

    location / {
        error_page 502 =502 /errorpages/discourse_offline.html;
        proxy_intercept_errors on;
        # Requires containers/app.yml to use websockets
        proxy_pass http://unix:/var/discourse/shared/standalone/nginx.http.sock:;
        proxy_set_header Host $http_host;
        proxy_http_version 1.1;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
ln -s /etc/nginx/sites-available/proxy.conf /etc/nginx/sites-enabled/proxy.conf
service nginx restart

Another bonus of such a proxy is to have a maintenance page without an ugly gateway error.

The full configuration can be found here.



Installation is a breeze – just add the installation calls into the app.yml file and rebuild the container.

# egrep -v "^$|#" /var/discourse/containers/app.yml
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
  db_default_text_search_config: "pg_catalog.english"
  LANG: en_US.UTF-8
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log
    - exec:
        cd: $home/plugins
          - git clone
          - git clone
          - git clone
  - exec: echo "Beginning of custom commands"
  - exec: echo "End of custom commands"

./launcher rebuild app

Akismet checks against spam posts as you know it from WordPress. We’ve learned that spammers easily crack reCaptcha, and the only reliable way is filtering the actual posts.

The second useful plugin is for accepting an answer in a topic, marking it as solved. This is really useful if your platform is primarily used for Q&A topics.


Getting Started

Once everything is up and running, navigate to your domain in your browser. The simple setup wizard greets you with some basic questions. Proceed as you like, and then you are ready to build the platform for your own needs.

The admin interface has lots of options. Don’t fear it – many of the default settings are from best practices, and you can always restore them if you made a mistake. There’s also a filter to only list overridden options 🙂

Categories and Tags

Some organisation and structure is needed. The old-fashioned way of choosing a sub forum and adding a topic in there is gone. Still Discourse offers you to require a category from users. Think of monitoring – a question on the Icinga Director should be highlighted in a specific category to allow others to catch up.

By the way – you can subscribe to notifications for a specific category. This helps to keep track only for Icinga related categories for example.

In addition to that, tags help to refine the topics and make them easier to search for.

Communication matters

There are so many goodies. First off, you can start a new topic just from the start page. An overlay page which saves the session (!) is here for you to edit. Start typing Markdown, and see it pre-rendered live on the right side.

You can upload images, or paste an URL. Discourse will trigger a job to download this later and use a local cache. This avoids broken images in the future. If you paste a web link, Discourse tries to render a preview in “onebox”. This also renders Github URLs with code preview.

Add emotions to your discussion, appreciate posts by others and like them, enjoy the conversation and share it online. You can even save your draft and edit it amongst different sessions, e.g. after going home.


Tutorials, Trust Level and Rewards

Once you register a new account (you can add oauth apps from Twitter, Github, etc.!), a learning bot greets you. This interactive tutorial helps you learning the basics with likes, quotes, urls, uploads, and rewards you with a nice certificate in the end.

New users don’t start with full permissions, they need to earn their trust. Once they proceed with engaging with the community, their trust level is raised. The idea behind this is not to have moderators and admins regulating the conversation, but let experienced members to it. Sort of self healing if something goes wrong.

Users who really engage and help are able to earn so-called badges. These nifty rewards are highlighted on their profile page, e.g. for likes, number of replies, shared topics, even accepted solutions for questions. A pure motivation plaything built into this nice piece of open source software.


Wiki and Solved Topics

You can change topics to wiki entries. Everyone can edit them, this way you’ll combine the easiness of writing things in Markdown with a full-blown documentation wiki.

Accepting a replay as solution marks a topic as “solved”. This is incredibly helpful for others who had the same problem.



As an administrator, you’ll get automated page profiling for free. This includes explained SQL queries, measured page load time, and even flame graphs.

If you ever need to reschedule a job, e.g. for daily badge creation, admins can access the Sidekiq web UI which really is just awesome.

Plugin development seems also easy if you know Ruby and EmberJS. There are many official plugins around which tested before each release.

Discourse also has a rich REST API. Even a monitoring endpoint.



You can create backups on-demand in addition to regular intervals. You can even restore an old backup directly from the UI.



Discourse is used by many communities all over the world – Graylog, Elastic, Gitlab, Docker, Grafana, … have chosen to use the power of a great discussion platform. So does as a #monitoringlove community. A huge thank you to the Discourse team, your software is pure magic and just awesome 🙂

My journey in building a new community forum from scratch in just 5 days can be read here 🙂 running Discourse is fully hosted at NETWAYS, including SSL certificates, Puppet deployment and Icinga for monitoring. Everything I need to build an awesome community platform. You too?


Michael Friedrich

Autor: Michael Friedrich

Michael ist seit vielen Jahren Icinga Developer und hat sich Ende 2012 in das Abenteuer NETWAYS gewagt. Ein Umzug von Wien nach Nürnberg mit der Vorliebe, österreichische Köstlichkeiten zu importieren - so mancher Kollege verzweifelt an den süchtig machenden Dragee-Keksi. Oder schlicht am österreichischen Dialekt der gerne mit Thomas im Büro intensiviert wird ("Jo eh."). Wenn sich Michael mal nicht im Monitoring-Portal helfend meldet, arbeitet er am nächsten LEGO-Projekt oder geniesst das schöne Nürnberg. Oder - at an Icinga Camp near you 😉

“Willkommen auf der dunklen Seite der Macht, Alexander …”

Nein, ich meine nicht die antagonistische Fraktion aus einer sehr populären Filmreihe (deren Name nicht genannt werden muss), sondern die Apple-Fraktion bei NETWAYS.

Letzten Monat habe ich nämlich meine Ausbildung zum Fachinformatiker für Anwendungsentwicklung abgeschlossen und wurde in die Development-Abteilung übernommen.

Neuer Status – neues Gehalt, neue Aufgaben … und neues Arbeitsgerät. Ich habe mich bewusst für ein MacBook Pro entschieden, weil auf dieser Plattform sehr vieles viel besser und einfacher funktioniert und ich nicht bei jeder Kleinigkeit erst den Linux-Kernel patchen und neu kompilieren, SELinux abschalten oder die IPTables-Regeln anpassen muss. 😉

Wer weiß wann ich mit dem nächsten Kunden von Angesicht zu Angesicht zu tun haben werde – diesbzgl. will ich keine Risiken eingehen.

Abstriche muss ich keine machen – schließlich bietet MacOS X alles, was das Entwickler-Herz begehrt. Außer vielleicht …


Auf Linux haben sie sich breit gemacht wie ein Türsteher: Docker, LXC, LXD und wie sie alle heißen. Auch BSD macht mit seinen Jails keine Kompromisse in Sachen Virtualisierung und Sicherheit.

Umso mehr wundert es mich, dass das BSD-basierende MacOS X dieses Feature nicht übernommen hat. Aber es wäre kein *nix-System, wenn man das nicht mit ein wenig Geschick nachrüsten könnte …

Dann eben so …

Auf meinem alten Arbeitsgerät habe ich Ubuntu 16.04 verwendet. Diese GNU/Linux-Distribution enthält von Haus aus LXD in den Paketquellen, womit ich in einer leichtgewichtigen und sicher isolierten Umgebung mal schnell was ausprobieren konnte.

Dieses Werkzeug kann ich mit einer Virtuellen Maschine problemlos weiterhin verwenden – nicht nur in der VM selbst, sondern auch vom Mac-Host aus. Genau dafür braucht man das o. g. “wenig Geschick” …

LXD Server einrichten

Man logge sich in die VM ein und erlange Administratorrechte. Dann installiert man LXD und richtet diesen darauf hin ein:

$ sudo -i
# apt install lxd
# lxd init
Name of the storage backend to use (dir or zfs) [default=dir]: 
Would you like LXD to be available over the network (yes/no) [default=no]? yes
Address to bind LXD to (not including port) [default=all]: 
Port to bind LXD to [default=8443]: 
Trust password for new clients: 
Do you want to configure the LXD bridge (yes/no) [default=yes]?

Fast alle Abfragen des Installations-Assistenten können bedenkenlos mit der Eingabetaste bestätigt werden. Die von mir hervorgehobene Frage allerdings muss mit “yes” beantwortet werden – ansonsten fällt die Nutzung vom Host aus ins Wasser. Außerdem muss ein einigermaßen sicheres Passwort gewählt werden.

Der Frage nach der “LXD bridge” folgt ein weiterer, “pseudo-grafischer” Installations-Assistent. Dessen Fragen kann man ausnahmslos mit der Eingabetaste bestätigen.

Des weiteren wird später die IP der VM benötigt:

# ifconfig
enp0s5    Link encap:Ethernet  HWaddr 00:1c:42:8b:e9:ba  
          inet addr:  Bcast:  Mask:
          inet6 addr: fdb2:2c26:f4e4:0:880a:a527:1a6:1130/64 Scope:Global
          inet6 addr: fdb2:2c26:f4e4:0:fc54:870c:56af:cba0/64 Scope:Global
          inet6 addr: fe80::fbc7:ab4d:d29f:e227/64 Scope:Link

lo        Link encap:Local Loopback  
          inet addr:  Mask:
          inet6 addr: ::1/128 Scope:Host

lxdbr0    Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          inet addr:  Bcast:  Mask:
          inet6 addr: fdb3:61e3:ffb6:f62f::1/64 Scope:Global
          inet6 addr: fe80::4024:c9ff:fe4a:691b/64 Scope:Link

LXD Client einrichten

Da die Entwickler von LXD sich bemüht haben, nicht von Linux-spezifischen Funktionalitäten Gebrauch zu machen, funktioniert der LXD Client auch auf MacOS. Den muss man nur noch vom LXD Server in Kenntnis setzen.

Sobald das getan ist, kann auch schon der erste Container in Betrieb genommen werden.

$ brew install lxc
$ lxc remote add u1604vm
Certificate fingerprint: a761f551dffdf47d9145da2aa16b4f6be242d960ce1697994c969e495b0724fd
ok (y/n)? y
Admin password for u1604vm:
Client certificate stored at server:  u1604vm
$ lxc launch ubuntu:16.04 u1604vm:test1
Creating test1
Starting test1ge: rootfs: 100% (37.78MB/s)
$ lxc exec u1604vm:test1 -- bash
root@test1:~# man perlfunc


Es muss nicht immer Docker sein.

Gerade wenn das Testsystem sich möglichst so verhalten soll wie eine VM oder eine Physische Maschine ist LXD auf jeden Fall einen Blick Wert – auch auf dem Mac.

Alexander Klimov

Autor: Alexander Klimov

Alexander hat 2017 seine Ausbildung zum Developer bei NETWAYS erfolgreich abgeschlossen. Als leidenschaftlicher Programmierer und begeisterter Anhänger der Idee freier Software, hat er sich dabei innerhalb kürzester Zeit in die Herzen seiner Kollegen im Development geschlichen. Wäre nicht ausgerechnet Gandhi sein Vorbild, würde er von dort aus daran arbeiten, seinen geheimen Plan, erst die Abteilung und dann die Weltherrschaft an sich zu reißen, zu realisieren - tut er aber nicht. Stattdessen beschreitet er mit der Arbeit an Icinga Web 2 bei uns friedliche Wege.