Tracing PHP memory usage with Xdebug


In addition to Xdebug’s great profiling and debugging features it also supports tracing memory usage. This helps to find functions which consume a lot of memory or to identify memory leaks. The following options enable tracing right away:

xdebug.auto_trace=1
xdebug.trace_format=1

Traces are written to /tmp by default. You may change this via the xdebug.trace_output_dir setting. With xdebug.trace_format=1 you tell Xdebug to write traces in an easy-to-parse tab separated format. The logged information includes the start and end times of function calls as well as the amount of used memory when entering and leaving the function. The last two numbers help to figure out which functions increase the memory usage a lot. Luckily you don’t have to interpret the traces yourself but use a script for that.

The script parses the trace files and aggregates the numbers by function name. It accepts a few different keys to sort the output: time-own, memory-own, time-inclusive, memory-inclusive and calls. You can also configure the number of elements to show:

php tracefile-analyser.php trace.662975268.xt memory-own 10

Showing the 10 most costly calls sorted by 'memory-own'.

                                                          Inclusive        Own
function                                          #calls  time     memory  time     memory
------------------------------------------------------------------------------------------
Icinga\Application\ClassLoader->loadClass             90  0.9151  2638176  0.0814  1262312
Zend_Loader::loadFile                                 17  0.1496  1183528  0.0345   882928
PDOStatement->execute                                 22  0.0102   395368  0.0102   395368
require                                               85  0.6066  1548112  0.0157   338512
require_once                                          22  0.0506   566296  0.0107   149128
Icinga\Application\ClassLoader->requireZendAutoloader  1  0.0045   154360  0.0023   120136
Composer\Autoload\includeFile                          8  0.0148    86544  0.0028    59448
Zend_Db_Select->_join                                 41  0.2599    59368  0.0627    57072
explode                                              135  0.0341    55672  0.0341    55672
Icinga\Application\Benchmark::measure                 94  0.2099    47448  0.0737    47352
Eric Lippmann

Autor: Eric Lippmann

Eric kam während seines ersten Lehrjahres zu NETWAYS und hat seine Ausbildung bereits 2011 sehr erfolgreich abgeschlossen. Seit Beginn arbeitet er in der Softwareentwicklung und dort an den unterschiedlichen NETWAYS Open Source Lösungen, insbesondere inGraph und im Icinga Team an Icinga Web. Darüber hinaus zeichnet er sich für viele Kundenentwicklungen in der Finanz- und Automobilbranche verantwortlich.

Event-driven, async I/O with PHP

Ever wondered whether it is possible to write asynchronous code in PHP? Good news, it is! One of the most promising libraries is ReactPHP. It provides the powerful concept of event-driven, non-blocking I/O in PHP. Its core is an event loop, on top of which it provides utilities like stream abstraction and async network clients and servers. The library is written in pure PHP and its architecture is well-suited for high performance network servers and clients. The documentation and examples of ReactPHP’s components are quite detailed so it should be no problem to get started. The following example is a simple asynchronous web server written in ReactPHP which responds with “Hello World” for every request.

$loop = React\EventLoop\Factory::create();

$server = new React\Http\Server(function (Psr\Http\Message\ServerRequestInterface $request) {
    return new React\Http\Response(
        200,
        array('Content-Type' => 'text/plain'),
        "Hello World!\n"
    );
});

$socket = new React\Socket\Server(8080, $loop);
$server->listen($socket);

echo "Server running at http://127.0.0.1:8080\n";

$loop->run();
Eric Lippmann

Autor: Eric Lippmann

Eric kam während seines ersten Lehrjahres zu NETWAYS und hat seine Ausbildung bereits 2011 sehr erfolgreich abgeschlossen. Seit Beginn arbeitet er in der Softwareentwicklung und dort an den unterschiedlichen NETWAYS Open Source Lösungen, insbesondere inGraph und im Icinga Team an Icinga Web. Darüber hinaus zeichnet er sich für viele Kundenentwicklungen in der Finanz- und Automobilbranche verantwortlich.

Filter for Multiple Group Memberships in SQL

In the upcoming Icinga Web 2 release the filter functionality becomes even more powerful.
Version 2.6.0 introduces the possibility to exclude hosts and services that are member of specific groups.
You now filter for hosts that are not part of the production host group for example.
You want to filter for hosts that are member of the host groups linux and database? That will be possible as well.

I’d like to show you how the latter is done with an example. We have a database with user groups, users and their group
memberships:

CREATE TABLE user_group (
  id int(10) unsigned NOT NULL AUTO_INCREMENT,
  `name` varchar(64) NOT NULL COLLATE utf8mb4_unicode_ci,
  PRIMARY KEY (id),
  UNIQUE KEY group_name (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE utf8mb4_bin;

CREATE TABLE user_group_membership (
  user_id int(10) unsigned NOT NULL,
  group_id int(10) unsigned NOT NULL,
  PRIMARY KEY (user_id,group_id),
  CONSTRAINT user_group_membership_user FOREIGN KEY (user_id) REFERENCES `user` (id),
  CONSTRAINT user_group_membership_group FOREIGN KEY (group_id) REFERENCES user_group (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE utf8mb4_bin;

INSERT INTO user VALUES (1,'john'),(2,'marc'),(3,'peter');
INSERT INTO user_group VALUES (1,'admins'),(3,'dev'),(2,'support');
INSERT INTO user_group_membership VALUES (1,1),(2,2),(2,3),(3,2);

User john is part of the group admins. Marc is member of the groups dev and support while Peter is part of the dev
group only. We now have the task to filter for users that are at least part of the groups dev and support. In this example
it’s easy of course because we only have three users and know the result without executing any queries.
But anyway how would you achieve this with SQL? Easy, let’s just JOIN the tables and add a WHERE IN condition:

SELECT * FROM `user` u INNER JOIN user_group_membership m ON m.user_id = u.id
INNER JOIN user_group g ON g.id = m.group_id WHERE g.name IN ('dev', 'support');

+----+-------+---------+----------+----+---------+
| id | name  | user_id | group_id | id | name    |
+----+-------+---------+----------+----+---------+
|  2 | marc  |       2 |        3 |  3 | dev     |
|  2 | marc  |       2 |        2 |  2 | support |
|  3 | peter |       3 |        2 |  2 | support |
+----+-------+---------+----------+----+---------+

The result is not surprising. Because of the WHERE IN condition we also get peter who is only part of the dev group.
So, we have to filter for users that produce two or more rows. For this aggregation, HAVING helps:

SELECT * FROM `user` u INNER JOIN user_group_membership m ON m.user_id = u.id
INNER JOIN user_group g ON g.id = m.group_id WHERE g.name IN ('dev', 'support')
HAVING(COUNT(group_id) >= 2);

+----+-------+---------+----------+----+------+
| id | name  | user_id | group_id | id | name |
+----+-------+---------+----------+----+------+
|  2 | marc  |       2 |        3 |  3 | dev  |
+----+-------+---------+----------+----+------+

Looks good! It’s best to move this to a subquery in order to be flexible if the query becomes more complex later on:

SELECT * FROM `user` u WHERE EXISTS (
  SELECT 1 FROM user_group_membership m
  INNER JOIN user_group g ON m.group_id = g.id
  WHERE m.user_id = u.id AND g.name IN ('dev', 'support')
  HAVING COUNT(*) >= 2
);

+----+-------+
| id | name  |
+----+-------+
|  2 | marc |
+----+-------+

Bonus question: how to filter for users that are member of dev and support but no other groups?

Eric Lippmann

Autor: Eric Lippmann

Eric kam während seines ersten Lehrjahres zu NETWAYS und hat seine Ausbildung bereits 2011 sehr erfolgreich abgeschlossen. Seit Beginn arbeitet er in der Softwareentwicklung und dort an den unterschiedlichen NETWAYS Open Source Lösungen, insbesondere inGraph und im Icinga Team an Icinga Web. Darüber hinaus zeichnet er sich für viele Kundenentwicklungen in der Finanz- und Automobilbranche verantwortlich.

Override Vagrant Config Locally

We are using Vagrant for most of our projects in order to provide the work environment for all people involved in the project. One of the things that we think is missing, is the option to easily override the Vagrant config locally. Developers could of course just change the Vagrantfile but this is not quite handy if it is managed via Git for example. Recently we came across the idea to include a local Vagrantfile if it exists:

Vagrant.configure("2") do |config|
  #
  # ...
  #

  if File.exists?(".Vagrantfile.local") then
    eval(IO.read(".Vagrantfile.local"), binding)
  end
end

This allows us to extend or override any Vagrant config in the file .Vagrantfile.local which developers exclude from Git. If you want to add a synced folder for example, the file could look like the following:

config.vm.synced_folder "../icingaweb2-module-director",
  "/usr/share/icingaweb2-modules/director"

config.vm.synced_folder "../icingaweb2-module-businessprocess",
  "/usr/share/icingaweb2-modules/businessprocess"
Eric Lippmann

Autor: Eric Lippmann

Eric kam während seines ersten Lehrjahres zu NETWAYS und hat seine Ausbildung bereits 2011 sehr erfolgreich abgeschlossen. Seit Beginn arbeitet er in der Softwareentwicklung und dort an den unterschiedlichen NETWAYS Open Source Lösungen, insbesondere inGraph und im Icinga Team an Icinga Web. Darüber hinaus zeichnet er sich für viele Kundenentwicklungen in der Finanz- und Automobilbranche verantwortlich.

Versteckte Perlen in Icinga Web 2

In Icinga Web 2 gibt es ein paar versteckte Parameter, die leider noch nicht dokumentiert sind aber sicher hier und da hilfreich sein können.

Entwicklermodus für JavaScript und CSS

Icinga Web 2 liefert eigentlich JavaScript und CSS komprimiert an den Client aus. Als Entwickler oder zur Fehlersuche ist aber hilfreich, die Kandidaten in ihrer Originalform auszuliefern. Dazu hängt man den Parameter _dev=1 an die URL.

Listen und Detail-Informationen exportieren

Die Listen zur Ansicht der Hosts, Services und Gruppen und der jeweiligen Detail-Bereiche lassen sich nach JSON und CSV exportieren. Dazu hängt man den format Parameter mit entweder json oder csv als Wert an, also z.B. icingaweb2/monitoring/list/services?format=json.

Vollbildmodus

Um eine Sicht in Icinga Web 2 in den Vollbildmodus zu bringen, hängt man einfach die zwei Parameter showCompact=1 und showFullscreen=1 an die URL. showCompact blendet die Kontrollelemente wie den Filter-Editor und Paginator aus und showFullscreen das Menü und den Header. Das ganz sieht dann so aus:

Eric Lippmann

Autor: Eric Lippmann

Eric kam während seines ersten Lehrjahres zu NETWAYS und hat seine Ausbildung bereits 2011 sehr erfolgreich abgeschlossen. Seit Beginn arbeitet er in der Softwareentwicklung und dort an den unterschiedlichen NETWAYS Open Source Lösungen, insbesondere inGraph und im Icinga Team an Icinga Web. Darüber hinaus zeichnet er sich für viele Kundenentwicklungen in der Finanz- und Automobilbranche verantwortlich.