May 25, 2017

A Reason to Use the Drupal Coder / PHP_CodeSniffer utility

Drupal Coder includes the command line utility PHP_CodeSniffer, which parses source code to detect violations of a coding standard.

Actually, there are two utilities in the package, one to detect violations, a second to automatically make changes for those that can be so fixed. 

I used the provided Drupal standard and ran it against all of the source code of the Optimizely module.

There are more than a hundred "sniffs" that are checked against. Individually, many are minor and relatively insignificant. A few are downright annoying. In the aggregate, though, I do feel that the resulting code was improved in terms of its readability.

The main benefit I've experienced so far is that I am being nudged into writing doc comments for all functions and classes. At first, I was resistent to doing so because good naming is often sufficient as documentation. As I edited file after file, though, I started to appreciate this kind of commenting as a desirable, consistent practice to adopt.

In one case, writing descriptions about a class and its methods helped me realize that the class was not entirely cohesive and maybe should have been written as two classes instead.

Retroactively applying the Drupal coding standard to the entire module was quite a bit of work. Moving forward, using PHP_CodeSniffer incrementally as a matter of habit should be much, much easier.

Sources:

Coder
https://www.drupal.org/project/coder

Installing Coder Sniffer
https://www.drupal.org/node/1419988

Apr 21, 2017

Using xDebug and Sublime Text with Docker

I've used xDebug with Sublime Text locally for quite some time but have started playing with Docker containers to instantiate instances of Apache, PHP, MySQL, and Drupal 8.

The Docker image I use has xDebug enabled for PHP, but I wanted to have xDebug running in the container to communicate with Sublime running on my local host system. This is not complicated, but it still took quite awhile for me to determine the correct settings.

Some of my confusion was due to the terms server and client as used in documentation and comments. Most of the time, server refers to xDebug running within PHP, and client refers to the IDE or text editor such as Sublime.

On the other hand, apparently it is xDebug that initiates the connection to the IDE, which makes xDebug act like a client. Also, xDebug has a setting called remote_host which sounds like a remote server that it is communicating with.

In the container I'm running, the xDebug settings are in

  /etc/php5/mods-available/xdebug.ini

Here are tips to get these components working together.

Run ifconfig in a local terminal to get the local ip address.

Working on my laptop connected to a home router, ifconfig shows eth0 with inet addr 10.0.0.3. I use that value as follows in xdebug.ini 

  xdebug.remote_host = 10.0.0.3

At a public library using their wi-fi, ifconfig shows wlan0 with inet addr 10.12.13.211, for example, but the address changes from session to session.

  xdebug.remote_host = 10.12.13.211

This is the key difference from running in a purely local way without Docker, where localhost is a typically used value for the remote_host setting.

Here are the settings that need to be present in xdebug.ini.

  xdebug.remote_enable = On
  xdebug.remote_host = 10.0.0.3
  xdebug.remote_port = 9000


If you have trouble getting your setup to work, use a log for xDebug to record errors and warnings. Enable the log by adding the following directive into your xdebug.ini file, e.g.

  xdebug.remote_log = /tmp/xdebug.log

This is useful for debugging. For example, at the beginning of the log there will probably be an indication of whether xDebug is even able to connect to the client, which is an important clue.

But use this key only as needed since it can generate a lot of log messages, some of which seem spurious.

For the Sublime editor, the key setting is path_mapping. Its value is an object that indicates corresponding paths. For example,

  { "/var/www/modules/custom/optimizely/": "/var/www/html/opti/modules/contrib/optimizely/" }

The key (the left side) is a path in the Docker container where the xDebug server finds its source code. Its value (the right side) is the path in the local system where Sublime finds the corresponding code.

In my use case, I am only interested in the code for the Optimizely module, so I'm only providing a mapping between the two root directories of the module.

Here are the settings that need to be present in Sublime for its xDebug package.

    "port": 9000,
    "path_mapping": { "/var/www/modules/custom/optimizely/": "/var/www/html/opti/modules/contrib/optimizely/" },

If you need to do troubleshooting, you might add the following setting as well in order for Sublime to output messages to its own local xDebug log.

    "debug": true

Also use this key only as needed since it can generate a lot of log messages, some of which seem spurious.

No port forwarding is needed.

The Docker documentation states: "By default Docker containers can make connections to the outside world ..." Since it is xDebug within the container that initiates the connection to Sublime running outside, there is no need to use port forwarding for the port between them.

Once you've got the correct settings for xdebug.ini, there are different ways to persist them. In my case, some settings don't change, but I work in different locations where the IP address for remote_host does vary, so I took a hybrid approach.

First, I used the docker commit command to capture the current state of a container into an image. In that container I had edited xdebug.ini with the settings that remain the same.

# docker commit distracted_dijkstra drupal-xdebug

Second, I use the docker run command with the -e option to provide the IP address as an environment variable when instantiating the image in a new container.

# docker run -e XDEBUG_CONFIG="remote_host=10.0.0.3" drupal-xdebug


Sources:

Xdebug 2 | Remote Debugging
https://xdebug.org/docs/remote#browser_session

martomo / SublimeTextXdebug
https://github.com/martomo/SublimeTextXdebug/blob/master/Xdebug.sublime-settings

Debug your PHP in Docker with Intellij/PHPStorm and Xdebug
https://gist.github.com/chadrien/c90927ec2d160ffea9c4

Apr 17, 2017

Initial Thoughts on Using Docker

I have started to use the Docker images from wadmiraal/drupal for local Drupal development, for example, wadmiraal/drupal:8.1.0 to use Drupal 8.1.0. These images are very well documented at Use Docker to kickstart your Drupal development.

There are other images for working with Drupal, but I happened to choose this set since it incorporates the versions of PHP, MySQL, and Apache that are close to what I've been using.

Here are some random notes on what I have experienced initially as someone who is new to Docker.

(1) Containers are easy and really fast to spin up
(At least on my Ubuntu-based system).

If you want to start completely fresh, run a new container, which means any changes you have made are lost. Sometimes, that's what you want, for example, when I want a fresh install of the module I'm working on and a clean database.

On the other hand, if you want to preserve changes to the file system of the container, you can stop it and then start the same container later. But be aware that it's really easy to accumulate clutter in the way of containers that you no longer want and have to manually remove. You can see all containers, running or not, by the command: docker ps -a

(2) There are different ways to communicate into Docker containers.

The ways I've used are port forwarding and volume mounting.

With port forwarding, when you run a container you specify which ports on the local host are passed on to a corresponding port of the container. For example, port 8080 locally can be mapped to the default port 80 of the container for http. Browsing to an address such as localhost:8080 then sends the request to the instance of the web server running in the container.

Volume mounting is a way to make local directories visible to processes running in the container. I mount my local development directory for the Optimizely module to a directory path in the container's file system under the Drupal site. This allows me to edit code locally without having to do so inside the container. Nice!

(3) Using ssh and scp

If you run the container with the appropriate port forwarding, you can ssh into the container. In the case of the image wadmiraal/drupal, once you have an ssh terminal the vi and vim editors are available.

However, other tools that you might want are not there. These can be added on the fly by using apt-get install, for example. But keep in mind that such changes will not necessarily persist, depending on how you manage the container.

If ssh is working, then so does scp for copying files back and forth between host and container. I have a one line php script that calls the function phpinfo(), which I scp from my local into the web root of the container for troubleshoot.

(4) Using xDebug and Sublime Text with Docker

The Docker image has xDebug enabled for PHP, but I'd like to have xDebug running on the container to communicate with Sublime on the local system so that I can edit and step through code locally.

So far, I am struggling to set this up. I expect to crack this nut eventually and will blog about it when I do.


Sources:

Docker Overview
https://docs.docker.com/engine/understanding-docker/

Docker Tutorial Series, Part 1: An Introduction | Docker Components
http://blog.flux7.com/blogs/docker/docker-tutorial-series-part-1-an-introduction

Use Docker to kickstart your Drupal development
http://wadmiraal.net/lore/2015/03/27/use-docker-to-kickstart-your-drupal-development/ 

Bind container ports to the host
https://docs.docker.com/engine/userguide/networking/default_network/binding/

Nov 22, 2016

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 72 bytes)

I ran into the above error message while modifying some custom Drupal code. At first, I thought I just needed to increase the memory limit due to some newly called core functions consuming more than what was allocated.

There's good discussion about different ways to change the PHP memory limit in the article  Fatal error: Allowed memory size of X bytes exhausted (tried to allocate Y bytes)...

On my local stack, one relevant file is /etc/php5/apache2/php.ini which had a limit of 512M that I had set about two years ago. That seemed like a lot already. I upped it to 640M, but got the same error at exactly the same line of code.

So I fired up a debugger to trace through. When execution reached the loop body containing the reported line where execution died (line 4 below), I single-stepped, single-stepped, single-stepped, ... and it didn't leave after the expected number of iterations.
1:  $dirname = pathinfo($current_path, PATHINFO_DIRNAME);
2:  while ($dirname && $dirname != '.') {
3:    $page['#cache']['tags'][] = 'optimizely:' . $dirname . '/*';
4:    $dirname = pathinfo($dirname, PATHINFO_DIRNAME);
5:  }
Line 3 kept adding entries to the $page array until memory was exhausted. It was my own coding mistake in inadvertently creating a runaway infinite loop.


Sources:

Fatal error: Allowed memory size of X bytes exhausted (tried to allocate Y bytes)...
https://www.drupal.org/node/76156

Nov 3, 2016

Using Drupal 8 Cache Tags for Page Caching

This is a small case study in converting how page caching is done in the D7 version of the Optimizely module to Drupal 8.

The purpose of the module is to manage the insertion of certain <script> elements into designated pages of a site. To do so, the user creates one or more projects. Each project has one or more url paths. When a project is enabled, all pages that match one of its paths have the <script> element added.

To specify project paths, use of a trailing * wildcard is allowed, as are special page designators. For example, these are all valid paths.

/node/2
/node/*
/admin/config/system
/admin/*
*
<front>

In the case of /admin/* it would match against any of

/admin/
/admin/config/
/admin/config/system/
/admin/people
/admin/people/create
  ...

In the D7 version, invalidating is done through calls to cache_clear_all(), which takes three parameters. The function is used by the module in three different ways.

(1) To invalidate a particular page, e.g.

    cache_clear_all('/node/2', 'cache_page', FALSE);

(2) To invalidate a path with a trailing wildcard, e.g.

    cache_clear_all('/node/*', 'cache_page', TRUE);

(3) To invalidate all pages of the site,

    cache_clear_all('*', 'cache_page', TRUE);

In Drupal 8 the Cache API is completely different. Function cache_clear_all() has disappeared.

After some research, it looked like using cache tags would be the way to go. The article  Cacheability of render arrays  was especially helpful in how to think about caching as applied to page rendering.

There are two facets to implementing this. The first is what needs to be done when a page is rendered, the second is what to invalidate when triggering changes occur.

For page rendering, I already had an implementation of hook_page_attachments() that checked for inserting the element into any page whose path matched against any of the enabled project paths.

This hook function is where cache tags could be added to the page. This turned out to be a little tricky. A page must be invalidated for two different use cases: when the page contains the element which now needs to be removed, and when the page does not contain the element but it now needs to be added.

For the first case, I decided to just use the matching project path act as the cache tag (there can only be one because overlapping project paths are not allowed).

But for the second case, I had to cover all the possible project paths that might be enabled in the future. So, for example, for the page at /node/2 there are three such possible paths.

  /node/2
  /node/*
  *

For every page rendered, it was necessary to attach all of these possible project paths as cache tags. Here is a snippet of code that shows how this is done in hook_page_attachments().

  // Site-wide wildcard.
  $page['#cache']['tags'][] = 'optimizely:*';

  // Non site-wide wildcards. Repeat for every directory level.
  $dirname = pathinfo($current_path, PATHINFO_DIRNAME);
  while ($dirname && $dirname != '/') {
    $page['#cache']['tags'][] = 'optimizely:' . $dirname . '/*';
    $dirname = pathinfo($dirname, PATHINFO_DIRNAME);
  }

  // The specific page url.
  $page['#cache']['tags'][] = 'optimizely:' . $current_path;


  // Finally, if there is an alias for the page, tag it.
  if ($current_path_alias) {
    $page['#cache']['tags'][] = 'optimizely:' . $current_path_alias;
  }


The optimizely: prefix follows the convention of prefixing a group name as part of the tag where appropriate. For example, cache tags from core include node:1 and config:node.type.article.

Finally, there is the matter of what and how to invalidate when changes to the projects and their paths are submitted. This turned out to be fairly easy to implement.

An array of all relevant project paths is passed to a function that carries out the following.

  $cache_tags = [];
  foreach ($path_array as $path) {
    $cache_tags[] = 'optimizely:' . $path;
  }
 

  \Drupal::service('cache_tags.invalidator')->invalidateTags($cache_tags);



For debugging purposes, outputting X-Drupal-Cache-Tags in HTTP headers was extremely useful. See my earlier post  Enable and Use X-Drupal-Cache-Tags in HTTP headers.

This post is about the caching that is done by Drupal itself. The D7 version also checks for the presence of the varnish module and calls a function of that module if it exists.  I did not pursue a replacement for that functionality, but the articles  Varnish  and  Use Drupal 8 Cache Tags with Varnish and Purge  look promising.


Sources:

Function cache_clear_all() has been removed
https://optimizely-to-drupal-8.blogspot.com/2014/07/function-cacheclearall-has-been-removed.html

Cacheability of render arrays
https://www.drupal.org/developing/api/8/render/arrays/cacheability

Cache tags
https://www.drupal.org/developing/api/8/cache/tags

Allow to set #cache metadata in hook_page_attachments() https://www.drupal.org/node/2475749

public static function Cache::invalidateTags
https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Cache!Cache.php/function/Cache%3A%3AinvalidateTags/8