Jul 30, 2014

Automated testing a jQuery / Ajax call

While converting some tests to class WebTestBase in Drupal 8, I came across an attempt to test a jQuery call for which the response was an object encoded in JSON.

Here is the solution I came up with for automated checking of this functionality. It does not use the jQuery code itself but mimics such a call to the server.

(1)  The normal jQuery / Ajax code that sets up the call.

var params = "target_oid=" + 7 + "&target_enable=" + 1;

$.ajax({
  'url': drupalSettings.path.basePath + 'ajax/optimizely',
  'type': 'POST',
  'dataType': 'json',
  'data': params,
  'success': function(data) {
    // Check data.status, update the DOM, ...
  },
  // Other properties ...
});


(2)  The server-side code that responds.

  $target_oid = $_POST['target_oid'];
  $target_enable = $_POST['target_enable'];


  // Process the request ...

  $options = array('status' => 'Updated', 'message' => 'Good job');
  return new JsonResponse($options);


(3)  The code in a WebTestBase test method to mimic the call and check the results.

class OptimizelyDefaultProjectTest extends WebTestBase {
 

  public function testListingAjax() {
 

    $params = array('target_oid' => 7, 'target_enable' => 1);
    $response = $this->drupalPost('ajax/optimizely',

                                  'application/json', 
                                   $params);
  

    $resp_obj = json_decode($response);

    $this->assertEqual($resp_obj->status, 'Updated', ...);
    $this->assertEqual($resp_obj->message, 'Good job', ...);

  }
}


N.B. WebTestBase::drupalPost() in Drupal 8 is not the same as DrupalWebTestCase::drupalPost() in Drupal 7.

An approach with better coverage would be to load the form and simulate the user action that triggers the Ajax call, which, in this case, is clicking on a checkbox.  That would test not only the server code but also the jQuery code on the client.

But I will save that problem for another day. 8-D


Update: see  http://optimizely-to-drupal-8.blogspot.com/2015/01/beta-4-drupalmatchpath-and.html


Sources:

protected function WebTestBase::drupalPost
https://api.drupal.org/api/drupal/core!modules!simpletest!src!WebTestBase.php/function/WebTestBase%3A%3AdrupalPost/8

json_decode
http://php.net/manual/en/function.json-decode.php

Jul 28, 2014

Use assertPattern() for testing

While converting automated tests to class WebTestBase in Drupal 8, several of the assertions were failing. This was happening in spite of the fact that none of the theme templates, element ids, or other relevant code had changed.

These assertions had been implemented with assertRaw(), which does literal string matching against the raw HTML of the response pages. It turns out that in D8, some form elements may be generated differently even though the final rendering is pretty much the same.

As a slightly simplified example, here's a tag that is output by the Drupal 7 version of the module.

<input disabled="disabled" type="text" id="edit-optimizely-project-code" name="optimizely_project_code" value="Undefined" size="30" />

But in D8, the corresponding tag is

<input aria-describedby="edit-optimizely-project-code--description" disabled="disabled" type="text" id="edit-optimizely-project-code" name="optimizely_project_code" value="Undefined" size="30" aria-required="true" />

Obviously, these two strings don't match.

The intent of the assertion is to check that the particular input tag exists, that it is disabled, and that it has a value of "Undefined". The other attributes are irrelevant.

So I re-wrote the assertion using assertPattern() instead, which takes a regular expression as its first parameter.

$this->assertPattern(
  ':<input .*? disabled="disabled" .*? id="edit-optimizely-project-code" .*? value="Undefined" .*? />:', ...);

This is not perfect. For one thing, it still relies on the three attributes being in a certain order. It also has some extra space characters that could break the match, but I'm leaving them here to make the regex more readable.

However, I believe implementing with assertPattern() makes the test more robust in the face of future changes as well as making the code better self-documenting.

Sources:

Assertions
https://www.drupal.org/node/265828

abstract class WebTestBase
https://api.drupal.org/api/drupal/core!modules!simpletest!src!WebTestBase.php/class/WebTestBase/8

Jul 26, 2014

Implementing an Ajax call on form checkboxes

On the Project Listing page of the Optimizely module, each project has a checkbox that the user can use to enable or disable the project. The handling of that user action is implemented by use of an Ajax call back to the server to validate the action and to update the database if the action is accepted. The server then returns the appropriate response.

The D7 code from hook_menu() is,

  $items['ajax/optimizely'] = array(
    'title' => 'Optimizely Administer AJAX',
    'page callback' => 'optimizely_ajax_enable',
    'access arguments' => array('administer optimizely'),
    'file' => 'optimizely.admin.inc',
    'file path' => drupal_get_path('module', 'optimizely'),
    'type' => MENU_CALLBACK,
  );


To do the conversion to Drupal 8, first, there was the need to define a route in the optimizely.routing.yml file to replace the above entry in hook_menu(). This route is similar to the ones that were defined for menu pages but there are critical differences.

The most important difference is that instead of the _form or the _content property, the _controller property is used. Whatever is returned by the indicated method is then returned as is without further processing. In this case it is a Json response to the Ajax call.

The other difference is that the _format property specifies that the request must be in Json.

ajax.enable:
  path: /ajax/optimizely
  defaults:
    _controller: \Drupal\optimizely\AjaxEnable::enableDisable
    _title: Optimizely Administer AJAX
  requirements:
    _permission: administer optimizely
    _format: json



Second, implementing the method to be called. The conversion of the code to Drupal 8 required replacing calls to drupal_json_output() with instantiating instances of class JsonResponse. The arguments remain the same.

namespace Drupal\optimizely;

use Symfony\Component\HttpFoundation\JsonResponse;
 

class AjaxEnable {

  public static function enableDisable() {


    // Code to validate the user's action, 
    // update the database, etc.

    if (. . . ) {


     $options = array('status' => 'updated', 
                        'oid' => $target_oid,
                        'target_enable' => $target_enable,
                        'message' => $message);

      return new JsonResponse($options);
    }
    else {
      $options = array('status' => 'rejected', 

                        'oid' => $target_oid,
                        'issue_path' => $target_path,
                        'message' => $message);

      return new JsonResponse($options);
    }



Finally, there was an edit that had to be made to the jQuery code on the client side. Drupal.settings is now drupalSettings, and the following line, 

    'url': Drupal.settings.basePath + 'ajax/optimizely',

was replaced with,

    'url': drupalSettings.path.basePath + 'ajax/optimizely',


Update: see  http://optimizely-to-drupal-8.blogspot.com/2015/01/beta-4-drupalmatchpath-and.html

Sources:

Routing system in Drupal 8
https://www.drupal.org/node/2122071

Structure of routes
https://www.drupal.org/node/2092643

drupal_json_output is removed in favor of Symfony\Component\HttpFoundation\JsonResponse
https://www.drupal.org/node/1665684

Ajax framework
https://api.drupal.org/api/drupal/core%21core.api.php/group/ajax/8.0.x
This is a good explanation of an Ajax framework that is provided in D8. 

replace all occurence of Drupal.settings with drupalSettings
https://www.drupal.org/node/1793648

Jul 21, 2014

Ajax request fails with 404 Not Found

When I started to convert a jQuery Ajax call that is used in the Drupal 7 Optimizely module, I found that it did not work in my local server environment, which consists of Apache 2.2 under Windows 7. The call was failing with a 404 Not Found error.

Using the Firebug extension for Firefox, I was able to see that the url for the Ajax request was missing the site root portion of the path. Specifically, the correct url is http://localhost/opti/ajax/optimizely where opti is the site root, but what was requested was http://localhost/ajax/optimizely . On this local system I have a number of different sites in subdirectories under the web server root.

I poked around in Apache to no avail, adding a RewriteBase directive in the site's .htaccess file.

I edited Drupal's settings.php to assign a value to $base_url, also with no effect.

Eventually, I stumbled on a casual side comment in the article cited below. It sounded like the same problem I was having, and it suggested the prepending of Drupal.settings.basePath in the url.

That solved the problem, as in the second line of code below.

$.ajax({
  'url': Drupal.settings.basePath + 'ajax/optimizely',
  'type': 'POST',
  'dataType': 'json',
  'data': post,

  // Other code ...

where the original line was,

  'url': '/ajax/optimizely',


I confirmed this issue and this particular fix on a fresh installation of Drupal 7.29 and the Optimizely module 2.14.

However, the original javascript code has been working elsewhere. I was concerned that this change would break the functionality in other server environments. So I tested on a Linux Mint 16 system with Apache 2.4. The ajax call also works there, which gives me some confidence that the fix is broadly usable without introducing a regression.

N.B. Drupal.settings.basePath provides a path that has a trailing slash character. There was one place in the code where it was necessary not to have a beginning slash in the string literal 'ajax/optimizely' even though the resulting doubled slash worked fine in other cases.

N.B. For this ajax call to work with Apache, the mod_rewrite module must be enabled. I had to manually do this since the module was not enabled by default when I installed Apache 2.4.


Source:

Ajax in Drupal using jQuery
https://www.drupal.org/node/305747

Jul 14, 2014

Building a delete confirmation form with a wildcard in its path

When the user clicks on a link to delete an Optimizely project, a confirmation form is returned with buttons to Delete or to Cancel.

The url for this delete confirmation form has a parameter as the last part of the path, which is the unique integer id for the project. So it's structurally very similar to the update form that was documented in the previous post  http://optimizely-to-drupal-8.blogspot.com/2014/06/take-two-building-form-using-route-path.html

Unlike the update form, I was able to implement the delete confirmation in a simpler way. It was not necessary to define an intermediate method to accept the project id as a parameter and then pass the id along as part of a further call.

(1)  Defined a route in the optimizely.routing.yml file:

optimizely.delete.oid:
  path: /admin/config/system/optimizely/delete/{oid}
  defaults:
    _form: \Drupal\optimizely\DeleteForm
    _title: Optimizely Delete Project
  requirements:
    _permission: administer optimizely


Note the use of the {oid} wildcard in the path, which represents the project id.

For this route I was able to use the _form property rather than the _content property. For reasons not apparent to me, because class DeleteForm is derived from ConfirmFormBase rather than FormBase, passing the value of {oid} is done as I would hope. See buildForm() below and its optional third parameter.


(2)  Defined class DeleteForm (with comments removed here; other abbreviations also made).

namespace Drupal\optimizely;

use Drupal\Core\Form\ConfirmFormBase;

class DeleteForm extends ConfirmFormBase {

  private $oid = NULL;

  public function getFormId() {
    return 'optimizely-delete-page-confirm';
  }

  public function getQuestion() {
    return $this->t('Delete');  // Actually, a heading.
  }

  public function getDescription() {
    return $this->t('Are you sure ...');
  }

  public function getConfirmText() {
    return $this->t('Delete');
  }

  public function getCancelRoute() {
    return array('route_name' => 'optimizely.listing');
  }

  public function buildForm(array $form, array &$form_state, 

                            $oid = NULL) {
    $this->oid = $oid;
    return parent::buildForm($form, $form_state);
  }

  public function submitForm(array &$form, array &$form_state) {

    // Use $this->oid to update the database, etc. ...

    // Return to project listing page
    $form_state['redirect_route']['route_name'] = 

        'optimizely.listing';

  }
}


Some of these methods have default implementations. For example, getConfirmText() defaults to returning 'Confirm' for the confirmation button.

There is also a default implementation of buildForm() but in this scenario I implemented it in order to receive the value of the project id as a param and to save the id in an instance variable. Then the method calls buildForm() of the parent class to do the actual form building. Note that that third parameter must have a default value declared, otherwise you get a syntax error because the signature of the method will not match that in FormInterface.

At the end of submitForm() the reference parameter $form_state is altered in order to redirect to a particular path after processing the form. For a little more discussion about redirection, see this other post on drupal_goto()  http://optimizely-to-drupal-8.blogspot.com/2014/06/function-drupalgoto-has-been-removed.html

Updates for Drupal 8, alpha 13 and 14:  The name of method getCancelRoute() has been changed to getCancelUrl(), and it has a different type of return value. The signatures for methods buildForm() and submitForm() have changed to use FormStateInterface. Redirection is implemented differently.
See  http://optimizely-to-drupal-8.blogspot.com/2014/08/drupal-8-alpha11-alpha13.html  and   http://optimizely-to-drupal-8.blogspot.com/2014/08/drupal-8-alpha-13-alpha-14.html

  use Drupal\Core\Form\FormStateInterface;
  use Drupal\Core\Url;


  public function getCancelUrl() {
    return new Url('optimizely.listing');
  }


  public function buildForm(array $form, 

                            FormStateInterface $form_state, 
                            $oid = NULL) { ... }

  public function submitForm(array &$form, 

                             FormStateInterface $form_state) {
    // ...

    // Return to project listing page
    $form_state->setRedirect('optimizely.setting');
  }

Sources:

Drupal 8: Forms, OOP style, August 12, 2013
http://effulgentsia.drupalgardens.com/content/drupal-8-forms-oop-style

Form API in Drupal 8
https://www.drupal.org/node/2117411

abstract class ConfirmFormBase
https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Form!ConfirmFormBase.php/class/ConfirmFormBase/8

Jul 11, 2014

Converting SimpleTest / Testing module classes for automated testing

The SimpleTest module became incorporated into Drupal 7 core as the Testing module. In Drupal 8 it retains its name of Testing. There have been a number of changes, some of which were made to follow conventions in D8.

As far as I can tell, the various flavors of assertions remain the same.

Here is the outline of a testing class for Drupal 8. I have omitted most of the actual code in order to focus on what is different. Explanatory notes follow the class definition.

File:  optimizely/src/Tests/OptimizelyAccessTest.php

namespace Drupal\optimizely\Tests;

use Drupal\simpletest\WebTestBase;

class OptimizelyAccessTest extends WebTestBase {

  protected $listingPage = 'admin/config/system/optimizely';
  protected $privilegedUser;
 
  public static $modules = array('optimizely', 'node');

  public static function getInfo() {
    // Same as before ...
  }

  public function setUp() {
   
    parent::setUp();

    $this->drupalCreateContentType(array(

      'type' => 'page', 'name' => 'Basic page'));
   
    $this->privilegedUser = 

      $this->drupalCreateUser(array(
        'access content',
        'create page content',
        'edit own page content',
        'administer optimizely'));

  }

  public function testUserWithPermission() {

    $this->drupalLogin($this->privilegedUser);
   
    $this->drupalGet($this->listingPage);
    $this->assertResponse('200',
      "User with 'administer optimizely' permission" .
      "
may access project listing page");
   
    $this->drupalLogout();

  }

}


Notes:

(1)  As a good organizing practice, create a separate folder for the test files. For example,
optimizely/src/Tests
The corresponding namespace declaration is then
namespace Drupal\optimizely\Tests;

(2)  Similar to other classes, each test class should be defined in its own file that has the same name as the class. For functional tests, the name of the test class must have the suffix "Test".


(3)  In D7 you declare the .test file in the .info file by using something like
files[]=optimizely.test
This is no longer necessary. The test files are auto-discovered by using the file and class naming conventions.


(4)  The core module class DrupalWebTestCase has been renamed to WebTestBase. The use statement so that that class can be referred to by its unqualified name is
use Drupal\simpletest\WebTestBase;
The other core base classes are UnitTestBase and DrupalUnitTestBase.


(5)  Instead of the following call in the setup() method of the test class
parent::setUp('optimizely');
add a declaration of $modules like the following to the test class definition in order to enable the module you're testing. Also, call the parent setup() method with no arguments in order to do other parental setup.
public static $modules = array('optimizely');

parent::setUp();

(6)  By default, testing is done using a low-weight testing profile. However, for example, if you need permissions that are defined by core modules such as node, the testing profile does not provide them. You will get an Invalid permission error.

One way to solve this is to use the standard profile by adding the $profile property to the test class.
protected $profile = 'standard';
Use of this profile provides an easy blanket solution, but some comments claim it may cause tests to run noticeably slower. Another approach is to explicitly enable those core modules that are needed, like this for the node module.
public static $modules = array('optimizely', 'node');
However, enabling the core module may not be enough. For example, I found that to be able to use permission create page content I also had to create the content type.
$this->drupalCreateContentType(array(
  'type' => 'page', 'name' => 'Basic page'));

(7)  If you have the xDebug extension enabled for PHP, disabling the extension will speed up the running of tests. I have reduced some test run times by almost half by disabling xDebug.


Update: see  http://optimizely-to-drupal-8.blogspot.com/2014/12/beta-3-beta-4-configuration-schema-and.html


Sources:

Converting D7 SimpleTests to Drupal 8
https://www.drupal.org/node/2166895

Drupal SimpleTest coding standards

https://www.drupal.org/node/325974

Namespace and naming changes for test classes of automated tests
https://www.drupal.org/node/1543796

New simpletest class property $modules to enable modules in tests instead of WebTestBase::setUp($modules)
https://www.drupal.org/node/1710766

Simpletest automated tests now use empty 'testing' profile by default
https://www.drupal.org/node/1911318

Jul 7, 2014

Drupal 8 requires PHP 5.4 or higher, and PHP traits


Having to migrate from an older version of PHP could be an issue for some hosting providers or for your local dev environments.

PHP 5.4 included major additions to the language. There's a long, amazing discussion about this in the second source article below, Require PHP 5.4, with almost two hundred comments by Drupal core maintainers and others!

Many of those comments dealt with the availability of traits. I did not know anything about that language feature before this. It's a bit of a controversial feature for which there is no shortage of opinions about how to use it and how not to use it.

For myself, as a simple learning exercise and in order to eliminate a small amount of duplication, I decided to refactor the code I had previously written to look up paths, as described in this older post:  http://optimizely-to-drupal-8.blogspot.com/2014/06/function-drupallookuppath-has-been.html

I had added two private wrapper methods to a class. I now extracted those methods into a new trait. Autoloading seems to work the same as for classes, i.e. define the trait in its own file, name the file the same as the trait, and place the file into the directory structure where the autoloader will find it.

The result was (with code comments removed),

File:  optimizely/src/LookupPath.php

namespace Drupal\optimizely; 

trait LookupPath  {

  function lookupPathAlias($path) {
    $alias = 

        \Drupal::service('path.alias_manager')
            ->getPathAlias($path);
    return (strcmp($alias, $path) == 0) ? FALSE : $alias;
  }
 

  function lookupSystemPath($alias) {
    $path = 

        \Drupal::service('path.alias_manager')
            ->getSystemPath($alias);
    return (strcmp($path, $alias) == 0) ? FALSE : $path;
  }

}


Syntactically, the definition of a trait looks a lot like a class.

Then, for example, in any class that needs these methods, use the trait, then call the methods as though they are defined in the class itself.

class ProjectListForm extends FormBase {

  use LookupPath;


  // . . .
      $path_alias = $this->lookupPathAlias($front_path);
  // . . .

}

This is a very nice way to reuse snippets of code and adhere to the DRY principle.

(At the moment, I am running Drupal 8 on Windows using PHP 5.4.15.)

Sources:

System requirements
https://drupal.org/requirements

Require PHP 5.4
https://drupal.org/node/1498574

Traits
http://php.net/manual/en/language.oop5.traits.php

PHP: Traits vs. Interfaces
https://stackoverflow.com/questions/9205083/php-traits-vs-interfaces

Jul 6, 2014

Setting up xDebug with Sublime Text 2 on Windows

I have gotten deeply enough into working with the PHP code of the Optimizely module that I missed having a debugger to step through the flow, examine the values of variables, and in general to dynamically control and understand the execution of the code.

Here are steps I took to successfully install and configure the xDebug PHP debugger to work with Sublime Text 2  (version 2.0.2) on a Windows 7 system.


(1) Installed xDebug 2.2.5 extension for PHP.

(a)  Downloaded php_xdebug-2.2.5-5.4-vc9.dll from  http://www.xdebug.org/download.php  and placed it into C:/php/ext (or wherever you might have PHP installed).

Since I wasn't sure which version of the dll to use, I followed the "custom installation instructions" on the xDebug site, which were very helpful and indicated which dll should work on my system.

(b)  Edited php.ini, adding the following line
zend_extension = ext\php_xdebug-2.2.5-5.4-vc9.dll
(c)  Restarted Apache. Loaded a page containing a call to phpinfo() to check that the extension was installed. On the page there appeared an xdebug section with several tables that showed settings and directives.


(2) Installed the Sublime package manager, "Package Control".

(a)  Within Sublime, clicked Preferences > Browse Packages, then browsed up one level to find the full path for the Installed Packages folder. On my system the path is

  C:/Users/Earl/AppData/Roaming/Sublime Text 2/Installed Packages

(b)  Downloaded the file Package Control.sublime-package from  https://sublime.wbond.net/Package%20Control.sublime-package

(c)   Copied that file into the Installed Packages folder.

(d)  Re-started Sublime.

In Sublime's Command Palette window, various Package Control commands were then available on the palette, such as Package Control: Install Package. You can also get to Package Control via the menu bar, Preferences > Package Control.


(3) Using Sublime's Package Control, installed the Xdebug Client package.

A couple of clicks. Done. In Sublime, this adds a submenu, Tools > Xdebug, or you can use the keyboard equivalents.


(4) Added further directives to php.ini for xDebug.

In php.ini created an [xdebug] section which includes the zend_extension directive added mentioned above.
[xdebug]
zend_extension = ext\php_xdebug-2.2.5-5.4-vc9.dll
xdebug.remote_enable = 1
xdebug.remote_host = "127.0.0.1"
xdebug.remote_port = 9000
xdebug.remote_handler = "dbgp"
xdebug.remote_mode = req
xdebug.remote_connect_back = 0
Because I only want to debug locally, I set xdebug.remote_connect_back to 0 to turn it off. I believe this should be set to 1 if you intend to debug a site running on a remote system, but my understanding of these directives is pretty fuzzy at this point.


(5) To start an xDebug session.

To start a debugging session, I am loading the Drupal 8 site that contains the converted Optimizely module by appending the query string ?XDEBUG_SESSION_START=1  to the usual url, like so:
http://localhost/opti/index.php?XDEBUG_SESSION_START=1
The value of  XDEBUG_SESSION_START  is a session name that is stored in a cookie, so it could be any name you choose.

There are other ways to start an xDebug session, but for getting started with using this debugger, this seemed like the simplest. Later, I may try to use one of the browser extensions that are available.

Finally, in Sublime itself, I do Start Debugging in order to activate the editor as an xDebug client.

Sweet!


Sources:

XDEBUG EXTENSION FOR PHP
http://xdebug.org/docs/remote

martomo / SublimeTextXdebug
https://github.com/martomo/SublimeTextXdebug

Package Control
https://sublime.wbond.net/installation#st2

Download:  Package Control.sublime-package
https://sublime.wbond.net/Package%20Control.sublime-package

Jul 1, 2014

Function cache_clear_all() has been removed

Drupal 8 has a new API for caching, so D7 functions such as cache_clear_all() have disappeared.

Function cache_clear_all() has a boolean third parameter that indicates use of a wildcard. If this third param is true, then the first param, which is the "cache_id", acts as though it has a trailing wildcard. That is, any cache IDs starting with the string value of  the cache_id param  are deleted in addition to the exact cache ID specified.

Unfortunately, that kind of wildcard functionality seems to be missing from the new cache API.

In one case, that didn't matter. For example,
cache_clear_all('*', 'cache_page', TRUE);
was easily replaced by
$cache = \Drupal::cache('render');
$cache->deleteAll();
On the other hand, the following call does not have an exact equivalent, where the $recursive variable can be either true or false. 
cache_clear_all($cid, 'cache_page', $recursive);
My replacement below is overkill when $recursive is true, but the performance hit may not be significant given the nature of the Optimizely module and how often it triggers cache clearing.

$cache = \Drupal::cache('render');
if ($recursive) {
  $cache->deleteAll();
}
else {
  $cache->delete($cid);
}
The D8 docs state that the render bin contains "cached HTML strings like cached pages and blocks". I am assuming that it therefore includes entries kept in the cache_page bin in D7, but I have not tested and confirmed this yet.

The new API has two different ways to clear the cache, which are to "delete" and to "invalidate". Cached entries that are invalidated are still potentially accessible, whereas deleted ones are completely removed. In our case, the entries should no longer be used at all, so I used the delete methods.

Update: Caching of pages has been properly re-implemented for the module. See post  Using Drupal 8 Cache Tags for Page Caching  


Sources:

function cache_clear_all
https://api.drupal.org/api/drupal/includes!cache.inc/function/cache_clear_all/7

Cache API
https://api.drupal.org/api/drupal/core%21core.api.php/group/cache/8.0.x

interface CacheBackendInterface
https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Cache!CacheBackendInterface.php/interface/CacheBackendInterface/8

Drupal 8: perf & ops - Cache tags & pluggable asset optimization, Wim Leers
http://wimleers.com/talk-drupal-8-perf-ops/