Dec 26, 2015

Drupal 8, Beta 15 --> RC 1: Eliminating use of checkPlain( )

Since SafeMarkup::checkPlain() is now deprecated and possibly intended to be removed along with other SafeMarkup methods, I decided to address this in migrating to RC 1.

In the common use case where D8's Twig templates are used for output, you can dispense with checkPlain() entirely and rely on Twig's autoescaping. That turned out to apply to our simple use of forms to input and render a few text values.

In the original code, checkPlain() was called both to process user-entered values as well as to sanitize those values before they were placed into render arrays. When I typed <script> into a form field, it was incorrectly escaped twice and displayed as   &lt;script&gt;

Removing those calls works fine and conforms to what is considered good practice for D8. Potentially unsafe markup is stored as is in the database, but Twig converts it before it is sent to the browser.

The article  SafeMarkup methods are removed  is extremely useful in how it breaks down the different use cases for checkPlain() and what needs to be done differently for each in D8 in order to prevent unsafe markup from being rendered.

Besides Twig templates, the other use cases are:

-  Text placed into a render array by using the #plain_text key
-  A mixture of escaped markup with markup not to be escaped
-  Non-HTML responses, eg. JSON

The discussion also applies to the check_plain() function in Drupal 7, for which checkPlain() was a replacement. If you're starting out with a conversion from D7, the article is a must read.

Sources:

SafeMarkup::set(), SafeMarkup::checkPlain(), and other methods are removed from Drupal 8 core
https://groups.drupal.org/node/478558

SafeMarkup methods are removed
https://www.drupal.org/node/2549395

Twig autoescape enabled and text sanitization APIs updated
https://www.drupal.org/node/2296163

Dec 19, 2015

Module: "Configuration inspector for Drupal 8"

In my recent post on Missing langcode in configuration schema I discovered that the langcode key in configuration schema is now required by default by Testing module automated tests.

The test results had error messages that said "Uncaught PHP Exception Drupal\Core\Config\Schema\SchemaIncompleteException" and "langcode missing schema".

As a result of that debugging, I got interested in a module called Configuration Inspector for Drupal 8.

Installing the module adds a new tab named Inspect to the page at  
/admin/config/development/configuration

That tab shows all of the Configuration Keys on the site, to which core contributes quite a few.

For the settings for our module, after I removed the langcode key from the .schema.yml file, the tab showed that the configuration key had 1 error. Going to the Raw Data page for the key then showed under Configuration Validation,

array ( 
  'optimizely.settings:langcode' => 'missing schema',
)

This is the gist of what the exceptions had said in the results from running the automated tests, but if I had used this config inspector early on to validate the schema, I would have spotted the error sooner rather than after the testing failed.

     * * *

The inspector also shows the types and the values of individual items. In our case, this includes a project id number whose value had been submitted through a form and stored programmatically by using the Simple Configuration API.

There was no entry shown for the langcode item, which is defined but had no value. I was able to provide one by adding code in hook_install(), again, by using the Simple Configuration API.

Sources:

Configuration inspector for Drupal 8
https://www.drupal.org/project/config_inspector

Configuration API in Drupal 8
https://www.drupal.org/node/1667894

Configuration schema/metadata
https://www.drupal.org/node/1905070
 

Dec 12, 2015

Drupal 8, Beta 14 --> Beta 15: Missing langcode in configuration schema

In migrating from Drupal 8 beta 14 to beta 15, the functionality of the module itself worked fine, but some of the automated tests were failing with error messages that included the following.
Uncaught PHP Exception Drupal\Core\Config\Schema\SchemaIncompleteException: "Schema errors for optimizely.settings with the following errors: optimizely.settings:langcode missing schema" at /var/www/html/opti/core/lib/Drupal/Core/Config/Testing/ConfigSchemaChecker.php line 98
In this message, optimizely.settings is the name of a group of configuration settings. It is also a configuration key in a .schema.yml file that is required for automated testing, which I blogged about earlier at Beta 3 --> Beta 4:  Configuration schema and metadata.

The article Fix config schema mentioned a very similar error message and stated "Because of a recent core change all tests are failing". I looked at the patches in that article, but I could not figure out what needed to be added in our case.

Then a search through the core code for the string "langcode:" led me to add langcode: as a key to the .schema.yml file as in the following.

optimizely.settings:
  type: mapping
  label: 'Optimizely Config Data'
  mapping:
    optimizely_id:
      type: integer
      label: 'Optimizely ID Number'
      translatable: false
    langcode:
      type: string
      label: 'Language code'


Problem solved. All automated tests passed after this change.

Update:  Also see my later post on  Module: Configuration Inspector for Drupal 8.

Sources:

Beta 3 --> Beta 4: Configuration schema and metadata
http://optimizely-to-drupal-8.blogspot.com/2014/12/beta-3-beta-4-configuration-schema-and.html

Fix config schema
https://www.drupal.org/node/2547365

All TestBase derived tests now enforce strict configuration schema adherence by default
https://www.drupal.org/node/2391795

Configuration schema/metadata
https://www.drupal.org/node/1905070

Nov 10, 2015

"Notice: Undefined index: und in eval() (line . . . . . /modules/php/php.module(80) : eval()'d code)"

On my local instance of a Drupal 7 site, I'd get warnings like the following on almost every page for a custom content type called Poem.

Notice: Undefined index: und in eval() (line 12 of /var/www/html/power-poetry/modules/php/php.module(80) : eval()'d code).

Although most probably innocuous, these warnings were also being logged numerous times in the system log, cluttering it up and making it harder to spot other, significant messages. It was enough of an annoyance that I decided to fix it.

I was able  to track this to a snippet of PHP code that is used in a Drupal block of ours. The code is part of the block's Visibility Settings. It determines if the current Poem being rendered satisfies certain criteria or not. If so, the block is made visible.

The offending line turned out to be

  $slam_id = $current_slam['und'][0]['target_id'];

The variable $current_slam was often an empty array. So by replacing the line with an additional check, the PHP warnings disappeared.

  if (!empty($current_slam)) {
    $slam_id = $current_slam['und'][0]['target_id'];
  }

The key to finding this quickly was remembering that we have this user-supplied snippet of code that needs to be evaluated at runtime

  - - - - -

While debugging, I wanted to see the type and the value of the variable $current_slam. The following worked to output it to the system log.

  watchdog('debug', print_r($current_slam, TRUE), array(),
           WATCHDOG_NOTICE);

The second parameter to print_r() determines what kind of return value the function passes back. By default, print_r() returns its status. But by setting the second param to TRUE, the return value is the output string instead.

  - - - - -

The site is hosted on Pantheon, which provides this status message:

  • PHP Filter: PHP Filter is enabled! Executable code should never be stored in the database, and support for this feature was removed in Drupal 8 - https://drupal.org/node/1203886
    Remove all executable code from your content and move it to your codebase.

Source:

Remove the PHP module from Drupal core
https://drupal.org/node/1203886

Nov 3, 2015

Malicious page redirects and Drupal's Filtered-HTML text format

One of the Drupal 7 sites I maintain is www.powerpoetry.org which is a platform for publishing poems and comments about them.

Recently, we heard from an irate poet. When browsing to one of her poems, the page would partially load, then automatically redirect to a movie streaming site that had nothing to do with her poem nor with the site in general. Her poem page somehow got hijacked, and readers could never read her work.

It turned out that all site users, even Anonymous ones, had permissions for the Full HTML Text Format. This allowed comments to be submitted that contained malicious code to redirect the page.

So far, I have found two different techniques that were used to implement redirection.

(1)  Use of the <meta> tag with an attribute of http-equiv="refresh". For example,

  <meta http-equiv="refresh" 
        content="2;url=http://scumbags.com/">

where the value of 2 means a delay of two seconds before the refresh.

(2)  Use of the onmouseover attribute to execute JavaScript. For example,

<a href="//tinyurl.com/nm8ugh
   onmouseover="document.location='//tinyurl.com/nm8ugh';">

As documented, this attribute and closely related ones are valid for almost all HTML tags, which would make it potentially a big problem where markup is allowed.

If you have good knowledge of HTML, these techniques may be obvious, but for me as a back-end developer they were new.

Because I have direct access to the Drupal database, I was able to use a SQL query such as the following to find malicious comments.

  select entity_id, comment_body_value
    from field_data_comment_body
    where comment_body_value like "%onmouseover%";

 - - - - -

To help plug these security holes, we decided to allow only Administrators to have access to all Text Formats. Depending on the needs of your site, you might want to disable the Full HTML format entirely.

For all other users, the Filtered HTML Text Format was permitted, configured using Limit Allowed HTML Tags so that only the following tags are processed.

  <p> <br> <em> <strong> <cite> <blockquote> <ul> <ol> <li>

Although not part of this security problem, we purposely disallowed <a> tags, deciding that they are not essential for comments on our site and almost always only appear in spam comments. As part of eliminating links, we also turned off Convert URLs Into Links.

Note that the user can still type in whatever they want, including any kind of HTML. The original text is stored unchanged as the content of the comment.

However, what the Filtered HTML format does is effectively strip out disallowed tags as part of the rendering process.

 - - - - -

Even with Filtered HTML, I was concerned that the onmouseover attribute could still be used inside one of the tags that is allowed. What about something like

  <p onmouseover="....">

But some testing and some stepping through core code revealed that onmouseover as well as other risky attributes are stripped out of those tags for output even though the tags themselves are processed.

The holes were plugged for these two exploits.

 - - - - -

The final thing left to do was to go back and check again the malicious comments, which I had left in the database while carrying out the above steps. I thought that applying Filtered HTML would neutralize them. I was wrong.

In the database table field_data_comment_body there is a column comment_body_format that stores the format in effect when the comment was created. The format apparently is used to render the comment even if it is no longer allowed.

As a result, those comments were still redirecting!

So I used SQL queries to find their entity ids, then deleted them by browsing to, for example, the following path where 30259 is the entity id of a comment.

  /comment/30259/edit

or

  /comment/30259/delete

The latter path goes directly to the delete confirmation dialog. In some cases I had to use it because loading the edit form for the comment triggered the redirection, so I couldn't even edit it.

 - - - - -

Michael Richardson at Pantheon support did an awesome job of doing the initial troubleshooting when I was at a loss as to where to start. It was he who uncovered the use of onmouseover on our site. Thanks, Michael!


Sources:

What is the Meta Refresh Tag?
http://webdesign.about.com/od/metataglibraries/a/aa080300a.htm

HTML onmouseover Event Attribute
http://www.w3schools.com/tags/ev_onmouseover.asp

Drupal Text Formats and Filters Tutorial
https://www.hostknox.com/tutorials/drupal/formats-and-filters

Text Filters and Input Formats
https://www.drupal.org/node/213156

Sep 21, 2015

Shareable URLs from Forward and the Twilio service: accessing an API endpoint on a localhost site

How do you have an external service access an API endpoint that is defined on a site that you're running locally?

One of the Drupal 7 sites (AllGoodText.com) I work on sends out SMS messages to users who register for weekly messages to be delivered to their cellphones.

The site uses the Twilio service to configure a virtual cellphone number from which the messages are apparently sent.

We also use the D7 Twilio module (version 7.x-1.10) to interface with the Twilio API. The module provides Drupal-specific elements such as hooks for implementing the desired functionality.

During local development, this worked fine as long as the site was only calling the Twilio API endpoints.

However, we wanted to enhance the site to define its own endpoints for the Twilio service to call back. For example, we wanted to be notified of and to be able to process STOP messages that our users were texting to our virtual cell number so that we could update the site's database.

For development purposes, I wanted to test as realistically as possible. I wanted my local site to interact with my cellphone via Twilio.

The problem was, I only run Apache on my system as a local server. At first, I thought I'd have to use our development site (it happens to be on Pantheon), constantly pushing code changes to it. That would have been tedious and messy.

Because I find the use of a debugger indispensable, I also would have had to figure out how to do so remotely.

Luckily, I saw a comment in response to a question about how to test the use of Twilio that mentioned "request forwarding utilities", of which Forward is one. The comment is clear and concise, so I'm just going to quote the commenter, Devin Rader.
Basically, request forwarding utilities like ForwardHQ do two things:
  1. It creates a publicly accessible URL, that you can tell Twilio about.
  2. It creates an SSH tunnel between a port on your machine and a server on the internet
When Twilio needs to make an HTTP request because it received an inbound SMS or voice call, it will request the ForwardHQ URL. Forward knows how to take that request and send it to the port running on your local machine. That port maps to your local web development server running in order to process the request and return the result to Twilio.

This sounded like exactly what I wanted. Here are the steps I took.

1.  Signed up for an account with Forward.

2.  Installed their browser extension for Chrome, which puts an icon on the toolbar to Open Forward.

3.  Browsed to the local site. Clicked on the Open Forward icon.

(I probably had to log in to Forward the first time I did this.)

4.  This brought up a special page from the browser extension. Clicked on Start Tunnel to create a tunnel between the site running on my localhost and Forward.

To access the site, the url wholewhale.fwd.wf was provided by default, where the subdomain wholewhale was specified by me.

5.  The Twilio module defines the path /twilio/sms using hook_menu() for processing incoming sms messages.

6.  So under my Twilio account, I specified http://wholewhale.fwd.wf/twilio/sms as the Request URL to notify the local site of incoming sms messages from users.

(On the live site, the endpoint will be http://allgoodtext.com/twilio/sms)

7.  I had an issue with the Twilio module logging the error message "Incoming SMS could not be validated" and not invoking the hook I had implemented.

I hacked around this by temporarily inserting a "return TRUE" statement in the module code so that it would continue as though no error had occurred. This worked fine for what I was doing.
 
I was now able to run the site and make code changes to it locally, run my debugger locally, and have the site interact with the Twilio service in both directions.

Sweet!!


Sources:

Forward
https://forwardhq.com/

twilio (the service)
https://www.twilio.com/

Twilio (the Drupal module)
https://www.drupal.org/project/twilio

How to test Twilio calling my REST API via POST when a user sends an sms to a short code?
http://stackoverflow.com/questions/17983398/how-to-test-twilio-calling-my-rest-api-via-post-when-a-user-sends-an-sms-to-a-sh

Accessing localhost from Anywhere
http://www.sitepoint.com/accessing-localhost-from-anywhere/

Aug 24, 2015

Beta 11 --> Beta 12:    _format property in route definition

Migrating to Beta 12, some ajax functionality was not functioning as expected. The log messages reported an exception,

Symfony\Component\HttpKernel\Exception\NotAcceptableHttpException: No route found for the specified format

Using a debugger to step through the function that throws the exception, I made guesses as to what was needed. Eventually, I stumbled on a fix.

In the .routing.yml file I changed the relevant route definition by removing the _format property as follows.

Before:

ajax.enable:
  path: /ajax/optimizely
  defaults:
    _controller: \Drupal\optimizely\AjaxEnable::enableDisable
    _title: Optimizely Administer AJAX
  requirements:
    _permission: administer optimizely
    _format: json


After:

ajax.enable:
  path: /ajax/optimizely
  defaults:
    _controller: \Drupal\optimizely\AjaxEnable::enableDisable
    _title: Optimizely Administer AJAX
  requirements:
    _permission: administer optimizely


Unfortunately, searching through the Drupal and the Symfony docs, I have not found anything to help me understand why this change to the route definition makes a critical difference.

From the module's point of view, it looks as though the property is actually not needed. The client-side JavaScript that makes the request to the server sets json as the dataType. And the server-side code always instantiates a JsonResponse object to pass back.



Aug 19, 2015

Beta 11 --> Beta 12: Leading slash required in paths for "Add Alias" page (and elsewhere)

The Add Alias page that can be accessed via admin/config/search/path/add  now requires that values entered for both of the fields Existing System Path and Path Alias must start with a leading slash. Otherwise, the form does not validate.

When filling in the form manually, error messages are displayed to that effect, so it's clear what the problem is.

However, when this is being done programmatically in an automated Testing case, it's not at all obvious what went wrong.

Here's a piece of original code to create a path alias to an existing node, written for a subclass of WebTestBase:

  $edit = array();
  $edit['source'] = 'node/' . $node->id();
  $edit['alias'] = $this->randomMachineName(10);
  $this->drupalPostForm($this->addAliasPage, $edit, t('Save'));

 And here's the corrected code:

  $edit = array();
  $edit['source'] = '/node/' . $node->id();
  $edit['alias'] = '/' . $this->randomMachineName(10);
  $this->drupalPostForm($this->addAliasPage, $edit, t('Save'));



There seems to be an overall pattern that paths relative to the site root must now start with a slash. Besides these fields for the Add Alias page, I wrote in a previous post about such a requirement in calls to some methods of class AliasManager. And there's recent mention of this in Update the DB dump to have a leading slash for the frontpage.



Aug 14, 2015

Beta 11 --> Beta 12: AliasManager expects leading slash

As soon as I enabled the Optimizely module under Beta 12, I'd get the normal setup messages from the module, but then followed by this error.

The website encountered an unexpected error. Please try again later.

There was no other indication on the page of what had gone wrong. Moreover, the site would be completely unusable such that I could not even browse to the front page.

Eventually, I stumbled on a way to access the log. By deleting the directory for the module after the error occurred and using drush to clear cache, the site became usable again.

The log showed this message:

InvalidArgumentException: Source path node has to start with a slash. in Drupal\Core\Path\AliasManager->getAliasByPath() (line 191 of /var/www/html/opti/core/lib/Drupal/Core/Path/AliasManager.php).

This made me realize that the parameter in my calls to getAliasByPath() has to start with a leading slash. This also applies to the sibling method getPathByAlias().

So, for example, I revised the following function,

trait LookupPath  {

  static function lookupPathAlias($path) {


    $alias = \Drupal::service('path.alias_manager')->getAliasByPath($path); 
    return (strcmp($alias, $path) == 0) ? FALSE : $alias;
  }



to call a new helper routine.

trait LookupPath  {

  static function lookupPathAlias($path) {

    $path = LookupPath::checkPath($path);
    $alias = \Drupal::service('path.alias_manager')->getAliasByPath($path);
    return (strcmp($alias, $path) == 0) ? FALSE : $alias;
  }

  static function checkPath($path) {
    return ($path[0] == '/') ? $path : '/' . $path;
  }

}


The fix was simple enough once I saw the error message in the log. The difficulty lay in getting to the log in the first place!

Aug 7, 2015

"The RSA host key for git.drupal.org has changed"

Just now when I tried to git push to drupal.org I got a long warning message that started with the following.

The RSA host key for git.drupal.org has changed, and the key for the corresponding IP address 140.211.10.43 is unknown.

The article  Drupal.org Git Server Migration  and its comments give a good explanation and discussion about how drupal.org has migrated to different servers as of early July.

The steps I took to fix the issue on my Linux Mint system were as follows, where the -R option for ssh-keygen means "remove the keys".

# ssh-keygen -R git.drupal.org
# ssh-keygen -R 140.211.10.43

# git fetch

The authenticity of host 'git.drupal.org (140.211.10.43)' can't be established.

RSA key fingerprint is 16:f5:44:6c:a1:c6:be:72:cd:98:b5:b7:7d:26:d6:14.


Are you sure you want to continue connecting (yes/no)? yes



After that, there were no further warnings in using git to access drupal.org.

Instead of git fetch probably any git command that accesses the remote repo would have the same effect, such as git pull or git clone.

Source:

Drupal.org Git Server Migration
https://www.drupal.org/node/2529356

Beta 10 --> Beta 11: String::checkPlain() moved to SafeMarkup::checkPlain()

It turns out that my blog post of last week was completely wrong in claiming that migrating the Optimizely module from Beta 10 to Beta 13 did not require any code changes. In fact, doing so completely breaks the site as soon as the module is enabled.

I'm not sure what went wrong during the migration. I must have had a brain burp and copied the wrong release into the test directory.

In any case, I have backtracked to Beta 11 and successfully migrated to it by making one coding change.

Method checkPlain() has been moved from class

  Drupal\Component\Utility\String

 to

  Drupal\Component\Utility\SafeMarkup

Jul 31, 2015

Beta 10 --> Beta 13: No code changes needed. But additional automated tests would help.

After a rather long digression of writing several posts about JavaScript in this blog, I got back to work on migrating the Optimizely module to Drupal 8, this time from Beta 10 to the recent Beta 13.

Once again, no code changes were needed.

Update: I made a serious mistake in migrating from Beta 10 to Beta 13. I thought that I had done so successfully without any code changes needed.

It turns out that I was testing against an older release, not Beta 13. In actuality, the Optimizely module completely breaks the site as soon as it is installed under Beta 13.

After discovering my mistake, I backtracked to Beta 11, where the method checkPlain() has been moved to a different class, that is, from String to SafeMarkup.



When I test this module against a new release of D8, I first carry out manual testing on the intended functionality, then run an automated suite of several tests written to the core Testing module.

When those automated tests all pass, they provide a lot of additional confidence that the module is working correctly.

At the moment, the suite consists only of the ones that I inherited when I started this conversion project. The suite lacks testing of two critical error conditions that must be detected when the user is editing projects.

Now, it's time for me to expand beyond that original set to provide some enhancements by adding new tests. For this module, D8 is a stable enough platform to start looking to new functionality.

Jun 13, 2015

Notes on object-oriented JavaScript for o-o developers (part 5)

This post continues with more notes on distinctive features of object-oriented JavaScript as described in the book The Principles of Object-Oriented JavaScript.

This is a follow-up to the earlier Part 4 and is my final post related to that book.

**  Private properties can be made by using closures.

In JavaScript, all properties of an object are publicly accessible. Unlike some other languages, there are no keyword access modifiers that correspond to public/protected/private.

However, you can get the same effect as private by defining an object in the context of a function that provides local variables and methods that you don't want to be public. The object is then used as a closure on those items that would otherwise not be accessible at all outside of the function.

So here's an example that I have copied from the above book, which nicely illustrates the idea.

  var person = (function () {

    var age = 25;

    return {
      name: "Nick",
      getAge: function () { return age; },
      growOlder: function () { ++age; }
    }

  })();

The outer, anonymously defined function is immediately invoked and returns an object which is assigned to the variable person.

The key concept here is that the two methods of that object refer to a local variable of the function. Because of those references, the lifetime of the local variable is extended, while at the same time it is not visible in any other code.

This use of closures is an example of the Module pattern in JavaScript.

This provides an individual object with a private property, but a similar technique can be used in a constructor so that every object created via the constructor gets its own private, instance property.

  function Person (initial_age) {

    var age = initial_age;

    this.getAge = function () { return age; };
    this.setAge = function (new_age) { age = new_age; };
    this.growOlder = function () { ++age; };

  }

  nick = new Person(59);
  lena = new Person(37);

  console.log(nick.age);  // undefined


Sources:

Closures
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Closures

Learning JavaScript Design Patterns
http://www.addyosmani.com/resources/essentialjsdesignpatterns/book/


Jun 1, 2015

Syntax explanation of (function ($) { ..... })(jQuery)

A few months ago when I had to modify some JavaScript, I ran across code like the following that just baffled me in terms of the basic syntax.

  (function($){
    $(document).ready( ..... );
  })(jQuery);


I'm now in a much better position to explain at the language level what's going on.

This explanation is not about jQuery nor about jQuery plugins, which a web search will show results for, such as the second article mentioned below.

Previously, I wrote that one of the ways to code the literal definition of a function is by using an expression that starts with the keyword function but does not include a name.

For example, this code defines an anonymous function and assigns it to a variable.

  var helloWorld = function() {
    console.log("hello, world");
  }

The right-hand side of the = operator must be an expression that provides a value. In this case, it's a function object.

Likewise, here's a definition of another function object. The parentheses around it force it to be treated as an expression rather than as a regular function declaration.

  (function ($) {
    $(document).ready( ..... );
  })

A single dollar sign $ is a valid identifier. So in this case, that's the name of the function's argument. That argument is apparently expected to be a function since $(document) looks like a call of $, whatever the actual argument is.

Let's add the final piece.

  (function ($) {
    $(document).ready( ..... );
  })(jQuery);

The (jQuery) part of this expression means call the anonymously defined function, with jQuery as the actual argument.

In general, this kind of construct is known as an Immediately Invoked Function Expression (IIFE).

There's a slightly different way to write this code. I prefer the above, and it's what I've been seeing in practice. But the following is also correct.

  (function ($) {
    $(document).ready( ..... );
  }(jQuery));


Sources: 

Immediately-Invoked Function Expression (IIFE)
http://benalman.com/news/2010/11/immediately-invoked-function-expression/ 

What does (function($) {})(jQuery); mean?
http://stackoverflow.com/questions/2937227/what-does-function-jquery-mean

May 25, 2015

Notes on object-oriented JavaScript for o-o developers (part 4)

This post continues with more notes on distinctive features of object-oriented JavaScript as described in the book The Principles of Object-Oriented JavaScript.

The post is a follow-up to the earlier  Part 3.

**  Prototype chaining provides inheritance.

Almost all objects refer to a prototype, from which methods and properties are inherited that are defined in the prototype rather than in the object itself.

But an object's prototype is itself an object with its own prototype (usually). This chain normally goes all the way back to Object.prototype.

The object inherits the methods and properties of each of these prototypes. When a method is called, the JavaScript interpreter walks the chain to search for it. The first occurrence that is found is the one that is used. (That occurrence shadows any others that may be defined farther along in the chain.)

This means that pretty much every object inherits generally useful methods such as valueOf() and toString() from Object.prototype.

Here's a simple example of inheritance via prototypes.

  function Pet(name) {
    this.name = name;
  }
  Pet.prototype.getName = function () { return this.name; };


  my_pet = new Pet('My Pet');

  function Dog(name) {
    this.name = name;
  }
  Dog.prototype = my_pet;

  Dog.prototype.constructor = Dog;

  my_dog = new Dog('Foghat');


  console.log(my_dog.getName());  // Displays 'Foghat'


An instance of Pet is created, then it is assigned as the prototype for the Dog constructor. So the getName() method is indirectly inherited by the instance of Dog.

(The constructor property of Dog.prototype has to be restored to refer to Dog. Otherwise, it would refer to Pet, which is not correct.)

A slightly more concise and cleaner version of this example is

  function Pet(name) {
    this.name = name;
  }
  Pet.prototype.getName = function () { return this.name; };


  function Dog(name) {
    this.name = name;
  }
  Dog.prototype =
new Pet();
  Dog.prototype.constructor = Dog;

  my_dog = new Dog('Foghat');


  console.log(my_dog.getName());  // Displays 'Foghat'


We still create an instance of Pet,but it is anonymously assigned directly as the prototype for Dog.

The argument to the Pet constructor can be omitted since it isn't used in this particular use case.

This example chains from one constructor to another so it is known as constructor inheritance. However, the prototype chain may also consist of objects that are not constructors.

Caveat: prototype chaining works well for inheriting methods, but not for inheriting data properties that are intended to be instance data, that is, data with values that are unique to each object. That's because prototype objects are shared, therefore, their data properties are also shared among all objects that have the same prototype. These are like static or class variables in other languages.

(Continued at the final Part 5.)

May 17, 2015

Notes on object-oriented JavaScript for o-o developers (part 3)

This post continues with more notes on distinctive features of object-oriented JavaScript as described in the book The Principles of Object-Oriented JavaScript.

The post is a follow-up to the earlier Part 1 and Part 2.


**  Although Javascript does not have classes, it does have constructors.

Here's a simple example of using a constructor to instantiate an object.

  function Mountain(name, height) {
    this.name = name;
    this.height = height;

    this.howTall = function () {
      return this.height;
    }
  }

  mt1 = new Mountain('Kilimanjaro', 19340);


A constructor is much like other functions, but it is different in the following ways.

-  By convention, the name of a constructor starts with a capital letter.

-  The constructor should be invoked via the new operator.

-  Within the body of a constructor, the keyword this refers to the object created by the new operator. There is no requirement to return a value because new does this.

The newly-created object is an instance of the constructor (which makes the constructor sound a bit like a class, doesn't it?).

This can be checked by using the instanceof operator. An object remains an instance of its constructor even as properties of the object are added and removed.

  console.log(mt1 instanceof Mountain);  // true

  // Add a property, and delete a different one.
  mt1.location = 'Earth';
  delete mt1.name;

  console.log(mt1 instanceof Mountain);  // still true


**  Prototypes fill a role similar to classes.

Using the Mountain constructor as defined above means that every object that is created by using the constructor gets its own copy of the howTall() method.

That redundancy can be eliminated by using the constructor's prototype.

Almost every function (except for a few that are built-in) has a prototype property that is a reference to an object. This includes functions that are used as constructors.

When an object is created, it is given a prototype property whose value is a reference to the same prototype object as its constructor. Therefore, the Mountain constructor and every object that is created by using Mountain have a reference to the same prototype object.

The properties and methods of the prototype are shared among all these objects and are visible through them.

So another way to provide the howTall() method is to do this:

  function Mountain(name, height) {
    this.name = name;
    this.height = height;
  }

  Mountain.prototype.howTall = function () {
    return this.height;
  }

  mt2 = new Mountain('Fuji', 12388);
  console.log(mt2.howTall());

That is, howTall() is available to all instances of Mountain through their prototype.

Because prototypes are just objects, you can add more methods to a prototype. Those methods then become available to objects which have that prototype, even after the objects have already been created.

  function Mountain(name, height) {
    this.name = name;
    this.height = height;
  }

  mt2 = new Mountain('Fuji', 12388);

  Mountain.prototype.heightInMeters = function () {
    return this.height * 0.305;
  }

  console.log(mt2.heightInMeters());

It's a bit like having classes that can change dynamically, adding and removing methods at will. Sounds like a lot of fun and some interesting debugging.

(Continued at Part 4.)

May 10, 2015

Beta 9 --> Beta 10: No code changes needed

For only the second time since Drupal 8 went beta, no coding changes for the Optimizely module were needed for a new release of D8.

It's pretty amazing that a small module like this one, which uses only a very small subset of the API's, can reveal quite a few changes just by attempting to run through its normal functionality, its error checking, and automated tests.

Back in July of last year, I wrote about function cache_clear_all() being removed along with its wildcard for subpaths. At the time, I simply replaced that function with a clearing of the entire cache where any wildcard is used in the D7 version.

However, that cache invalidation must be re-written for Drupal 8 so that it works in a targeted manner and does not potentially cause such a big hit to performance.

Now that D8 has become much more stable, it's time I stopped procrastinating and do it right.

May 4, 2015

Notes on object-oriented JavaScript for o-o developers (part 2)

This post continues with more notes on distinctive features of object-oriented JavaScript as described in the book The Principles of Object-Oriented JavaScript.

The post is a follow-up to the earlier Part 1.


**  Regardless of how a function is defined and what its declared signature, it can be called with any number of parameters without triggering a runtime error.

Almost always, you do want to pass and to use the parameters as declared and as named. If for no other reason, do this for the sake of making the code better self-documenting.

Otherwise, within the context of each function there is a special array called arguments[] that contains the actual params for the call. You can use arguments[] to get a count of params and to access their values.


**  There is no function overloading.

If a function is defined more than once, it is the most recent definition that is used. The others are effectively overwritten.


**  There are special methods on functions.

Since functions are objects, it's possible that those objects have methods. There are at least three:  call(), apply(), and bind().

They are a bit too complicated to write about for these concise notes, so I won't describe them any further. I just wanted to mention that they exist and to reinforce the notion that functions are objects to which methods can be applied.


**  Objects can be made non-modifiable in different degrees.

(1) You can prevent properties from being added to an object by calling Object.preventExtensions() on it. In other respects, the object is still modifiable.

  var obj = new Object;

  Object.preventExtensions(obj);

  obj.greeting = 'hello, world';  // Not allowed.

(2)  You can prevent properties from being added or removed from an object by calling Object.seal() on it. The values of the properties can be changed.

This makes the object similar to those created in a strongly-typed language with classes.

  var obj = new Object;
  obj.name = 'Turtle';

  Object.seal(obj);

  obj.greeting = 'hello, world';  // Not allowed.
  delete obj.name;                // Not allowed.
  obj.name = 'Tortoise';          // Okay.

(3)  You can prevent properties from being added or removed, and also prevent the values of properties from being changed by calling Object.freeze() on it. In general, this is sometimes called an immutable object.

  var obj = new Object;
  obj.name = 'Turtle';

  Object.freeze(obj);

  obj.greeting = 'hello, world';  // Not allowed.
  delete obj.name;                // Not allowed.
  obj.name = 'Tortoise';          // Not allowed.

(Continued at Part 3.)

Apr 29, 2015

Notes on object-oriented JavaScript for o-o developers (part 1)

I've started reading The Principles of Object-Oriented JavaScript by Nicholas C. Zakas.

So far, it's been excellent, one of the best technical books I've read. It's clear, concise, and doesn't have any fluff, a solid tutorial with fewer than one hundred pages.

Coming from having programmed a lot in the strongly-typed language C++ a couple of lifetimes ago, I find JavaScript very fluid and sometimes very different in certain aspects.

It has some commonalities with PHP, which is what I'm mostly working in these days and still learning.

The purpose of this post and its follow-up is to highlight those JavaScript features  described in this book that might be of particular interest to someone with a background similar to mine.



**  Javascript does not have classes nor class definitions in the usual sense.

It does have objects, which are instances of reference types. The built-in reference types include Object, Array, Date, Function, RegExp, and Error.


**  An object is a dynamic collection of properties and their values.

Properties can be added or removed from an object at any time. In other words, by default, an object is not constrained by whatever type was used to instantiate it.

For example, create a Date object, then add an arbitrary property to it. Note that assigning a value to a property that didn't exist before implicitly adds that property to the object.

  var date = new Date;
  date.blessing = 'May you live in interesting times';

Then delete the property. The date object is now back to its initial state.

  delete date.blessing;


**  The primitive types, which include string, number, and boolean,  sometimes can be treated as objects even though they are not objects.

Each of these primitive types has a corresponding primitive wrapper type, respectively, String, Number, and Boolean.

The JavaScript  interpreter creates temporary objects of these primitive wrapper types as needed, uses them, and then discards them immediately. This is how properties and methods can seemingly be invoked on primitive types.

For example, extract the first character of a string using a method call.

  var first = 'hello'.charAt(0);


**  Functions are first-class objects.

That is, functions are objects. So functions are values that can be assigned to variables, passed as parameters, be returned, and appear in expressions, just like values of other types.


**  There are two literal ways of defining functions.

The first is the familiar declaration that starts with the keyword function and must include a function name. For example,

  function helloWorld() {
    console.log("hello, world");
  }

The second way is an expression that also starts with the keyword function but does not include a name. For example, this code defines an anonymous function and assigns it to a variable.

  var helloWorld = function() {
    console.log("hello, world");
  }


**  A function declaration may appear after a call to that function.

For example, this is allowed.

  helloWorld();

  function helloWorld() {
    console.log("hello, world");
  }

The function declaration is effectively hoisted to the top of its context.

But this only works for function declarations, not for function expressions. The code below results in a runtime error.

  hello();

  var hello = function() {
    console.log("hello");
  }

(Continued at Part 2.)

Apr 25, 2015

Beta 9: automated tests, module_invoke(), and drush

When I tried to run the automated tests, none of them showed up in the list of available tests for the Testing module's configuration.

It turns out that there is a new requirement for writing test classes, namely,
Your test class needs a phpDoc comment block with a description and a @group annotation, which gives information about the test.
So, for example, by adding @group in the following comment block, the test class was recognized and listed under the Optimizely grouping. The lead paragraph of the comment block is treated as a description and is displayed accordingly in the listing.

/**
 * Create users with no, some, and optimizely permissions
 * to test access to module related pages.
 *
 * @group Optimizely
 */
class OptimizelyAccessTest extends WebTestBase { . . . . }



This @group annotation seems to have been created specifically for the Testing module and is not part of the general documentation standards.

Sources:

Automated tests
https://api.drupal.org/api/drupal/core!modules!system!core.api.php/group/testing/8

API documentation and comment standards
https://www.drupal.org/coding-standards/docs



Fatal error: Call to undefined function Drupal\optimizely\Tests\module_invoke()

Function module_invoke() has been removed. I replaced this call,

  $schema = module_invoke('optimizely', 'schema');

 with this one.

  $schema = \Drupal::moduleHandler()
              ->invoke('optimizely', 'schema');

Generally speaking, interface ModuleHandlerInterface and class ModuleHandler provide the functionality that was previously in functions such as this one as well as module_list() and module_implements() which have also been removed.

Sources:

Remove bootstrap.inc  module_invoke()
https://www.drupal.org/node/2330181

Module/hook system functions replaced with module_handler and module_installer
https://www.drupal.org/node/1894902 



Running drush on a freshly-installed Beta 9 site gave the error message, 

Drupal\Core\DependencyInjection\ContainerNotInitializedException: \Drupal::$container is not initialized yet. \Drupal::setContainer() must be called with a real container. in Drupal::getContainer() (line 129 of /var/www/html/opti/core/lib/Drupal.php).

I was using a version of drush 7 from January. Downloading and installing the current drush 7.0-dev  fixed this problem.

Sources:

Installing Drush for Drupal 8
http://optimizely-to-drupal-8.blogspot.com/2015/01/installing-drush-for-drupal-8.html

Apr 6, 2015

"An Introduction to Entities"

An Introduction to Entities
https://www.drupal.org/node/1261744

I might be particularly dense about this aspect of Drupal, but this is the first article I've read that clearly and succinctly describes the closely related concepts of entity type, bundle, node, content type, field, and entity.

For those who are familiar with object-oriented programming, there is a comparison that is helpful, even though the analogy is not a perfect match.

Kudos to the authors.

Mar 8, 2015

PHP reference in foreach loop -- gotcha!

When I wrote PHP code similar to the following, I thought it was innocuous, but I was wrong.

  $arr = ['A', 'B', 'C', 'D'];

  foreach ($arr as $index => &$value) {
    $value = strtolower($value);
  }


  foreach ($arr as $index => $value) {
    . . .

  }

Besides lowercasing each string as intended, the last element of the array was overwritten by the next to last element, i.e. the array ended up with the values ['a', 'b', 'c', 'c'].

This happens consistently with any array with two or more elements. The last element is overwritten with the value preceding it.

Although I don't fully understand what's going on, I know that it has something to do with the use of the reference in the first foreach loop. After the first loop finishes executing, the variable $value continues to be a reference, and that leads to the unexpected behavior.

I have omitted the body of the second loop because it is irrelevant to the issue at hand.

There are at least three ways to fix this problem.

1)  In the second loop, use a different variable.

  foreach ($arr as $index => $item) {
    . . .

  }
 
2)  Unset the variable before using it further.

  unset($value);

3)  Use the variable as a reference also in the second loop, rather than a non-reference.

  foreach ($arr as $index => &$value) {
    . . .

  }

I don't find any of these fixes ideal, but all of them do eliminate the problem.

Source:

What References Do
http://php.net/manual/en/language.references.whatdo.php

Feb 22, 2015

Beta 6: ConfigFactoryInterface::getEditable()

Migrating to D8 beta 6 (from beta 4), when I tried to submit a form to change a field value that is persisted by using the Simple Configuration API, I got the log error message,

Can not set values on immutable configuration ... Use \Drupal\Core\Config\ConfigFactoryInterface::getEditable() ...

As of beta 6, by default, the values of configuration objects cannot be changed. So the line of code from the previous release,

  $config = \Drupal::config('optimizely.settings');

had to be replaced with the following in order to retrieve a mutable object.

  $config =
    \Drupal::configFactory()
      ->getEditable('optimizely.settings');

Calls to get() and set() then work as expected.

  $id = $config->get('optimizely_id');

  $config->set('optimizely_id', $id);


Sources:

Configuration objects by default are immutable.
https://www.drupal.org/node/2407153

Working with Configuration Forms
https://www.drupal.org/node/2206607

Functions variable_set() and variable_get() are removed
http://optimizely-to-drupal-8.blogspot.com/2014/06/functions-variableset-and-variableget.html

Jan 19, 2015

Setting up virtual hosts for multiple domains in Apache

On my local Linux Mint system running Apache, the server root is at /var/www/html. Because I have multiple sites, each site is installed in a subdirectory under that root.

To browse to a site, I would use a url such as localhost/opti for the Optimizely module, which happens to be an instance of Drupal 8.

Lately, I've had problems with some site roots being subdirectories under the server root rather than being at the server root itself. So I figured it was time to learn how to set up virtual hosts.

The following steps worked for Linux Mint 17 with Apache 2.4.7.

(0)  Enabled the Apache module vhost_alias.

#  a2enmod  vhost_alias

(1)  Created a virtual host.

# cd  /etc/apache2/sites-available
# cp  000-default.conf  opti.conf


Edited opti.conf so that within the <VirtualHost *:80> directive, two changes were made.

Inserted the line for ServerName, where opti.local will be the url to use for the local site.

  ServerName  opti.local

Edited DocumentRoot to set the site root to its path.

  DocumentRoot  /var/www/html/opti


(2)  Enabled the site.

#  a2ensite  opti

(3)  Directed the new url to the local system.

Edited /etc/hosts to add  the following line.

127.0.0.1  opti.local

(4)  Restarted Apache.

#  service  apache2  restart

(5)  Browsed to opti.local instead of localhost/opti.

Caveat: after I set up the virtual host, I had problems with code that does matching of paths when I absent-mindedly went back to using the url localhost/opti. Although I haven't really explored this, there appears to be confusion that results from two different notions of what the base url is. If you've got a virtual host, it's best to use its url consistently.

(Many thanks to Darren Lee for mentioning Apache's virtual hosts to me.)

Sources:

Setting Up Virtual Hosts for Apache on Ubuntu for Local Development
http://blog.code4hire.com/2011/03/setting-up-virtual-hosts-for-apache-on-ubuntu-for-local-development/

How To Set Up Apache Virtual Hosts on Ubuntu 12.04 LTS
https://www.digitalocean.com/community/tutorials/how-to-set-up-apache-virtual-hosts-on-ubuntu-12-04-lts