How to deploy safe ZF 2 applications on Amazon Elastic Beanstalk

Laisser une réponse

One question that often arise is how to deal with multiple environments (production, development…) and how not to compromise your credentials. This short article focus on how to deploy your application on Amazon Elastic Beanstalk, which is a PaaS system that I love.

The first mistake not to do is to commit configuration files that contain credentials (like your database password, or the secret key to your mail provider). This is why Zend Framework 2 introduced the concept of local and global config files (learn more about this here). To be short, those files are saved in your autoload folder, and allow to configure a module. local files should NEVER be pushed to your version control system.

Let’s take the example of configuring Doctrine 2 ORM using Doctrine Module. We are going to create two files: doctrine_module.config.global.php and doctrine_module.config.local.php. The global file will be pushed to Git (and to Elastic Beanstalk as it uses Git deployment), and will contain production settings. On the other hand, the local file will contain development settings and access to your local database. As local files overwrite global files, it works nicely.

Here is the global file, with production settings (this one will be on your computer AND will be pushed to Elastic Beanstalk):

Notice that it has APC cache, does not regenerate proxies… One interesting thing is the $_SERVER['RDS_*'] lines. If you create your Elastic Beanstalk environment with a RDS database, Elastic Beanstalk will automatically give you those environment variables for free! This means that this file can be committed without explicitly writing the production credentials, which is nice.

Here is the local files (this one is only on your development computer, and is not committed to Git nor Beanstalk if you configured the .gitignore correctly):

This works well because Elastic Beanstalk provides RDS_* environment variables for us. But let’s say we want to do the same with Mandrill, an email provider. We don’t want to commit our secret key.

As for before, let’s create a “mandrill.global.php” file:

While the mandrill.local.php file contains the secret key hardcoded (once again, remember it’s not committed !):

The problem is that Elastic Beanstalk does not know the “MANDRILL_KEY” environment variable. In order to do that, let’s create a folder called “.ebextensions” at the root at your website, and create a “script.config” file. Those files are used to customized the underlying EC2 instance without ever log to the instance. Learn more on that here.

In this script.config file, add the following YAML content:

This code simply create a new environment variable with a default placeholder (don’t write your real credential here, as this file is pushed to Git !). Deploy the new application to Elastic Beanstalk. Then, log in to your Amazon account and, through the console, choose “Edit/load configuration”, and open the “Container” tab. Scroll down and it added a new option:

elastic-beanstalk-environment-variables

 

Now, you can safely write the real credential !

I personally find this way very convenient. It adds a security layer by not pushing credentials to Git, and it keeps deployment simple.

Understanding the Zend Framework 2 event manager

2 réponses

La version française de cet article est disponible ici.

Introduction

The event manager is without any doubt one of the least known Zend Framework 2 component. Nevertheless, it is a very powerful component which offers a lot of flexibility when used correctly. This short article aims to help you to use it.

Why using the event manager?

The event manager allows event driven programming. It allows to flexibly connect different parts of the code to each other, without having a wrong, inverted dependency.

The event manager is extensively used internally (that’s why we said that ZF 2 is an event-driven framework). Thus, contrary to Zend Framework 1, most of MVC elements (routing, dispatching, view…) are not called one after another. Instead, framework triggers events (“dispatch”, “route”…). Other objects “listen” those events and, in turn, do something in response of those events. We can extract some wording:

  • Objects trigger events. Those events are named (“route”, “dispatch”, “sendTweet”…) and often contain additional parameters (for instance, an event called “sendTweet” could contain the tweet’s content).
  • Other objects listen (they are called listeners) those events to do something. In other ways, we attach objects to events.

The TweetService example

The nice thing about the event manager is that not only the framework can use them! Let’s take a simple example: we have written a TweetService class whose goal is to send a tweet thanks to the Twitter API.

But this crazy marketing guy said us that it could be fun to send an email to the tweet’s author too:

But it’s not enough for him, he also wants SMS:

Some problems quickly arise:

  • Our TwitterService now depends on the application. This specific application requires that a mail and SMS are sent, but maybe another one will just send the tweet, while another one will add a message to a queue. This is exactly what we want to avoid when we create a “generic” module whose aim is to be a “drop-in” module.
  • Furthermore, the TweetService depends on two other services too: an email service and SMS service. This means: every time we want to use the TwitterService module, we also must download the EmailService and SMSService modules, even if we don’t use them.

How to use it

Trigger an event

A very elegant solution to this problem is directly bundled into Zend Framework 2 : the event manager. Thus, we are going to simply trigger an event to say “hey mate, a tweet has been sent” instead of hard-coding what to do. Our code is then modified this way:

As we can see, we now only trigger an event called “sendTweet”. It’s up to the mail and/or SMS service to listen to this event to send an email/SMS.

You can also notice that this class implements the EventManagerAwareInterface interface. When we create an object that implements this interface through the service manager, Zend Framework 2 will automatically inject a new event manager for us by calling the setEventManager method.

Add listeners

Most of the time, listeners are added in the Module.php file, more precisely in the onBootstrap method. This is the easiest and recommended way. Intuitively, this is what most people do when they are first introduced to the event manager:

This code is pretty simple: we retrieve the application’s event manager, and add a callback (through a closure) that is called when the event sendTweet is triggered.

Unfortunately… this does not work. I know I know, you may think this is not intuitive, but there are reasons for that. Let’s introduce the shared event manager !

The shared event manager

Let’s go back to the TweetService. Earlier, I said that when a class implements the EventManagerAwareInterface, ZF 2 automatically inject a new event manager. I need to emphasize on the word new !

As a consequence, when we trigger the event sendTweet in the TweetService class, because the event manager is different (once again, it is a new event manager), the TweetService’s event manager has absolutely no knowledge about listeners that could have been added to other event managers. And this is absolutely what we were doing in the previous code snippet, as we were attaching a listener to the application’s event manager (which is different from the TwitterService’s one).

You may ask yourself why ZF 2 does not inject the same event manager everywhere. This way, the problem would be solved. But if you think about it more, this could rise even more problems. For instance, let’s imagine an event called send. Multiple objects may trigger an event called send, but for completely different purposes (send could be an event’s name used to send an email, a SMS, a HTTP request or whatever!). This would mean that listeners could receive event for things they would not be interested at all.

That’s why each object has its own event manager, with its own events.

To solve our previous example, we need to use a so-called shared event manager. A shared event manager is a manage which is unique across the application, and that is injected into each event manager (yeah I know, it’s not easy to grasp !). Let’s modify our Module.php code in order to attach the event into the shared event manager instead:

First, we retrieve the shared event manager from the application’s event manager. Then, we attach a listener. The subtlety is the first parameter whose value is here Tweet\Service\TweetService. Indeed, currently, without this parameter, the event manager of our TweetService has no way to “get” the listeners of the event it triggers.

Thus, we need to slightly modify our setEventManager method in TweetService:

Now, everything is linked properly. We are adding here a new identifier whose value is get_called_class() (in this case, it is equals to Tweet\Service\TweetService). When this service will trigger the event sendTweet, here is what will happen internally:

  1. The TweetService’s event manager will check if objects are listening to sendTweet, which is not the case here because the listeners have been added to the shared event manager (remember the Module.php code!).
  2. The event manager will then retrieve the shared event manager (remember it is unique !). Then, for each of its identifiers (in our case, it only has one, which is get_called_class), it will check if an event was added to the event sendTweet with the given identifier. In other words, it will check if there is a registered listener for the event sendTweet with the identifier Tweet\Service\TweetService (=== get_called_class()). This is exactly what we did in the onBootstrap method!
  3. Finally, for each listener, the callback will be executed (in our example, we simply called a var_dump($e))

This identifier mechanism is really powerful. For instance, let’s imagine we want to trigger the event sendTweet in several services. We could add another, more generic, identifier:

Now, update the Module.php class:

The special case of MVC events

I said earlier that we should use the shared event manager. But there is one specific case: the event manager we retrieve from the onBootstrap method is the MVC event manager. This means that this event manager knows the events triggered by the framework. This means that if you want to add listeners to the events of the Zend\Mvc\MvcEvent class, you can do it without using the shared event manager:

Let’s clean that…

In the previous example, we used a closure to define what the listeners must do. This is quick and easy, but if you have a lot of listeners, this can quickly be a mess. Furthermore, if you want to attach the same listener to multiple events, it will irremediably lead to a code duplication.

Hopefully, we can do it better by creating new classes that implement Zend\EventManager\ListenerAggregateInterface interface (starting from ZF 2.2, you can instead extend the abstract class Zend\EventManager\AbstractListenerAggregate). This interface asks you to write two methods: attach and detach. Here is the class that adds a listener to the sendTweet event:

This way, the onBootstrap method is simplified, and allow to move logic to specific classes instead of polluting the Module.php class:

Conclusion

Events are extremely powerful and are often a nice solution to some problems. However, please don’t abuse them: it makes the code harder to read and complicate the workflow (you need to know all the listeners that are registered for all the events, good luck if the code is not documented!).

From my personal experience, in your application-specific code, you’d rather hardcode the actions in your services, while you’d send events in generic modules.

Links

Zend Framework 2: Getting Closer With EventManager

Prevent CodeKit to refresh pages when using PhpStorm

Laisser une réponse

I’m a fervent user of PhpStorm and CodeKit on my Mac. Those tools make my job so much easier that I could not work without them today. However, one of the nicest feature of CodeKit, auto-refresh, does not work well with PhpStorm. Instead of injecting new content, it always refresh the page, which is annoying.

Hopefully, Caleb Fidecaro have shared the solution to prevent this. In the IDE settings, General tab, uncheck the “Use safe write” option. And boom, no more problem ! ;-)

How to automatically tag EC2 instances launched by Beanstalk’s Auto-Scaling ?

Laisser une réponse

At the time of writing, Amazon Beanstalk does not support to tag any new EC2 instances created by Auto-Scaling. This makes it rather inconvenient when you rely on tagging for billing, for instances.

EDIT 14th May 2013 : it appears I’ve found a better and simpler solution. This new solution assumes that your environment is configured with an IAM EC2 instance role that have enough privileges to add EC2 tags.

Now, add a .ebextensions folder at the root of your application, and add a script.config file. Copy-paste the following code:

Easy, isn’t it ?

Older solution

I recommend you the previous, newer solution because it’s better. Firstly, the older solution (described below) needs to be repeated each time the auto-scaling group is created. Secondly, it’s better to be able to change tags through a config file, which the older solution does not allow.

As Amazon Beanstalk does nothing more than launching underlying services, it’s rather easy to overcome this problem by using the Auto-Scaling CLI tool (because Auto-Scaling DOES support tag created instances).

To do this, you need to first download the Auto-Scaling CLI tool and configure it according to the tutorial there (if you are a OS X user, you can use Homebrew to install it more easily with the command “brew install aws-as”. Don’t forget to change the region if you are not using the default one (us-east-1). To change it, you need to modify the variable environment value.

Now that we have installed the CLI tools, let’s launch our Beanstalk environment. It will launch a first EC2 instance, without any tag. By default, Beanstalk automatically add some tags to your EC2 instances, as you can see below:

elastic-beanstalk-tags-1

Let’s add a tag for this first instance, for instance “client” => “my-best-client”:

elastic-beanstalk-tags-2

 

So far so good. The problem is that if our Auto scaling launches new instances based on criterias set for Beanstalk, new instances won’t contain this new tag. This is a problem.

Firstly, we need to find the auto-scaling identifier. In the screenshots above, this is the value for the key “aws:autoscaling:groupName”. Copy-paste this value. Then open the terminal, and type the following command:

as-create-or-update-tags –tag “id=AUTO_SCALING_NAME, t=auto-scaling-group, k=KEY, v=VALUE, p=true”

The AUTO_SCALING_NAME is the value of the “aws:autoscaling:groupeName”, KEY is the… key of the tag you want to add for every instances, and VALUE the value. Don’t forget to set the p to true, otherwise this tag won’t be propagated to any new instances launched by Auto-scaling:

terminal-beanstalk

 

To try this out, let’s force Beanstalk to create a new instance by modifying the environment settings and changing the min instance from 1 to 2:

beanstalk-auto-scaling-config

And magic ! The tag has been added automatically to the created instance:

elastic-beanstalk-tags-3

Now, you may want to remove the tag from being added to each new instances. Just write the following command (of course, if you want it to be removed from the already launched instances, you must do it manually, it won’t propagate to already launched instances):

as-delete-tags –tag “id=AUTO_SCALING_NAME, t=auto-scaling-group, k=KEY”

I hope Amazon will allow to add default tag directly in Beanstalk environment setting but for now, it’s the only solution that we have. Hopefully, it’s not that complicated !

Comprendre le gestionnaire d’évènements de Zend Framework 2

6 réponses

The English version of this article can be found here.

Introduction

Le gestionnaire d’évènements est sans nul doute l’un des composants le plus mal connu de Zend Framework 2. Pour autant, il est extrêmement puissant et offre une très grande flexibilité lorsqu’il est correctement utilisé. Cet article a pour vocation de vous initier à son utilisation.

Pour quoi faire ?

Le gestionnaire d’évènements est utilisé de manière abondante au sein du framework (on dit que ZF 2 est event-driven). Ainsi, les différents éléments de la boucle MVC (préparation, routing, dispatch, vue…) ne sont plus appelés les uns après les autres, comme c’était le cas sur ZF 1, mais à la place, le framework lance à différents endroits des évènements (“dispatch”, “route”…). Ces évènements sont “écoutés” par d’autres objets qui, en réponse, effectuent des traitements. Nous avons donc extrait la terminologie importante des évènements :

  • Des objets “lancent” des évènements (en anglais : trigger). Ces évènements ont systématiquement un nom (“route”, “dispatch”, “sendTweet”…) et, souvent, contiennent des paramètres additionnels (par exemple, un évènement “sendTweet” pourrait contenir le contenu du tweet).
  • Des objets “écoutent” ces évènements pour y réagir (en anglais : listen). En d’autres termes, on attache des objets à des évènements.

Le cas du TweetService

Nous avons vu que le framework utilise abondamment les évènements. Mais il est également possible d’en tirer parti dans notre propre code. Prenons un exemple simple : nous avons écrit un TweetService dont le seul et unique rôle est d’envoyer le contenu d’un tweet via l’API Twitter :

Puis on se dit que finalement, il serait également utile d’envoyer un e-mail à l’auteur du tweet pour l’avertir :

Et puis finalement, il serait peut-être également sympa de lui envoyer un SMS :

Bref, vous avez compris le soucis, ou plutôt les soucis :

  • Notre TwitterService est maintenant complètement couplé à notre application. Cette application requiert d’envoyer un mail et un SMS, peut-être qu’une autre se contentera juste d’envoyer un tweet. Il sera ainsi nécessaire d’aller modifier le code pour chaque application, ce qui est évidemment ce que nous souhaitons éviter lorsque l’on écrit un module “générique”.
  • De plus, notre TweeterService est maintenant fortement couplé avec deux autres services : un service d’envoi de mail et un service d’envoi de SMS. Ceci signifie : écrire des fabriques plus complexes pour tout injecter, et installer systématiquement les dépendances vers des modules supplémentaires.

Comment s’en servir ?

Lancer un évènement

Heureusement, il existe une solution extrêmement élégante à ce problème : le gestionnaire d’évènements. Ainsi, plutôt que de coder en dur les actions à réaliser, nous allons lancer un évènement qui va signaler qu’un tweet a été envoyé :

Comme nous pouvons le voir, nous nous contentons maintenant juste de lever l’évènement “sendTweet”. A charge de notre service de mail et/ou de SMS de s’abonner à cet évènement et d’y envoyer correctement le mail.

Notez également que notre classe implémente l’interface EventManagerAwareInterface. Lorsque nous créons un objet implémentant cette interface, Zend Framework 2 va automatiquement y injecter un gestionnaire d’évènements tout neuf.

Ajouter des listeners

La plupart du temps, vous allez ajouter des listeners à des évènements dans le fichier Module.php, plus précisément dans la méthode onBootstrap. C’est la méthode la plus simple et celle qui est recommandée. Voici, de manière intuitive, ce que la plupart des débutants font :

Ici, nous récupérons le gestionnaire d’évènements de l’application, et nous attachons une fonction à réaliser dès que l’évènement “sendTweet” est levé.

Hélas… ceci ne fonctionne pas ! C’est la partie la plus compliquée du gestionnaire d’évènements, et l’astuce consiste à utiliser un gestionnaire d’évènements partagé.

Le gestionnaire d’évènements partagé

Revenons à notre TweetService. J’ai dit que lorsque l’on implémente l’interface EventManagerAwareInterface, ZF injecte automatiquement pour nous un gestionnaire d’évènements. La subtilité, c’est que ZF 2 injecte un nouveau gestionnaire d’évènements à chaque fois.

Conséquence : lorsque nous lançons l’évènement sendTweet dans le TweetService, comme le gestionnaire d’évènements est différent, celui du TweetService n’a aucune connaissance des listeners qui ont pu être ajoutés dans d’autres gestionnaires d’évènements (dans notre cas, celui de l’application).

Pourquoi est-ce que ZF 2 n’injecte tout simplement pas le même gestionnaire d’évènements partout ? Ainsi, le problème serait réglé… Hélas, si nous faisions ceci, nous aurions rapidement des problèmes. Imaginons un évènement tel que “send”. Plusieurs objets pourraient lancer un évènement nommé “send”, mais de nature complètement différentes (“send” peut aussi bien concerné un envoi de mail, que de SMS, ou d’un envoi de requête ou que sais-je !). Ce qui signifie que nos écouteurs (listeners) recevraient des évènements qui ne les concernent absolument pas ! C’est pourquoi chaque objet dispose de son propre gestionnaire d’évènements, avec ses propres évènements indépendants aux autres gestionnaires d’évènements.

Pour résoudre notre problème précédent, il faut utiliser un gestionnaire d’évènements partagé. Un gestionnaire d’évènements partagé est un gestionnaire qui, lui, est unique, et qui est injecté automatiquement dans chaque gestionnaire d’évènements. Modifions notre code de Module.php afin d’enregistrer l’évènement auprès du gestionnaire d’évènements partagé :

En premier lieu, nous avons d’abord récupéré le gestionnaire d’évènements partagé à partir du gestionnaire d’évènements. Puis nous avons attaché un listener. La subtilité, c’est le premier paramètre : Tweet\Service\TweetService. En effet, à l’heure actuelle, sans ce paramètre, le gestionnaire d’évènements, lorsque l’on appel trigger dans notre TweetService, n’a aucun moyen de “récupérer” les listeners de l’évènement qu’il déclenche.

Ainsi, nous allons devoir légèrement modifier notre fonction setEventManager dans notre TweetService :

Maintenant, le lien va pouvoir être effectué correctement. Nous ajoutons dans notre $eventManager un identifiant qui ici vaut get_called_class() (donc Tweet\Service\TweetService). Lorsque ce service va lancer l’évènement sendTweet, voici ce qui va se passer :

  1. Le gestionnaire d’évènement de TweetService va vérifier si des objets écoutent l’évènement sendTweet, ce qui n’est pas le cas car les écouteurs ont été ajouté dans le gestionnaire d’évènements partagé.
  2. Le gestionnaire d’évènements de TweetService va ensuite récupérer le gestionnaire d’évènement partagé, unique à tous les gestionnaires d’évènements. Pour chacun de ses identifiants (dans notre cas, il n’y en a qu’un), il va vérifier si quelqu’un a ajouté un évènement sendTweet pour l’identifiant donné. En clair : il va vérifier si il y a un écouteur pour l”évènement sendTweet attaché à l’identifiant Tweet\Service\TweetService. Bingo ! C’est exactement ce que nous avons fait dans la fonction onBootstrap !
  3. Pour chaque écouteur trouvé, la fonction va être effectuée (dans notre cas, un simple var_dump($e) est effectué dans Module.php.

Ce mécanisme d’identifiant est très puissant. Par exemple, imaginons que nous souhaitons envoyer l’évènement sendTweet dans plusieurs de nos services. Nous pouvons ajouter un identifiant complémentaire et plus générique :

Maintenant, mettons à jour onBootstrap :

Le cas des évènements de la boucle MVC

J’ai dis plus tôt qu’il fallait utiliser le gestionnaire d’évènements partagé. A une exception : le gestionnaire d’évènements que nous recevons dans la méthode onBootstrap est justement le gestionnaire d’évènements de la boucle MVC principale. Ce qui signifie que ce gestionnaire d’évènements connaît les évènements lancés par le framework. A ce titre, si vous souhaitez ajouter un listener aux évènements décrits dans la classe Zend\Mvc\MvcEvent, vous pouvez le faire sans passer par le gestionnaire d’évènements partagés :

Nettoyons un peu tout ça…

Dans les exemples précédents, nous avons définit ce que doivent faire nos écouteurs en utilisant une fonction anonyme (le deuxième paramètre dans le cas du gestionnaire d’évènements, ou le troisième dans le gestionnaire d’évènements partagés).

Or, si vous avez beaucoup d’écouteurs différents à enregistrer dans la méthode onBootstrap, le code peut rapidement devenir illisible. De même, il est possible que vous souhaitiez enregistrer le même écouteur pour plusieurs évènements, ce qui entraînera irrémédiablement une duplication de code.

Heureusement, nous pouvons nettoyer ceci en créant des classes implémentant l’interface Zend\EventManager\ListenerAggregateInterface. Cette interface impose d’implémenter deux méthodes : attach et detach. Voici la classe permettant d’écouter l’évènement sendTweet du gestionnaire d’évènements partagé :

Du coup, notre onBootstrap est simplifié, et le SendListener peut être réutilisé. Cela permet de déplacer la logique dans des objets spécifiques plutôt que de tout foutre dans Module.php, ce qui est toujours une bonne chose.

Conclusion

Les évènements sont extrêmement puissants et s’avèrent parfois la solution idéale à certains problèmes. Toutefois, il est important de ne pas en abuser : il rend la compréhension du code plus difficile car il est nécessaire de voir quels sont les écouteurs enregistrés pour chaque évènement lancé.

De manière générale, dans votre code métier (donc les modules non génériques), préférez réaliser autant d’actions que possibles dans vos services, plutôt que de lancer des évènements partout.

Liens

Zend Framework 2: Getting Closer With EventManager

Beanstalkd and Amazon SQS queues with Zend Framework 2

Laisser une réponse

I’m always amazed by the flexibility of Zend Framework 2. Modules really brought Zend Framework to another level, and you should not be shy to use modules.

Speaking about that, I’ve completely refactored SlmQueue, thanks to Jurians Sluiman. Previously, it only supported Beanstalkd queue systems, but it now supports Amazon SQS too, thanks to a flexible architecture.

I’ve put all my efforts to make those modules as high-quality as possible, and follow the latest best practices (yes, the ones I described in my previous article). Therefore, you may be interested to read the source code.

SlmQueue is now splitted into three modules: SlmQueue is the base module that provides interfaces and abstract classes shared by every queue systems, while SlmQueueBeanstalkd and SlmQueueSqs provides implementation for those systems.

Feel free to try those modules !

Some tips to write better Zend Framework 2 modules

10 réponses

Update 4th february 2013 : add a new tips about dependencies

Update: once again thanks to the amazing Oleg, this article is available in Russian.

While working on various Zend Framework 2 modules (currently, my biggest project is working on ZfrRest, which aims to provide a powerful REST module) with many very talented developers (Matthew Weier O’Phinney, Marco Pivetta…), I really learnt a lot of things about PHP and how to write better, safer and more flexible code.

This article is more targeted to ZF 2 beginners who would like to spread to the world their super-über-fantastic module. However, if you would like people to use it (and hence, make it better !), you really need to ship something that is good. I hope that those few advices (some are really obvious, but I see too many modules that do not respect even the simplest rules).

There’s a module for that !

Before writing your module, check if somebody did not write it before ! Instead of doing your own module, just make the existing ones better by sending pull requests. You can check ZF2.modules and find for a module with keywords !

For more information about this topic, check Martin Shwalbe’s blog post about modules.

Follow conventions

Zend Framework 2, as any other modern frameworks, has strict code conventions. Of course, the framework respect PSR-0/1/2 conventions, and you should must too. But we also have some standard conventions about where should we put some files.

For instance, many modules are configurable. Once you have many options, it’s a good idea to centralize those options into an Options class (that extends Zend\Stdlib\AbstractOptions). It’s common practice to save all those files into an “Options” folder so that each options classes have their own namespaces. Of course, you could put those classes wherever you want, but as it is how we do it, it is much more easier for developers to read the code if you save those classes in predictable places.

Furthermore, I’ve seen many people that put a lot of files into the same folder just to avoid create new one. Seriously, just create new folders. It won’t hurt anyone and will simply help people to more easily discover the files. For instance, if you create a Paginator adapter, just mimic ZF 2 structure and save it into YourModule\Paginator\Adapter\MySuperAdapter, not into YourModule\Form\Adapter\MySuperAdapter, just because you’re lazy to create a new folder :-) .

For example, see the DoctrineModule options classes.

Allow to configure everything that needs to be configurable

Modules are meant to be shared and reused by other people. And people don’t always have exactly the same needs as you are. Whenever a thing “could” be modified by other people, add options to allow to modify it. For instance, if your module generate some HTML page with some inline style that is copied from a CSS file shipped in the module… just set a way so that people can set the path to their own CSS file (this way they won’t need to manually tweak the CSS you ship in the module).

When you ship your module, you should ship two files into the config folder. One file called “module.config.php” that provides “standard” configuration and configuration that should not be modified by the user AND another file called “my_module.global.php.dist” that exposes all the options that can be overridable by the user (this file is typically copy/pasted to the config/autoload folder of your application).

A great example of that is the ZfcUser config file. As you can see, pretty much everything can be overridden, so the module is really flexible.

Set your hard dependencies in constructor, avoid initializers and setters/getters

The service locator of Zend Framework 2 really allows to cleanly handle complex dependencies, mocking… it’s really a powerful tool that is not that hard to use once you understand it. However, ZF 2 shipped with something called initializers. I’m pretty sure you’ve encountered them already. For instance, they are ServiceLocatorAwareInterface, or EventManagerAwareInterface… Whenever a class implements those interfaces, the service manager will automatically inject (through setters) the dependencies.

While this is nice, it suffers from one problem that can lead to a lot of pain and bad use from your module’s users. For instance, in ZfrRest, we have classes that are used to parse HTTP request/response object. All those parsers extends an abstract class called AbstractParser. You can see the class here.

As an example, here is the constructor :

Instead of the constructor, we could have written a setter/getter to set the DecoderPluginManager, and inject the dependency through the service manager or using an initializer. BUT this assumes that your user will effectively use the service manager to create your object.

Until the day someone create the parser like this:

And badaboom. He didn’t call setDecoderPluginManager nor didn’t create the instance through the service manager. The dependency is not set correctly and this code will throw an exception.

BECAUSE the DecoderPluginManager is an essential dependency of this object (it cannot work without this dependency), just ask to set the dependency directly through the constructor. So that people can’t use your object badly. Even if they want to.

Reduce your closure usage for factories

It’s common usage in ZF 2 to handle dependencies through the use of closures as factories in the getServiceConfig of your Module.php class. Instead, you’d better use explicit class factories, and set define the factories in the module.config.php file (in the service_manager key). This is slightly more efficient (because closures are instantiated, it’s faster to create a string than a closure), allow you to cache config file (closure are not cacheable), and remove the unreadeable Module.php with tons of closures.

Use service manager as much as possible to put dependencies

This advice closely follows the one on “allow to configure everything”. Sometimes, people really want really weird things. Trust me. You sometimes ask yourself “why the hell do you want this feature ?”. But they want. Of course, you won’t add this feature to the core of your module because you still want to kick this guy’s ass for suggesting such a strange thing, but at least you must make it easy for him to override everything.

And with ZF 2 architecture, this is really simple : create most of your objects through the service manager. For instance, in ZfrRest, we add some listeners that provide useful features in REST context. The first way to do it is simply this:

However, think about this strange guy who want to also send a mail to his grand-mother whenever an exception is thrown. How can he do it currently ? Well… he can modify your module code, but this is not really good as whenever he will update your module, he will need to make the changes again.

Fortunately, using the Service Manager solves this:

Now, the listener is pulled from the service manager. Of course, ZfrRest has a default and sane implementation of this listener, by adding an invokable to the service manager config:

Now, if someone want to override the listener, he just needs to override the invokables in his own config file:

Write unit tests

Write unit tests for your module (yes, it’s boring). I’ve already use untested modules and it’s really pain in the ass. Whenever I updated it, my whole application failed because an upgrade to the module break something. This is even more important for modules that people want to use professionally. If they are untested, it’s nearly a no-go.

Once again, write unit tests for your module. Full stop.

Host your module on Packagist (for Composer use) and zf.modules

Many people use Composer to handle dependencies between modules. While you may hate it, just remember that many people love it. So please post your module to Packagist if you think it’s good enough.

Furthermore, you can add your module to ZF2.modules page.

Only set dependencies on what you require

Zend Framework 2 is a modular framework. This means that people don’t need to download the full framework to be able to use it. For instance, some people may be interested only about the filter and validator components (and using the Symfony 2 Mvc component). All of this thanks to Composer.

Of course, people will be angry if using your module means downloading the full ZF 2 framework. This is why instead of setting the whole framework as a dependency, just set the specific components you need. For instance, let’s say that your module uses the ZF 2 InputFilter component, nothing else. Just replace this in the composer.json of your module :

by :

Now, it will download ONLY the Zend Input Filter (well, not exactly, because the InputFilter component itself has some other dependencies, but this is not your problem and this is handled by composer, because each ZF 2 component has its own composer.json file !).

If you have any other advices, don’t hesitate !

MONO à la Maroquinerie (9 décembre 2012)

Laisser une réponse

Dans l’univers un peu fermé du post-rock, MONO fait figure de référence, aux côtés de quelques grands noms tels que Mogwai, Explosions In The Sky ou GY!BE. Pour autant, MONO a ce petit quelque chose d’épique que les autres n’ont pas. Peut-être leur collaboration avec le Wordless Music Orchestra qui donne cette dimension unique à leurs titres. Du coup, j’étais un peu inquiet quant à leur capacité, à 4, à retranscrire toute la dimension de leur musique.

Et j’avais tort… Comme souvent dans le post-rock, et à fortiori chez MONO, les intros au glockenspiel (d’ailleurs fortes jolies dans le silence d’une salle de concert), ne sont qu’un prétexte pour des envolées foudroyantes (pour lesquelles la salle ne rendait d’ailleurs pas honneur, tant le son souvent saturait). L’imposant gong qu’ils se trimballent en live apporte un son peu inhabituel dans le post-rock, et les deux guitaristes du quator japonais (Takâkira Goto et Hideki Suematsu) donnent tout ce qu’ils ont pendant les (nombreuses) envolées des chansons, et parviennent largement à combler l’absence des violons présents, notamment, sur Hymn To The Immortal Wind et le magnifique album live avec l’orchestre Wordless Music.

Niveau setlist, nous avons eu le droit leur (très bon) EP For My Parents, ainsi que plusieurs chansons phares d’Hymn To The Immortal Wind dont Follow The Map et l’épique Ashes In The Snow.

A noter également que MONO est un groupe extrêmement réservé (et qui entretient certainement cette image). Nous avons pu boire un verre sur la table juste à côté d’eux, qui étaient tranquillement en train de lire. Et sur scène, on ne peut pas dire que MONO soit très communicatif : les deux guitaristes restent assis, leur regard constamment sur leur guitare et le visage caché par leur longue chevelure, ou encore le batteur qui allonge sa tête sur sa batterie lorsqu’il ne joue pas, et un simple salue japonais en guise d’aurevoir ! Et pour autant, ça ne m’a pas gêné un seul instant. La puissance et la beauté de leur musique compense largement leur inactivité totale sur scène.

Bref, un groupe que je souhaitais voir depuis longtemps et qui ne m’a pas déçu (si ce n’est par moment l’acoustique de la salle), et qui définit parfaitement ce qu’est pour moi le post-rock. A voir, d’autant plus qu’ils repassent en France le 19 février 2013 à Feyzin, le 26 février à Nîmes, le 27 février à Limoges, le 28 à Amiens et le 1er février à Mulhouse.

mono-on-stage

Installer PHP, Apache et MySQL sur Amazon EC2 + RDS

Laisser une réponse

Mise à jour 26 février 2013 : Je viens de découvrir il y a quelques jours un autre service d’Amazon AWS appelé Amazon Beanstalk. Ce service automatise la plupart des tâches que je décris dans cet article, avec quelques trucs en plus (l’installation d’un load balancer entre autres). Du coup, je vous conseille d’avantage d’utiliser ce service.

Merci à Jérémy Pinhel et Florent Blaison pour leur aide ^^.

Nous réfléchissons avec mon associé d’héberger notre futur site via les services Amazon AWS. Comme c’est gratuit pendant 1 an (à condition de n’utiliser que les plans “micro” pour chaque service), on s’est dit qu’on allait tenter (et puis, soyons honnête, l’aspect serveur n’est ni mon fort, ni celui de mon associé).

J’ai réussi à installer sans quasiment aucune encombre la base de données MySQL (via le service Amazon RDS), et le serveur (sur Amazon EC2, en installant Apache et PHP, ainsi que quelques extensions PHP indispensables). Cet article est surtout un pense-bête pour moi même, mais il peut être utile à certains.

Amazon RDS (base de données)

  1. Launch DB instance : création de l’instance.
  2. Choisir MySQL.
  3. Génération de mot de passe : https://www.grc.com/passwords.htm
  4. Lancer l’instance, et créer un Security Group afin de permettre à la machine ou au serveur S3 de se connecter à la base. Il faut ensuite modifier l’instance afin d’ajouter le groupe de sécurité créé.

Amazon EC2 (serveur web)

  1. Launch instance : création de l’instance.
  2. Choisir Classic Wizard, puis sélectionner Ubuntu Server (ou n’importe quelle autre, mais les étapes suivantes n’ont été testées que sur Ubuntu Server 12.04).
  3. Ajouter un nom d’utilisateur pour la clé SSH, et la télécharger.
  4. Pour se connecter à l’instance : sudo ssh -i [FILE NAME].pem ubuntu@ec2-[IP ADDRESS].compute.amazonaws.com

Sur OS X, on peut stocker le fichier SSH :

  1. Copier le fichier .pem dans le dossier .ssh (taper la commande open ~/.ssh pour ouvrir le dossier).
  2. Maintenant on peut taper : sudo ssh -i ~/.ssh/[FILE NAME].pem ubuntu@ec2-[IP ADDRESS].compute.amazonaws.com
  3. On ajoute ensuite une configuration SSH afin de permettre aux applications de connaître l’emplacement du fichier SSH. Pour cela on ouvre/créé le fichier ~/.ssh/config, et on ajoute la ligne IdentityFile "~/.ssh/[FILE NAME].pem"
  4. On change les permissions du fichier .pem en tapant la commande chmod 700 ~/.ssh/[FILE NAME].pem
  5. Redémarrer la machine. On peut ensuite se connecter directement sur Transmit en tapant le nom du serveur, et ubuntu comme nom d’utilisateur (ou root sur un autre serveur Linux), et en laissant le mot de passe vide.

Par défaut, les autres dossiers du serveur ont comme propriétaire “root”, ce qui signifie qu’on ne peut pas y transférer de fichiers depuis un FTP, par exemple. Généralement, on balance ses sites dans le dossier /var/www. On va donc changer le propriétaire juste pour ce dossier :

  1. Aller dans le dossier /var en tapant cd /var
  2. Et taper : sudo chown -R ubuntu:ubuntu /var/www (en assumant que l’utilisateur s’appelle ubuntu, ce qui est le réglage par défaut lorsque l’on crée une instance Ubuntu Server sur EC2).

Création de l’IP

Afin de rendre le serveur accessible, il faut créer une Elastic IP à partir du panneau de contrôle d’EC2. Puis associer l’IP à l’instance (note : du coup, l’accès SSH change car l’IP a été modifiée vers une IP publique).

Il ne faut pas oublier d’ouvrir le port 80 de l’instance EC2 :

  1. Dans l’onglet “Network & Security” de EC2, cliquer sur Security Group.
  2. Dans l’onglet “Inbound” de la page de droite, ajouter une règle TCP pour le port 80.

Installation d’Apache

  1. sudo apt-get install apache2
  2. sudo /etc/init.d/apache2 start

Par défaut, certaines extensions utiles ne sont pas activées :

  • Pour mod_rewrite : sudo a2enmod rewrite
  • Pour mod_expires : sudo a2enmod expires
  • Pour mod_headers : sudo a2enmod headers
  • Pour mod_filter : sudo a2enmod filter

Pour installer l’extension mod_pagespeed de Google, jetez un coup d’oeil à ce lien. Au cas ou le lien venait à disparaître :

  1. cd ~
  2. wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-beta_current_amd64.deb
  3. sudo dpkg -i mod-pagespeed*.deb
  4. rm mod-pagespeed*.deb
  5. Configurer le module : sudo nano /etc/apache2/mods-available/pagespeed.conf

Puis redémarrer Apache : sudo /etc/init.d/apache2 restart

Installation de PHP

Note : les commandes suivantes permettent d’installer la dernière version de PHP 5 (à l’heure où j’écris, cela m’a installé PHP 5.4.9). Pour installer la dernière version “embarquée” par Ubuntu, c’est une autre commande (dont je ne me souviens plus) :

  1. sudo add-apt-repository ppa:ondrej/php5
  2. sudo apt-get update
  3. sudo apt-get install php5

Pour installer Intl (une fois Apache installé) :

  1. apt-cache search php5
  2. sudo apt-get install php5-intl
  3. sudo /etc/init.d/apache2 restart

Pour installer APC (une fois Apache installé) :

  1. sudo apt-get install php-apc
  2. sudo /etc/init.d/apache2 restart

Installation de MySQL

Même si nous utilisons RDS pour la base de données MySQL (la base ne sera donc pas stockée sur notre instance EC2 mais sur l’instance d’Amazon RDS), on doit quand même installer MySQL pour récupérer, entre autre, l’extension MySQL pour PDO :

  1. sudo apt-get install php5-mysql
  2. Logiquement, le fichier /etc/php5/mods-available/pdo_mysql.ini a été automatiquement créé. Si ce n’est pas le cas, ajouter la ligne extension=pdo_mysql.so dans le fichier /etc/php5/mods-available/pdo.ini.

Quelques chemins utiles…

  • /var/www : dossier pour mettre vos sites
  • /etc/php5/apache2/php.ini : fichier de configuration PHP
  • /etc/apache2/apache2.conf : fichier de configuration Apache
  • /etc/apache2/sites-enabled : dossier dans lequel mettre vos Virtual Hosts
  • /etc/hosts : fichier hosts

Création de l’AMI

Je n’ai pas réussi à créer une AMI à partir de l’instance EC2. Si quelqu’un sait comment le faire, je suis preneur !

Envoi d’un site

Il suffit de créer un site dans le dossier /var/www, puis d’ajouter un virtual host dans /etc/apache2/sites-enabled, et enfin de redémarrer Apache en tapant sudo /etc/init.d/apache2 restart.

Il se peut que lorsque l’on tente d’accéder au site, une page blanche s’affiche avec une vilaine erreur 500. La solution : chez moi, le dossier “data” de mon site (qui contient entre autre les templates Twig, les données de cache…) n’était pas

Redirection

Si vous avez un nom de domaine chez OVH que vous souhaitez rediriger vers votre serveur EC 2 :

  1. Aller dans le manager OVH.
  2. Cliquer sur Domaine & DNS.
  3. Dans la partie Zone DNS changer la ligne dont le type est ‘A’ en mettant l’IP publique de votre serveur Amazon.

NOTE : apparemment il est également possible de gérer ses DNS depuis le service Route 53 d’Amazon.

Une fois ceci fait, il faut modifier le virtual host :

Et d’ajouter la ligne suivante dans le fichier /etc/hosts : 127.0.0.1 [mondomaine.com].

Questions and thoughts about client-side rendering from a ZF 2 developer

6 réponses

Disclaimer: I’m completely new to client-side rendering and, in general, to JavaScript. This article is just here to share my initial thoughts about client-side rendering and get help from people that better understand it.

Everything began last week on a sunny monday, when Marco Pivetta (aka ocramius), my dear #zftalk friend, talk to me about client-side rendering (he’s building a CMS).

To be honest, I’ve never took time to investigate client-side rendering before, so I was quite lost. But after one week of research, I’m kinda feeling stupid. I feel like I just discovered a completely different way of thinking websites, and how to make them. I hope this article will make people realize this new trend, because it really does sounds exciting and you could be asked to use it rather sooner than later.

For those who know me, I’m a Zend Framework 2 contributor, so I may refer to Zend Framework, but of course it applies to any server-side technology.

Client-side rendering vs server-side rendering

Server

There are chances like, that me, your workflow when thinking website was similar to this:

client-side-1

This is easy to understand:

  1. Client makes a request to your server.
  2. The framework dispatches the request to an action.
  3. Your action may load a lot of different objects (in this example I use a forum – because I’m making a Zend Framework 2 forum module (check it out and help us if you want !): categories, threads in the categories…
  4. Those data are sent to your server-side templates, renders it, and returns everything to the client.

As a consequence, as soon as the page is loaded, the user already has access to all information. And it’s SEO friendly too. Of course, when the user changes page, this round-trip to the server has to be made again, which slows done the user experience.

Like me, you have surely played with jQuery to make Ajax query. And, I suppose you understand me when I say that it is painful. Basically, you end up to bind a lot of elements and doing a boilerplate code that quickly becomes unreadable and unmaintenable. I won’t show you my JS code in my current website for your mental health…

Client

Client rendering, on the other hand, is a way to reduce this round-trip to the server, so that your webpage (nearly) never reloads. Of course, it won’t work this magically. Most of the time, you will be using a Javascript MVC framework that offer you a MVC pattern in Javascript. Some popular frameworks are ExtJS, Ember.js, Knockout, AngularJS… I’m personally trying to use AngularJS.

Here is how it works with client-side rendering:

client-side-2

As you can see, this is not as intuitive. Here are the steps that happen:

  1. Client makes a request to the server.
  2. Most of the time, whatever the request (/forum/categories/1, /forum/threads/4…) the request will be dispatched to the same action, that will just returns a minimal HTML (most of the time, just the layout).
  3. Your JS framework will be initialized, and your JS framework will “route” the request (notice that routing is moved from server-side to client-side too…). For instance, the URL /forum/categories/1 may be bound to a specific controller in JavaScript and a specific HTML template.
  4. The JS framework will make a request to a REST API (or any other endpoint, but REST plays really nice) to get the categories, and another one to get the threads (you could return both at once, but this may lead to a bad API design).
  5. The server will returns the data as JSON/XML.
  6. The browser will parse the JSON, and bind the JSON data to a JavaScript template.

There are a lot of things that I really had a hard time to understand. First of all, if you have JavaScript disabled, you’ll basically receive an empty page. If you return a layout, you will have a minimal layout, but nearly nothing more. This means that the user have to wait until he can actually sees data. If your page depends on a lot of different data (this means making different REST calls), the page may appear in “chunks”. Of course, you can just implement a “spinning icon” while all the data has not been loaded, but it is completely different to what I used to know before. Furthermore, this means that SEO is harder and you need to use some tricks to make it crawlable (more about this in this article - this is targeted to AngularJS but may apply to any other JavaScript framework -).

As a consequence, instead of one single-request that would load everything, with client-side rendering you will have several small HTTP requests that are made to the server. Depending on the context, I suppose this may be slower.

BUT there are some advantages to client-side rendering:

  • Less work to the server. It does not have anything else to do other than sending back JSON data. It does not have to take care of the rendering.
  • When the user changes pages (for instance, when he goes to the second page of categories), instead of reloading the whole page, the JavaScript framework will just make another Ajax call to your REST API, and just update the part of the page that needs to be updating.

The hard parts

What confused me a lot, is that templating is not done on PHP side, but on JS side. This means that you just do PHP, because the templates are just fetched by the JS framework directly and “compiled” by JavaScript. Obviously, there are things that I don’t know how to do yet in client-side.

Complex conditional renderings

In one of my application, I have a part that is rendered according to “complex” criterias (if the user is logged, show this, if he is logged AND is the author, show this…). Such conditional renderings use several entities, and I don’t really know how to efficiently map this to client-side rendering and REST API, that basically returns one kind of resource.

i18n (internationalization)

When using server-side template and a framework like Zend Framework 2, internationalization is easy. You have some view helpers that will fetch translation, for instance, in a Gettext file. This means that it is really easy to extract the strings to translate from your website, send the file to translators, get back the file, update the translations…

On the other hand, it’s much harder in JavaScript. Some JavaScript frameworks like AngularJS have a built-in component to translations, but translations strings have to be put in a JS file. This is really a pain in the ass, as people have to download another JS file (and what if there are hundreds/thousands of strings to translate ? It’s going to be super-slow to download !).

Authentication

Obviously, authentication is not as easy to do. On server-side, using Zend Framework 2 identity helper, you just can do something like this:

It’s not as easy in client side.

Some solutions to the above problems

One of the problem that annoyed me the most is that when you are going to an URL, let’s say /forum/categories/2, you’ll basically have an empty page without any content. This is sad because, after all, an initial request is made… just to return, at most, a stupid empty HTML layout. Why not use this request to load some data and bind it to the view ? It appears that I’m not the first one asking myself this but it’s not that easy to do (I didn’t manage to do this using AngularJS, while some other manage to do it natively).

The thing is that, to me, it appears that it’s a “everything or nothing”. You either render your template in server-side, or in client-side, but you can’t do both.

So… when to use client-side rendering ?

I’m asking myself this question. On the website I’m working, it’s really a “standard” website. Some pages have very complicated conditional rendering, with data coming from several tables in the database… I just fetch all the data I need in one action, render it and voilà ! I don’t really see how I could integrates this to my website. I’ve found some solutions, but they rely on complex solutions (for instance using another JS library like PhantomJS that would render the JS templates in the server).

One solution I’ve thought about is doing a “hybrid” approach. I didn’t try this approach yet, so give me feedbacks if you think it would work or not. My idea is to have two URLs. For instance, let’s go back with my categories forum example. I would have to different routes :

  • /forum/api/categories/2 : this is a purely REST URL. It would be the API and would simply returns JSON data.
  • /forum/categories/2/my-category : this is a “standard” action, dispatched using ZF 2 router, and with a more SEO-friendly URL. It would act as a “normal” action, so it would load not only the category 2, but also all the threads of this category, so that the server renders a completely constructed page.
Of course, if the user navigate to page 2, I would like NOT to reload the whole page, but being able to use my JS framework ! In AngularJS, I could use something like that. I would attach an event handler to the /forum/categories/2/my-category?page=2 link so that when the user click it, it makes the Ajax request, and replace the actual content using, this time, an Angular template ! When the user goes to page 3, as this is already an Angular template, you have nothing to do.
The drawback of this approach is that you have to maintain 2 templates (and 2 routes, one for the API and one for the “first-time request”) : one template that would be rendered by the server (with <?php echo ; ?> code), and one using AngularJS binding mechanism. But on the other hand, I think that having a correct REST API is cool too, as you could use it for anything else (desktop application, native mobile application…).
Of course you have more work to do, but I think this approach would allow to have the best of both worlds, although I really hate duplicating the templates. But I’m not sure if this would work completely =)…

How to use Zend Framework 2 for client-side rendering applications ?

As you can imagine, using a Javascript framework delegates a lot of work from the server, to the client. This means that you will be using less-features of the server-side.

AbstractRestController

You will be using this a lot. And fortunately, it’s pretty good and intuitive. Instead of extending from AbstractActionController, your controllers will extend AbstractRestfulController. You will need to implement 5 abstract functions (getList, get, update, post and delete). Those functions will be automatically call by ZF 2 depending on the HTTP verb action (POST, GET, DELETE and PUT). If a parameter named “id” is matched, it will select get function (/forum/api/categories/2 for instance), otherwise it will select getList (/forum/api/categories for instance).

Forms

You will no longer use forms too (haha, quite ironic for me as I am one of the maintainer of the Zend\Form component ^^). In fact, as you will most likely render your form on client-side, you won’t be able to generate them using ZF 2 view helpers.

HOWEVER, you will still need to validate data ! In ZF 1 it was problematic because the validation rules were defined in the Form object itself. In ZF 2, this task was delegated to the InputFilter object. Therefore, you are encouraged to use the Zend\InputFilter component to validate/filter your data on your server-side. As a consequence, if you want to use client-side framework, instead of defining your input filter by implementing the InputFilterProviderInterface in your fieldsets:

Just define your input filter in a different class that extends Zend\InputFilter. This will allow you to create form but only to instantiate the InputFilter and use it in your REST actions (most likely update and create).

Hydrators

You’ll likely still use hydrators, most of the time in your REST actions to convert array data to an object, that will be consumed by your services.

Conclusion

Do you think client-side rendering should be only used for web-app (I mean, applications that look like desktop/mobile applications), or do you need that it could be used in standard website too ? Of course, it would make the user experience so much smoother, but there are a lot of things I don’t really figure how to make them properly using a client-side library.

My friend and me are still thinking about what to use for our ZF 2 Forum module, so please give me some feedbacks on this ;-) .