Standardizing development environments with Vagrantpar Clément Mégnin le 09/12/2015 Labs
This is the first article of a series in which we'll explain our development process industrialisation. We'll describe how we went from a team of individuals doing things together but artisanally, to a more industrial and qualitative approach. This first topic talks about how we standardized our development environments with Vagrant.
When working in a team, every person has its own habits, methods and tools. Without any direction, if you have x developers, you'll end up with x versions of the same solution for a given problem. Diversity is good but when we try to put together a productive development team of more than 5 persons, you'll have to ensure that any developer is able to understand and pursue the work of everyone else quickly and efficiently.
That's why you need to establish a common ground. You come together and reflect on workflows, processes, good practices, and establish rules that define a way of working in this team, born from the input and adhesion of everyone with the most sense as possible.
We've been doing this with our code base for a long time at NOE. The whole team gathers once a month for example, to discuss the best way to handle a set of challenges encountered in the last month. That makes everyone aware of the team solution when they'll face it on their own in the future. We discuss workflows as well, to make sure we agree on our methods, and criticize them when we feel they are not optimum.
Regarding our development environment on the other hand, we never really took the time to reflect on the matter. The first step we took a long time ago, was to choose a common IDE to code, with the same writing settings like encoding and indentation. We've done so to avoid that a file coded by one person, would be totally misaligned and wrongly encoded when opened on another machine. But since then, we didn't really had any problems with our setup.
So what was the setup like at NOE?
We could summarize it in two word: heterogenous and artisanal. First of all, we have different machines, MACs and PCs alike, on which some very different development environments are installed. Some have ready made solutions like mamp or wamp, some have installed PHP and MYSQL on their machine directly. We all have a different PHP versions, different MYSQL versions. And cherry on top, none of these environments match the setup we have in place on our own dedicated server.
We rode along like this for about four years. It didn't affect us much. Sure when a new dev or a new machine comes in, it takes some time to setup it up properly, then it has some bugs here and there that needs fixing, like enabling a particular php extension for example. But overall, it was allright.
Then, recently, WordPress changed its database charset to utf8mb4 on servers running MYSQL above version 5.5.3. This introduced some problems for us, because some of our developers were below this version on their local machine, and some were above. And there you have it. You are walking on a straight line, everything looks sort of fine, and when a bummer comes in, your sense of balance falls apart, and half of your team can't push database changes into production, or work together anymore. It became clear to us that we had to align our development machines with our server configuration, and that it would be more efficient to use a common standardized development setup.
Searching for a solution
When thinking about setting up and deploying an environment that could be defined once and for all and easily installed on any machine, virtual machines come to mind. You could provide a full blown machine, with Apache, PHP, MYSQL, even a graphical user interface and the tools like the IDE, grunt... But this idea comes with its cliches and shortcomings. Virtual machines can be slow. Slow to boot, slow to run, they can slow down their hosting machine... It takes disk space, memory, CPU time... It's quite a heavy setup really, and when you compare them with your wamp app, you can tell this solution seems so much more complex, that adopting it will come with heavy frictions.
Could there be a simpler way to virtualize just what we need? A LAMP stack, that would be lighter than a full virtual machine, not slowing us down?
This is where we found out about Vagrant.
But what is Vagrant?
As it states on its website, Vagrant helps you create and configure lightweight, reproducible and portable development environments. Needless to say this sounded exactly what we were looking for, and not because it's hype, but because it answers our needs pretty well.
How does it do that?
Well it is a virtual machine indeed, to make it portable essentially. But where Vagrant is different and adds value, comes from its main file, the Vagrant file that is going to be used to describe how to configure this VM automatically. For example, it's in this file that you specify which virtual machine system you are going to use, how much resources you'd like to allocate to it, which folders you'd like to sync with your hosting machine and so on. You'll even be able to configure its network settings as well.
In a second step, you'll tell Vagrant how to provision your Virtual Machine, in other words, what you wish to install on it, which software for example and you'll even have the capacity to describe how you you'd want them configured.
Let's get back to our practical case here : we want to replicate our production server with Vagrant. That means we'll write a Vagrant file to run a CentOs Virtual machine. Then, we'll write a provisioning file to install on this machine the right PHP and MYSQL version, as well as phpmyadmin. We'll then provide configuration files for those services in order to make sure they are configured exactly like the production server.
Now that our Vagrant environment seems correctly configured, it's time to run it for the first time. Vagrant is depending on Virtual Machines Environment to run properly, that means before running Vagrant, you'll need to install VirtualBox or VMware first.
Once you have this dependency, let's run one of Vagrant's main command:
As this is the very first time this Vagrant Box is being used, the virtual machine box will be downloaded and provisioned entirely. This will surely take quite some time to run to finish, we'll have to be patient. In other words, Vagrant will download our centOs image, install it, download all the dependencies we have defined in the provisioning file, install them, and configure them.
It's often quite long. But it's doing all of this in just one command line, and that's where it's pretty handy. Not to worry though, it only happens the very first time you launch the box. Next time it'll just use what's already installed until you decide to destroy the Virtual Machine entirely.
Once the Vagrant box is up and running, it gives you access to a local Virtual machine, on a private IP (192.168.33.10 by default). Your system thinks it's a different machine, that you can ping.
Installing local websites
Usually for us, without Vagrant and with a LAMP stack, when we want to run a website locally, we need a few things:
- The site's files and folders
- A host entry pointing the local URL to your local machine, for example: 127.0.0.1 local.noe-interactive.com
- A virtual host mapping the local URL to the local folder
- A database.
How is it different with Vagrant?
- You can keep your same folder, and sync it with Vagrant. This done thanks to synced folders. It's not that different for this point. But that's a good thing because the whole point of this is to keep local website administration simple, and close to what we are used to and like with the locally installed LAMP stacks.
- You'll still need a host line on your machine, but it'll have to point to the Virtual Machine's private URL. For example: 192.168.33.10 (instead of 127.0.0.1)
- The virtual host will conceptually no longer be on your local machine but on the virtual one. As for site folders, you can sync a vhost folder between your local machine and the VM.
- The database will be hosted on the VM, where you'll have a phpmyadmin, which would be accessible through the virtual machine's IP address (again, not conceptually on your local machine).
Into practice : Insights after a few weeks
Once the vagrant file and the provisioning files are done, it's true that giving a folder to a dev and telling him to run vagrant up in it is straightforward. No additional setup is required in theory, and you are sure to master the perimeter of this new local environment. To get there however, is not that easy, the provisioning file especially can be rather difficult to get right. The aim is to replicate the server environment by entering shell scripts to download and configure softwares. If you don't have a minimum of sysadmin knowledge, that can be tricky.
We've had a few issues during the setup phase. Mainly focused on synced folders, which allow you to share a folder between your local machine and the virtual one. On some machines it didn't work because of the virtual box version we chose, some other times because of the centOs image version, some other times because of the guest additions plugin. Well it didn't go that smoothly. But once again, after we figured out what the correct and compatible version of everything was, it did work as simply as running vagrant up.
Unlike a local WAMP or MAMP install, a vagrant virtual machine should be destroyable at any time if necessary. When thinking about your files that's absolutely fine because they either won't be destroyed if shared, or will be part of the provisioning. But what about your database? By default, if you destroy your box, your local databases will be destroyed as well. That's an issue because it's not what we want, we want our box to behave like a MAMP install, with persisting databases.
You have several solutions to this problem. You could version your database files and load them at provisioning, or, like we did, use triggers to export your database on destroy, and load the export if found at provisioning. That way your database content is never lost.
Like we said earlier, vagrant provides a virtual environment. That implies that your physical machine is hosting the virtual one, and does so by allocating some of its resources to it. It's important to try to get the balance right. If you don't allocate enough resources to it, the virtual machine will be slow. If you allocate too much, you physical machine will be put under stress. We also need to keep in mind that we're setting up this environment to equip our devs with. Allocating resources needs to be done with our colleagues capacity in mind as well so a setup that might be perfect for you could actually be too heavy for another machine. Or maybe it could be a good argument to upgrade this machine if is not powerful enough to welcome the standard development environment.
- Vagrant helps you create and configure lightweight, reproducible and portable development environments. It's a virtual machine, and it requires vmware or virtualBox to run.
- Two main files are especially important, the vagrant file and the provisioning file. They can be quite tricky to setup but once done. Your environment runs with a single command:
- It helped us standardize our development environment, which is the first step we took to industrialize our development processes.