LAMP, for those who don’t know, is a software architecture commonly deployed for high availability web applications. It’s entirely open source (meaning free to use) and is very inexpensive to get started with (some hosting plans offer LAMP for as little as $2-3/mo). I’ve worked with a number of websites running on LAMP and find as a developer I think very little about what’s going on under the hood. Recently I had a horrible experience (fortunately one that was reversible) deploying a site onto FreeBSD — afterall, there’s a reason this is called LAMP and not FAMP. Before I get into details and start an OS war, I’d like to declare that I find there are different tasks that are best suited for different environments. It’s not about selecting the tools that I use most of the time, it’s about using the tool that works best in a given task. I also understand that sometimes, those tools are not available and your forced to make due with what you have. While screwdrivers don’t generally make good hammers, if it’s all you have to hang a picture on the wall, it can get the job done… but I digress.

Recently I was working on an infrastructure move. It involved a few servers which distributed the LAMP stack into a typical database application server model, segmenting rolls out over equipment. Nothing super complex, but sufficiently large and dealing with a large enough volume that the move was done in stages to minimize downtime. Working with our very talented team of System administrators we planned out the entire move and experienced only 1 minute of downtime. The gotcha wasn’t code based, but OS based. The only difference between our two infrastructures was our OS layer (the ‘L’ in LAMP), we had opted for FreeBSD for a number of valid reasons (which are beyond the scope of this rant.)

We began by replicating the database using standard MySQL replication from our production master into the new infrastructure, this went without much difficulty and the system was able to handle the query load nicely. The database replicated and was very quick. We moved our codebase over and made necessary configuration changes to work within the new hardware environment (mostly path & security changes). After we did this, we were able to successfully test and use the application in the new environment – all seemed well. Queries performed faster than they had in our old setup – long and short we were pleased. We flipped the switch and basked in the glory that is a smooth transition. The site worked wonderfully, load was well within reason and we had solid performance from the application.

Two or three hours after the transition (as our load started to increase with traffic) we began experiencing issues with the latency of pages going up and up and up. Working with our ISP we were able to locate the problem. Threading in FreeBSD != threading in Linux. After two more hours working on the equipment, we decided to implement our rollback plan and revert to our old equipment (which was standing by for just such an occasion.) We were able to reapply the data changes to the old equipment and revert just about everything back in just a few minutes and bring the site back online. Unfortunately we learned the hard way, you shouldn’t use (at least as of right now) current stable releases of MySQL and FreeBSD together. While there are a number of hacks to make this work – we decided that we wouldn’t be the guinnea pig who proved these solutions in a production environment. While I have nothing against FreeBSD as an OS, this threading difference was a make or break for us. In summary, if your using FreeBSD and MySQL, and you are expecting (or at least hoping) to become a high volume site, you’ve been warned if things stop working. It’s called LAMP for a reason, move to CentOS, RedHat or another Linux distribution for your database hardware – it will save you many headaches.