These days, it's becoming increasingly harder to find web applications that are homogenous in terms of the tools they use to Get Things Done. The ability to build the web front-end of your site using PHP but a critical part that requires asynchronous processing using Node.js is something that is both exciting and, well, practical. Loosely coupled components, passing messages to each other, is great architecture to try and build if you have both the skills and patience to make it work.

For a project at work, I am using PHP (specifically Zend Framework) for the front-end but are using Python scripts run as a cron-job (and also on-demand when statistical corrections occur) to collect raw stats for a variety of sports, and then generate fantasy point totals for the games we run. I'm already using PHPUnit for tests of the front end, and I decided to to use py.test to test my Python scripts.

Setting up tests in Python was pretty simple. Here's one of my test scripts:

PLAIN TEXT PYTHON:

1. import py
2. import baseball_scoring
3.  
4. def test_batter_empty_data_set():
5.     expected_points = 0
6.     test_data = dict()
7.     test_points = baseball_scoring.batter_points(test_data)
8.     assert expected_points == test_points
9.  
10. def test_batter_simple():
11.     test_data = {
12.         'hits': 4,
13.         'doubles': 1,
14.         'triples': 1,
15.         'home_runs': 1,
16.         'runs_scored': 1,
17.         'rbi': 1,
18.         'stolen_bases': 1,
19.         'league': 'bluejays2010'
20.     }
21.     expected_points = 11
22.     test_points = baseball_scoring.batter_points(test_data)
23.     assert expected_points == test_points
24.  
25. def test_pitcher_empty_data_set():
26.     expected_points = 0
27.     test_data = dict()
28.     test_points = baseball_scoring.pitcher_points(test_data)
29.     assert expected_points == test_points
30.  
31. def test_pitcher_simple():
32.     test_data = {
33.         'wins': 1,
34.         'losses': 0,
35.         'saves': 0,
36.         'strikeouts': 7,
37.         'complete_games': 1,
38.         'shutouts': 1,
39.         'league': 'bluejays2010'
40.     }
41.     expected_points = 25
42.     test_points = baseball_scoring.pitcher_points(test_data)
43.     assert expected_points == test_points

Very similar to tests with PHPUnit, right? So now that I had both PHPUnit tests and py.tests tests (hrm, is there are better way to say that?) to run, I had to figure out how to automatically run them. More specifically, how to get our installation of Hudson to run them.

Getting PHPUnit to play nice with Hudson was relatively easy. I installed the NUnit plugin for Hudson, made sure I installed phpunit, and then I added it's use to my build scripts. However, the strength of Hudson is that with the use of another plugin I could read reports of all those tests. So when things failed, I would not have to look at the console output to figure things out. There's a place in the Hudson config where you can configure this:


Now, I figured that the same thing could be done with py.test. It had an option so that at run-time you could tell it where to put JUnit-compatible test result files. After a little tinkering, I got it to work. First step was adding execution of it to my build script. Here is the latest-and-greatest version of that script:

mkdir /var/www/games-hudson/${BUILD_ID}
cd ${WORKSPACE}/games
/usr/local/zend/bin/php doctrine-cli migrate
cd ${WORKSPACE}/games/tests
/usr/local/zend/bin/phpunit --log-junit=${WORKSPACE}/build/logs/phpunit-results.xml
cd ${WORKSPACE}/games/scripts
/usr/bin/py.test --junitxml=${WORKSPACE}/build/logs/pytest-xmlrunner.xml
cp -R /var/lib/hudson/jobs/${JOB_NAME}/workspace/games/* /var/www/games-hudson/${BUILD_ID}
chmod 777 /var/www/games-hudson/${BUILD_ID}/tmp
rm -rf /var/www/games-hudson/current
ln -sf /var/www/games-hudson/${BUILD_ID} /var/www/games-hudson/current

Next, I then told Huson where it could find the JUnit-compatible files generated by py.test:

So there you have it. Now, when I do a commit and trigger a Hudson build, both my PHPUnit and Python tests get run. And there is output to check, so I don't have to dig through console output to figure things out.
PHPUnit

In discussions with my friend Kevin, I have come to realize that we have arrived at the same conclusions about the use of web application frameworks. Much of these conclusions have driven from 12+ years in building applications using them, with various rates of success. Suffice it to say, only the most l33t programmers out there are using their own frameworks, be it a custom one or contributing to the creation of an open-sourced framework. Everyone else is like me: a user of frameworks, and extremely rare contributor back to those frameworks.

Since Kevin and I are travelling together through the world of snake handling, we have been looking at the available options for building web applications in that environment. Our conclusions have been surprising to ourselves, but I think they can be very instructive to others. These two lessons are applicable to other programming languages as well, so feel free to substitute your favourite language where applicable.

Lesson 1: Full-stack gets you in the door, lightweight lets you find your niche

If you want to learn Python for the web, the first place you go is Django. It is a full-stack solution for building things in Python. Awesome documentation, and with some help from Python's online documentation, you can build an app in a reasonable time frame without knowing a ton of Python. Awesome tutorials and code samples FTW! I have built a Django app and briefly made it public. It was, like so many other things, a piece of crap missing functionality so I am trying to rectify that in my ever-dwindling spare time. I did not find any serious obstacles to building this app, except for figuring out how to do some dynamic form-field-generating code. It took me a while to find the right info (amazing how having the correct keywords in your search helps) but I did.

As an old warrior of frameworks (10 and counting across 3 languages) I found that because I already understood the concepts behind MVC (or MVT in the case of Django) I could concentrate on learning Python first, and then the unique features of the framework after that.

In many ways, my early experiments with Rails (back in 2004-2005) helped me understand frameworks a lot better. All frameworks have their own magic methods for doing things, with Rails making heavy use of Ruby's ability to create Domain Specific Languages to make Rails the incredibly useful framework that it is today. But eventually, your skill with the language and the framework gets to the point where you see the limitations of the conventions and magic methods the framework uses.

Now that I've gotten my hooks into building stuff with Django, I now recognize the same seductive promises that I found in things like CakePHP: the full-stack takes you away from actually figuring out how to code things, and instead you find yourself just using all the built-in methods. Don't get me wrong, the built-in stuff for Django is awesome. But if I want to push my skills forward so I can say "I can build stuff for the web using Python" instead of "I can build stuff for the web using Django" I need to use other tools. What if I want to run my stuff on Google App Engine? Sure, there are these hacks available for Django that make it "usable" on GAE, but that's not what I want. And what about the use of WSGI instead of relying on mod_python? Forgive me if this stuff comes across as technobabble. It was technobabble to me at one time too.

So now the next step is (after finishing the current Django app) is to build an application using a much more lightweight framework. Why? Because it's time to actually learn how to do something instead of relying on magic methods. Because once I learn to write code to duplicate those magic methods, I'll be able to use Django *and* Flask. And that, my friends, is how you make yourself useful. That and the ability to run the app both as a standalone WSGI app or on Google App Engine seems like a bonus to me.

So now that you've gone from full-stack to lightweight, it's time to look at the next lesson...

Lesson 2: Don't use anything made by One Guy

This will undoubtably be a touchy subject for some. Let me try and explain it.

In the quest for the lightweight solution, I ran across web2py. Nice and lightweight, awesome documentation, but it's all driven by ONE GUY. Experience has taught me that projects driven by ONE GUY end up looking like how the ONE GUY would do it. Sometimes the ONE GUY is brilliant and it all works out. That is rare, in my experience. Open source projects are littered with the wreckage caused by egos and control freaks who were unwilling to accept outside help to solve problems.

So look for projects where it appears there are solid contributors beyond ONE GUY. web2py sure has an impressive list of people who contribute, but is Massimo the ONE GUY when it comes to actually making non-trivial code changes? I sure hope not.

There will be no test afterwards

While I do not expect other people's experiences to be the same as mine, I'm pretty sure I am not the only person who has learned these lessons. So the next time you are evaluating using a web application framework (in a new language or an existing one) think back on these lessons.

The other day I had a nice meeting in my basement lair with my co-workers. We laughed. We cried. We had some BBQ'd burgers, and nobody got food poisoning. As we discussed the implementation of our new fantasy gaming platform, I managed to figure out that one requirement was for an admin to upload a header and footer file for a game. "The idea here is to be able to set up a new game for someone without programmer intervention".

After I got over being insulted that a programmer was not required for every tiny tweak to the site (I'm kidding) I tried to figure out how I was going to accomplish this. I'm not a big fan of storing actual HTML content in a database. Call me old-fashioned, but I feel my HTML content belongs on the filesystem, and nowhere else. Besides, it makes it harder to do deployments of code on multiple servers if all the templates aren't there to begin with.

The solution was, of course, obvious: create a custom Zend_View and override stuff until you get it behaving the way you want! The logic for doing this is actually quite simple:

1. Check to see if the script in question exists in the file system
2. If not, figure out what template we need to pull from the database.
3. Read in contents of that script from the database
4. Write contents to the filesystem

NOTE: I am not sure if this is the most elegant solution for solving this particular problem. I am open to a different solution so long as it meets the same needs.

Since I've made a commitment to doing TDD for this project, naturally I started out with a test:

PLAIN TEXT PHP:

1. <?php
2. require_once 'PHPUnit/Framework.php';
3. require_once 'Zend/Application.php';
4.  
5. // Define path to application directory
6. if (!defined('APPLICATION_PATH')) {
7.     define('APPLICATION_PATH', realpath(dirname(__FILE__) . '/../../../application'));
8. }
9.  
10. // Define application environment
11. if (!defined('APPLICATION_ENV')) {
12.     define('APPLICATION_ENV', (getenv('APPLICATION_ENV') ? getenv('APPLICATION_ENV') : 'test'));
13. }
14.  
15. // Ensure library/ is on include_path
16. set_include_path(implode(PATH_SEPARATOR, array(
17.         realpath(APPLICATION_PATH . '/../library'),
18.             get_include_path(),
19.         )));
20.  
21. class Xmlteam_ZFViewTest extends PHPUnit_Framework_TestCase
22. {
23.  
24.     public function setUp()
25.     {
26.         $this->bootstrap = new Zend_Application(
27.             'testing',
28.             APPLICATION_PATH . '/configs/application.ini'
29.         );
30.         parent::setUp();
31.     }
32.  
33.     public function tearDown()
34.     {
35.         parent::tearDown();
36.     }
37.  
38.    
39.     public function testRenderView()
40.     {
41.         $templateData = array(
42.             'league_short_name' => 'testing',
43.             'type' => 'header',
44.             'created_at' => new Doctrine_Expression('NOW()'),
45.             'content' => '<div id="header"><b>header</b></div>'
46.         );
47.         $testTemplate = new Template();
48.         $testTemplate->fromArray($templateData);
49.         $testTemplate->save();
50.         $testView = new Xmlteam_ZFView();
51.         $testView->setScriptPath('./application/views');
52.         $content = $testView->render('testing-header.phtml');
53.         $this->assertTrue(file_exists('./application/views/testing-header.phtml'));
54.         $this->assertEquals($content, $templateData['content']);
55.         $testTemplate->delete();
56.         unlink('./application/views/testing-header.phtml');
57.     }   
58. }

For those wondering, yes all the tests pass.

The "Template" is a Doctrine (1.2) model that represents the table where I'm storing information about the templates. Yes, it has unit tests too. That pass.

So how do we accomplish this? First, I hit up my peeps on Twitter and the advice from Matthew Weier-O'Phinney himself was to simply create my own Zend_View and override what needed to be overriden. Also, he was very wise to tell me to look at Zend/View.php itself for guidance. That and some helpful messages from exceptions during testing.

Here's the initial implementation, for which all tests pass.

PLAIN TEXT PHP:

1. <?php
2.  
3. // Custom view that reads in templates from the database and
4. // writes them to the filesystem
5.  
6. class Xmlteam_ZFView extends Zend_View_Abstract
7. {
8.     private $_filter = array();
9.     private $_file = null;
10.  
11.     public function __construct($config = array())
12.     {
13.         parent::__construct($config);
14.     }
15.  
16.     /**
17.      * Custom method to see if template is in the file system
18.      * and then load it into memory, write out to the file system
19.      * if it doesn't
20.      *
21.      * @param string $name The script name to precess
22.      * @return string THe script output
23.     */
24.     public function render($name)
25.     {
26.         $scriptPaths = $this->getScriptPaths();
27.         $filename = $scriptPaths[0] . $name;
28.        
29.         if (!file_exists($filename)) {
30.             list($league, $tmp) = explode('-', $name);
31.             list($templateType, $extension) = explode('.', $tmp);
32.             $q = Doctrine_Query::create()
33.                 ->select('t.*')
34.                 ->from('Template t')
35.                 ->where('t.league_short_name = ?', $league)
36.                 ->andWhere('t.type = ?', $templateType);
37.             $q->setHydrationMode(Doctrine_Core::HYDRATE_ARRAY);
38.             $results = $q->execute();
39.  
40.             if (count($results)> 0) {
41.                 file_put_contents($filename, $results[0]['content']);
42.             }
43.         }
44.        
45.         // find the script file name using the parent private method
46.         $this->_file = $this->_script($name);
47.         unset($name); // remove $name from local scope
48.  
49.         ob_start();
50.         $this->_run($filename);
51.  
52.         return $this->_filter(ob_get_clean()); // filter output
53.     }
54.  
55.                     /**
56.      * Applies the filter callback to a buffer.
57.      *
58.      * @param string $buffer The buffer contents.
59.      * @return string The filtered buffer.
60.      */
61.     private function _filter($buffer)
62.     {
63.         // loop through each filter class
64.         foreach ($this->_filter as $name) {
65.             // load and apply the filter class
66.             $filter = $this->getFilter($name);
67.             $buffer = call_user_func(array($filter, 'filter'), $buffer);
68.         }
69.  
70.         // done!
71.         return $buffer;
72.     }
73.  
74.     /**
75.      * Includes the view script in a scope with only public $this variables.
76.      *
77.      * @param string The view script to execute.
78.      */
79.     protected function _run()
80.     {
81.         if ($this->_useViewStream && $this->useStreamWrapper()) {
82.             include 'zend.view://' . func_get_arg(0);
83.         } else {
84.             include func_get_arg(0);
85.         }
86.     }
87. }

So, what did I learn from this process?

1. I needed to create my own private _file and _filter variables to match what exists in so that if I create any custom filters they can be applied to my custom view *and* to keep compatibility with existing code.
2. Because the _run method is protected AND abstract, I needed to implement my own _run method in my custom view, or else it would spit out errors. Even if it doesn't do anything special.

You can see that the logic in my render method is pretty much as I outlined before. I realize I will take a slight hit the first time we go to load a particular template. I'm assuming at some point I will want to run a test to see just how long it does take to pull in the template from the database and write it to the file system. I'm guessing this will not be a serious performance hit, but you never know. All those file_exists() calls can't be that good for performance,

I look forward to seeing other potential solutions to this problem

Zend Framework

The folks at Packt Publishing asked if I was interested in reviewing their upcoming title "CodeIgniter 1.7: Professional Development" due to my experience with CodeIgniter. While it is not my favourite framework (for reasons that I have explained on this blog), I still have to work with it going forward. I was interested to see what direction such a book would take. After going through it, I was happily surprised.

When I read a book about a programming framework, I'm expecting to see that they start off with the basics of using the framework, then keep expanding on the abilities of the framework. By the end of the book, you should be able to build something OTHER than the example application (if they provide one). I mean, if I just want a list of functions then I will hit up Google/Bing/Duck Duck Go to find the info I want. This book starts with the basics of the framework, and patiently builds on the concepts you previously learned. Good, clear examples of how to do things.

I thought another nice touch was the expanding into concepts like scaling the application, and extending the framework itself. If there is one thing I've learned building web apps over the years, it's that you (a) inevitably need to worry about scaling the application and (b) you end up having to make your own modifications to the framework itself to meet unique challenges for the application.

All in all, I recommend this book to anyone who is planning on using CodeIgniter 1.7 (and beyond) to build web apps. The sections on the first steps towards scaling your app, and extending CodeIgniter itself make this book stand out from the crowd.

(Note, this example was using Zend Framework 1.10, so things might change going forwards).

As a lapsed tester, I've made the commitment to build out our Zend Framework powered application using tests going forward. The first two modules for the app *should* have been done with tests, and for that I hang my head with shame. Now that I've invested the work in creating a continuous integration environment using Hudson, there is no longer an excuse to not write tests. Especially when I can get tests run automatically every time I commit code.

Anyway, I was asked to implement a new feature for the application: a list of all transactions for a fantasy baseball league, sorted by date. What a perfect excuse to write some tests! The initial problem though was how do I simulate logging in a user so I can see this page, which was protected by authentication using Zend_Auth. So I started my scouring the internet for answers.

I found quite a few examples on how to test a controller, using the $this->dispatch('/path/to/action') method but found that I wasn't getting redirected properly to the post-login page. I was passing proper credentials in and everything. Then finally after deciding to go back to basics and read up on just how to test Zend_Auth. After some swearing a few face-palms, I realized what I had been doing wrong.

The tl;dr version: I was forgetting that the unit test itself needed to manually log in the user by speaking with Zend_Auth directly. Here's the testing code for that particular controller:

PLAIN TEXT PHP:

1. <?php
2.  
3. require_once 'Zend/Test/PHPUnit/ControllerTestCase.php';
4. require_once 'Zend/Application.php';
5.  
6. // Define path to application directory
7. if (!defined('APPLICATION_PATH')) {
8.     define('APPLICATION_PATH', realpath(dirname(__FILE__) . '/../../../application'));
9. }
10.  
11. // Define application environment
12. if (!defined('APPLICATION_ENV')) {
13.     define('APPLICATION_ENV', (getenv('APPLICATION_ENV') ? getenv('APPLICATION_ENV') : 'test'));
14. }
15.  
16. // Ensure library/ is on include_path
17. set_include_path(implode(PATH_SEPARATOR, array(
18.         realpath(APPLICATION_PATH . '/../library'),
19.             get_include_path(),
20.         )));
21.  
22. class TransactionControllerTest extends Zend_Test_PHPUnit_ControllerTestCase
23. {
24.  
25.     public function setUp()
26.     {
27.         $this->bootstrap = new Zend_Application(
28.             'testing',
29.             APPLICATION_PATH . '/configs/application.ini'
30.         );
31.         parent::setUp();
32.     }
33.  
34.     public function loginUser($login, $passwd, $shortLeague)
35.     {
36.         $authParams = array(
37.             'login' => $login,
38.             'password' => $passwd,
39.             'short_league' => $shortLeague
40.         );
41.         $adapter = new Xmlteam_Auth($authParams);
42.         $auth = Zend_Auth::getInstance();
43.         $result = $auth->authenticate($adapter);
44.         $this->assertTrue($auth->hasIdentity());
45.     }
46.  
47.     public function tearDown()
48.     {
49.         parent::tearDown();
50.     }
51.  
52.     public function testOverallBaseballList()
53.     {
54.         $this->loginUser('test@test.com', '*****', 'bluejays2010');
55.         $this->dispatch('/baseball/bluejays2010/transactions/list');
56.         $this->assertModule('baseball', 'In baseball module');
57.         $this->assertController('transactions', 'In the transactions controller');
58.         $this->assertQuery('#tblTransactions', 'Transaction table exists');
59.     }
60. }

I'm using a custom authentication adaptor with Zend_Auth, as we need to limit access for a user to a particular league. Perhaps better to do it with Zend_Acl, and since the platform is likely to expand I will end up using that as well. But I am drifting off topic here.

The magic is in that loginUser() method. What I did not realize at the time was that running a test where I asked it to dispatch users to the login page and passed along what would've been entered in the form was not working. Perhaps it was creating an authenticated session OUTSIDE the scope of the test environment. Perhaps it is my sucky code. Either way, it wasn't working.

So, by creating a authenticated login Zend_Auth *inside* the test, I could then safely dispatch my testing code to look at pages requiring authentication. I hope this blog posts helps out others trying to write tests for controllers that are hidden behind Zend_Auth.

Like many old sk00l developers, I like the security blanket a bookshelf full of technical books gives me. Whenever I have a problem, I check to see if I can figure it out using the reference materials on hand. Just looking at my shelves. Since She Who Must Be Obeyed Around Here frowns on boxes constantly showing up at our house from various online book retailers, I usually only purchase books that I think are going to be good additions to the library.

Taking a quick peek at my bookcase, these are a few of the books that I consider essential reference guides for me:

• The Pragmatic Programmer
• Practices of an Agile Developer
• Mastering Regular Expressions
• Beautiful Code / Beautiful Architecture
• Javascript: The Good Parts

I have more books, but those are the ones closest to my desk.

There is now a new book that has earned a spot on my bookcase, and should be in one of the slots closest to my desk. I am reading a borrowed copy of it, but I plan on rectifying that soon. "Building Scalable Web Sites" by Cal Henderson (he of Flickr and "Why I Hate Django fame) is that book.

I don't know how I can describe how awesome this book is in this blog. Just check out the topics in the table of contents:

1. Introduction
2. Web Application Architecture
3. Development Environment
4. i18n, L10n, and Unicode
5. Data Integrity and Security
6. Email
7. Remote Services
8. Bottlenecks
9. Scaling Web Applications
10. Statistics, Monitoring, and Alerting
11. APIs

Pardon my vulgarity, but that is pretty much every single fucking thing you need to worry about when building a web site that is going to be used by anyone other than you and your close circle of friends.

This book is old in internet time: published in 2006, meaning it was probably written in 2004-2005. A lot of technology has changed, but the principles themselves have not. Also, a lot of the code inside uses PHP for it's examples, but I really think this applies to any programming language that is stateless and/or promotes a "shared nothing" architecture. That means (as far as I'm concerned) PHP, Python and Ruby. It's very simple: scaling anything is difficult, but scaling something where there is information that needs to be shared between nodes is extremely difficult. The best way to use this book is to ignore the specific technologies being mentioned and focus on the ideas and practices that are being promoted. I mean, let's be brutally honest: Cal Henderson helped build one of the most massively scaled web sites, using a programming language at it's core that the hipster programmer crowd sneers at. Ignore his advice at your peril.

Go buy this book. Right now. You will not regret it.

I had an awesome time at Tek-X in Chicago last week. It was worth the 8 hour drive to get to Chicago from my basement lair. Of course, it helped that the wait at the border was under 10 minutes both times. There were good talks, but even better *people*.

A programming language is only as good as the community that supports. By supporting it, I mean willing to get together several times a year for conferences and bust their asses to have people share interesting talks with everyone else. Look at Python and Ruby. Awesome languages to work with. But even more awesome are the conferences that serve as anchors for the community.

There seems to be lots of little regional Ruby conferences, which give the language itself a unique flavour in return. Of course, it helps that Ruby can be used quite easily outside the traditional web application building environment. I don't see so many regional Python conferences, just PyCon and DjangoCon both in the US and over in Europe. It could be that I'm not looking hard enough.

As for PHP, to be fair, it shows up at pretty much any open source web-related conference. OS Bridge. OSCON. Codemash. I could go on, but I am not in the conference-promotion business. I mean, why is there no Great Canadian PHP Conference? I know there was phpworks in Toronto back in the mid 2000's. It was my first conference I ever went to. But I'll tell you why there is no Great Canadian PHP Conference: there is no PHP community in Toronto.

Oh sure, there are tons of PHP jobs to be had in Toronto. Lots of code monkey positions to be filled. But the Toronto PHP User's group has been dormant since last year. I sent an email to the organizer telling him I wanted to help out at least 3 days ago. Haven't heard a damn thing back. Perhaps it is time to go around such a person and just do it myself.

See, without the PHP community I do not have a career. Once I took the plunge, submitted a talk, and got accepted (I gave a talk about what the PHP community could learn from Ruby on Rails) I found I had suddenly joined the PHP *community*. Instantly I had people asking me questions AND people willing to answer my questions. Like I was their long-lost buddy. That was an awesome feeling, let me tell you. But even more important were the people I met.

As a result of the community, I have found myself in the awesome position that every time I needed a job, it took less than a week to get one. It was a combination of my blogging and my work in making those connections in the community that put me in this position, but it is sure a nice feeling to be able to convince your nervous wife that we were not about to face a rough patch while I struggled to find work. So it's a no-brainer for me to continue to submit talks to conferences and try to attend even when I don't speak. It's about giving something back to the community that has helped me grow my career.

So don't just *use* your language of choice. Try and become part of that community. Doesn't matter if it's Python, Ruby, or good-old-solved-the-web-problem-first PHP. You'd be surprised what doors are opened up by your participation.

Tomorrow (May 18) I throw all my junk in the trunk of my trusty steed (a 2000 Ford Taurus that loves to chew up the highway miles) and hit the road for Chicago for TEK-X, the 10th edition of this conference. I have spoken at previous editions (is that the correct word?) of this conference and I'll be honest: I'm a bit bummed that I'm not speaking.

See, I normally only go to conferences when I speak. Why? Being away from the family is tough to arrange when you work from home. I've got two little girls, and they are used to Daddy being around to help out with stuff. Like taking them to day care, or picking them up from school. Making dinner. Helping with school lunches. Grocery shopping. I'm sure you get the picture. So to abandon the family to go to a conference requires lots of logistical work.

But damnit, I hadn't been to a conference since March of last year (the old PHP Quebec conference) I was determined not to get shut out. Luckily my boss agreed to foot the bill and so off I go. I'd love to meet any of my readers, and even sign paper copies of my book if you got a print-on-demand version. Hell, I don't even have one. So don't be intimidated by my size, appearance, or lack of knowledge on certain topics. Come and say hi. I'll the big, bald, grumpy Canadian with a pot-scrubber on his chin stomping around the conference.

Anyway, I wanted to talk about the sessions I'm looking forward to seeing. In no particular order, here they are:

Building Real-Time Applications with XMPP

I've known Travis for a while through the PHP community, and never seen him do a presentation. However, his numerous IM sessions helping me understand decorators in the context of Django and talking me through fixing totally busted Git commits lead me to one conclusion: he knows his shit so I should pay attention to what he has to say. The building of our next-gen fantasy sports gaming platform starts when I get back so I'm hoping he imparts some pearls of wisdom.

Continuous Inspection and Integration of PHP Projects

Being the only guy working on projects gets frustrating: nobody to share ideas with and also very hard to make the case for the types of tools and procedures that the TDD/BDD and continuous integration crowd advocates. I mean, all this work so Chris stops complaining about bugs slipping through the QA cycle? Besides, I always have a soft-spot for Ze Germans who are always at the PHP conferences.

Graphs, Edges & Nodes: Untangling the Social Web

I first met the presenter (Joel Perras) at PHP Quebec last year through the CakePHP community. Now that he's moved on to the Lithium project, I see his ego has grown to the point where he is okay giving presentations. Don't worry whippersnapper, I'll cut you down to size during this (probably awesome because it deals with stats and nodes and graphs and all this cool stuff I've been reading in "Programming Collective Intelligence") talk.

Now, I'm sure that I will be attending other sessions while I'm there. But those are the main ones I'm looking forward to seeing. We are spoiled for choice at this conference, and I am definitely looking forward to seeing old friends and making some new ones.

In a previous blog post I showed some very basic techniques for using Phing for doing deployment. As with any good deployment process, once you let people know that there is a flexible deployment tool other people start asking you to new things.

"Hey Chris, can you set things up so that whenever someone runs one of those deployment scripts, we send an email to our support mailbox of all the files that were pushed up since the last deploy?" After pausing for a minute to figure out how I would do it (the quick answer was "no idea") I said "Sure, I'll see what I can do."

So I stood in front of my whiteboard and proceeded to sketch out an idea of how I would want to do such a thing. Luckily I could cheat and look at how Capistrano did it. Because the way that they do it makes so much sense, so I duplicated it quick-and-dirty style. A big thanks goes out to my friend Derek for pointing me in the right direction.

So, the way to make this happen is as follows:

1. Read in a file inside your current deployment that contains the date of your last deployment
2. Check your code out of your version control system into /releases/
3. Symlink /releases/ to /current
4. Write a file into /current that contains the date of the last deploy.
   

Here's what it looks like as a Phing script

PLAIN TEXT XML:

1. <?xml version="1.0"?>                                                                               
2. <project name="build" default="main">
3.     <property name="exportdir"  value="/var/www/app" />
4.     <property name="svnpath" value="/usr/bin/svn" />
5.     <property name="repo" value="svn+ssh://path/to/repo" />
6.     <tstamp>
7.         <format property="build.time" pattern="%Y%m%d%H%I%S" />
8.         <format property="newdeploy.date" pattern="%Y-%m-%d %H:%I:%S" />
9.     </tstamp>
10.     <property file="${exportdir}/current/deploy.properties" />
11.     <target name="main" depends="svnexport">
12.         <exec command="rm -f ${exportdir}/current" escape="false" />
13.         <exec command="ln -s ${exportdir}/releases/${build.time} ${exportdir}/current" escape="false" />
14.         <exec command="chmod -R 777 ${exportdir}/current/tmp" escape="false" />
15.         <exec command="chown -R scriptkiddie:company ${exportdir}/current" escape="false" />
16.         <exec command="chmod -R ug+rw ${exportdir}/current" escape="false" />
17.         <exec command="echo 'deploy.date=${newdeploy.date}'>> ${exportdir}/current/deploy.properties" />
18.         <exec command="svn log -r '{${deploy.date}}':'{${newdeploy.date}}' ${repo} | mail -s 'Deployment Of FizzBuzz in production' support@fizzbuzz" />
19.     </exec></target>
20.     <target name="svnexport">
21.         <svnexport svnpath="${svnpath}"
22.             username="${username}"
23.             password="${password}"
24.             nocache="true"
25.             repositoryurl="${repo}"
26.             todir="${exportdir}/releases/${build.time}"/>
27.     </target>
28. </project>

The key was figuring out the best way to send SVN log information via email. Leave it to the UNIX way of piping output to provide with the best solution totally within Phing.

I have been suffering from a major case of blogger's block, so I reached out to the lucky people who follow me on Twitter to ask them about some topics to talk about on the blog. Thanks to them I've got at least two blog posts coming. Credit to this one goes to Neil Crookes (who I met at the first CakeFest conference in Orlando) who suggested I talk about deployment issues

What's that you are saying? You thought I was using Capistrano to do deployments at work? Gentle reader, things change. All my work in trying to automate deployments is designed to allow me to do a deployment either from my own laptop or remotely on the server where it's run. Tools like Capistrano and Phing let me do that. Sure, I could cobble together my own solution using rsync or shell scripts, but my experience telecommuting has taught me that if you want others to use your tools, they have to be tools that can be used by others.

While Capistrano is certainly up to the task of deploying my own personal projects, I had nothing but problems getting people OTHER than myself to get it up and running. You wouldn't think that getting Ruby and then assorted gems installed on a non-Windows machine would be that hard. Apparently it is. So what was my alternate solution? Create something that can be run on the server where the code is going to be deployed.

As a guy who is always up to his neck in the devops environment (with a tip of the hat to Brian Moon for bringing this term to my attention) I am determined to NOT be the only one who can do deployments. It's 2010, the year we make contact, and there is no reason that my boss, when he alters some marketing copy on our main web site, should not be able to push those changes himself (with the side-effect being leaving me out of it). With a little work and some digging, I got Phing to do it for me.

Like many awesome tools with lots of documentation, Phing suffers from a lack of easy-to-find tutorials covering the exact things I wanted to do. Or sure, I eventually tracked it all down (with the help of the best support group in the world, Twitter) but I'm guessing this a trend that will not go away.

So, being the lazy sort, I set out to duplicate how Capistrano does their deploys:

1. Check out the latest version of code from HEAD on your repo (SVN in this case)
2. Create a new "current" directory
3. Symlink the latest checkout (which is in release/yyyymmddhhmmss) to current

There are some other steps in here that I use as well that are specific to my applications but these are the basics.

Why the symlink method? Capistrano allows you to also rollback to a previous deployment, but I haven't gotten around to implementing that feature with the new Phing-based setup. I never make mistakes that need to be rolled back like that. Yes, I'm kidding.

So, what would a build script to cover the basics look like?

PLAIN TEXT XML:

1. <?xml version="1.0"?>                                                                               
2. <project name="build" default="main">
3.     <property name="exportdir"  value="/var/www/foo" />
4.     <property name="svnpath" value="/usr/bin/svn" />
5.     <property name="repo" value="REDACTED" />
6.     <tstamp>
7.         <format property="build.time" pattern="%Y%m%d%H%I%S" />
8.     </tstamp>
9.     <target name="main" depends="svnexport">
10.         <exec command="rm -f ${exportdir}/current" escape="false" />
11.         <exec command="ln -s ${exportdir}/releases/${build.time} ${exportdir}/current" escape="false" />
12.     </target>
13.     <target name="svnexport">
14.         <svnexport svnpath="${svnpath}"
15.             nocache="true"
16.             repositoryurl="${repo}"
17.             todir="${exportdir}/releases/${build.time}"/>
18.     </target>
19. </project>

I know there are XML haters out there, but that's a readable configuration file right there. One of the things I did was to remove any user name and password information from the config file. You never know if you'll get cracked by a script kiddie exploit and someone reads a configuration file full of critical information. Call me paranoid, butyaneverknow.

So, after you install Phing then to run things you just need to type 'phing -f '. In our case, I then get prompted for a password, as the user names on the servers are the same as for the SVN repo.

So what else do I do when I deploy? There is always some clean-up work to be done after the fact, so I add a few more actions to the file:

PLAIN TEXT XML:

1. <exec command="chmod -R 777 ${exportdir}/current/tmp" escape="false" />
2.         <exec command="chown -R user:group ${exportdir}/current" escape="false" />
3.         <exec command="chmod -R ug+rw ${exportdir}/current" escape="false" />

I won't get into the details of what I'm doing there, but you can execute any command you want on the server, so long you have permissions to do so.

So, sometimes you are lazy and don't want to mess around with symlinks and figuring out how to roll things backwards. Here's how you could do a deployment using Phing that simply updates a directory that you checked out using SVN already:

PLAIN TEXT XML:

1. <?xml version="1.0"?>
2. <project name="build" default="main">
3.         <property name="svnpath" value="/usr/bin/svn" />
4.         <property name="username" value="codemonkey" />
5.         <property name="exportdir"  value="/var/www/bar" />
6.         <target name="main" depends="svnupdate" />
7.         <target name="svnupdate">
8.                 <svnupdate svnpath="${svnpath}"
9.                         username="${username}"
10.                         nocache="true"
11.                         todir="${exportdir}"/>
12.         </target>
13. </project>

Again, very straight-forward. Phing is a very powerful tool that can be used for a lot more than just deployments. Browsing through the documentation gave me some ideas for extending Phing further to meet our deployment needs. If you deal with PHP all the time, I cannot think of a better tool to help with complex build-and-deploy tasks than Phing.