What's In The Box?

ZDC is a much more talented developer than I am, so I frequently throw my code his way for review. In reviewing some of the tests I wrote for Jolt, he had some criticisms. One test in particular, testRouteNamesAreValid(), was quite smelly.

It was incredibly verbose and tested two things at once: valid named routes and invalid named routes. He suggested I refactor it to use a data provider and to split it up into two distinct test. The updated code is much more concise, easier to read, and more descriptive.

Read the new unit tests to see what I changed. Adding a new route to test as either valid or invalid is now simply adding another element to the array in either of the data providers.

I used this method in a few other test classes as well to test exceptions are being thrown properly.

Writing Unit Tests has definitely tested my discipline as a developer more than any other tool I’ve ever used. As I’m becoming more familiar with them, they’ll become second nature like using source control. If you’re not using them, do yourself a favor and add them to your toolbelt.

Over the weekend, I worked on the routing code for Jolt. Following a strict TDD design, I wrote the tests first and then wrote the code to match the tests.

I primarily worked on the Named Route code. Jolt will have two types of routes: Named and RESTful. Named Routes are exactly that, they’re explicitly specified and can be routed to any controller and action. They do have a specific format, and they won’t let you create a route if the format is invalid.

Named Routing

Take the example of a simple messaging application. You want to send messages to the server, but don’t wish to use a purely RESTful interface. Simply create a create-message and send-message route and you’re on your way.

namespace Jolt;
 
$router_config = array(
  'site_root' => 'http://joltcore.org',
  'site_root_secure' => 'https://joltcore.org'
);
 
$named_route_list = array(
  new Named_Route('/create-message', 'Message', 'create', Route::METHOD_POST),
  new Named_Route('/send-message/%d/%d', 'Message', 'send', Route::METHOD_POST)
);
 
$router = new Router();
$router->setConfig($router_config);
$router->setNamedList($named_route_list);
$router->dispatch();

All calls to http://joltcore.com/create-message through a POST method along with any associated data (which the controller is responsible for loading) will be passed to the Message controller and the create() action. If the method is called through GET, an HTTP 400^[#1] will be returned because it is a bad request.

Additionally, a call made to http://joltcore.org/send-message/10/18 through a POST method will call the Message controller and the send() action, which would, for example, send the last created message from User #10 to User #18.

As you can see, Named Routes are clearly very powerful. All named routes must start with a “/” and can be nested infinitely. Values can be replaced with normal printf() replacements: %d for digits, %s for strings would be the most common.

RESTful Routing

RESTful routing is very powerful, but less flexible than Named routing. RESTful routing lets you define a route and a resource. From there, Jolt handles everything else. Assuming the setup from above, adding a RESTful route for managing Users would be simple.

$restful_route_list = array(
  new Restful_Route('User')
);
 
$router->setRestfulRouteList($restful_route_list);

By defining the User resource as RESTful, several methods will be automatically available.

• http://joltcore.org/user GET – Get a list of all User models (objects).
• http://joltcore.org/user/1 GET – Retrieve data for a specific user.
• http://joltcore.org/user POST – Create a new user.
• http://joltcore.org/user/1 PUT – Update a user’s data.
• http://joltcore.org/user/1 DELETE – Delete a user.
• http://joltcore.org/user DELETE – Delete all users.

You’ll be expected to write the actual controller to handle these requests. Following a formal RESTful interface, you should not add additional resources. If they are required, please us a Named route.

All routes will take into account the Accept and Content-Type HTTP headers. The resulting controller will then use these to determine the format of the incoming data and the format of the result data. Additionally, each controller will return the correct HTTP status code.

Routes can accept multiple formats as well. If the Content-Type header specifies the incoming data is text/json and wants the data back as XML (the Accept header specifies text/xml), the controller will be able to handle that.

Conclusion

Building an application in Jolt will be seamless and easy. Bundled with the framework is a set of tools to facilitate this. One command line tool, jolt-app will read in your list of Routes and build all of the controller and action skeletons for them. It is my hope you’ll become an active and interested participant in the Jolt project. Please follow it on Github or subscribe to this blog.

References

1. HTTP/1.1: Status Code Definitions

Programmers are (generally) defined by the language they know best or use most frequently. I’m a PHP programmer, John Resig is a JavaScript programmer, DHH is a Ruby programmer, and Randal Schwartz is a Perl programmer. Every programmer needs to know several languages to be successful, but programmers usually know one or two very well.

Right now, PHP is not very respected. As a PHP developer, hearing language like that frankly hurts.

We have to put up with php as a language for web apps, but it’s a rotten language whose use should not be encouraged. Or supported beyond what’s necessary.

Ouch. Hearing that the language I use daily, make my living with, and the one I want to build a really cool framework in is “rotten” sucks. One might respond that it shouldn’t matter what other people think about your language of choice, and I suppose I really shouldn’t, but also implied in that statement is that not only the language sucks, but the developers who use that language suck. And that does hurt.

The PHP namespace (of developers) is filled with a lot of cruft, but there is also some great PHP code written. While there’s no concrete way to describe what is rotten code, I would venture to say most of the PHP released is rotten (I’ve personally worked with some of the worst PHP out there). This is mainly what gives us a bad name. Poor code can be written in any language, however. Many argue that PHP allows you to do that easier, but I don’t think it’s PHP that lets you do that. I believe it’s the entire stack of applications: you can begin writing PHP programs in minutes on any system. Combine that with inexperience, and you have a strong force to be reckoned with. An inexperienced developer can write a very usable application in PHP very quickly, release it to the world, let it become popular, and then watch as its ripped apart by crackers; the same isn’t true for C, C++, or Java for example.

So, what should PHP developers do about this? Can we make the language respected? Can we make PHP developers respected? Let’s analyze some ways we can do this.

Break Everything

PHP7 should break everything. PHP developers are used to things breaking between releases, but the PHP team should announce that PHP7 will most likely break everything. PHP7 will be a highly respected language, and in doing so, it will not allow a lot of backward compatibility. Here’s how.

• Create a concrete, core language. Remove all library methods, and keep centralized core methods on objects. You should be able to compile PHP7 without any external libraries or extensions and have a nice complete language for basic input/output, string manipulation, and math. Anything outside of that should come through approved extensions.
• Treat everything as an object. Take from Ruby, Smalltalk, and (mostly) Java and treat everything as an object. Integers are objects, strings are objects, and each of them have methods that can operate on them. I don’t believe PHP needs the ideas of Ruby and Smalltalk where objects pass messages between each other; calling methods on objects is fine.
• Consistent naming of methods and classes. Because one of the biggest complaints of PHP is constantly having to check if its (needle, haystack) or (haystack, needle), or some_function(), or function_some(), or someFunction(), a consistent format needs to be chosen. I prefer camelCase, but will live with whatever decision is chosen.
• Make things strict. Try to pass a string as a float to a method? That’s a warning. Let me use strict/static typing if I want to. Let me define a method with a return type and throw an error if the value returned isn’t that type.
• Everything is Unicode. I believe in PHP6 all strings are Unicode, but if not PHP7 should ensure this.
• Central start point. Create a Main or Init class where all code execution originates from.
• Clean up the C code. I’m not a C expert, but if you compare the Ruby C code to the PHP C code, it’s easy to understand the internals of Ruby over PHP. I would love that for PHP so I could begin writing my own extensions easier.
• Get rid of eval(). eval() is evil. If you’re using it, something is wrong.
• Support operator overloading. Because everything are objects, operators are simply methods on objects. Thus, I could have the + operator do something entirely different on an object.
• Allow method signatures. Allow true method signatures, so programmers can have methods of the same name with a different argument list or return type.

  class A {
    public int function doSomething(int $a, float $b) {
      int $c = $a * $b->to_int(); // Same as $a->*($b->to_int());
      return $c;
    }
   
    public float function doSomething(int $a, float $b, float $c) {
      float $d = $a * $b * $c; // Same as calling $a->*($b->*($c)); since * is a method on each object $a and $b.
      return $d;
    }
  }

• Build a PHP Virtual Machine (PVM). I’m not entirely sure this is possible as I’m not a language designer, but it would be nice to have a PHP Virtual Machine. It could execute PHP bytecode, and allow an explicit heap and stack.
• Remove copy-on-write (COW). COW is a fairly foreign concept to newer developers, and can cause problems if you’re not aware it exists. With a PVM, PHP could have an explicit stack and heap, allowing programmers to declare variables statically on the stack, and dynamically on the heap. The garbage collector would then collect variables on the heap.
• Publish an official PHP spec. Similar to the W3C HTML5 spec, but much more concise and designed by a smaller committee, the PHP spec would allow developers to implement their own versions of PHP and would ensure there’s concrete examples to compile against.

Respect the Language

An effort should be made to get the language respected. We should try to recruit developers who can be very vocal about great features of PHP7. We should release great code that’s secure, simple to read, and teach new developers the right way to do things. PEAR should be cleaned up, and a great set of core libraries written in PHP7 should be released.

If we can get higher profile developers to respect the language, we’ll be in the right direction. They don’t have to use the language, but just some mutual respect. I’m not crazy about Ruby, but I respect the language. I see it’s power, and I see where Ruby on Rails is a very nice framework, it’s just not my favorite. If we could get high profile developers (say, Martin Fowler) to endorse or respect the language, it would go far in a lot of people’s eyes.

Respected Developers

Continuing from above, we need a core team of very highly respected PHP developers. As they release code, give talks, and show people the “right way” to do things, this team will grow and soon many developers will be respected. Similar to what John Resig has done to JavaScript, these developers would do with PHP. Take the language to new heights and show how powerful it can be.

Conclusion

I’m excited about the future of PHP. I highly doubt any of my ideas would be implemented, but I truly believe they’d help the community as a whole. I was very excited when PHP announced HipHop-PHP. Having the second largest website in the world announce they were helping the PHP community that much is great. PHP isn’t going away, there’s simply too much code written in it to go away for a long time. However, lets try to improve it at each release, and maybe one day, it will be a highly respected language.

Finally, I’m not a language designer. My ideas could be entirely horseshit, or valid. If I’m incorrect somewhere, please politely let me know and I’ll be happy to talk about it. Let’s work together to make PHP a well respected, powerful, fast, and efficient language.

Happy hacking.

I’m hosting Jolt’s Git repository on my server rather than using Github. To do this, I’m using Gitosis, a small and efficient program in Python to manage your Git repositories. The brilliant thing about Gitosis is it uses Git to manage your repositories. You can learn to properly host your Git repositories quickly and easily.

What that article doesn’t mention is how to start up and shutdown the git-daemon properly on an Ubuntu system. The git-daemon program is what is used to serve a Git repository. I wrote a small init script to handle starting and stopping git-daemon. Place this script in /etc/init.d/ as gitd and make sure it is executable (chmod +x gitd).

#!/bin/sh
 
BIN=git-daemon
USER=git
GITDIR=/home/git
REPODIR=$GITDIR/repositories/
PIDFILE=$GITDIR/.git-daemon-pid
 
case "$1" in
  # Start command
  start)
    echo "Starting $BIN"
    sudo -u $USER $BIN --base-path=$REPODIR --pid-file=$PIDFILE &
    echo "$BIN started successfully"
  ;;
 
  # Stop command
  stop)
    echo "Stopping $BIN"
    if [ -f $PIDFILE ]; then
      sudo -u $USER kill -9 `cat $PIDFILE`
      sudo -u $USER rm -f $PIDFILE
    fi
    echo "$BIN stopped successfully"
  ;;
 
  *)
    echo "Usage: /etc/init.d/gitd {start|stop}"
    exit 1
  ;;
esac
 
exit 0

Change the directories to wherever you have Gitosis installed. Install the script by running update-rc.d.

update-rc.d gitd defaults

And now it will be executed on startup and shutdown.

I do freelance work from time to time to pick up some extra money for my family. It’s generally not much, just small changes here and there on mostly existing websites. Rarely I write software from scratch for a client. I always get a surprised look when I explain how much I charge. I generally use my hourly rate to determine an overall cost for the project as well (if the project is very large, I’ll give an estimated project price). I’ll do my best to determine the number of hours I think it’ll take to complete it and multiply it by my hourly rate.

My last bid was for changing some functionality of an existing shopping cart built on Joomla and VirtueCart. I’m not familiar with either of those platforms (from a developers perspective, I’ve taken cursory glances at both), so doing the changes the client wanted was going to take longer. That learning experience will be passed on to the client. When doing freelance work, you should treat yourself as an individual corporation (and if you’re not incorporated, you should be). Corporations generally don’t do things for free, and you shouldn’t either.

I also build software correctly. Even if I’m just working alone, I try to maintain an agile approach. I write tests (this is definitely my weak spot, but I’m rapidly improving), I track bugs, I use source control. If you’re not doing these things, you’re not building software correctly. That work is time passed on to the client. If you, the client, wish for properly developed software, you will have to pay for it.

VirtueCart is horribly written. I can tell it’s horribly written just by glancing at it. I’ve been writing PHP for 10 years now, I know what crap PHP looks like quickly (conversely, I know what great PHP looks like quickly, and it’s very possible to write great PHP). The code [VirtueCart] is typical PHP garbage, very little coding standards, globals littered everywhere, no true object oriented design, etc. (Joomla has a test suite, and seems much better architected). There’s very little design that went into VirtueCart’s development, and it appears to be a horrible experience to work with. Having to work in that environment is going to cost the customer. Some refactoring will be required to make the software bearable, which will cost the client.

In the end, I lost the bid (fortunately). What was unfortunate was this was functionality the customer really wanted. This means they’ll approach a cheaper developer to get it completed. Hopefully, the developer maintains a high level of development standards, but it seems like in the web world, that just doesn’t happen often.

When I first started to learn about agile approaches and test driven development, the ideas sounded absurd: no client is going to pay for that! They throw a fit when it costs them even the slightest change. But that’s not how it works. If you wish to have properly developed software, you need to spend the money to get it. There is too much shit out there, and it’s only going to get worse unless developers become active, passionate, and involved in their craft. The client does not have to understand or know you’re writing unit tests, they simply need to know how much it will cost them.

Software development should not be a “thing I do for a living”. It should be your passion. If it’s not, please leave the industry and find something else to do. I charge what I do because software development is my passion, and not one I take lightly.

So, yes, that is what I charge, no, I’m not changing it: take it or leave it.

Interceptors will be a new feature added to Jolt. When dealing with a classic Model-View Controller, the Router should determine both the type of content coming in (via the Content-Type header), and the type of data the client with accept (via the Accept header).

If an HTTP/1.1 request contains these headers, the RESTful application should respond correctly.

Content-Type: text/json
Accept: text/json

The MIME-type application/json is also valid, I believe. The Router needs to interpret this headers and tell the Controller how to respond accordingly. Of course, the default Content-Type and Accept headers are text/html in Jolt, but in the cases when they are different, what should be done?

The concept of Interceptors are introduced. An Interceptor takes the resulting object from the Controller and determines what View to send it to and how. By default, an Interceptor will take the entire Model and turn it into the appropriate type.

class User_Controller extends Jolt_Controller {
  public function bookListGet($user_id) {
    // Assuming the Accept header is text/json, Content-Type is normal text/html
    $user = new User($user_id);
    $book_list = $user->getBookList();
 
    // Render accordingly
    $this->book_list = $book_list;
    $this->render('book-list');
  }
}

The render() method will know what type of content the client accepts. If it’s the default HTML, that View will be loaded, rendered, and returned. In this example, the JSON Interceptor would turn the $book_list object (or array) into a JSON object and return that to the client.

What if the resource were to only return a single object? How might that look?

class Post_Controller extends Jolt_Controller {
  public function postGet($post_id) {
    $post = new Post($post_id);
 
    $this->post = $post;
    $this->render('post');
  }
}

The client would receive a JSON object that contains the post data.

{"date_create":1266926552,"date_modify":0,"title":"My First Post","content":"Here is the body of my first post."}

This is a fairly straight-forward example, but what if you didn’t want the date_create or date_modify fields returned in the object? The default JSON Interceptor does a 1:1 conversion between a PHP object and JSON.

An extended Interceptor would be written for that resource that would remove those fields.

class Post_Interceptor extends Jolt_Interceptor_Json {
  public function run() {
    // $this->post is the object from above passed into here.
    $this->post->remove('date_create', 'date_modify');
 
    // Encode everything to JSON and return it to the client.
    parent::run();
  }
}

The above are just examples as I’m fleshing out Jolt, but this should allow for systems that can be quickly and easily built. Basic models will easily handle themselves. More complex models may require further action; an Interceptor could run additional operations on an object before it was returned to the client.

As Jolt becomes more mature, I promise the methods above will change; this should give you an idea of how a small system would work.

I released Artisan System in November 2008. Ever since I released it, I was never crazy about the name. I always felt like it would be difficult for developers to say they used Artisan System. I intended it to be a “system” level framework, where it handles many of the system requirements of Internet software. It has since morphed from there to something more, so I feel its time for a name change.

Coming up with a new name is very difficult. It seems like every other framework out there has a great name, except mine. I tried going the ancient god’s route, and thought Gnosis was good, but there are already plenty of Open Source projects named that, one of which is related to PHP development. So that was out. I found a Hindu god of knowledge and creativity, Saraswati, and thought about the name Swati. It was unique, easy to pronounce and spell, but after talking to ZDC, I determined it wasn’t a fitting name.

Ultimately stuck, I turned to my wife. I explained to her my dilemma, and within about three or four attempts, she came up with Jolt. I loved it. Easy to spell, four letters that mean a lot. Speed, intensity. The things I wan to convey with my framework.

Jolt.com is, of course, taken (and for sale consequently). After coming up with a name is finding the right domain to go along with it. This one was easier than expected. I like the name “core” associated with my projects since I want Jolt to be the core framework you use for your project. Unfortunately joltcore.com is taken, but not used for anything. Joltcore.org was free, and since this is a non-project Open Source, a .org works great for it.

I’ve reserved a sizable Linode server for hosting. Issue Tracking, Wiki, Technical Docs, Mail, Git, and the main website will be hosted there. Copies of Jira and Confluence will be used for the issue tracker and Wiki, respectively. I’ll be using Gitosis for Git hosting, and Doxygen will be used for documentation.

This is an exciting time for my software. With a rename comes an entirely new roadmap and scope for Jolt.

What’s New For Jolt

Jolt aims to be an entire solution for building Web based software quickly and painlessly. Jolt will support an entirely RESTful interface for building a front facing website or an API.

More will announced later this week as I develop the roadmap and timeline even more. If you’re a developer and wish to contribute, I’d love to hear from you.

I’m very excited about this change and what it will bring to software development. Who knows, maybe I’ll even win a Jolt Award.

Happy hacking.

Facebook recently open-sourced their new software, HipHop-PHP. HPHP compiles PHP source to very efficient C++, which is then compiled to an executable binary. After a few hiccups, building HPHP is surprisingly easy. HPHP must be compiled on a 64bit platform. If you only have a 32bit platform and you really want to play with it, set up a VM at Slicehost or Linode and compile it there (if you have a 32bit platform, you can’t install a 64bit VM). I eventually used my Linode account and built it there.

First, you’ll need to install Git if you haven’t already.

# You might as well drop into root. You'll be doing everything as root
# so I won't prepend everything with sudo.
sudo su
apt-get install git-core

Ubuntu comes with a handy package called build-essential which installs all of the tools you need (make, gcc, g++, and more) to build software from scratch.

apt-get install build-essential

Navigate to the /opt directory and clone the project from Github.

cd /opt
git clone git://github.com/facebook/hiphop-php.git

It comes with another Git module, libmbfl, which will cause you a lot of headaches if you don’t initialize it.

cd hiphop-php
git submodule init
git submodule update
cd ..

HPHP requires a patched version of cURL and libevent, so you’ll have to patch and compile those normally.

# cURL first, this will be installed into /usr/local/lib/,
# manually change the prefix in ./configure to install it elsewhere
wget http://curl.haxx.se/download/curl-7.20.0.tar.gz
tar -xvzf curl-7.20.0.tar.gz
cd curl-7.20.0
patch -p0 < ../hiphop-php/src/third_party/libcurl.fb-changes.diff
./configure
make && make install
cd ..

# libevent will be installed to /usr/local/lib/,
# manually change the prefix in ./configure to install it elsewhere
wget http://www.monkey.org/~provos/libevent-1.4.13-stable.tar.gz
tar -xvzf libevent-1.4.13-stable.tar.gz
cd libevent-1.4.13-stable
patch -p0 < ../hiphop-php/src/third_party/libevent.fb-changes.diff
./configure
make && make install
cd ..

HPHP requires libicu, a Unicode library. However, it requires 4.2+, and the latest version in the Ubuntu repositories is 4.0, so you’ll have to manually compile that as well.

# libicu will be installed to /usr/local/lib/,
# manually change the prefix in ./configure to install it elsewhere
wget http://download.icu-project.org/files/icu4c/4.2.1/icu4c-4_2_1-src.tgz
tar -xvzf icu4c-4_2_1-src.tgz
cd icu/source
./configure
make && make install
cd ..

Oniguruma is a Unicode regular expression library that you’ll need to compile and install.

# oniguruma first, this will be installed into /usr/local/lib/,
# manually change the prefix in ./configure to install it elsewhere
wget http://www.geocities.jp/kosako3/oniguruma/archive/onig-5.9.2.tar.gz
tar -xvzf onig-5.9.2.tar.gz
cd onig-5.9.2
./configure
make && make install
cd ..

Head back into the hiphop-php source tree. We’ll be able to install the rest of the required libraries from Ubuntu’s repository.

cd hiphop-php
 
# Libraries
apt-get install cmake flex bison re2c libmysqlclient16 libmysqlclient16-dev libxml2 libxml2-dev libmcrypt4 libmcrypt-dev binutils binutils-dev libcap2 libcap2-dev libgd2-xpm libgd2-xpm-dev zlibc libtbb2 libtbb-dev libpcre3 libpcre3-dev libpcre++-dev libssl0.9.8 libssl-dev
 
# Boost, in reality you do not need *all* of Boost, only a few
# libraries, but you might as well install all of it to play with.
apt-get install libboost-date-time1.40.0 libboost-date-time1.40-dev libboost-filesystem1.40.0 libboost-filesystem1.40-dev libboost-graph1.40.0 libboost-graph1.40-dev libboost-graph-parallel1.40.0 libboost-graph-parallel1.40-dev libboost-iostreams1.40.0 libboost-iostreams1.40-dev libboost-math1.40.0 libboost-math1.40-dev libboost-mpi1.40.0 libboost-mpi1.40-dev libboost-program-options1.40.0 libboost-program-options1.40-dev libboost-python1.40.0 libboost-python1.40-dev libboost-regex1.40.0 libboost-regex1.40-dev libboost-serialization1.40.0 libboost-serialization1.40-dev libboost-signals1.40.0 libboost-signals1.40-dev libboost-system1.40.0 libboost-system1.40-dev libboost-test1.40.0 libboost-test1.40-dev libboost-thread1.40.0 libboost-thread1.40-dev libboost-wave1.40.0 libboost-wave1.40-dev

Define some environment variables for compiling.

# Ensure you are in /opt/hiphop-php/ at this point
export CMAKE_PREFIX_PATH=/usr/local
export HPHP_HOME=`pwd`
export HPHP_LIB=`pwd`/bin

Compile and install everything.

cmake .
 
# Assuming cmake doesn't give you any errors.
make && make install

After you’ve compiled HPHP, the binary is located in hiphop-php/src/hphp/ and is aptly named hphp. From here, you can follow the instructions on http://wiki.github.com/facebook/hiphop-php/running-hiphop on how to use it.

So far, it’s very promising. Of course, you probably don’t fit the use-case for this. Regardless, I believe this will take PHP application in new directions. No longer will you need a Zend Encoder or Optimizer, application deployment can be installing a single binary, and it will be possible for companies to sell their applications without giving away the source code. Everything I release will still be entirely Open Source, however, a HPHP binary may be attached for easy deployment.

I recently read an article about a problem with blogging: very outdated and inaccurate articles (particularly technical articles) are still indexed by Google, so when you’re browsing for a solution to your answer, and you find an article from 2001, you end up becoming more frustrated.

Because this is a technically oriented website, I’ll be validating all of my articles, and doing it on a routine basis. You’ll see a banner above each article explaining that it is still technically valid should you happen onto it by way of a search engine.

Valid articles will see a describing the date the article is valid until. If the current date is after that date, that doesn’t mean the article isn’t invalid, simply that it hasn’t been validated since that time.

Invalid articles also have a header describing them.

Facebook has officially released HipHop for PHP on GitHub. I’ve spent all morning attempting to compile it on my local machine (Ubuntu 9.10 32bit) with no luck. I’ll try another computer in a moment, and then post a detailed instruction set on how to build it fully.

If you’re not familiar with compiling programs on a Linux environment, it definitely has a steep learning curve.

Stay around for updates throughout the day.

Update: You must compile this on a 64bit platform.