Archive for August, 2005

Last minute changes

Published on August 24, 2005 in Stuffed Tracker. 0 Comments

So it’s quite possible that we will meet the 1st of September deadline for launching Stuffed Tracker 2.0. Almost everything is ready, and I am currently preparing a User Guide which would be in PDF format. We originally wanted to create the guide in Adobe InDesign, but due to lack of time we might as well go with an auto-converted-from-Word version, sacrificing quality a little.

Anyway, at the beginning of April 2005 when the main tracking framework was ready, we started collecting statistics data on several large and small friendly sites. 5 months later we’ve managed to accumulate almost 500 megabytes of data!

Database stats in Stuffed Tracker

And this actually helped us to identify the bottleneck in our original database design. We have designed the database in such a way that the same main 6 tables are used for storing tracking data for all sites defined in the tracker. However, this proved to be not so good after all, since this design penalized small sites.

Even if Stuffed Tracker displayed statistics for a small site with little amount of statistical data, we still had to extract these stats from huge common tables, sometimes containing several millions of records! So we did a terrible thing — we’ve made a major product design change only one week before it’s release.

Thankfully, due to a good code design, the change took only 1 day and went smoothly (as far as we can see). Now statistics for different companies and their sites is stored in separate company tables and doesn’t interfere with statistics for other companies and their sites.

This new design adds a little more work for the tracker in several places, but what is much more important — the statistical reports generation is now much more efficient in situation when a company with a large site and a company with a small site are tracked using one Stuffed Tracker installation..

PHP key generator for ShareIt

Published on August 17, 2005 in General. 4 Comments

We are currently working in a “death march” mode in order to launch our new site, Stuffed Tracker 2.0 and Stuffed Sync on the 1st of September. Also we are going to launch a new sales system, integrated with our payment-taking partner ShareIt.

The integration is quite simple actually — whenever a customer pays for our product, ShareIt’s own keygen server should make a POST request to our script to get the key (which is just a short text string in our case) that will be presented to the customer at the end of the payment process. This whole process is described in the SDK that you can download from ShareIt if you are a registered author in their system.

I’ve already done this integration before, but I’ve used Perl back then. The new version of our sales system is built with PHP. And unexpectedly we’ve got a big problem because of this yesterday. I’ve actually had to spend the whole day on fixing it, instead of doing other important things.

So, to save time for someone who might also want to use PHP to write a keygen for ShareIt, here is the whole story.

In order to install a new keygen for a particular product in ShareIt’s system, you need to write them an email, in which you should specify the URL of your keygen. Then they use a special tool, called KeyTest.exe (provided with their SDK), to check that your keygen is actually working. This tool is quite simple; it just submits several test values to your keygen via a POST request and then displays the response.

So we’ve got stuck on this testing phase yesterday. We’ve manually tested our keygen with several POST requests containing the required fields that ShareIt should provide to us and everything worked perfectly. So we happily submitted the keygen to ShareIt and quite soon got a response from them that our keygen is not working in their “test environment” (which is their slang for this KeyTest.exe tool).

This was a surprise to us, but we really haven’t tested our keygen using KeyTest.exe up to this moment. So we’ve launching this tool and tried to submit a test POST request to our keygen with it. And it didn’t work just as ShareIt has told us.

Here is the most interesting thing: after we’ve started debugging our script, we’ve quickly realized that PHP doesn’t populate $_POST or $_REQUEST variables when KeyTest.exe makes its request. Yes, that’s right — we’ve got nothing in these variables, just as if no parameters were submitted at all.

It was quite strange, to say the least. Obviously ShareIt uses this tool to test keygens everyday and they should have noticed that it doesn’t submit any parameters in its POST requests. So we’ve continued digging deeper.

I’ve opened up Ethereal (which is a great network traffic monitoring tool, btw) and started capturing TCP traffic during KeyTest.exe POST requests. And immediately I’ve found the problem – KeyTest.exe wasn’t specifying Content-Type with its request, so PHP didn’t know how to handle the data submitted via POST.

Thankfully, in cases like this, PHP creates a special variable called $HTTP_RAW_POST_DATA in which it stores all data that was passed to it in the POST request (basically everything it gets from STDIN). And, lo and behold, we’ve found the missing parameters there. It was trivial after this to make our keygen work with ShareIt’s KeyTest.exe tool.

But our problems were not over yet. We’ve wrote a new email to ShareIt saying that we’ve finally made our keygen work. But in their response ShareIt told us that the keygen still doesn’t work.

What the hell, we thought and looked again at the values that KeyTest.exe submits. For some reason one parameter called PURCHASE_ID was specified as equal to 0. We had a test in our keygen that specifically tested that this parameter is not 0; if it was, then our keygen returned an error.

We’ve added this test because the description of this parameter in the SDK says this: “Type: decimal string representation of an unsigned 32bit integer number > 0” (emphasis is mine).

Not surprisingly any more, after we’ve removed the test for this parameter and sent yet another email to ShareIt, our keygen was finally accepted.

And that’s the happy end of this story. Hope it helps someone.

Process of creating user interface

Published on August 5, 2005 in Software Engineering and Stuffed Tracker. 0 Comments

I personally think that user interface is the most important part in any software. You can have the best functionality on the market, but if the UI in your product sucks then no one will buy it. Who needs software that is impossible to use?

In Stuffed Guys, we have a 2-tier process of creating user interface.

The first version of the interface is created during initial product development. It is done only by us; professional designer is not involved. The first version evolves a lot during the time when the product is developed. We constantly test it on ourselves as the first users of the product and make improvements where we can.

At the end of the product development we usually have an overall good UI, that as we think is easy to use and it usually looks good too. But we realize that since we were involved in the development of the product and know everything it does and how it does it we probably don’t see things that are still inconvenient for a new user.

So then we handle functionally finished product to the professional designer. One of the best advantages for us is that this designer is completely new to the product, he hasn’t used it yet. Based on his fresh look on how things work and should work, he creates an improved user interface, adding all required eye-candy to it in the process as well.

With Stuffed Tracker, we are currently at the last stages of the second tier described above. Professional designer is almost finished with his version of the UI and we are currently implementing it in html. Now, since the new version of Stuffed Tracker is build on our own Nova System framework that supports skins, we decided to keep both versions of the interface as two different skins both of which would be available in the final product.

1st version

1st version of the interface

2nd version

2nd version of the interface

Synchronize MySQL Databases

Published on August 2, 2005 in SyncSQL.com. 4 Comments

As developers of our own web software and contractors that currently support a large online reservations system (www.twibo.com), we’ve always had one major problem during the software deployment – synchronization of beta and live database structures.

There are usually some changes in the structure of the database tables between release X and release Y. It’s quite simple to add a few new tables to make the structure of the databases in sync, but it becomes a real pain when indices were updated in some tables, fields renamed or added or descriptions of the fields have changed.

Having searched for a web based solution for this problem and been unable to find one, I’ve decided to create my own. That’s how Stuffed SyncDB was born.

Stuffed SyncDB

Shortcomings:

  1. Built with Perl — requires Perl with MySQL support to be available on the server where SyncDB is going to be installed.
  2. Can only synchronize MySQL databases structure (for now).
  3. Doesn’t take into account types of tables (was only tested with MyISAMI tables).

Features:

  1. Synchronizes two databases in 2 easy steps — select a source and target database from a list, click a button and you are set.
  2. Generates synchronization strategy (SQL statements that would make target database’s structure equal to the source database’s structure) that can be edited before it will be executed.
  3. Supports renaming of the fields — understands that if a field is missing in the target database it doesn’t necessary mean that it should be dropped.
  4. Allows to compare a database structure of a real database with a database structure dump.
  5. Allows to compare two database structure dumps with each other.
  6. Database structure dumps could be pasted in a textarea or uploaded as a file.
  7. You don’t need to have direct access to a remote database to synchronize it with your local one — just install Stuffed SyncDB on the same server with the target database and upload a dump of the source database structure.

Now the good news is this: Stuffed SyncDB will be free and would be released under a GPL license or similar. This would happen at the day of Stuffed Guys Relaunch*, when we will release Stuffed Tracker 2.0 and also show our freshly redesigned web site to the world.

* Hopefully this day would be 1st of September 2005.