The Sean Code

The Entire Problem with Microsoft

2008-11-06T18:32:00.003-06:00

Steve Ballmer has always had the uncanny ability to take something complicated and sum it up in a nice, simple phrase that explains it so everyone can understand. In dismissing Android as a competitor, he's done it again:

"Google doesn't exactly bubble to the top of the list of the top competitors we've got going in mobile. They might someday. But right now..." he said.

In other words, Microsoft doesn't bother trying to compete with someone until they're already beaten. It worked when they created Windows after they'd been beaten by the MacOS, but since then it's been far less successful (IE, Vista, search, Zune, etc).

Perhaps they should try competing with something while it's still new and weak? Maybe they could finally win again? Like when the 360 beat the PS3 by attacking it before it was ready?

Just a thought.

Breadcrumb History in Django

2008-10-26T14:29:00.002-05:00

This afternoon I wanted to add a breadcrumb-like feature to an application I've been writing in Django. I figured the best way to do this is with sessions, so the first step is to turn those on.

The way I wanted it to work is like a rolling history of pages visited. So there's no implied hierarchy of pages, it just shows you the pages you've visited most recently. I figured limiting it to four pages was reasonable, but I can certainly see myself increasing that number in the future.

The first thing to do is to create a helper function that adds the current page to the history. You want to create the default empty history if there's nothing for it in the session, and you want to make sure it doesn't get too long. Also, you access the session and the address of the current page through the HttpRequest object that's available to every view in Django, so we just pass that in.

Here's my function:

# the breadcrumb history is a list of the last pages the user has visited
# the newest page is at the end of the list
# the length of the list is limited; currently it is at four pages
def add_breadcrumb_history(request):
    history = request.session.get('breadcrumb_history', [])

    # if the last item in the history is the current page, we don't want to add this page to the history
    # if it's not the last item in the history, we do add it
    if len(history) == 0 or history[len(history)-1] != request.path:
        history.append(request.path)

    # if there are more than four items in the history, pop the first one
    if len(history) > 4:
        history.pop(0)

    # save the history to the session
    request.session['breadcrumb_history'] = history

    # return the current breadcrumb
    return history

And in your views, its usage is simple enough. You call:

breadcrumb = add_breadcrumb_history(request)

in your view, preferably after you verify the user's permission to view that page. Then you want to add it to the list of variables exposed to your template, usually in your call to render_to_response().

Then you have to add the breadcrumb handler to the template. I put mine in my base.html, so it's available on every page.

{% if breadcrumb %}
<p class="breadcrumb">
{% for page in breadcrumb %}
<a href="{{ page }}">{{ page }}</a>
{% endfor %}
</p>
{% endif %}

Very simple; if there's no breadcrumb history, it doesn't display anything. If there is, it displays the links in order.

I like this feature; while I recognize it's close to useless in some situations (namely those where the data actually adheres to a strict hierarchy), in others it can be really useful. It means I have to do a lot less state checking on my pages to try to "calculate where the user came from" to get to that page, and can trust that users can easily navigate back to recent pages even if they POSTed information in between. (While I like that "You POSTed information on this page, do you really want to hit the back button?" feature, it can be really annoying in practice.)

Just thought I'd toss this up there in case anyone found it useful -- or is aware of similar functionality elsewhere.

Asus 1000H vs Acer Aspire One ... A Pack of Lies

2008-10-02T21:48:00.001-05:00

I just saw this "brilliant" visual comparison of the Eee 1000H vs the Acer Aspire One, and I can't just let it pass. I mean, I understand that the netbook marketplace has become absurdly overcrowded, and the main differentiators between the machines are 1) price, 2) software, 3) packaging. Asus is clearly annoyed that despite virtually inventing the category, they are left as the overpriced competitor much maligned for their confusing marketing and unpredictable product cycle.

Meanwhile, Acer has pumped out exactly one netbook, priced it competitively, and just let people buy it and enjoy it. And here comes Asus to rain on Acer's parade in the most immature way possible. (They couldn't even bring themselves to spell out Aspire, instead spelling it "A*****" -- um, lame?)

As the owner of an Asus laptop and an Acer Aspire One, I find myself compelled to take on the comparison point by point.

1) Battery life

Asus: 6 cells, 7 hours
Acer: 3 cells, 2.25 hours

I haven't used the Eee 1000H, but my Aspire One gets 3.5-4 hours of battery life when I use it normally (ie, wifi 100% of the time, internet browsing, typing). And that's with the default 3 cell battery, not the add-on 6 cell. If the Eee really gets 7 hours of battery I'd be impressed, but I wouldn't be at all surprised if they're lying about their own battery life to the same extent they're lying about the Acer's.

2) Hard drive

Asus: 160 GB
Acer: 120 GB

Firstly, big f'ing deal. It's a netbook. If you're storing a bunch of media locally, you're doing it wrong. Secondly, the Acer defaults to a nice little SSD that more than gets the job done and allows me to 1) not worry at all about the fact that I keep dropping it, and 2) boot/sleep really quickly. 40 GB is meaningless on a laptop, moreso on a netbook.

3) RAM

Asus: 1024 MB
Acer: 512 MB

Yes, I could go for more RAM. It's not like I've been dying without it, but this is a pretty clear win for the Asus.

4) Size

Asus: Large touchpad, large screen, large keyboard (92% standard), wider palm rest
Acer: Tiny touchpad, small screen, small keyboard (80% standard -- although it's actually 85%), tiny palm rest

Every single point here is true ... and made possible by the fact that the Asus is a 10.2" netbook while the Acer is an 8.9" netbook. I wonder how Asus feels about its own 7" netbooks. Also, more lying ... the keyboard is bigger than Asus claims it is. And Acer's sub-9" netbook is certainly more portable than the over-10" Asus. Which, I think, is the point of netbooks.

5) Um, miscellaneous networking, I guess

Asus: Bluetooth & 802.11n, 1.3 megapixel webcam, "Digital Array Mic"
Acer: No Bluetooth & 802.11n, 0.3 megapixel webcam

Frankly I wish my Acer had Bluetooth, but I can't say I miss it all that much -- I don't even use it on my full-size Asus. And 802.11n isn't that meaningful to me, since I don't have an 802.11n router (and neither does anyone else). Oh, and the webcam? My full-size Asus has a 1.3 MP webcam, and it looks awful compared to the 0.3 MP on the Acer, when both are using Skype on Linux. Seriously, Acer's 0.3MP webcam is surprisingly awesome. And what the hell is the "Digital Array Mic?" Is it something that solves the horrible feedback problem my full-size Asus has? Because if it is, that's nice, but the Aspire One doesn't have any feedback problems with its mic.

6) Location of touchpad buttons

Apparently the familiar position of the Asus' touchbad buttons allows easy usage, while the odd location of the buttons on the Aspire One leads to "unsmooth usage." I've heard other people complain about this too, and I just don't get it. It seems to me that it's an innovative use of space given the obvious size constraints, and it's not exactly counterintuitive -- the "right click" button is "on the right side" of the touchpad. Holy crap, I'd never find it!

7) Temperature

Asus: Gives off no heat at all -- the whole thing is green!!
Acer: It's on fire! Look at all the red!

In reality ... I can't speak to the Asus, but the Aspire One doesn't give off any heat. I've used it for 10 hours straight, plugged in for 4, then 3 hours on battery, then plugged in for another 4, and the temperature remained constant the entire time. And it remained constantly the same temperature as it was when it was off. Both on the top and bottom, it's comfortably cool the entire time. I don't know how they came up with this test, but the only way it could be true is if the Asus has The Al Gore Anti-Global-Warming Super-Prius Hybrid Engine built into it, or something, and actively sucks heat and carbon emissions into it and converts them into sunshine and virgins. I call bullshit.

8) "Work Efficiently"

Asus: Boots in 30 seconds, ready in 38 seconds, ready to launch "AP" in 41 seconds, shuts down in 15 seconds
Acer: Boots in 45 seconds, ready in 62 seconds, ready to launch "AP" in 74 seconds, shuts down in 30 seconds

Apparently time goes faster in my apartment, because the Aspire One actually boots in about 10 seconds. And shuts down in about 5. And goes from "off" to "browsing the web" in 12-15. I don't know what "AP" is, so I'll go ahead and skip it. I guess I don't use it. Hopefully Asus' users really like it, because it seems to take a really long time. The only computers I've seen get going faster than the Aspire One are Macbooks. And, um, those are in a slightly different market segment, and I don't think I'd compare them.

9) "Quiet Computing"

Asus: Idle, 25.1 dB; Media Player, 28.8 dB
Acer: Idle, 30.7 dB; Media Player, 31.5 dB

I guess the 2.7-5.6 dB difference is simply massive, the difference between "silent" and "a fucking jet engine." But I have two things to say: 1) I haven't heard the Aspire One since I've owned it; I wasn't even aware it had a fan in it. 2) If you're using the media player and not wearing headphones, you're doing it wrong; it's not like these things come with great speakers. Also, 2.7 dB? Did you know that normal humans can't tell the difference?

Finally, the major differences as I see them:

1) Size

The Acer is a 9" netbook, and is smaller, thinner, and lighter (2.2 lbs vs 3.2 lbs) than Asus' 10" entry. Really, Asus? That extra 1" of screen space weighs an entire pound? Are you sure you're not just bad at this?

2) Price

I bought my Aspire One for $400 a couple months ago, and it's down to $330 from Amazon. On the other hand, the Eee costs $530 and is available for pre-order. Raise your hand if you can tell which is better.

3) Linux

The Aspire One runs Linux, and runs it well. The Eee 1000H runs Windows XP, and I'm sure it runs it worse than computer with 2-5x its specs that can barely keep themselves on their feet. Acer wins.

In conclusion, the choice of a netbook really comes down to personal taste and how much you want to pay for it -- you might prefer a 10" Windows machine to a 9" Linux machine, and you might not. But this kind of childish lying about yourself and your competitor is the stuff of Microsoft and Republicans. Is that really who Asus wants to model themselves after? Oh yeah, and I thought false advertising was illegal?

The Contractor Experiment Ends, Plus My First Wordpress Plugin

2008-08-31T13:51:00.004-05:00

It's been a while since I've posted anything, but I figured I should mention this: The Contractor Experiment is over, after just one contract. I found myself in a place I liked, with a good working environment and good co-workers ... and I became a full time employee last week. I plan on working exactly the same way as an employee as I did when I was a contractor -- why else would they have hired me? -- and it should remain a lot of fun.

Meanwhile, I think I should also mention a little side project I've been working on, and have just released. It's a Wordpress plugin that downloads baseball player's stats and displays them on your blog.

It can be seen in action at my baseball blog, Fire Gardy. To learn more about the program, you can visit the Fire Gardy Stat Grabber home page. Or just jump straight to the project page at Github. (It's GPL software.)

When I wrote it, the first step was to write a Baseball-Reference.com parser, which grabs the requested player's page (via PHP's CURL), and runs through it line by line to get the player's batting statistics (pitching stats will have to be added later).

Once that was there, it was time to turn it into a Wordpress plugin. It was actually a lot easier than I thought it'd be.

First you go to your blog's wp-content/plugins directory. Create a new directory inside it for your plugin (mine was called "fire-gardy-stat-grabber"). Inside that new directory, create a PHP file with the same name as the plugin directory ("fire-gardy-stat-grabber.php").

Open it up and put a comment inside it that will tell Wordpress the details of your program. Mine looks like this:

/*
Plugin Name: FireGardy Stat Grabber
Plugin URI: http://stat-grabber.firegardy.com
Description: Download a player's stats from Baseball-Reference.com and display them on the page.
Version: 0.1a
Author: Sean Schulte
Author URI: http://seancode.blogspot.com
*/

Really, only the "Plugin Name" line is required.

You need to define a function that will display what you want -- because for this plugin we'll be using a "Template Tag" (which allows us to put it anywhere on the page by editing the theme's template files).

It's a simple PHP function, that can take any parameters you want. It doesn't return anything, and needs to echo your desired HTML output to the screen. Mine has the following signature:

function fire_gardy_stat_grabber($playerName, $playerId, $year)

I'm not going to print all the code for it here -- you can go check it out at Github.

You can use the Wordpress database in this function -- which is useful for caching the results you get back from Baseball-Reference. There are two options for using the database. One is to create your own tables and use actual SQL queries; that'd be useful for more complicated plugins. The other is to use the Wordpress options table which stores anything -- accessible as part of a simple name-value pair.

get_option($key);
delete_option($key);
update_option($key, $value);

The plugin also uses a Wordpress Plugin hook to load some CSS. A function like this writes the CSS:

function fg_baseball_reference_css()

And this line makes it load:

add_action('wp_footer', 'fg_baseball_reference_css');

Once you've got that in place, just toss it into your sidebar.php in your theme's template:

<?php fire_gardy_stat_grabber('Nick Punto', 'puntoni01', 2008); ?>

And that's that! Now that I've finally written a Wordpress plugin I expect to do it significantly more often. I expected there'd be a lot more BS involved in hooking extensions into the Wordpress system, and realistically anyone who can write even the simplest PHP functions can do this.

Flex Injection: Passing a Function to a Component

2008-07-04T15:05:00.005-05:00

Sometimes, in Flex, I find myself in the situation of using a ViewStack to define different screens of my application. And I want to be able to navigate between those views easily, in addition to using components to make development simpler.

I've found that having a "Back" button inside a component can be useful, but if the back button is hidden inside the component, it can't access the ViewStack (since it's in the parent).

I thought it would be useful to be able to pass a little bit of ActionScript to the component that would jump back to the previous view. It would effectively inject functionality from the parent to the child. Here's what my component looks like:

<?xml version="1.0" encoding="utf-8" ?>
<mx:VBox xmlns:mx="http://www.adobe.com/2006/mxml" width="100%" height="100%">
 <mx:Script>                                                                                                                 
 <![CDATA[                                                                                                                   
                                                                                                                             
     [Bindable] private var _labelText:String;
     [Bindable] private var _buttonClick:Function;                                                                           

     public function set labelText(str:String):void {                                                                        
         _labelText = str;                                                                                                   
     }                                                                                                                       
                                                                                                                             
     public function set buttonClick(func:Function):void {                                                                   
         _buttonClick = func;
     }                                                                                                                       
                                                                                                                             
 ]]>                                                                                                                         
 </mx:Script>                                                                                                                
                                                                                                                             
 <mx:Label text="{_labelText}" />                                                                                            
 <mx:Button label="Back" click="{_buttonClick()}" />
</mx:VBox>

As you can see, this component allows you to define some text to display and the back button's click handler. The click handler has to be a Function object.

Note that when you call this Function object, you have to put () at the end. It thinks this is actually a method that it can call, so you have to call it like any other method.

But when you instantiate this component, you can't just do something simple like buttonClick="viewStack.selectedChild=anotherView" ... while that may be the ActionScript code you want to execute, and it may work when you're defining the actual click event, the component will interpret this as a String instead of a Function, and it won't work. So you actually have to create a Function object in the parent.

<?xml version="1.0" encoding="utf-8" ?>
<mx:Application
      xmlns:mx="http://www.adobe.com/2006/mxml"
      xmlns:components="components.*"
      width="100%"
      height="100%">
  <mx:Script>
  <![CDATA[
      [Bindable] private var backToViewZero:Function =
function():void { viewStack.selectedChild = viewZero };
  ]]>
  </mx:Script>

  <mx:ViewStack id="viewStack" width="100%" height="100%">
      <mx:VBox id="viewZero" width="100%" height="100%">
          <mx:Label text="Start here!" />
          <mx:Button label="One" click="viewStack.selectedChild=viewOne" />
          <mx:Button label="Two" click="viewStack.selectedChild=viewTwo" />
      </mx:VBox>
      <components:MyBox id="viewOne" labelText="One" buttonClick="{backToViewZero}" />
      <components:MyBox id="viewTwo" labelText="Two" buttonClick="{backToViewZero}" />
  </mx:ViewStack>

</mx:Application>

Here, we create an anonymous method and assign it to the Function object, and then we can pass it to the component when we instantiate it.

This works splendidly, and is a pretty cool way to inject functionality down into your components that wouldn't otherwise be possible. (The component doesn't have to reach up to the parent somehow, which means your component doesn't need to know what the ViewStack is called, or about any of the other views; instead, it just needs to know that it's going to do something, and the parent is responsible for knowing what.)

It leaves me wondering, though, if there's a way to get around the problem of passing in code and having it interpreted as a String rather than the preferred Function. Perhaps by having the buttonClick setter take a String and create the Function object at that point, using something like eval(). Unfortunately, ActionScript 3.0 doesn't have eval(), so I don't know if this is possible. Maybe I'll figure something out later. In the mean time, I'll be creating Function objects in the parent.

The Freedom Framework Bends to Your Will

2008-02-26T16:28:00.000-06:00

On Sunday, I came up with the concept for the Freedom Framework, which consists of PHP Object Generator, EpiCode, and Smarty. I'd like to take some time and go into what I feel are some of its actual advantages, over other frameworks and over not using a framework at all.

First, you define your data model from the perspective of the database, rather than from the code. While you don't actually have to write any SQL, you're defining the fields and their types as if you're just creating a table. It adds to this with the Parent/Child/Sibling concept, but doesn't go as far as other frameworks which give you the impression that you're just creating and working with objects, and then try to generate the database from that (or make you write both the database and the models).

When you've done this, you edit the config file with your database's login information and run setup. It tests your database connectivity, creates the tables, and then runs some unit tests to make sure everything's working. At this point you have a set of classes that give you CRUD functionality for the tables you've defined.

And now, EpiCode comes in to define your controllers. This is the best part of the "framework." Since you define the controller classes yourself (rather than having them generated), you have a little extra freedom there. And since you actually define the routes table (rather than generating it or being stuck with a convention), you gain more control over the URLs that'll be used by your application. If it's a small application, you can use just one controller class, even if you have a bunch of differently named URLs pointing to its methods.

That freedom allows for excellent refactoring support. As I was feeling my way around the new "framework," I had only one controller class. After my methods started growing, I decided it would be best to separate them into multiple classes. This was the work of a few seconds (made easier since these are static methods); then I just had to update the routes table to point the URLs to the new classes and methods ... and voila! The project graduates from being a simple proof of concept to being a maintanable MVC application.

Even Smarty allows this sort of flexibility for the view. When I was putting together my proof of concept, I just threw the .html files into the root templates/ directory. As it grew, and I decided that this was more than just a tiny project, I created subdirectories underneath the templates/ directory and moved the .html files into them. (It's probably simplest to name these subdirectories in such a way that they match the URLs, but you don't HAVE to.)

The goal of the Freedom Framework is to minimize the things you're forced to do to work with it. There's no generated filesystem structure or controller classes; you can name them and structure them however you please. The framework grows with each project, starting out as just a little bit of glue and expanding into a well defined structure for your code (and since it's YOUR code, you get to define that structure). You don't have to know (or be told) the final structure before you start working on it.

The more I think about this "framework," the more I like it. The more I use it, the more I like it. It's the first I've found that doesn't inherently limit the way I want to work. The tool bends to my will. That's what makes it a good tool.

POG + EpiCode + Smarty == The Freedom Framework

2008-02-24T21:05:00.001-06:00

I generally have a problem with web frameworks, like Rails, CakePHP, Symfony, and the millions of others that have sprung up lately (moreso in PHP than in Ruby, as Ruby hadn't really been used for web development until Rails came along). As a rule, they force you to do things a certain way, and -- more importantly -- to think in a certain way. In most cases, this "way" is mimicry of Rails, but in all cases, this "correct way of thinking" is the way the framework developers think. That's the aspect of frameworks that I take issue with, while at the same time acknowledging the productivity gains that are possible with these frameworks.

Since everybody likes productivity gains ... I decided to go out and see if there was some way to get the same increase in productivity without using a framework. I want to use tools in such a way that makes sense to me, not to the tool.

The place I started was with the model; I needed to find a good ORM tool for PHP. I hadn't found one that I like yet, so this search took me a while. I still haven't found a real ORM library that I like, but I have come across an interesting tool. The PHP Object Generator allows you to define a database table (with an interface similar to phpMyAdmin, which I don't care for, but at least it's a familiar concept), and from that it generates not only the SQL to create the table, but also a PHP class with the standard CRUD methods. It handles relationships between tables with a parent/child concept (a parent can have multiple children, but a child may only have one parent). At first, it didn't support has_and_belongs_to_many relationships, but added that in with the concept of "siblings." I think that terminology is a bit of a stretch, and the documentation on it isn't the best, but it works well enough.

For the controller, I wanted something really light. I considered just making my own, and that probably would have worked. But by chance, I discovered EpiCode, which is just about exactly what I would have wanted to create on my own. It uses a .htaccess file to direct all HTTP access to the index.php file, which contains an array called $routes, which in turn wires paths to functions (or static methods, which works even better). It doesn't touch the query string parameters, so you get to process those however you like.

For the view, I stuck with the old Smarty standby, and it works like a charm for this. You have to define which variables to open up to the template, as well as which template file to use; despite having to type a couple of extra lines of code, I prefer this to being forced into a particular directory structure and naming convention that is common among the true frameworks.

In my first foray into the POG+EpiCode+Smarty "framework," I made an IMDB clone in about an hour and a half, and I didn't have to write a single line of SQL (table creation or queries). In fact, I didn't even have to log into the database (as the POG setup process creates the tables and verifies that everything's working). I'd say that's a fairly significant increase in productivity over my previous methods, and I still got to define my workflow for myself, rather than kneeling before some anonymous framework creator.

I know a lot of people like frameworks, and a lot of people use them. Because of this, my opinion on the matter probably isn't very popular. But I think this freedom is really valuable. And I find it difficult to believe that my opinion is unique.

PHP: Reading Key/Value Pairs from a File

2008-02-23T16:40:00.001-06:00

I'm working on an application in which I have to parse a file with key-value pairs, looking for a single value. I decided to make the datafile look like this:

key0|val0
key1|val1
key2|val2
...
key2000|val2000

I don't expect to have more than 2000 values in the file, but just in case I set up a second sample file with 100000 values just to see how that would affect things. (That kind of growth in usage would be awesome.)

My initial thoughts on this are to keep it simple. Performance is important, but I don't think spending the time for a binary search file seek algorithm would be worth it; also, that would be premature optimization. So I want to check out the performance of a couple simpler options first.

My first option was to loop through each line, explode on the | character, and compare the keys. Like this:

function searchLinearly($filename, $key) {
    $lines = file($filename);  

    foreach ($lines as $line) {
        list($k, $v) = explode('|', $line);

        if ($k == $key) {
            return $v;
        }
    }

    return false;
}

That's just about as simple as you can get. But how does it perform? I searched for a few keys throughout the file, and measured how long it took to find them.

In a 2000 line datafile:

Key 10: 0.00215s
Key 150: 0.00233s
Key 900: 0.00555s
Key 1700: 0.00866s

In a 100000 line datafile:

Key 10: 0.11047s
Key 1500: 0.04429s
Key 50000: 0.11832s
Key 99000: 0.18710s

What's really strange about this is that key 1500 is consistently faster than key 10, whether I search for key 10 first or second, or just search once per run (in an attempt to separate caching issues). I can't explain that.

Another option is to use fscanf() instead of just using file() to get all the lines. The idea is to define a regex that matches the format of the lines in your file, and "scan" through the file, extracting the pieces of data along the way. That version looks like this:

function searchFscanf($filename, $key) {
    $f = fopen($filename, 'r');
    while ($line = fscanf($f, "%[^\|]|%[^\|]\n")) {
        list($k, $v) = $line;

        if ($k == $key) {
            fclose($f);
            return $v;
        }
    }

    fclose($f);
    return false;
}

It's only slightly more complicated; does it perform faster?

In a 2000 line datafile:

Key 10: 0.00052s
Key 150: 0.00101s
Key 900: 0.00572s
Key 1700: 0.01033s

In a 100000 line datafile:

Key 10: 0.00052s
Key 1500: 0.00947s
Key 50000: 0.12980s
Key 99000: 0.20901s

I think these results are pretty interesting. Using fscanf() can actually be considerably faster, and it isn't affected by the size of the file. However, it is drastically affected by the key you're searching for; if you have to scan through a lot of lines, it slows down significantly and steadily.

In contrast, the file() method is more affected by the filesize than by which key you're searching for (searching for later keys does take longer, but it's not nearly as significant).

So which option is better? As the number of records increases, it appears the fscanf() method becomes more and more advantageous (assuming that early records are hit as often as later records, considerably less time is spent searching than with the file() method). The same consideration remains true even if the number of records remains constant, it just isn't as noticeable or significant at that level.

Unsurprisingly, fscanf() beats file(). But not by as much as I'd like. In order to maximize performance, another method is going to have to be devised; either a binary search that minimizes the number of comparisons made (O(ln n) versus O(n)), or perhaps just using something like memcached to keep the datafile in memory at all times. Maybe both.

Something to think about.

Using the Velocity Template Engine

2008-02-18T20:46:00.002-06:00

Templates are really important for separating the presentation of your program from the actual code. In PHP, I prefer Smarty (mostly because it's the one I'm most familiar with). In Java, I've taken to using Velocity.

The advantages of templates are very quick to see. Imagine you're creating a web page. You could pretty easily put the <html> code right in along with your PHP or Java code. In PHP this is absurdly easy, just by using the <?php ?> tags. The problem is that this very quickly gets cumbersome when you want to change things. Changing the way the display looks requires changing the way the program works. That's not really what you want at all. So you use templates to separate the display from the logic; that way you can change the two independently.

But templates aren't just for web development. I recently found a reason to use templates in a Java program. It is intended to create content for a web-delivered program, but it doesn't have to be dynamically generated; instead, we can pre-generate all the necessary content periodically. When I first threw the content generator together, I simply used a StringBuilder to put the content together; then I had to change something, and it was horrible. I had to parse these long append() lines, then re-compile the entire program. It became especially obvious that something needed to change because my program has to support multiple languages ... and I only speak one. I'll be getting translations from other people, and I'm not that interested it pasting translations into my code (or giving commit access to the translators). In steps Velocity. I moved the content out into its own file:

my_template.vm:
------------------------------
This is my content. My name is $name.

Of course, you can have any number of these template files, and you can load any of them at runtime just by specifying the filename.

In order to do that, you have to set up your program to load the template files at runtime using a classloader.

Properties p = new Properties();
p.setProperty("resource.loader", "class");
p.setProperty("class.resource.loader.class",
       "org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader");

(That's an important step. A lot of Velocity documentation leaves that out. I figured I'd put it before the more "significant" stuff that actually "does the work." Without this, the work doesn't happen.)

Speaking of which, let's get to the meaty part.

VelocityEngine ve = new VelocityEngine();
ve.init(p);

Here we set up the VelocityEngine. Note that we pass the Properties object to the init() method. That's where the classloader setup actually happens, so don't overlook that little character.

VelocityContext vc = new VelocityContext();
vc.put("name", "Sean");

The VelocityContext is where you put all the data you want to expose to the template. It's essentially a set of name-value pairs; like most templating engines, Velocity allows you to use conditionals and loops within the templates. These are almost always necessary in non-trivial templates, but you should start from the standpoint of avoiding it if at all possible. After all, before you're used to using templates, it's natural to attempt to put your program's logic into the template. That totally defeats the point.

StringWriter writer = new StringWriter();
Template template = ve.getTemplate("<package_name>/templates/my_template.vm");
template.merge(vc, writer);
System.out.println(writer.toString());

Here you load the template, merge it (which basically matches up all the name-value pairs in the VelocityContext with all the $tags matching the names in the template), and output it. In a web application, your output would be printing it back to the browser. In my case, it was using a FileWriter to output the text to a file. I've used PHP's templating a whole lot more than Java's, but I've already found Java to be considerably more flexible in what you do with the output after it's been generated.

I hope that example helped if you wanted to start using Velocity ... or if you hadn't considered using templates. It's pretty essential.

Java Properties Files

2008-02-16T18:29:00.001-06:00

Yesterday I posted about using JSch to SCP a File in Java, and the code I'd written at the time was pretty simplistic. It only sent a file to a remote server, and it even foolishly had string constants in the code that specified the location of the known_hosts file and the information about the remote server. I closed with a call to action to move that to a config file. Well, I've now done that, using the simple but effective Properties files that comes built in with Java.

I added a properties file to the package, and it opened right up for editing. The basic concept is that you fill it out with name-value pairs. Here's the properties file:

knownHostsFilename=/home/sschulte/.ssh/known_hosts
numServers=1
server0_host=172.16.40.128
server0_username=user
server0_password=user

I designed it with the idea in mind that I could send files and commands to multiple servers at once. I'd just have to increment the numServers field and add a new set of server info fields. Pretty nice.

To load the properties file, I just add these lines:

Properties configFile = new Properties();
configFile.load(this.getClass().getClassLoader().getResourceAsStream("remoteappmain/serverconfig.properties"));

Note that "remoteappmain" is the name of my package, which is why it's in the path to the properties file. To get one of the values, you just call the getProperty() method, passing it the key. They're strings, so in the case of numServers I had to parse it into an integer using Integer.parseInt(). No big deal.

To simplify my classes and to standardize their interfaces, I created a RemoteAuth class that encapsulates the hostname, username, and password values. And there's now an abstract base class called RemoteConnector that the FileSender extends. The constructor takes a RemoteAuth object and a known_hosts filename.

Creating an abstract base class allowed me to quickly create another class with useful functionality. This one is RemoteExecutor, which allows you to execute a command on a remote server. Stuff like "ls -la" or "rm my_file" or anything else you might type into the command line.

RemoteExecutor re = new RemoteExecutor(auth, knownHostsFilename);
re.execute("rm AllPhysics.wmv");
re.execute("ls -la");
System.out.println(re.getResult());

That's what you'd do if you wanted to delete a file, then call ls on the directory, and display it to the screen (which is what the getResult() method does).

I think that pretty much covers what I wanted. The main thing I want to add now is a FileGrabber class so I can download a file from a server as well as upload one. I feel like these classes are going to be pretty useful; I guess I'll know more about what I missed here once I use them in a few projects and decide on what they're lacking.

JSch: SCP a File in Java

2008-02-15T22:18:00.001-06:00

If you want to upload a file to another computer, SCP is an excellent way to go. And if you want to do it from within a Java program, your best bet is to use the JSch library from JCraft. They've implemented the SSH protocol purely in Java, and it works splendidly. I've written a nice little class that encapsulates the action of sending a file.

Using the class is pretty simple. It goes like this:

FileSender fs = new FileSender("172.16.40.128", "user", "user");
boolean ret = fs.sendFile("/home/sschulte/Desktop/AllPhysics.wmv", "AllPhysics.wmv");

The constructor takes a host, a username, and a password. The sendFile() method takes a source filename and a destination filename. It returns true if the upload was successful, false if it failed.

I developed it in Netbeans, and in order to use JSch, the first step is to download the JAR and add it as a library in your project. (When I first tried it, I downloaded the ZIP instead, and added the source to my project; this works, but it's not necessary to compile this library along with your project unless you plan to edit it. I didn't.)

Most of the code in this class is taken from the ScpTo example included in the ZIP file. I recommend reading it. Something to pay close attention to, however, is the SSH known_hosts file. Their example file doesn't take this into account, so if you just run their code, you get an unknown host exception and it doesn't work. So be sure to include the following:

String knownHostsFilename = "/home/sschulte/.ssh/known_hosts";
jsch.setKnownHosts(knownHostsFilename);

(This is, of course, after you instantiate your JSch object.)

I like it, and it works. But it definitely needs a few things before I'm satisfied.

Read the known_hosts filename from a config file, rather than compiling it into the code.
Have a config file for the host/username/password information, possibly for multiple hosts at once.
Throw exceptions for various types of failures, rather that simply returning false.

Pretty cool stuff. The JCraft guys did a great job with this library.

Using CURL to Download a File in PHP

2008-02-11T21:39:00.001-06:00

Today at work I came across an interesting problem. I had to make a REST-style call to a backend server, which would do some processing and return a file. The filename and content type of that file is not known to me at the time I make the call, but the file must be downloaded properly directly to the client. And the file could easily be large, meaning that I can't load it completely into memory before sending it to the client (and even if I could, that would be unwise due to the double-time download).

I'd used CURL in PHP before, but was having problems getting the content type. If I typed the REST URL directly into the browser, it would download successfully, but display a useless byte stream instead of the file itself. After discovering the CURLOPT_HEADER option, I came up with the initial solution of making the call twice: the first time I just got the headers (and not the body, using CURLOPT_NOBODY), and then set the headers using header() before making a second call which would get the body. This worked ... except that the backend team informed me that it's possible that the processing could get pretty extensive, and they didn't want it to be doubled every time. Needless to say, that made my solution pretty stupid.

That's when I dug deeper and discovered CURLOPT_HEADERFUNCTION. With this, you register a callback function which will process the headers.

curl_setopt($c, CURLOPT_HEADERFUNCTION, array(&$this, 'applyHeaders'));

The third parameter here is the function that processes the header. If you're using an instance method, you pass an array containing a reference to the current object and the name of the method. In this case, my callback function looks like this:

function applyHeaders($c, $header) {
    header($header);
    return strlen($header);
}

It simply calls the header() function on the file's headers and returns the length of the header (which is necessary for the callback function to work).

This way, you can keep CURLOPT_RETURNTRANSFER false, meaning you don't have to load the file into memory and it's streamed directly to the browser, and the applyHeaders() method set the content type and filename properly, so it gets downloaded just as it's supposed to. Best of all, you only have to make one call to the backend, and the processing is only done once.

Excellent. And the documentation on this isn't the best, so I figured I should post it on here to help out. And to help me remember.

An Early Update into the Life of a Consultant

2008-02-10T12:27:00.001-06:00

I've now been on my current contract for about 2.5 months, so I figure it's a good time to come back and post an update.

The work isn't that challenging. The complicated problems are on the backend, and that team is constantly frantic and has been falling behind as the codebase becomes larger. Unfortunately, I'm not on that team, and can't help out. All I'm doing is making XML-RPC calls to the backend and displaying the data to the user. My days are currently spent waiting, either for something to be finished on the backend so I can finish it on the frontend, or for a bug to be found so I can fix it. On some days, neither happens.

But enough of the bad. I don't much care for complaining.

The people are great. The developers I've been working with and talking to are talented, and the managers I report to are very technical. From a personnel standpoint, this is just about ideal. However, I've begun to think that this might actually be a problem. Sooner or later, this contract will end and I'll be moving on, possibly never working with these guys again. I'll have to start over, left to hope that the people at the next contract are as talented and fun as they are at this one. This is the life of a consultant, I suppose.

This has led to me eating lunch by myself, eschewing the chance to get close to my coworkers and managers. This is probably unwise, but I feel it's a natural response to the situation I'm in, where I'll probably be leaving them soon. I should try to make an active effort to change my behavior and start spending more time socializing with my coworkers, starting at lunchtime.

And things are looking up, workload-wise. The current project is close enough to being complete that I'm apparently going to be placed onto another one this week. I think this is excellent, and hopefully I can slide over easily to the new project and get its wheels moving as fast as possible. That should feel pretty good.

Flex: Automatically Compile and Deploy to Server

2008-01-08T13:23:00.000-06:00

One thing that's been irking me a little bit about my Flex development process is that I have to upload the SWF to the server in order to test it. Normally you don't have to, but I'm using HTTPService to load XML from the server to populate the components. I really like that feature, but for security reasons, it doesn't allow cross-domain access.

One option is to put a crossdomain.xml file on my web server, but since I'd be testing from a local machine, I'd have to define the allowed domains as "*" ... which probably isn't the best idea. Also, I'm not that excited about hard-coding the domain name of my server into the application. (I currently have them just using absolute URL's, without the domain, ie "/server/api.php".)

Up until now, I've been uploading the file manually with scp after the compile successfully finishes. For that, I have to enter the compile command, check if it was successful, enter the upload command, type my password, and then I'm done. Figuring there must be a way to automate that, I learned that Flex works with Ant, which can do all these things (and allegedly supports FTP and SCP).

After setting up my build.xml file, I learned that Ant doesn't support FTP and SCP without some esoteric libraries ... which I didn't really feel like hunting down. I thought to myself ... "Why do I have to use Ant, when there must be a way to do this with roughly the same effort as this build.xml file, but that actually does the work?"

Enter Python.


#!/usr/bin/python

import commands
import pexpect

sourceFile = name of source file
outputFile = name of output file

username = remote username
password = remote password
serverAddress = remote server
remoteDir = path to destination folder on remote server

print "BUILDING..."
(compileStatus, compileOutput) = commands.getstatusoutput('mxmlc -file-specs ' + sourceFile + ' -output="' + outputFile + '"')

if (compileStatus > 0):
 #the build failed, display the output
 print "BUILD FAILED!"
 print compileOutput
else:
 #the build was successful, upload it
 print compileOutput
 print "UPLOADING..."
 scpCommand = 'scp ' + outputFile + ' ' + username + '@' + serverAddress + ':' + remoteDir + '/' + outputFile

 scp = pexpect.spawn(scpCommand)
 scp.expect('Password:')
 scp.sendline(password)
 try:
  scp.interact()
 except:
  print "(exception)"
 finally:
  print "Done!"

This script uses pexpect, which is a cool little program that lets you programmatically interact with the command line. This is necessary to enter your password into scp (which doesn't allow you to pass it into the command as a parameter, for obvious security reasons).

It's pretty simple. Basically, you compile using mxmlc, and if it was unsuccessful you display all the errors. If it worked ... you upload it to the server and you're done. Much easier than doing everything manually every time.

You have to set the constants in the file before executing, which isn't really a problem. I could make it so it takes parameters, but that almost defeats the point. I'm looking at this as a build.xml replacement, not as a full-bore program. So there shouldn't be a problem with setting some constants inside.

Enjoy the 3-5 seconds you'll repeatedly save! I know I have.

An Awesome Flex HTML Escaper

2008-01-05T19:32:00.000-06:00

One of the problems I frequently come across when trying to post code online is that the characters need to be escaped. For example, when entering a bunch of MXML code, the browser vainly attempts to display the code in the mistaken belief that it's actually HTML.

So one of the simple applications I like to make in any new language is an HTML escaper. Since I'm playing with Flex right now, I figured I'd make one (and hopefully it'll be useful, since I'll be using it to post the code).

It took a couple of minutes, but here's the code:


<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" backgroundColor="#f6f6f6">
  <mx:Script>
  <![CDATA[

  public function escapeHTML(str:String):String {
      str = str.replace(/&/g, "&amp;");
      str = str.replace(/</g, "&lt;");
      str = str.replace(/>/g, "&gt;");
      return str;
  }

  public function outputClickHandler(event:Event):void {
      System.setClipboard(output.text);
  }

  ]]>
  </mx:Script>

  <mx:VBox>
      <mx:Label text="Original" fontSize="14" fontWeight="bold" />
      <mx:TextArea id="input" width="800" height="200" />

      <mx:Label text="Escaped" fontSize="14" fontWeight="bold" />
      <mx:TextArea id="output" text="{escapeHTML(input.text)}" click="outputClickHandler(event)" width="800" height="200" />
  </mx:VBox>
</mx:Application>

The cool things about this little application are:

The data binding

Note that the second TextArea's text attribute is defined as text="{escapeHTML(input.text)}". What that means is that the two text fields are bound together, with the ActionScript function between them as a sort of filter. Any change you make in the first box will immediately show up in the second box.

The event handler

The second box has an event listener set inline by the click attribute. It calls the outputClickHandler() method, which simply sets the your clipboard to be the result of the escaping. That way you don't have to manually select everything when you're ready to copy the results.

While this is a really small, simple application, it demonstrates the power of Flex pretty well. The event model is nice, clean, and simple ... but the data binding is what gives you real power. It's the best kind of magic: the kind that you very explicitly tell it what you want, and raise you eyebrows in pleasant surprise when it works better than you could have imagined.

This is by far the best HTML escaper program I've written, and it's all thanks to Flex. Awesome.

(Feel free to use the HTML Escaper Application to your heart's content.)

Flex Layouts Make Things Much Simpler

2008-01-05T17:45:00.000-06:00

A while ago, when I discovered Flex, I wrote about a simple application that calculates a QB Rating for both the NFL and the NCAA. It was meant to demonstrate how you get started using Flex.

From my perspective, the main limitation of that program was that it used absolute layout, meaning each element needed to be manually placed using the x and y attributes. Frankly, that's pretty lame. Even as I wrote it, I knew I'd have to learn about Flex's layouts.

Since that time, I have. And thus far, they seem to be both powerful and simple. They beat the pants off of both HTML and Swing, which were the UI technologies I had compared Flex to.

The layouts are based on the concept of containers. A container is an element in which other elements can be placed (whether they're other containers or actual UI components). The containers have different properties that control how things are laid out inside them.

For example, if you wanted to put two buttons right next to each other, you'd use an HBox, like this:


<mx:HBox>
  <mx:Button label="One" />
  <mx:Button label="Two" />
</mx:hbox>

That's much nicer than having to define the x/y coordinates and the width of both buttons.

So what did I decide to do to update my QB Rating application? I used a Form Container, with a set of FormItem containers inside it, to lay out the TextInput fields where you enter the QB's stats. And below that, I have a VBox for the results. It looks like this:


   <mx:Form left="10" top="10">
      <mx:FormHeading label="Calculate QB Rating" fontSize="20" fontWeight="bold" />

      <mx:FormItem label="Completions">
          <mx:TextInput id="completions" />
      </mx:FormItem>

      <mx:FormItem label="Attempts">
          <mx:TextInput id="attempts" />
      </mx:FormItem>

      <mx:FormItem label="Yards">
          <mx:TextInput id="yards" />
      </mx:FormItem>

      <mx:FormItem label="TDs">
          <mx:TextInput id="touchdowns" />
      </mx:FormItem>

      <mx:FormItem label="INTs">
          <mx:TextInput id="interceptions" />
      </mx:FormItem>

      <mx:FormItem>
          <mx:Button id="calcButton" label="Calculate QB Rating" click="{calcRatings();}" />
      </mx:FormItem>

      <mx:VBox>
          <mx:Label id="nflRating" text="" fontsize="16" />
          <mx:Label id="ncaaRating" text="" fontsize="16" />
      </mx:VBox>
   </mx:Form>

And my oh my, is that nicer than the previous version. If you looked at the old version of the QB Rating application, it had a Label and a TextInput for each field, and the left and top attributes were set for each one. The new version actually takes up a few more lines of code, but it requires you to have less extensive knowledge of the size of the elements. For example, if you decide to change the font size of one of the elements, you won't have to change the "top" attribute on everything below it. If you change the title of one of the labels, you won't have to change the "left" attribute on all the TextInputs so they all line up and don't overlap their labels. I think that's worth a couple more lines of code.

So ... why did I learn about the absolute positioning first? Well, that's how the examples on Adobe's sites all work. At first glance that doesn't make sense, but I've recently learned that absolute positioning is the default in Flex Builder 2 (which I don't have). So it looks like the guys writing these examples are building their applications in Flex Builder 2 and then going through the generated code in their articles. I don't have a problem with that in general, but it seems to me that if you're ostensibly teaching someone about writing the code, it should be code that was actually written. That's a pretty minor quibble though, especially since Adobe's online documentation about layouts and containers is pretty excellent.

If you want to check out the new app or source code, by all means do so.

Flex: MXML Highlighting in VIM

2008-01-05T17:23:00.000-06:00

For the first couple of weeks I've been using Flex, I've had to do it without syntax highlighting. My editor of choice is vim, which doesn't support MXML and ActionScript out of the box. However, syntax highlighting is a great invention, and now that I'm in the middle of a sizable Flex project, I figured I needed to get my hands on some syntax highlighting. (It'll be a whole lot cheaper than buying Flex Builder 2!)

The first thing you need to do is download the following two files:
actionscript.vim
mxml.vim

(Even if you only plan on editing MXML files, you need both of these. The MXML syntax depends on the ActionScript syntax, because MXML files can include ActionScript code.)

(Thanks to Abdul Qabiz for these files.)

You need to put these files into vim's syntax folder. On Ubuntu 7.10, it's /usr/share/vim71/syntax. (Make sure you install vim-full, using sudo aptitude install vim-full ... for some reason my installation of Ubuntu came with vim-tiny, which doesn't support syntax highlighting.)

Once that's done, you need to make sure the following lines are in your ~/.vimrc file:


au BufNewFile,BufRead *.mxml set filetype=mxml
au BufNewFile,BufRead *.as set filetype=actionscript
syntax on

Now ... the next time you open up an mxml file in vim, it'll have syntax highlighting for you. Excellent. My development environment just got a little bit better.

Modal Dialog Box in Javascript

2008-01-03T13:54:00.000-06:00

Occasionally you want to be able to pop up a modal dialog box over your web page, so you can force the user to enter something before they continue. You could just use confirm(), but that's pretty lame and doesn't allow you to exercise much control over the user's input options. Yes/No? Okay/Cancel? Lame.

Well, the way to do it, of course, is with a transparent div! That'd be the background, and then you have another div that you put right in the middle of the screen, mimicking a popup window. You'll be able to see through the background div (kind of), but you can't click through it ... which makes it perfect.

First, you need to throw in the HTML:


<div id="modalBackgroundDiv" class="modalPopupTransparent" style="display: none;"> </div>

<div id="modalWindow" class="modalPopupWindow" style="display: none; width: 560px;">
    It's awesome that you can put whatever you want in here.
</div>

Then you need some CSS to make it so they actually work as planned:


div.modalPopupTransparent {
    filter: alpha(opacity=75);
    -moz-opacity: .75;
    background: #CCCCCC;
    position: absolute;
    top: 0px;
    left: 0px;
}

div.modalPopupWindow {
    position: absolute;
    background-color: #FFFFFF;
    border: 2px solid black;
    padding: 8px;
    text-align: left;
}

The most important thing here is that the background div's style has its opacity set to 75%. It has to be done twice because IE and Firefox use different commands for this. (Typical.) The second most important thing to notice is the absolute positioning. That's what allows them to display on top of everything else. It's not that useful if your transparent div shows up below the page, is it?

It's also not that useful if the modal dialog has no functionality and can't pop up. Which is why we now have to add in the Javascript:


function openModalWindow() {
        var div = $('modalWindow');
        var bgDiv = $('modalBackgroundDiv');

        var docDim = Element.getDimensions(document.body);

        //get the size of the window and calculate where the box should be placed
        var wDim = getBrowserWindowSize();
        var dDim = Element.getDimensions(div);

        div.style.top = ((wDim.height - dDim.height*2) / 2) + 'px';
        div.style.left = ((wDim.width - dDim.width) / 2) + 'px';

        if (docDim.height > wDim.height) {
            wDim.height = docDim.height;
        }

        bgDiv.style.width = wDim.width + 'px';
        bgDiv.style.height = wDim.height + 'px';

        Element.show(div);
        Element.show(bgDiv);
}

function closeModalWindow() {
    Element.hide('modalWindow');
    Element.hide('modalBackgroundDiv');
}

function getBrowserWindowSize() {
    var winW = 630, winH = 460;

    if (parseInt(navigator.appVersion)>3) {
        if (navigator.appName=="Netscape") {
            winW = window.innerWidth;
            winH = window.innerHeight;
        }
        if (navigator.appName.indexOf("Microsoft")!=-1) {
            winW = document.body.offsetWidth;
            winH = document.body.offsetHeight;
        }
    }

    var rval = {
        width: winW,
        height: winH
    };

    return rval;
}

To open the modal window, the first thing we need to do is grab a reference to the window, the background, and the document body. We then determine the size of the visible browser window (using a custom function, shown above) and the size of the modal dialog (using Prototype's handy method).

Since we want the transparent background to cover the entire page (not just what's visible), we set the height of the background to be the height of the document body instead of the browser window -- if necessary. That way the user can't scroll down and click something lower on the page if there's a modal dialog up.

Then we center the popup, make the background as big as it should be, and display them. Simple enough. Similarly, in order to close the window, we just hide both elements.

And there you have it. A nice, simple modal dialog, with a transparent gray background that dims out everything else on the page so the user knows what to focus on. I wouldn't use it too often, but in some cases it can be exactly what you want.

Enjoy!

PHP usort(): Sort an array of objects

2008-01-02T14:08:00.000-06:00

PHP is nice enough to include a nice, fast sort() function that works on simple datatypes like numbers and strings. Saves you the time of having to write your own sorting functions. However, how often is your important data in an array of integers or strings? Considerably more often, you have an array of more complex data structures that you'd like sorted. And the sort() function just can't do it.

One option you have is to write your own sorting function, as I've already mentioned. In fact, I've done this numerous times. But why should I have to type out a quicksort algorithm -- slightly modified to compare objects or associative arrays -- rather than having the libraries do it and move on to more important things. This isn't CS class any more. I shouldn't have to "prove" that I understand the quicksort algorithm ... I need to get things done!

Well, as it turns out, PHP does indeed offer the capability to sort an array of objects. And it's not so bad. Check out usort. The basic concept is that you pass it the array of objects and a function (or static method) that'll act as a comparison function between two objects. This function is something you write yourself, since you're the one who knows about your classes. I usually put it right in the class itself, acting as a static method. Here's a quick example:


class Item {
 var $a;
 var $b;

 function Item($pA, $pB) {
    $this->a = $pA;
    $this->b = $pB;
 }

 function _cmpAscA($m, $n) {
    if ($m->a == $n->a) {
        return 0;
    }

    return ($m->a < $n->a) ? -1 : 1;
 }

 function _cmpDescA($m, $n) {
    if ($m->a == $n->a) {
        return 0;
    }

    return ($m->a > $n->a) ? -1 : 1;
 }
}

//let's make an array
$arr = array();
$arr[] = new Item(1, 2);
$arr[] = new Item(4, 3);
$arr[] = new Item(2, 3);

//now we can sort it!
$sorted = usort($arr, array('Item', '_cmpAscA'));

Note that the second parameter to the usort() function is an array. This is because I'm calling a static method. The first item('Item') denotes the class, and the second item('_cmpAscA') denotes the method name. If I had simply made a comparison function that wasn't in a class, the second parameter to usort() would have been a string, not an array.

I typically make two comparison functions per sortable field in my objects; I want to be able to sort ascending and descending, without having to muck with strrev(). My way is a bit faster ... but may force you to use a bit more code. It seems to me that that's a matter of personal preference. And I have this preference because I feel that it allows you to more accurately describe what you want. If you want to sort in descending order, you say "sort this in descending order," rather than "sort this and then reverse it."

Anyhow, usort() is a powerful tool to add to the toolbox. I know I'll be using it in the future for all my object-sorting needs. Hope you find it as useful.

I Finally Discovered Flex

2007-12-16T22:35:00.001-06:00

Just a little while ago I was considering the possibility of pre-built PHP Display Components, which would allow me to build applications faster and re-use a lot of the boilerplate code that I currently spend too much time on. And once those components are built, they could be improved individually, which would make all my applications better without having to do anything.

It seemed like a good idea, and while considering how I might go about doing it, I discovered Adobe Flex. I'd heard of it before, as "Flash for developers," but had never been that curious. Having investigated Flash before, I was not interested in having to purchase a visual development environment and then messing around with timing, tweening, layers, and all the other crap Flash puts you through.

But Flex isn't like that at all. In fact, it's essentially just a set of visual components you can use in building a visual interface, along with a slew of classes for connecting to web services and easily binding XML data to the visual components. Additionally, ActionScript appears to be what JavaScript wishes it were ... which is nice.

Best of all? They provide a free SDK with a command line compiler. So as long as you're willing to actually write the code (rather than drag-drop-connect, which seems to be the direction the IDE companies want programming to go), you don't have to pay a dime and can check Flex out. So I did.

I decided to write a quick program to calculate quarterback ratings. Since the ratings formulae are different in the NFL and in college, I figured I'd have it calculate both.

Here are links to the app and the source code.

The first thing I did here was set up the visual interface, which is at the bottom. I used Flex's Label, TextInput, and Button components to build a simple form. I used absolute positioning because it's supposed to be better for performance (and because all the examples on Adobe's site use it, which means I haven't learned much about their automatic layouts yet -- I certainly would like to).

These components should be pretty familiar to anyone who's used HTML and Swing ... the tags seem to be a mix between the two, which is pretty interesting.

After writing the interface (and compiling and testing it to make sure it looked the way I wanted it to), it's time to add the functionality itself. The calculations will be done by ActionScript, in the <mx:Script> tag. It's a lot like JavaScript, except that you can define strong types and public/protected/private status, both of which are nice additions. The functions in here are simply designed to pull the data from the input fields, convert to Numbers, apply the formula, and set the label.

The one slightly non-basic feature I used in this program is NumberFormatter. Flex claims you can instantiate any component either with tags or in the script code, but the only way I got this working was to make an <mx:NumberFormatter> tag and reference it from the script. It's actually pretty interesting, and once I get used to having to do that, I may grow to like it.

I then went ahead and compiled it, and it worked quite nicely. Flex looks like a promising way to build web applications -- it's even possible that I could grow to prefer it to PHP + AJAX for my own applications.

PHP Display Components

2007-12-08T17:00:00.001-06:00

Just about every GUI language/framework has standard classes for displaying common datatypes. There are datagrids, sortable datagrids, paginated lists, searchable lists, input forms, display forms, etc. You instantiate them, set up some information about them, and connect them to your data source. Then you call some sort of display() method on the object, and it displays everything on the screen.

The reason I mention this is because PHP has nothing like this (at least not that I know about). In PHP, the View part of MVC is merely an HTML-based layout file with some markup in it to display your data. But isn't is possibly to create a set of components that generate the HTML on their own?

Obviously, we could all just continue to code the HTML in our Smarty templates, recreating the same functionality over and over again in "low level" HTML. On the other hand, we could create components at a higher level of abstraction, and when we figure out a better/cooler way to do that searchable+paginated list (maybe adding some Javascript), we do it once -- in the component -- and it works everywhere we're using it. That's much better than having to change HTML code everywhere we're displaying a list of data (um, a lot).

So what are the goals of these display components?

Define a set of reusable components that handle the display of data on a web page and generate the HTML
Concentrate the HTML/JS of your application in component code, out of sight
Ability to easily create new components, or edit existing ones as needs arise
Skinnable -- either set CSS values in the code, or assign a CSS file to the object (also allow for a default CSS file to be set sitewide)
Allow you to stay at a higher level of abstraction, so you can get more done with less code

These seem like reasonable goals. They're not trying to shoot the moon ... and they don't seem to demand that you develop a certain type of application if you use these components. However, the problem of defining what components are necessary then quickly arises.

Lists of data

Paginated
Searchable
Sortable

Header
Footer
Sidebar
Input form
Display form
Calendar
Date selector

Single date
Date range

Time selector

Single time
Time range

Popup dialogs

Modal
Non-modal

AJAX

Periodical updating
Response to user action
Connect a URL-defined data source to a component

That's a start, anyhow. More components are probably necessary.

There should probably be a general Page class, or something, that you would attach the other components to. The point of that would be to set pagewide CSS and only being required to call the display() method once per page.

In the interest of separating this from the concept of "scaffolds," I think the way it should work is that each component has a standard/default HTML template that it draws from, and the file it uses is an object variable. If you want, you should be able to create a brand new HTML template and make your component use it instead -- it would be nice to have that kind of flexibility.

Right now ... I don't know if this is feasible. I don't even know if it's a good idea. I'm sure people have tried to do this before, and I've never heard of the results. At the moment, my plan is to think about it for a while longer, and if I continue to think it's a good idea, it may be worth spending a bunch of time on to see what I can come up with. Could be interesting. (Let me know your thoughts, if you have any.)

The First Week -- Mostly Waiting

2007-12-08T16:32:00.001-06:00

I just finished my first week on the job. I don't know how typical an experience it was, but I'll run down a quick overview of how it went for me.

It was "Team Week," which means that everyone who works in offices around the country all get together in Chicago for a week of project planning (during the day) and partying (at night). As a general rule I would have jumped on the partying train after work, but I'm adapting to a new lifestyle -- one that involves waking up early and commuting to work -- and have been pretty tired. I don't want to throw a hangover on top of that process. It's already hard enough to prevent myself from drinking coffee.

I really like the team I'm working with, so it's a bit of a shame that they've now all gone back to their respective homes and we'll have to work remotely. But they've been successfully doing it for a while, so I'll just have to slide right in and contribute.

And on that note, I was completely unable to "slide right in." I expected it to take a day, maybe a day and a half, before everything was set up and I would be working. Alas, no. There were numerous problems with waiting for IT, and one entire day was lost because the one guy who knew what we needed to take a step forward missed a day (spent it with his son in the ER -- apparently everything's okay now). I spent most of the week reading source code, but with no access to the repository or a development server, nor any direction on what to do. Finally, on Thursday, I was assigned projects, and on Friday, I was told what they actually want. But I'm still waiting on CVS access.

In my first week on the job, the vast majority of my time was spent on non-work tasks. I don't think that's very unusual, but it feels weird to have to "wait" like that after you've started the job. From the reactions of the rest of the team, they seemed to expect that it would take 2-3 days. By the end of the week they were getting as frustrated as I was that I still couldn't work.

Hopefully everything gets set up quickly, and next week goes a bit better.

Becoming a Consultant

2007-11-16T12:47:00.000-06:00

Last week I decided to look for work. From what I can tell, the best way to do that is to throw your resume up on dice.com and wait for the calls to roll in (well, the best way is probably to know someone who'll get you a job more easily, but I don't). I didn't have to wait long until I had a couple of promising contacts with what I assumed were both recruiting firms. I set up face to face interviews with both of them.

As it turned out, the first firm was in fact a recruiting firm who had a permanent position in mind for me. Everything seemed to go well, and when I left the meeting I felt good (which was significantly different from how I felt when I went in ... I was quite nervous). They were going to send my information along to their client, and I'd have an interview on Monday. While waiting for my next interview a couple hours later, I decided to have lunch at Potbelly Sandwich Works, who make delicious sandwiches for a low low price. (I guess it's not as low as Jimmy John's ... but Potbelly actually cooks their sandwiches.) I got my usual, the "Italian on Wheat." I love restaurants where you can order something that sounds almost non-sensical, and they know exactly what you mean. While eating my sandwich, I took the opportunity to get nervous again. After all, the second interview was in the Sears Tower, which is designed to be intimidating ... and is successful. It has as much security as an airport, and is much, much taller.

I stood outside the massive tower, unable to see the top, for about an hour and a half. This did not help the state of my nerves. Finally, I decided to call my contact at the recruiting firm to get this show on the road. He came downstairs and we held our meeting in the Starbucks in the lobby. I declined the offer for coffee, since I don't drink the stuff, but he insisted "Come on, make me feel good," and I was obligated to order a small bottle of orange juice. It was delicious. You've got to love expense accounts.

Early into our meeting, I learned that this company was not really a recruiting firm, but was instead a consulting firm. It was SPR, and he spent a considerable amount of time convincing me about the greatness of the company. And you know those cases where someone tries really hard to make you think they're great, but the whole time you're sitting there thinking "That doesn't sound so great," and waiting for them to let you leave? Well ... this wasn't one of those times. He didn't have to try very hard at all, because he had a lot of "we're great and it's pretty self-evident" material. He went on to describe the client I'd be working for, and it sounded like a great situation and a great person to work for.

I left that meeting excited about working with SPR, and again nervous since I had to pass a technical screening followed by a face to face interview with the client if the technical screening went well. I had to wait a few days (it was a weekend) for the phone screening, and as it turned out I didn't need to be so nervous (as had been the case every time). I had the interview the next day, and so the next morning I hopped on the bus and rode downtown. I didn't know exactly where to get off, so I sat there the whole time with my iPhone out, matching the streets on Google Maps with the stops on the way. That worked fine, because I managed to get off on the correct street.

This interview was mostly over lunch, and was with the guy I'd be working for as well as another developer I'd be working with. It turned out that they were both vegetarians, and I ordered a meat ravioli (in other words, something that you know exactly what it is when you hear the words, which I don't like as much). At first I thought "What if they think I'm one of those heartless meat-eaters?" Then I immediately thought "If the fact that I'm not a vegetarian is any kind of an issue, would I really want this job?" Neither of them had a second thought about it. That was excellent. So was the rest of the lunch interview. After lunch, the boss went back to work and the developer gave me a tour of the office, and after a little while I went home feeling pretty good about my chances.

When I talked to my dad, he tried to manage my expectations and keep my feet on the ground (though they already were). "Don't get too excited, you don't have the job yet." Yeah. I know. Don't remind me. After that phone call, I was on the edge of my seat, staring anxiously at the phone. It took about two hours, but it rang and the voice on the other end said "I have good news and good news." Well, needless to say, that was good news! He told me the deal, and when I hung up the phone I was a consultant.

The whole way through the process I was nervous, and I shouldn't have been. The key is just to be yourself, and everything will go well. Now I have to wait two weeks before I can start (professional courtesy to the current employer), and then I get to get down to the part I'm good at, which is actually doing the work. It's all very exciting. And since I'll be paid by the hour, the number of 12 hour days will probably decrease just a bit (down from "every day except Sundays").

After I start, I'll know how I like it. I'll post my thoughts here, naturally. Until then, have fun.

Apache Rewrite Rules

2007-11-15T18:49:00.000-06:00

Everybody likes websites that have good, clean URLs. On a minor little project I'm working on on the site, I wanted to give that a try. So instead of seeing a URL like /index.php?username=user1&view=feeds, you'd see /user1/feeds. It's kind of cool ... and remarkably easy!

One assumption this particular projects makes is that index.php is the only page that'll be viewed. It checks the query string parameters and changes its behavior depending on that. I don't personally like this method, but it makes it easier to handle the rewriting rules.

Here's the .htaccess file you're going to want to put into your webroot directory:

RewriteEngine on
RewriteRule ^([a-zA-Z0-9]+)/?$ index.php?username=$1
RewriteRule ^([a-zA-Z0-9]+)/([a-zA-Z]+)/?$ index.php?username=$1&view=$2
RewriteRule ^([a-zA-Z0-9]+)/([a-zA-Z]+)/([a-zA-Z]+)/?$ index.php?username=$1&view=$2&action=$3
RewriteRule ^([a-zA-Z0-9]+)/([a-zA-Z]+)/([a-zA-Z]+)/([0-9]+)/?$ index.php?username=$1&view=$2&action=$3&id=$4

So that seems simple enough. The first option on each line is the regular expression that you want to match. Each match within a set of parentheses corresponds to the $1, $2, etc, flags in the second option. The ^ at the beginning and the $ at the end are regex symbols signifying the beginning and end of the string, respectively. the /? right before the $ on each line makes it so the rule will match the URL whether there's a trailing slash or not. (For example, /user1 and /user1/ will work in exactly the same way.)

Thanks ... and enjoy!

PHP Profiler

2007-10-14T02:14:00.000-05:00

At ZedZone, we found we're having a few problems with performance on a few of our pages. So I needed to find a way to determine what parts of our pages needed optimization so we could speed everything up without wasting our time optimizing functions and queries that don't need it.

There are a few profiling packages available for PHP, but they require installing modules or PEAR packages, and it seemed to me that that's not necessary to profile your performance. So instead, I decided to write my own class.

What I wanted from my profiling class was the ability to measure the entire execution time of the script, as well as the execution time between any given points in the script. I wanted there to be any arbitrary number of measurable points in the script; it should be up to me to decide what parts of the script I want to measure, as well as how many such parts.


class Profiler {
    var $startTime;

   var $endTime;

   var $marks;

   var $report;

   function profiler() {

       //initialize the object

       $this->marks = array();

function getMicroTime() {

       $t = microtime();

       $time = explode(' ', $t);

       return doubleval($time[0]) + $time[1];

function start() {

       //start the profiling

       $this->startTime = $this->getMicroTime();

       $this->mark('Start');

function mark($title) {

       //mark the current spot

       $this->marks[] = array(

'title' => $title,

           'time' => $this->getMicroTime()

);

function finish() {

       //shut it down

       $this->endTime = $this->getMicroTime();

       $this->mark('Finish');

function generateReport() {

$report = array();

$lastTime = $this->startTime;

$totalTime = $this->endTime - $this->startTime;

foreach($this->marks as $mark) {

$mark['diff'] = $mark['time'] - $lastTime;

$mark['percentage'] = ($mark['diff'] / $totalTime) * 100;

$lastTime = $mark['time'];

$report[] = $mark;

$report = sortArray($report, 'percentage');

$report = array_reverse($report);

$this->report = $report;

}
}

So the first thing to notice about the script is that when you initialize it, it only sets up the array for the profiling marks. In general use of the script, you call the start() method right after initializing the Profiler object, and the finish() method after all the code on the page.

Then after every function call, query, or section of code that you want to measure, call the mark() method, passing a string that identifies what's being measured between this mark and the previous mark. At the end of the script, call generateReport(), and you can either var_dump/print_r the report instance variable, or you can create another method called printReport() that builds a table to display the results of the profiling.

Note that the sortArray() function is not included in this script. In my code, it points to a quicksort that sorts an associative array by the values in the given key (as opposed to the sort functions that simply sort an array of strings or numbers, rather than associative arrays). I might post that later, but at this point the Profiler class only does this to sort the report by the sections of code that affect performance the most.

This class has worked pretty well for me so far, and hopefully it can be helpful to you too. If I find I need to add to it, I'll post further. Happy coding!

Getting iTunes to Recognize Your TV Shows

2007-01-15T18:52:00.000-06:00

I have quite a few movies on my computer, as I've mentioned in previous articles. I've encoded most of them in H.264, which I find gives the best quality. Since I never felt like looking up my Quicktime 7 key (which I legally purchased when it came out, hoping it would allow me to encode DVDs more easily), I always played all my videos using VLC (because Quicktime doesn't let you do full screen). The process was annoying: I had to find the movie in the Finder, then drag it down to the VLC icon in the dock, then go to full screen. And VLC doesn't have an onscreen controller during full screen playback (at least not on Mac, I don't know about the status of other operating systems).

Needless to say, I started to find this pretty cumbersome. When iTunes first got the ability to play video, I jumped at the chance to have a program manage my video playing needs. I was grossly disappointed by how slow it was, and after a few hours of trying to set it up, went back to my Finder-VLC approach. Well, it's been a few iTunes updates since then, so I decided to give it another go.

The first thing you need to make sure you do is to set iTunes so it doesn't automatically move your files into the iTunes directory. This won't work at all when your main drive is full and most of your movies are on an external hard drive. Plus, it takes a really long time to transfer those files.

Adding the movies is very simple. Switch to the Movies tab on iTunes, then select all your movies, and drag them on over to iTunes. It'll chug for a little while, then give you a list of all your movies. You can set it to automatically play them full screen, which I like, and they give you a nice onscreen controller during full screen playback, which I like more.

But in addition to my movies, I also have quite a few television shows on this thing. And while iTunes does support TV shows, they seem to prefer that you buy them from the iTS rather than play your own. But I don't feel like repurchasing all these shows from Apple, so I need to get iTunes to recognize that they are, in fact, TV shows instead of Movies.

For an individual episode, this isn't too bad. You go to Get Info, then Video (a tab on top). Here you can set it to TV Show, and also enter the name of the show and the season number. Cool.

But that's really cumbersome if you have a lot of TV shows. You know, like a season. Or ten. Or, in my case, many many more. There MUST be a better way!

Doug's Applescripts give us Set Video Kind of Selected, which allows you to do this very thing to an arbitrary number of files. Awesome. Just download it, toss the Applescript file into /Users//Library/iTunes/Scripts, restart iTunes, and you're ready to get started.

Select your files:

When you drag them to iTunes, remember to select them all again and choose the script from the Applescript menu:

This will bring up a little dialog box where you can change the Video Kind to TV Show, the Show Name to, for example, "The Simpsons" and the Season Number to, say, 6.

It chugs for a few seconds, and then it says it's done. Popping over to the TV Shows tab in iTunes shows us that it did in fact work nicely, turning 5 or so minutes of tedious data entry into just a handful of seconds of just telling your computer what you want.

And the TV Show interface is kind of slick. Maybe it's possible for it to be a bit nicer or more usable, but come on ... it's easily a dozen times better than Finder+VLC. And iTunes certainly isn't as slow as it used to be.

And now I'm off to watch a bit of television. In iTunes. Tally ho!

Is the NFL Fixed?

2007-01-14T16:20:00.000-06:00

I feel like I'm the only person in America who hates the New England Patriots. That's like hating apple pie and Neil Armstrong! How could anyone hate the Patriots? They're an American flag with a football, and they're so good as to be unbeatable. But does anyone remember a time before their dynasty started, and exactly when it happened?

In the year 2000, the Patriots were 5-11. (2000 Standings.) They weren't a good team. At the beginning of the 2001 season, nobody expected them to do anything, and for good reason. They started off 0-2, as expected. Then the unthinkable happened. The World Trade Center towers got knocked down, and everyone needed to pull together and be patriotic. And ... the Patriots started winning. A lot. Tom Brady, a nobody, a quarterback from the Big Ten (which is not very well known for producing great quarterbacks these days), who didn't play at all in 2000 (not technically true; he played in 2 games, and was 1-3 for 6 yards and 0 touchdowns), suddenly became the best quarterback in the league. People are now trying to go so far as to say he's the best quarterback of all time. (This is complete crap, by the way. Johnny Unitas, Joe Montana, Dan Marino, Brett Favre ... at least you can say Brady's name in the same sentence, but he's not as good as any of them.)

In 2001, the Patriots won the Super Bowl, after an amazing turnaround from the previous season. They started playing well after a big tragedy. And the "dynasty" has been going strong ever since. After the 2001 season, I had begun to suspect that something fishy may be going on in the NFL. How did the Patriots get so good all of a sudden? Why did a team called the "Patriots" suddenly become the best team -- by a long shot -- right when everyone in the country was supposed to start being really patriotic? It seemed like too much of a coincidence, but I had to sit and wait until something happened again before I could point any fingers.

Then, a year and a half ago, Hurricane Katrina devastated New Orleans. Despite all the pain, death, and damage, what was the main focus by the people charged with rebuilding the city? The Superdome. They'd been planning to demolish the Superdome anyway, since it's so old and crappy. The Saints wanted to build a new stadium, and since they might not have gotten one in New Orleans, they were considering leaving for a city with a better fanbase. It didn't matter where. New Orleans fans had always been bad to their team.

Why did everyone care so much about rebuilding the Superdome and keeping the Saints in New Orleans? Surely the money could have been better spent elsewhere in the city ... unless the Saints had a good season for the first time in their history and actually got fans into the stadium, made some money, and united the population of New Orleans.

Last season, the Superdome wasn't ready. The Saints didn't play any home games all season. They were miserable, going 3-13. This team had no future, no quarterback, no coach, no talent. Then they drafted Reggie Bush -- despite all the hype, there was no way he could be ready for the NFL already. And they picked up Drew Brees from the San Diego Chargers. It was the first time in NFL history that a team acquired a quarterback who had a passer rating of 90 or better for the previous two seasons. I was surprised by this until I realized that if you have a quarterback doing that well, you're not going to give him up. Well, the Chargers did (and it wasn't a bad move by any means, thanks to Phil Rivers), and the Saints really lucked out.

This year they made the playoffs, got a first round bye, and even won a game, sending them to the NFC Championship game against the Chicago Bears. It was the second playoff game the Saints had ever won. It will be their first trip to the NFC Championship. And it is the most remarkable one-season-to-the-next turnaround in NFL history. No other team had a worse record in the year preceding a trip to the Championship game.

We'll see how the Saints do in the rest of the playoffs. Will they manage to defeat the Chicago Bears, at Soldier Field? It's possible; the Bears haven't looked good in the second half of the season, despite their early dominance. And depending on the outcome of tonight's game between the Patriots and the Chargers, as well as next week's AFC matchup between the winner of that game and the Colts, it's just possible that the Saints will be matched up in the Super Bowl against -- get this -- the New England Patriots. The two teams who actually benefited from tragedy will be facing each other. Will the torch be passed from New England to New Orleans? From the Patriots to the Saints? From Belicheck to Payton? Only time will tell.

Another possibility is that the Chargers will beat the Patriots, and then the Colts (which is likely, considering LT's awesomeness and Indy's lack of a defense), and go on to face the Saints in the Superbowl. And we'll get to see if San Diego made a mistake by getting rid of Brees, or if they made the right move in going with Rivers. (By the way, in case anyone was paying attention, Drew Brees is another Big Ten quarterback.)

Frankly, I'm rooting for the Chargers over the Patriots. No surprise, I despise the Patriots. But that's a passing thing. Five years from now, my hatred will have subsided. But for the sake of the sport, I hope that on Superbowl Sunday we don't see a scoreboard with the Saints and Patriots both on it. That would just be too much. For all us football fans out there, who have spent so many years of our lives following our teams and our favorite players ... is it really so meaningless week in and week out that the only thing that you need to do to have a Superbowl caliber team is to suffer some horrible tragedy?

TVMini HD

2007-01-13T22:01:00.000-06:00

A while back I bought the Miglia TVMini HD, which allows you to view HDTV streams on your computer. It's essentially an HDTV tuner. There are many other such products, many of which have more impressive features than the TVMini; but this one gets to brag about its size (for what that's worth).

I have a cheap 37" HDTV (doesn't have an ATSC tuner, supports 1080i/720p only), but I don't have cable TV. I use the television as a computer monitor for my G5 tower. It works pretty well in that function -- all I use it for is watching movies and television shows that I have saved to the computer. But now it's time for playoff football, so I need to be able to watch television. So I had to get the TVMini working again.

My G5 doesn't have wireless (I bought an AirPort card, but there must be something wrong with either the card or the computer, as it won't work ... I bought them both refurbished, so it doesn't bother me much), so I hadn't had it connected to the internet. But apparently the EyeTV software that comes with the TVMini requires an internet connection. I bought a 25' ethernet cable and ran it from my Mac Mini to the G5 and turned on internet connection sharing. Instantly, the G5 was online, and the EyeTV software came alive.

It scanned the airwaves for what channels were available, then downloaded the program information and displayed it. The interface is pretty, but not quite *that* intuitive. You have to right click the channel name on the left of the screen to get the options you want, and there are 2 options that always stand out in my mind. "Go To Now" and "Tune To" ... it turns out that "Go To Now" does not mean "go to this channel now," but rather "change the program display such that what's currently on television is displayed." I don't know what would be a better name, but I found that confusing. "Tune To" did what I wanted: go to the channel and view it. It seems to me that if you double click the channel name, it should tune to it.

I must say, ATSC over-the-air broadcasts look really, really good. Of course, not all stations are broadcasting in HD, so I'm getting some SD content too (broadcast over HD frequencies), but those programs that actually are HD are stunning. I'm watching the Eagles-Saints game right now, and the picture quality is fantastic.

But, there's a problem with the move to digital television, and that seems to be the very nature of "digital" itself. Unlike your father's TV sets, if the signal isn't quite good enough, you never get static and the picture quality is never compromised. It just stops and waits for the signal strength to get back up to a sufficient level again.

According to a cursory study of the signal strength graph, the TVMini cuts out if the signal strength drops below 50%. Unfortunately, the antenna they give you really bites, and I have yet to see a signal higher than 66%. Worse, every 5-10 seconds it drops down to 30-40% for a few seconds, meaning that the television viewing experience is pretty jarring.

The EyeTV software includes some really cool features, like scheduling/recording shows, pause/resume live television, encode for iPod, etc. (I tried encoding down to H.264 several months ago, and it's _slooow_ ... much slower than the x264 codec used by Handbrake.) But I can't use any of these features because the feed I'm getting through this antenna is too miserable.

I don't want to sign up for cable (because I hate Comcast), so I may have to buy a higher quality ATSC antenna. And I'll have to get a good one that does both VHF and UHF, because I'm in Chicago and CBS, for some reason, is the only UHF station and is also broadcasting at extremely low power. There are technical reasons for this, but I don't understand the political reasons for them being put into place.

It's too bad, too. The TVMini HD is a pretty cool little product, hampered by its generic remote control which relies on the functionality of mostly unlabelled buttons and a bargain basement antenna that destroys the quality of your signal. I can't recommend this product by itself unless you have cable (but you can't use it with a cable box, you need to plug it directly into the cable and hope you're getting an unencrypted digital Clear QAM signal) or are planning to buy a high quality ATSC antenna along with it.

But I did get to watch football on my HDTV today, even if the CBS nonsense prevented me from watching the early game (it was AFC so it was on CBS) and the bad antenna made the late game (NFC, so it's on Fox) nearly unwatchable. It's still football, and it's still the playoffs. And it was a really good game, to top it all off!

Until next time, take it easy.

Installing TurboGears on Ubuntu

2006-06-06T16:17:00.000-05:00

I recently came across a new web framework called TurboGears that I think seems pretty interesting.

The basic concept is to integrate several different elements of a framework into one simple, easy install. It uses Python's SQLObject to interact with the database (SQLite, MySQL, PostgreSQL, and Firebird are supported), Kid for templating, the MochiKit Javascript library, and CherryPy for the actual server functionality. They attempt to blend front-end and back-end development in a way that seems like it would allow a developer to be much more productive. Anyhow, it looked good to me and I decided to install it.

The installation is supposed to be easy, and for the most part it is. However, at least if you're using Ubuntu Linux (I'm running the newly released Dapper Drake), there are a few things you need to make sure you have installed.

First, you need python2.4, and python2.4-dev, and libc6-dev. These can all be gotten from the apt repositories.

sudo apt-get install python2.4-dev libc6-dev

should do it for you.

Then you simply follow the instructions here, which I will summarize:

1. Download the ez_setup.py script.
2. Run sudo python ez_setup.py -f http://www.turbogears.org/download/index.html --script-dir /usr/local/bin TurboGears

After that, it should just blow through the installation and you'll be ready to rock. Now I'm off to see what I can do with it!

My Own JSON/PHP Web Services: SSWebService

2006-05-19T10:24:00.000-05:00

Web Services. It's a new buzzword. Everyone needs them, because they'll make everything better, right?

Well, nuts to that.

What web services really amount to is nothing more than remote procedure calls that you can make from a web browser. That's cool enough. You make a request, and get a response. Instead of sending HTML that you render, the web service just returns the data. Most web services today use XML to package that data, though a few places -- like Yahoo -- are starting to use JSON instead.

JSON has a few major advantages over XML. First, it is smaller: less bandwidth is wasted by 1.0 crapola. Second, it is very easy to parse it in Javascript. In fact, JSON stands for "Javascript Object Notation," and is native to Javascript. Parsing a JSON-encoded object takes very little time. I've written Javascript applications that need to parse an XML file, and let me tell you, parsing XML in Javascript is not trivial and is a major performance bottleneck.

For an upcoming project, I wanted to develop a web service. There will be PHP interacting with the database, and I want to make simple calls from the browser. The frontend will be a Javascript application, and it is not page-based at all. So I needed a way to pass data back and forth between the browser and the server simply and quickly.

I investigated using SOAP, but was underwhelmed. I decided to design my own web service in PHP, which would use JSON for all its data passing.

The first thing I needed was to be able to register functions with the service, such that only functions that are actively made public are available. I quickly put together a PHP class with a "methods" instance variable and a "register" instance method.

Each function will take a single parameter, but this parameter is expected to be an arbitrary JSON object, which can hold any amount of data. So rather than calling a function like sum(1,2), you would call the function sum($par), where $par = {one:1, two:2}. Then you parse that parameter object in the sum() function and handle it appropriately. It's an extra step and a bit more to remember, but I feel that it adds plenty of flexibility. For example, I wrote a sum function using this brand of parameter passing, and it was utterly trivial to make it work for an arbitrary number of values.

function sum($par) {
    $sum = 0;
    foreach ($par as $key=>$val) {
        $sum += $val;
    }
    return array("sum"=>$sum);
}

Oh yes. I forgot to mention how the return values work. Rather than simply returning a single value, I will always be returning an associative array/object (the PHP associative array type is most closely related to the Javascript object type). This way I know what the return value is (it's called the "sum"), and for more complicated functions I can return as much data as I want.

In order to serve a request, which should be a POST with two parameters ("method" and "param"), I need to decode the param parameter from JSON into a PHP-native object, check that the method is registered, and then call the method with the parameter. I then take the function's return value, JSON-encode it, and return it to the browser.

Here is the code for the web service (sswebservice.php):

<?php

require_once("JSON.php");

class SSWebService {
    var $methods;
    var $json;
   
    function initialize() {
        $this->json = new Services_JSON(SERVICES_JSON_LOOSE_TYPE);
    }
   
    /*
    add a function to the server, so it can be accessed
    */
    function register($name) {
        $this->methods[$name] = true;
    }
   
    /*
    set a registered function to be inaccessible
    */
    function deregister($name) {
        $this->methods[$name] = false;
    }
   
    /*
    execute the given method, passing its single parameter
    JSON-encodes the return value, which should be an object or associative array
    */
    function call($name, $param) {
        if ($this->methods[$name] == true) {
            $evalstring = $name."(\$param);";
            eval("\$rval=".$evalstring.";");
            return $this->json->encode($rval);
        }
    }
   
    /*
    decode the JSON param into a native object, and call the given method
    return the JSON-encoded object to the browser via echo
    */
    function serve($method, $param) {
        $obj = $this->json->decode(stripslashes($param));
       
        if ($this->methods[$method] == true) {
            $res = $this->call($method, $obj);
        } else {
            $res = $this->json->encode("Not a registered function.");
        }
       
        echo $res;
    }
}

?>

And here is a demonstration web service that uses it (jsontest.php):

<?php

require_once('sswebservice.php');

$server = new SSWebService;

$server->initialize();

function helloWorld($par) {
    return array("string"=>"Hello, world!");
}

function hello($par) {
    return array("string"=>"Hello, ".$par["name"]."!");
}

function sum($par) {
    $sum = 0;
    foreach($par as $key=>$val) {
        $sum += $val;
    }
    return array("sum"=>$sum);
}

$server->register("sum");
$server->register("helloWorld");
$server->register("hello");

$method = $_POST["method"];
$param = $_POST["param"];

$server->serve($method, $param);

?>

As you can see, this web service has three available functions: sum, hello, and helloWorld. Now I just need a way to call them from the browser.

I tend to use the Prototype/Scriptaculous libraries for my web development, so I first whipped something up using Prototype's Ajax.Request object, but that turns out to be quite a bit of code every time I want to make a call to my web service. So I created a very simple class with nothing in it but a class method (ssclient.js):

function SSClient() {

}

SSClient.call = function(url, method, obj, callback) {
    var param = "method=" + method + "&param=" + obj.toJSONString();
    new Ajax.Request(url, {
        parameters: param,
        onSuccess: function(req) {
            var rval = req.responseText.parseJSON();
            callback(rval);
        },
        onFailure: function(req) {
            alert("Call failed.");
        }
    });
}

So when I want to make a call to my web service, I simply need to supply the URL, the method I want to call, the Javascript object I want to send, and a callback function to execute when it returns. The callback function should expect to receive a JS-native object, since I parse the JSON-encoded object before calling the callback function.

Here is an example of the code to call the web service:

function button1() {
    SSClient.call("jsontest.php", "sum", {
        one: 1,
        two: 6,
        three: 9
    }, sum_callback);
}

function sum_callback(r) {
    alert(r.sum);
}

As you can see, the SSClient.call() function is very easy to use, and I can create an anonymous object to pass along. In this case, the callback function simply pops up the return value. Note that it is accessed by r.sum, which is because of how I set up my return values in jsontest.php. Neat.

This requires Prototype, json.js from http://www.json.org/json.js, and JSON.php from http://mike.teczno.com/JSON/JSON.phps.

And now I'm off to build an application with my new web service!

Intel's Cheap Laptop Proposition

2006-05-02T14:27:00.000-05:00

By now we've all heard quite a bit about MIT's One Laptop Per Child initiative, and its sub-$100 laptop. In order to reach such a low cost, the machine uses a slow and inexpensive AMD Geode processor, and runs Linux. Naturally, this has led Microsoft and Intel executives to call it "a gadget" and to insist that it is inadequate for children in developing nations.

Now Intel has backed up their statements with a plan to produce their own version of a cheap laptop computer for students in developing nations, called Edu-Wise. They claim that it will cost $400 or less, and will run some version of Microsoft Windows on an Intel processor faster than the AMD in MIT's inexpensive offering.

Another key differentiator between the two machines is that the MIT laptop uses 802.11 b/g wireless mesh networking to provide connectivity between machines, whereas the Intel laptop will include a WiMAX chip, allowing it to connect from much farther away from any access points and removing the requirement that it be used in a classroom setting.

While I understand Intel's claim that "that Wi-Max deployments will leapfrog stages of development in the nonindustrialized world," I find it somewhat dubious that the nonindustrialized world will gain access to WiMAX deployments any time soon. After all, there have only been minimal deployments of the technology even in industrialized areas, and widespread adoption does not appear to be imminent. The inclusion of WiMAX in an ultra-cheap device before it is even an option on high end equipment is an indication that this is little more than a stunt to keep developing nations from signing on with MIT's proposal, which happens to actually exist.

Another question is the actual cost. MIT promised a sub-$100 notebook, and have reported that it currently costs them $135 to build one. They haven't made their goal yet, but they're pretty close. On the other hand, Intel is starting from the $400 price point (which, by the way, is already what cheap laptops cost on the US market today), and their recent record with hitting price targets is not good. For example, the recent UMPC was projected to cost around $500, but is actually available for sale in the $1100 range. Engadget expects that Intel's new machines will cost close to $750, rather than $400. So the questions Intel has to answer are:

How is this machine an improvement over those that already exist at the same price point?
Is the announced price point remotely possible?
Is $400 affordable for students in developing nations?
Do students in developing nations really need more processing power than the AMD Geode can offer?
Do students in developing nations really need the expensive Microsoft Windows installed on their computers, or would they be better served by a Linux-based solution which would drive costs down?

At this time, none of these questions have been answered. The closest Microsoft or Intel will come to addressing these issues is to publish statements such as

"We don't think you cross the digital divide with old technology," he said. "It doesn't need exotic technology and it runs real applications."

made by Paul Otellini, which implies that they want to somehow use new (expensive) technology to address the demands of this new (poor) market.

This venture is doomed to fail. It is nothing but another scare tactic from Intel, designed to cripple its competitors without having to produce a competing product. They did the same thing with the Itanium, and it worked. Now they are doing it with Edu-Wise. A real product may or may come of this. If it does, it probably won't meet the announced price point. And even if it somehow does, it will still be too costly for the market they are trying to address. The real test of whether this is a successful move by Intel will be whether or not developing nations delay their purchases of the OLPC in favor of waiting for Intel's notebook. If that happens, Intel and Microsoft have already won (and they know this better than anyone).

Democracy - Television for the Internet Age

2006-05-01T12:21:00.000-05:00

Last week I came across a rather new video player with an interesting twist: it is meant to be a platform for distributing internet-based television. Democracy's goal is to be a democratization of the media, allowing anyone with a video camera and an internet connection to put together their own channel, producing art, entertainment, or news.

Rather than simply playing videos downloaded elsewhere (this is, in fact, not possible in the current player, though I have been told that it is planned for the next release in mid-May), in Democracy you subscribe to channels and videos are downloaded automatically as they are made available. It is the work of a moment to begin watching in a window or in full screen mode. And as an added bonus, all downloaded videos play one after the other so your television experience is not interrupted by having to start the next video.

In an effort to curb hard drive use, Democracy has implemented an expiration system based on Tivo's, in that downloaded videos expire in 6 days (by default, though that number can be changed), unless they are explicitly saved for longer. And the website reminds you that you can always download expired videos again if you want to see them.

I have only used the program to a somewhat limited extent thus far, and have not fully explored the channel options. However, the content I have found is mostly good, and in one case is better than the options on television (Channel Frederator). I am certainly looking forward to the creation of new and interesting video content, and the distribution opportunities being made available by standards such as RSS and innovative new players such as Democracy.

While I don't think it has the potential to replace the networks, the ability for anyone to produce their own news should force the news networks to remain diligent and the ability for anyone to produce and publish their own shows should force the studios to remain creative. This democratization of the media, by putting the tools into the hands of the individual, can only be a good thing.

Democracy is open source, released under the GPL, and is available for Windows, Mac OS X, and Linux.

Google Complains About IE7's Default Settings

2006-05-01T12:05:00.000-05:00

Just saw this article about Google officially complaining that Microsoft will set IE7 to use MSN search by default. The claim is that it is unfair to limit choice and drive users to MSN rather than Google.

This is a specious argument coming from Google, considering the fact that Google search is the default in Firefox, Opera, and Safari. The only difference would be that Microsoft owns both the browser and the search engine ... but if MS owns both then they should be allowed to link them as they see fit, yes? It certainly wouldn't make any sense to require Google search to be the default on IE7, when MSN search is the only option Microsoft has if they want to default to a non-competitor.

"The market favors open choice for search, and companies should compete for users based on the quality of their search services," Marissa Mayer, the vice president for search products at Google, told the Times. "We don't think it's right for Microsoft to just set the default to MSN. We believe users should choose."

Yes, users should be able to choose. But you still need a default, and for once Google isn't going to be that default. Perhaps that's what's gotten under their skin.

For the record, Microsoft did what they could not to come off well either:

Microsoft told the Times that giving users of the new version of IE an open-ended choice could add complexity and confusion to the browser set-up process, while offering a few options would be arbitrarily limiting.

Aside from once again demonstrating their lack of respect for their own users, how arbitrarily limiting can offering a few choices possibly be? The options would probably be:

MSN Search
Google Search
Yahoo Search
Add another search engine...

Perhaps all Google was trying to do was to raise awareness of the issue, and to force Microsoft to offer options. However, they should avoid whining like this, lest they lose their darling status. Google should get back to their roots: winning by being better, rather than by having better lawyers and journalist shills.

Netflix Online Video Distribution

2006-02-17T11:30:00.000-06:00

Netflix has a problem, in that it's getting harder and harder for them
to make a profit. They break even on a renter if he/she gets 5 DVDs
per month. So, they need to find some way to augment their service
somehow.

One possible way they could do this is to start getting into the
online distribution game. They have all these DVDs, and they're a
known commodity for the studios. They would need to license the movies
for online distribution, but the fact that they also distribute DVDs
should help in those negotiations. Then, once they can offer the
movies online, they have a few options as to how to get them to their
customers.

One possibility is to have each movie streamable from the Netflix
website as long as the customer has the DVD checked out. So when they
ship you a movie, they also make available a stream so you can watch
it online. The problem with this is that they then incur the cost of
shipping and of online distribution, making it somewhat non-viable.

A second possibility is to offer an added level of service that
includes streaming movies, allowing people to watch a movie instantly,
without having to wait for the disc to come in the mail. This would
free Netflix from having to ship the movie, saving them some money.

A third possibility is to offer the movies for download, for a fixed
cost per movie. This way, Netflix would be able to recoup the cost of
the transfer in the purchase fee. However, they would have to
implement some kind of DRM scheme.

They would want to build the DRM system so that it's as fair as
Apple's FairPlay. Since the best codec for distributing video online
at the moment is H.264, and they would want their files playable on
the iPod anyway, they should try to make a deal with Apple. Apple lets
the iPod play Netflix's DRM'd video files and possibly shares
bandwidth, licensing and advertising costs, while Netflix only
supplies video in this format, helping to ensure iPod dominance. It
would be a win for both companies, while allowing them both to do what
fits within their business model.

For the client side, Netflix would need an application to play the
streamed/downloaded video. There are a few possibilities here.
1) They could roll their own client which plays H.264 files and
decodes the Netflix DRM. They would probably need to make it talk to
the iPod somehow, but this would presumably be part of the deal with
Apple. Making it cross platform is important.
2) They could do some kind of bundling deal with Apple for Quicktime.
This would give them the benefit of getting advertisement through
Quicktime even for non-subscribers, but would be difficult to do and
would get into the thorny full screen issue (do you get Quicktime Pro
when you subscribe to Netflix?).
3) They could update an open source client, such as VLC. VLC already
has support for H.264, and for streaming video over the network, as
well as being very cross platform. There are two issues here, which
are related. The client needs to be able to decode the DRM, and VLC is
an open source program. The way I see it, DRM doesn't really work when
the code is open. Even if the technical aspect were completely figured
out and the DRM was uncrackable (dubious, considering that all DRM has
been cracked), the studios would never agree to it. Say what you will
about open source (I know I like it), it doesn't instill confidence in
the major content owners.

A solution could be developing a program that opens the
downloaded/streamed file, decodes the DRM, and passes the video along
to VLC, or any other video player. This greatly simplifies the
software from Netflix's perspective and gets them away from having to
open source the decoder. However, there could be a problem with the
player receiving the unencrypted file, with no way to prevent the user
from saving the stream to the hard drive. (Of course there will be
people who figure out how to do it, but the point of DRM is to make it
at least somewhat difficult, so the process doesn't enter the
mainstream.) Because of this issue, developing their own player seems
to be the best option.

Netflix needs to do something to help the bottom line, other than
attempting to screw their customers. Withholding discs from their
happiest customers is not a good way to keep those customers happy.
This would be a good way to make more money by offering more services,
rather than fewer. And partnering with Apple would been a boon for
both companies; in fact, it's difficult to tell who would benefit
more.

Obviously, I have no inside information about this. I say it because
it's what I would do if I were in a position of authority at either
Netflix or Apple. And as a user of both, I'd certainly like to see it
happen.

Apple Making Intel Uncomfortable?

2006-02-10T11:29:00.000-06:00

Just saw a story over on Ars about Apple's new ads making Intel uncomfortable. Basically, the premise is that some analysts are worried that Apple's ad (which insults Intel's other partners like Dell & HP by calling them "dull little boxes") will alienate Intel from their other, larger customers.

They cite Intel's "public love-fest" with its newest customer, the 1000 employees dedicated to The Switch, and the fact that Apple got the first Core Duo chips as other reasons Intel may be in trouble with its other customers.

But I'd like to point out another problem with their claim. Intel's in the driver's seat in these relationships. Dell only uses Intel chips. And they can't even threaten to go to AMD, because Intel can pull their sweet prices out from under them, or even stop selling them chips entirely, leaving Dell to find some way to get 10 million processors per quarter from AMD. I'd be willing to venture that AMD isn't capable of supplying that many processors. And the same goes for HP, Lenovo, and everyone else. If you want to offer AMD chips, then you can, but you have to stick with Intel for the same reason Apple chose Intel: manufacturing capability. Intel is the only place to go if you want a guaranteed supply.

So I'd say that Intel has nothing to worry about. In fact, they're probably excited that Apple is lighting a fire under the other computer makers. The CPU-comparison is now gone, so the difference maker is not processing power any more. That puts an onus on the designers of the whole system to put together a better offering ... and whether it's Apple's Front Row or Dell/HP/etc VIIV, Intel stands to benefit. A lot.

And I think everybody knows it. That's why none of the computer makers are saying they're insulted. Dell's spokesperson had this to say:

"As far as their marketing goes, that's marketing," said Dell spokesman Jess Blackburn. "We continue to have a very solid relationship with Intel and nothing has changed in that respect. We probably are their largest customer; we shipped 10 million systems in the fourth quarter. I don't think any supplier is going to ignore a customer that is building that many systems and using that many of their parts in them."

My guess is that Intel is quietly encouraging Apple to be as insulting as they want. If Dell and HP lose customers to Apple, it's probably better for Intel. They'd prefer their chips to be part of the high end, high margin space that Apple aims for, because it means more money for them. But this entire issue is just meaningless controversy invented by analysts. I guess they have to justify their paychecks somehow.

HandBrake Wrapper Script

2005-12-14T16:53:00.000-06:00

I've been encoding my DVD's lately using HandBrake for Linux. As far as I can tell, there isn't a GUI for the Linux version, though there is a very nice one for OS X. However, you don't need the GUI at all.

Once you have compiled HandBrake from source, you have everything you need. Except that the default options in the CLI program (HBTest) aren't exactly the greatest in the world. So every time you want to encode a movie, you have to type a long string of command line options which aren't strictly necessary, since they're the same every time. (I'm assuming you encode all your movies the same way, because that's what I do ... and who wants to worry about encoding each movie differently?)

I encode the video using the x264 codec at 700 kbps, and the audio using the faac codec at 96 kbps. I find that this gives a high enough quality rip that the artifacts aren't typically very apparent. In most cases, my encoded version is indistinguishable from the MPEG-2 version on the DVD, except for the small size.

Normally, I would have to type:
/home/sean/Desktop/HandBrake-0.7.0/HBTest -e x264 -E faac -2 -b 700 -B 96 -w 512 -i /dev/dvd -o This\ is\ the\ movie\ title.mp4

That's a lot of typing, especially considering the fact that all of it is the same every time except for the -o argument.

To save myself some time (it's a valuable couple of seconds!), I wrote a wrapper script in Python to use my defaults instead of HandBrake's.

Here it is.

Now all I have to type is:
./hbencode.py 'This is the movie title.mp4'

Which of course is a lot faster.

I also decided that I might at some point forget to type a title for the movie, but I'd still want the movie to be ripped and encoded, and I could name it later. So my solution was that if nothing is passed in at the command line, it would get the current timestamp and use that as the filename. That should be unique enough that I'd be able to repeatedly encode things without setting a title and be able to go back and set their names later. It has the added bonus of keeping the files in chronological order, according to when they're ripped.

The next step is to take more command line options than just the movie title, so that the defaults can be changed if necessary. The first addition would be the ability to set the title to rip, in case you're ripping a TV show, or a movie where the main feature is (for whatever dastardly reason) not the first title.

This is a very simple script, but it makes my video encoding life a little simpler, and a little more pleasant. Hopefully it does the same for you.

Web Based Remote Control

2005-12-06T10:48:00.000-06:00

I spend a lot of time at my computer, but I spend even more time near my computer but not quite at the keyboard. It's a Mac Mini, and most of the time it's playing music in iTunes or a movie in VLC. I wanted a way to control those applications remotely, so I could pause without going all the way over to the computer. Since I have a Nokia 770, I thought the easiest way to do this would be to write a web page that can control these applications, and connect to it from the 770.

The first thing I needed was a way to control the applications. This was quite simple, of course, using Applescript. I made a small script for each action I wanted to perform. For example:

tell application "iTunes"
   playpause
end tell

All the scripts I wrote were similar in complexity, but more complicated scripts can of course be written.

The next step was to run these scripts from a web page. PHP handles this quite nicely, using the system() command. However, it won't work right out of the box, because OS X runs the web browser as a different user than the person running iTunes (for obvious security reasons). So I had to add a line to the /etc/sudoers file:

www ALL = NOPASSWD; /usr/bin/osascript

This gives the user running Apache the right to run osascript (which executes Applescript files) without typing in a password. This is probably not the best thing to do, security-wise, so it's pretty important to make sure that your machine is only accessible to computers on your local network.

Next, the PHP. It's quite simple. I only need one page, which takes a script and executes it as if it were on the command line. Here it is (execute.php):

<?php

$script = $_REQUEST['script'];

system($script);

?>

I'm planning on sending requests to this PHP file via XMLHttpRequest, of course. I'm using my standard xmlhttp() function which you can get from a previous blog. The only other Javascript function I actually need is the execute() function, which sends a request to execute.php. Here it is:

function execute(script) {
   var req = xmlhttp();
   var param = 'script='+script;
 
   //this should be made async
   req.open('POST', 'execute.php', true);
   req.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
   req.send(param);
}

This is a pretty simple function, sending a single parameter asyncronously. Since it's only sending the request, and doesn't really need to return anything, I don't even bother checking the return. We just shoot off the request and wait for more input. You'll know if it worked it the music starts playing.

It was important to me that I didn't limit this program to any set of functionality that would be difficult to augment later. So I created an XML file that holds records for each program that can be controlled and the scripts to run in order to control it. Originally, I wanted the interface to be built dynamically via Javascript, so I crafted the XML file such that it had no attributes, only elements, because my JS XML parser doesn't read attributes (I should work on that). Unfortunately, it seems that the browser on the 770 doesn't support dynamically created DOM elements, so this approach didn't work. Plenty of wasted work there (and a lot more Javascript than I've included here).

This was when it dawned on me that I don't actually need to build the interface every time the page loads. It only needs to be done when the XML file is changed. So I wrote a Python program to take the XML file, parse it, and output the same HTML as would have been produced by the Javascript. I recreated the XML file to use attributes instead of just elements, because I find that to be simpler to read, and it's much easier to handle that in Python's xml.sax library. Here's the Python (buildinterface.py):

import os, sys
from xml.sax import saxutils, handler, make_parser

#set up the environment variables
config_file = "server2.xml"
html_file = "index.html"
output_file = file(html_file, 'w')

#globals programs dictionary
programs = []

class Program:
   def __init__(self, name):
       self.name = name
       self.commands = []
     
       programs.append(self)
     
   def addCommand(self, cmd):
       self.commands.append(cmd)
     
class Command:
   def __init__(self, name, script):
       self.name = name
       self.script = script
     
class ConfParser(handler.ContentHandler):
   def __init__(self):
       handler.ContentHandler.__init__(self)
     
   def startElement(self, name, attrs):
       global temp_prog
       if name == "program":
           temp_prog = Program(attrs['name'])
       elif name == "command":
           temp_prog.addCommand(Command(attrs['name'], attrs['script']))
         
def write_html(out = sys.stdout):
   out.write("<html><head><title>Remote Control</title>")
   out.write("<link rel='stylesheet' type='text/css' href='remote.css' />")
   out.write("<script type='text/javascript' src='remote.js'></script>")
   out.write("</head><body>")
   for prog in programs:
       out.write("<fieldset><legend>" + prog.name + "</legend>")
       out.write("<div id='" + prog.name + "_div'>")
     
       for cmd in prog.commands:
           out.write("<span class='link' onclick='execute(\"" + cmd.script + "\");'>" + cmd.name + "</span>")
           out.write("<br />")
         
       out.write("</div></fieldset>")
   out.write("</body></html>")

parser = make_parser()
parser.setContentHandler(ConfParser())
parser.parse(config_file)

write_html(output_file)

output_file.close()

As you can see, it outputs directly to index.html, which is convenient. Every time you run buildinterface.py, you have a fully updated interface waiting for you to load up in your browser. And in case you were wondering what those span tags are all about, they are to mimic links without the bother of actually putting a link on there. Here's the CSS for that:

.link {
   text-decoration: underline;
   font-size: 10pt;
   color: black;
   cursor: pointer;
}

Each link has its onclick event set to call execute() with the script to run. It's a fairly simple scheme, and when I loaded it in my browser to test it out, it worked swimmingly. I could control iTunes, VLC, Quicktime, and the system volume. I pulled out my laptop and connected, and found that it worked from remote systems. Now it was time to try it out on the 770. I fired up the browser and opened my page. This time the interface loaded, which was of course promising. However, none of the links worked. So the remote control program works, and is extensible, but I haven't been able to get it working on my Nokia. It seems that the browser doesn't want to send the XMLHttpRequests, which is weird, because it's Opera, and I tested this in Opera on both OS X and Linux, and it works there. I'll be trying to get it to work on the 770, but until then, here's the working code.

Things that could be added:
1. Get it to work on the Nokia 770.
2. Write plugins for other programs, to build out the capabilities of the remote.
3. Have the PHP return something, so it's possible to tell if the command has been executed or not. This will allow for more interesting possibilities on the front end.
4. Anything else ... ?

You can download the code here. Let me know how it goes, or if there are any problems you find with it. And let's here about those plugins!

Associative Array Doubly Linked List

2005-08-22T16:12:00.000-05:00

This isn't going to be a demonstration of an application of any kind, but rather I'm going to show a data structure I developed the other day to help out with my web applications. I implemented it in Javascript, but it should be trivial to port it to other languages. (If the language doesn't support associative arrays on its own, then you'll have to implement that first.)

The idea behind an Associative Array Doubly Linked List (AADLList) starts with an associative array and adds a doubly linked list to it. In an associative array, you have a bunch of key/value pairs, where the value is accessed by its key (normal arrays can be viewed as an associative array where the keys are integers starting with 0). In a linked list, you have to define an object that has not only its data, but also a pointer to the next object in the list. A linked list will also have a pointer to the front of the list. A doubly linked list takes this a step further, by giving each object a pointer to the object before it AND after it, and the list must have pointers to the first and last items in the list.

I wanted to make this data structure independent of the application you're going to use it in, so instead of defining information fields in the objects as well as the pointers, the nodes in this list will have another variable inside that you can fill with whatever object you want.

First, we have to define the list objects, AADLListNode and AADLList:

function AADLListNode() {
    this.node = null;
    this.next = null;
    this.prev = null;
}

function AADLList() {
    this.list = new Array();
    this.first = null;
    this.last = null;

Note that I omitted a trailing bracket at the end of the AADLList definition. That is because the rest of the code goes inside the AADLList class definition.

As you can see, the ListNode class has a node object and its two pointers. The List object has a list (which is an array) and its two pointers. At first, the first and last pointers are null, which means the array is empty.

The first thing we need to do is add nodes to the list. When we're adding the first item in the list, we have to take more things into consideration. When the first node is inserted, the list's first and last pointers have to change from null to the new first node. So we write our first function:

/*
    add a node that is the first and therefore only node in the list
    */
    this.addFirst = function(key, val) {
        var node = new AADLListNode();
        node.node = val;
        node.next = null;
        node.prev = null;
        
        this.list[key] = node;
        
        this.last = node;
        this.first = node;
    }

After our first node has been added, we can start adding more nodes. Since it's a doubly linked list, we have the desirable ability to add nodes to either the front or the back of the list. (Actually, we will be able to insert a node anywhere in the list.) So we write two functions, to add a node to the front or back of the list.

/*
    add the given key/val pair as the first element in the list
    */
    this.addToFront = function(key, val) {
        var temp = this.first;
        
        var node = new AADLListNode();
        node.node = val;
        node.next = this.first;
        node.prev = null;
        
        if (this.isEmpty()) {
            this.addFirst(key, val);
        }
        
        this.list[key] = node;
        
        if (this.first) {
            this.first.prev = node;
        }
        this.first = node;
    }
    
    /*
    add the given key/val pair as the last element in the list
    */
    this.addToBack = function(key, val) {
        var node = new AADLListNode();
        node.node = val;
        node.next = null;
        node.prev = this.last;
        
        if (this.isEmpty()) {
            this.addFirst(key, val);
        }
        
        this.list[key] = node;
        
        if (this.last) {
            this.last.next = node;
        }
        this.last = node;
    }

The code in these functions is pretty simple. We just need to set the prev or next pointers of the newly created node, and update the first or last pointers of the list. As you can see, these functions use the isEmpty() function to determine whether they should call addFirst() or not. We have to create isEmpty() for ourselves, like so:

/*
    returns true if the list is empty
    */
    this.isEmpty = function() {
        if (!this.first && !this.last) {
            return true;
        } else {
            return false;
        }
    }

Simple enough. If the first and last pointers are both null, then the list is empty. It should, in theory, be possible to also just access the this.list.length value and check that it is zero, but unfortunately, when you remove an entry in a Javascript array, it does not disappear. The key continues to exist, but points to null. So the this.list.length field will be incorrect if a node is ever removed from the list.

I often don't care whether my nodes are inserted at the front or the back, so I threw a convenience function in:

/*
    simple convenience function
    just calls addToBack
    */
    this.add = function(key, val) {
        this.addToBack(key, val);
    }

Now we can add as many nodes as we want, either at the front or the back of the list. But how do we access the nodes once they're there? One way is to simply go through the object's members and use them, but this is generally not an awesome way to go (if you really wanted to do it, you'd have to do something like this: myList.list[myKey].node.myField ... etc. Clearly not what you want to be doing every time you want to retrieve something from the list). So we can write a function that will retrieve a given node for us based on its key.

/*
    try to get the node for a given key
    return the node if it exists, null if not
    */
    this.getNode = function(key) {
        for (var id in this.list) {
            if (id == key) {
                return this.list[id];
            }
        }
        
        return null;
    }

But that isn't always what we want, because it will return an AADLListNode object, which you probably won't want to use in your application. So we write another, similar function, to just return the object inside the node (without the pointers).

/*
    try to get the node value for a given key
    return the value of the node or null, if there is no node
    */
    this.getNodeValue = function(key) {
        var node = this.getNode(key);
        if (node) {
            return node.node;
        } else {
            return null;
        }
    }

Note that if we were simply doing a linked list (or even a doubly linked list), this operation would have an O(n) efficiency (worst case), because we would have to traverse the list and check each node to see if it was the one we're looking for. But since it's also an associative array, we can access it in O(1) time, greatly speeding up our data access. This is good.

I'm sure you've noticed that what this is missing is a way to remove a node from the list. So we're going to add that function now. We need to make sure we check that the node actually exists, then change the next object's prev pointer to point to the removed object's prev, and vice versa. (Don't know how to explain that very well, except to use the term "switcharoo," which I think should cover it pretty well.)

/*
    remove a node from the list
    return true on success, false on failure (if the given key isn't in the list)
    */
    this.removeNode = function(key) {
        var node = this.getNode(key);
        
        //if the node doesn't exist, you can't remove it
        if (!node) {
            return false;
        }
        
        var tempNext = node.next;
        var tempPrev = node.prev;
        
        //if the node doesn't have next/prev, you can't change them
        if (tempNext)
            tempNext.prev = tempPrev;
        if (tempPrev)
            tempPrev.next = tempNext;
        
        //if it's the first node, set the next node to be the first
        if (key == this.first.node.id) {
            if (tempNext) {
                this.first = tempNext;
            } else if (tempPrev) {
                this.first = tempPrev;
            } else {
                this.first = null;
            }
        }
        
        //if it's the last node, set the prev node to be the last
        if (node == this.last) {
            if (tempPrev) {
                this.last = tempPrev;
            } else if (tempNext) {
                this.last = tempNext;
            } else {
                this.last = null;
            }
        }
        
        //null out the entry in the array
        this.list[key] = null;
        
        return true;
    }

As you can see, this is by far the most complicated function in the class (and it's not even very complicated!).

And don't forget that trailing bracket!

}

Now we have a working AADLList, which gives us all the advantages of a doubly linked list and an associative array in one easy to use package. You have O(1) access to any node in the list, but you can also traverse from the back or the front and add nodes in whatever order you want (although that isn't implemented in this version).

So what would you want to add to the AADLList to make it a little better?

How about an insert() function of some kind, so you can decide where in the list you want a node to be added?
Build on your insert() function and make the AADLList a sorted list.
Give it some queue-like or stack-like functionality, with push() and pop() operations. (Who wouldn't want the ultimate data structure?)

Feel free to use it, add to it, whatever. Let me know if you have any issues with it, or problems using it or implementing new features for it.

Skype's Purchaser

2005-07-29T21:00:00.000-05:00

I just read Cringely's theory about Skype's near purchase by Rupert Murdoch. He thinks it was to put Skype into play as a takeover target by another telecom company. His guess is that it will be either a mobile phone company like Vodaphone or NTT DoCoMo, or a cable company who wants to get into the VOIP market, like Comcast.

I must say I disagree with him. Comcast already has a digital phone service, which uses VOIP. What use would they have for Skype? I don't buy it that they want those 20 million subscribers to sell movies to. Why would Skype's customers switch to Comcast? Many of them don't have broadband, and more are international (and Comcast isn't). So they wouldn't be eligible for such downloaded movies.

And a mobile phone company? They'd be just as interested in killing Skype as the normal telecom companies. Skype can be used by any device that can connect to the internet, basically. People in coffee shops can use Skype on their laptops. People with PDAs can talk whenever they have wireless coverage, as if their PDA was one of those PDA-phones that cost much more.

That gets me to my counter-theory: Skype will be purchased by a company that wants to expand into a new market and sell their own product to go along with Skype. They will blanket the US and possibly Europe with a wireless signal (WiMAX?) and sell PDA-like devices whose primary use is as a phone, but it also inherently has full access to the internet.

Who would do something like this? I would guess that it would be either Intel or Microsoft. Intel wants to build and sell WiMAX transmitters and cards. They want to expand into more markets, including the home. Basically, they want to put an Intel chip in as many places as possible. They'd get their WiMAX chips in every consumer's laptop, PDA, and especially the Intel-branded PDA-device that goes with the service. They could offer such a service at similar cost to a cellular phone, available both in the home and on the road. They could take over the mobile phone market and the traditional telecom market it one fell swoop. And they'd make a lot more profit, because they wouldn't have to launch satellites like the cellular companies or invest in underground or expensive infrastructure. They would only have to put a WiMAX station or two in a few score major cities and get a good amount of bandwidth.

But Microsoft could do the same thing. They're looking for ways to grow the company, and they always love expanding into new markets. Additionally, they just announced that they're planning to acquire more companies in the near future to use their technology. Skype could be one of these purchases. Microsoft sells the mobile OS that Skype runs on. They'd love to be able to build their own phone and sell it to people directly, getting hardware revenues instead of software licenses. Of course, they would also continue to license the OS, so they're still getting those revenues. They'd use the same strategy that Intel would, but would probably push it a lot harder and market it a lot better. And they'd tie it directly into Windows, and the MS-phone would cooperate with it perfectly ... when you're sitting at your Windows Vista computer with your MS-phone in your pocket and you get a call, you see it on your screen (maybe it pauses your music or movie), and you can either answer it with your computer or use your MS-phone as the handset.

I can see Microsoft spending a few billion dollars to make that happen. If they did this (with their OS monopoly), they could rule the world for quite a while longer.

So I say be wary of MS looking into Skype. I'd much rather have Intel do it, or have Skype continue under its own sails. There's nothing wrong with a private company that makes $70 million dollars a year and has awesome growth potential. So there's no need to sell it.

Dashboard Escaper

2005-07-26T14:41:00.000-05:00

So I've done a little work on the HTML Escaper I released here a few days ago. While it might be convenient to have a web page open that lets you escape your code for the web, it would be even MORE convenient if you could just do it in a widget. So I've decided to do just that.

The Dashboard Escaper is basically the same program as the HTML Escaper, except that instead of being rendered by your web browser, it is run by Apple's Dashboard.

I decided to make a couple of other changes: there is no longer a button to force it to escape the code. It does it automatically. It escapes the code once per second, and also any time a key is pressed while the source text field is active. I think this automatic, real-time behavior is more suited to the Dashboard concept.

Making the widget was actually quite easy. I already had the web application written, and just wanted to run/display it in a new way. I used a special Dashboard IDE called Widgetarium (link), which contains an editor for your Javascript, HTML and CSS, a widget packager, a Dashboard debugging emulator, and a source level Javascript debugger. Apparently you can insert breakpoints into your JS code, but I haven't tried it yet (I haven't put any Javascript with bugs in it into the editor or debugger).

After downloading Widgetarium and running it, I started a new project. I called it Escaper and went right along.

It asked me for the size of my widget, and I gave it 300x200, as well as the color. It automatically created a PNG to use as the background of the widget.

The widget was now ready to be written. The first thing I did was open Escaper.html. I had to change the line which includes the .js file, because it was including $_NAME_.js instead of Escaper.js. Then I just had to put in my HTML where it directed me to do so ... my Escaper.html looks like this:

<html>
<head>
<!-- The style sheet should be kept in a separate file; it contains the design for the widget -->
<style type="text/css">
 @import "Escaper.css";
</style>
    <script type='text/javascript' src="/System/Library/WidgetResources/button/genericButton.js"></script>
    <script type='text/javascript' src="Escaper.js"></script>

<title>Escaper</title>
</head>
<body onload='setup();'>
    <div id="front" onmouseover='mousemove(event);' onmouseout='mouseexit(event);'>
     <img span="backgroundImage" src="Default.png">
     <textarea id="source" rows="10"></textarea>
     <textarea id="resultcode" rows="10"></textarea>
     <div id="result"></div><!-- need to make this scrollable! -->
    </div>


</body>
</html>

I executed the widget by selecting Run from the Project menu, and my widget popped up in the pseudo-Dashboard in the Widgetarium environment. Unfortunately, it was blank.

I needed to fix this, and I discovered the problem by using the IDE's DOM inspector (on the right hand side of the application). I found that the elements I had defined were off the image, and so would not be displayed. I opened the Escaper.css file and positioned my elements so they would be sure to show up on the widget. Here is my CSS file:

body {
    margin: 0;
}

.backgroundImage {
    position: absolute;
    top: 0px;
    left: 0px;
}

#widgetButton {
    font: 10px "Lucida Grande";
    font-weight: bold;
}

#source {
    position:absolute;
    left:30px;
    top:10px;
}

#resultcode {
    position:absolute;
    right:30px;
    top:10px;
}

#result {
    position:absolute;
    left:30px;
    top:150px;
}

Note that I only wrote the last two rules. The rest are generated automatically by Widgetarium.

Now running the widget shows everything properly. Now I just needed to implement the functionality. Luckily for me, I already had code to do the heavy lifting, which I quickly moved from my old Escaper program into this one.

_$, do_escape, and html_escape were easily added, and I didn't have to change them at all. But remember when I said I didn't put a button in? How is the code going to be escaped?

I wanted to do it once per second, automatically, and then whenever something changes in the source box. So I set up a timer that executes once per second and calls do_escape(). That function is:

function everySecond() {
    if (timerID) {
        clearTimeout(timerID);
    }
    
    do_escape();
    
    timerID = setTimeout("everySecond()", 1000);
}

(Note: timerID must be declared globally.) And to the function to handle keypresses:

function keyPressed(event) {
    do_escape();
}

So I have it all in place, but nothing would happen if I were to run it now. The keyPressed function has to be connected to the onkeypress event, and everySecond has to be called for the first time to start the timer.

function setup() {
    // put your setup code here...
    
    _$("source").onkeypress = keyPressed;
    
    //start checking for changes every second
    everySecond();
}

That was easy enough. I ran it in the IDE, and it worked. I went to the Finder and double clicked Escaper.wdgt, and it opened up in Dashboard and worked there, too.

And there you have it, folks. Basically anything you can do with a web application can be done in a Dashboard widget. And web applications can do most of the the things widgets can do (the difference is that widgets can make system calls that normal Javascript can't). Which you write should depend on the application in question. How do you want it displayed? Who's going to use it? Do you want quick access to it, or should it be open and running all the time? Et cetera.

To do:

Make it resize itself automatically, so there is no empty space at the bottom when nothing has been escaped.
Make a scrollbar, so that the code doesn't go off the bottom of the widget when it's displayed.

Graffiti

2005-07-25T15:43:00.000-05:00

You’ve probably heard that web pages can do a lot more than they used to … but I haven’t really shown you that yet. Simple Javascript applications can be fun and useful, but it’s about time to move on to bigger, better things.

Graffiti is a program I wrote that is designed to be an online chat/forum that allows people to make posts which will be propagated in real-time to everyone who has the page open. It has a Javascript interface that you’re used to by now, a PHP middle layer, and a SQLite database. It communicates with PHP using JSON, which I’ll explain in a later post.

The Idea

Why would you want a realtime chat/forum? Well, what if you’re at work, and the AIM port is blocked? Or you want to have a chat with several people, but not all of them are on the same IM network, or one of them has an old version of the client software that doesn’t support group chat? What if you’re reading a forum, and would never know if there have been new posts made since the last time you loaded the page? In any of these cases, it would be nice to have a program that will get around those problems. Graffiti is one such program.

Adding a new post appends it to the bottom of your page and sends it off to the server. In a standard client-server situation, it would be the server’s responsibility to send that post to every connected user. However, HTTP is limited, in that the server cannot “push” data down to clients. The clients must “pull” it from the server. How can we simulate a “push” when we can only “pull”?

By polling for new content. Each client will sit in a loop and check repeatedly for changes to the state of the program. In the interest of not killing the server, we have the clients checking once per second for new content. If there is no new content, the server will say so. If there is new content, the server will send it. This way, everyone who has the page open will be looking at exactly the same page, regardless of who edits it and when.

The Data

The first thing we’ll do is design the database. There aren’t a lot of demands for our data, so it won’t be too much of a problem. I’m using SQLite as my database, but this is SQL 92 compliant code, so you should be able to use it (or very easily port it) to whatever database you prefer.

Our database will have 3 tables and 3 triggers. The first table is the “entries” table, which holds all the posts that are displayed on the page. It needs a field for the unique id, the author, the content of the post, and the time the post was made. The other two tables are the “instracker” and “deltracker” tables, which are to keep track of the posts that have been made and deleted over the course of the program running. These tables help keep everything sync’d up, by giving the client a place to look when they need to know if anything has changed since the last update.

create table deltracker (id integer primary key, entry integer);
create table entries (id INTEGER PRIMARY KEY, author VARCHAR(255), data TEXT, time DATE);
create table instracker (id integer primary key, entry integer);

To use this code in SQLite, just open up your SQLite command line interface, and type these lines in verbatim. Other databases will obviously have different ways of doing it, so you’re on your own there.

% /usr/local/bin/sqlite graffiti.db

Now let’s put our triggers in. We have one trigger that will put the date into the entries table, so that each new post has an exact date that goes along with it. The other two triggers update the instracker and deltracker tables whenever a post has been added or deleted.

CREATE TRIGGER delete_entries_tracker AFTER DELETE ON entries
BEGIN
        DELETE from instracker where (entry=old.rowid);
        INSERT into deltracker (entry) values (old.rowid);
END;
CREATE TRIGGER insert_entries_time AFTER INSERT ON entries
BEGIN
        UPDATE entries SET time = DATETIME('NOW') WHERE rowid = new.rowid;
END;
CREATE TRIGGER insert_entries_tracker AFTER INSERT ON entries
BEGIN
        INSERT into instracker (entry) values (new.rowid);
END;

You can add these triggers to your SQLite database from the command line.

% /usr/local/bin/sqlite graffiti.db < triggers.sql

Our Javascript code needs to support this database as well. We will define a class, called Entry, that corresponds to the data in the entries table.

function Entry() {
    this.type = "";
    this.id = "";
    this.author = "";
    this.data = "";
    this.time = "";
}

Note that there is a member in our class for each field in the table.

The Interface

Now that our data is all set up and ready, let’s define the interface. It’s not very complicated, but we will have to modify the page at runtime quite a bit.

The page itself doesn’t need much in the way of code. We simply have two div’s, one of which will hold the list of all the posts, and the other will be where the user can add new posts.

<div id="outer"></div>

<div id="add_div">
    <input type="submit" value="Add new" onclick="openAdd();" />
</div>

The CSS to style this program is equally simple. We just define classes for an entry, author, entrypanel, and the bottom. The entrypanel is where the delete button for each post will go, and the bottom is where the content of the post will be displayed.

.entry {
    position:relative;
    border:1px solid black;
}

.author {
    position:relative;
}

.entrypanel {
    position:absolute;
    top: 0px;
    right: 0px;
}

.bottom {
    position:relative;
}

The Functionality

Now that we have the basic interface scoped out a little, what do we want it to do? The button in the add_div will open the Add Post region of the page at the bottom, below all the posts. The user types his name and a message, and clicks the submit button. The post shows up immediately on his page, and is sent off to the server. When someone else posts a response, it will automatically show up at the bottom of the list. If he decides he doesn’t like what he wrote, he can hit the delete button on his post, and it will disappear from everyone’s page.

Communication between the application and the database is enabled by the XMLHttpRequest object. Originally developed by Microsoft and implemented by the other browsers, this allows the browser to request and receive data from a server while the page is loaded, and the browser can do something with said data without refreshing the page. XMLHttpRequest can work synchronously or asynchronously, which means that you can define whether a request should stop the application from running while it waits for data, or if it should continue on its business even if it is waiting for a response from the server. We’re going to use both in this program.

You’ve probably heard the hype already, and I won’t go over it any more. Now … have you actually used the XMLHttpRequest object? Naturally, Microsoft does it a bit differently from everyone else. We’ll use a function that detects whether the browser supports Microsoft’s version or the Mozilla version (Safari and Opera use the Mozilla version), and returns the proper object.

function xmlhttp() {
    var treq;
    if (window.XMLHttpRequest) {
        try {
            treq = new XMLHttpRequest();
        } catch(e) {
            treq = false;
        }
    } else if (window.ActiveXObject) {
        try {
            treq = new ActiveXObject("Microsoft.XMLHTTP");
        } catch(e) {
            treq = false;
        }
    }
    return treq;
}

You can do several things with the XMLHTTP object, but you only actually need a few of them. After you get an object from xmlhttp(), you’ll open it, set its header (needed on some browsers for some types of requests), and send a parameter string. When you open it, you define the action type (GET or POST), the URL to which to send the request, and whether the request should be processed asynchronously or not. If you’re using POST, you have to set the request header, otherwise, you don’t. If you’re using asynchronous, you have to define a callback function. Otherwise, you don’t.

Before I get too far into this and forget to set you up on PHP/SQLite, you need to create a PHP file that can talk to your database. For an application using SQLite and JSON, as we are, the top of your PHP file will look like this:

<?php
require_once('JSON.php');

$json = new JSON();
$file = "graffiti.db";

$db = new SQLiteDatabase($file) or die("Could not open database");

...

unset($db);
?>

This tells PHP to use the object-oriented version of its SQLite adaptor, which we will find very convenient when we need to transfer data. The way JSON works is that it packages up an object, and since our Entry class exactly mirrors our entries table, PHP will see the exact same class structure coming out of the database and the application.

The first thing we need to do when we load the page is load all the entries that already exist in the database. We want this to be a synchronous request, because the program should not run until all the posts have been successfully loaded and put onto the page.

function loadEntries() {
    var req = xmlhttp();
    var param = "load=1";

    //this is synchronous, because we want it to block until all the entries
    //are loaded.
    req.open("POST", "graffitisub.php", false);
    req.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
    req.send(param);

    var res = req.responseText;
    entries = JSON.parse(res);
    for (var i=0; i < entries.length; i++) {
        buildEntry(entries[i]);
    }
}

In case you’re wondering what that “load=1” business is, you’re about to find out. That’s the parameter string for the request. If you were loading this in a browser window, that would follow the “?” in the URL. It is also known as a query string. In this case, we’re sending a message to our PHP page that tells it to return an array of Entry objects.

if (!empty($_POST['load'])) {
    $query = "select * from entries";
    $result = $db->query($query) or die("Error in loading");
    $arr = array();
    if ($result->numRows() > 0) {
        while ($obj = $result->fetchObject()) {
            $arr[] = $obj;
        }
        echo $json->encode($arr);
    }
    unset($arr);
}

This sends a query to the database, requesting everything from the entries table. It puts each row in the database into its own object, and puts all those objects into an array, which it returns to the application.

You may also have noticed that for each post that is in the database, it calls a function called “buildEntry”. What that function does is simple: it builds an entry and puts it onto the page. There are two ways to put something onto the page; one way is to use an element’s innerHTML property to simply place HTML into an object and have the browser render it. The other way is to build elements using the DOM’s createElement and appendChild functions. For the buildEntry function, we’ll be using the latter.

We create the div for the entry, then its author span, its entrypanel span, and its bottom div, populate all of them with the proper content, and put it into the page by appending it to the outer div (defined in the HTML).

function buildEntry(entry) {
    var outer = document.getElementById("outer");
    
    var div = document.createElement("DIV");
    div.id = "entrydiv_" + entry.id;
    div.setAttribute("class", "entry");
    
    var top = document.createElement("DIV");
    
    var a_span = document.createElement("SPAN");
    a_span.setAttribute("class", "author");
    var at = document.createTextNode(entry.author);
    var tt = document.createTextNode(entry.time);
    a_span.appendChild(at);
    a_span.appendChild(document.createElement("BR"));
    a_span.appendChild(tt);
    top.appendChild(a_span);
    
    var t_span = document.createElement("SPAN");
    t_span.setAttribute("class", "entrypanel");
    var del = document.createElement("input");
    del.type = "submit";
    del.value = "Delete";
    del.onclick = function() {
        delEntry(entry.id);
    }
    t_span.appendChild(del);
    top.appendChild(t_span);
    
    var bottom = document.createElement("DIV");
    bottom.setAttribute("class", "bottom");
    var d_span = document.createElement("SPAN");
    d_span.innerHTML = entry.data;
    bottom.appendChild(d_span);
    
    mostRecent = entry.id;
    
    div.appendChild(top);
    div.appendChild(bottom);
    outer.appendChild(div);
}

Now that we can load all the pre-existing entries, we need to be able to add our own. The single button isn’t really going to allow that, so we need a function that will expand the add_div div so that new posts can be entered. openAdd() uses the innerHTML property to build its HTML.

function openAdd() {
    var add_div = document.getElementById("add_div");
    var ih = "";
    ih += "<input type='submit' value='Cancel' onclick='closeAdd();' />";
    ih += "<input type='submit' value='Save' onclick='addEntry();' /><br />";
    ih += "Author: <input type='text' id='add_author' /><br />";
    ih += "<textarea id='add_data' cols='100' rows='15'></textarea>";
    add_div.innerHTML = ih;
}

If you can open it, you should be able to close it, so we have a corresponding closeAdd() function.

function closeAdd() {
    var add_div = document.getElementById("add_div");
    add_div.innerHTML = '<input type="submit" value="Add new" onclick="openAdd();" />';
}

As you see above, the openAdd function calls addEntry() when its button is clicked. We now need to write that function. We get the values of the author and data fields in the add_div, and send it off to the PHP page using XMLHttpRequest. For this we want to use asynchronous processing, so that the page doesn’t get slowed down when we click the Save button. This function also updates the global mostRecent variable, which is always the id number of the post most recently added to the page.

function addEntry() {
    var req = xmlhttp();
    var author_e = document.getElementById("add_author");
    var data_e = document.getElementById("add_data");
    
    var e = new Entry();
    e.author = author_e.value;
    e.data = data_e.value;
    var param = "insert=" + JSON.stringify(e);
    req.open("POST", "graffitisub.php", true);
    req.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
    req.send(param);
    req.onreadystatechange = function() {
        if (req.readyState == 4) {
            if (req.status == 200) {
                var res = JSON.parse(req.responseText);
                
                buildEntry(res);
                data_e.value = "";
            }
        }
    }
}

Our PHP now has to be updated to support this. We build our query from the passed Entry object, ship it off to the database, and (this part is kind of tricky) send another request to the database to get the entire object that was just inserted. We return this object to the application. The reason we do this is because we want to get the time and id number of the post, which are only known after it has been inserted. Notice above that the addEntry function is designed for this.

if (!empty($_POST['insert'])) {
    $input = $_POST['insert'];
    $value = $json->decode($input);
    $insQuery = "insert into entries (author,data) values (\"".$value->author."\", \"".$value->data."\")";
    $insResult = $db->query($insQuery) or die("Error in query");
    $query = "select * from entries where (id=".$db->lastInsertRowId().")";
    $res = $db->query($query) or die("Error in query 2");
    if ($res->numRows() > 0) {
        $obj = $res->fetchObject();
        $value->id = $obj->id;
        $value->time = $obj->time;
        echo $json->encode($value);
    }
}

Now that we can add posts, we need to be able to delete them. We wouldn’t want our page to get cluttered up with useless crap (especially when we’re testing it out, right?). The delete button on each post calls the delEntry function. This sends a request to delete an entry to the PHP. We use asynchronous processing here because we don’t want to block the rest of the program, but it doesn’t need a callback because nothing is returned from a delete request.

function delEntry(id) {
    var req = xmlhttp();
    var param = "delete=" + id;
    
    req.open("POST", "graffitisub.php", true);
    req.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
    req.send(param);
    //don't need to set a callback, because nothing needs to be done after
    //the post has been removed.
    
    removeEntry(id);
}

The PHP code to handle a deletion is probably the simplest thing you’ll have to do. It just takes the id number you sent it from the application and deletes it from the database.

if (!empty($_POST['delete'])) {
    $delQuery = "delete from entries where (id=".$_POST['delete'].")";
    $delResult = $db->query($delQuery) or die("Error in deletion");
}

Once the post has been deleted from the database, we have to remove it from the page. We do this by calling the removeEntry function. This updates the global deletions array (which is supposed to match the deltracker table) and removes the post from the page using the DOM removeChild function.

function removeEntry(id) {
    deletions[deletions.length] = id;
    var outer = document.getElementById("outer");
    var div = document.getElementById("entrydiv_"+id);
    if (div) {
        outer.removeChild(div);
    }
}

Keeping the different clients sync’d with the database and each other is the hardest part of this application. We’ll do it with a check to the database once per second for each client (the timer uses a global timerID variable). We send the most recently added entry id to the database, and it responds with a carefully crafted array. There are always two items in this array. The first is an array of new entries added after the one most recently added by this client (this will often be empty). The second is another array of entry id numbers that have been deleted. This mirrors the deltracker function.

function checkForEntries() {
    var req = xmlhttp();
    
    if (timerID) {
        clearTimeout(timerID);
    }
    
    var param = "last=" + mostRecent;
    req.open("POST", "graffitisub.php", false);
    req.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
    req.send(param);
    if (req.responseText) {
        var res = JSON.parse(req.responseText);
        for (var i=0; i < res[0].length; i++) {
            buildEntry(res[0][i]);
        }
        mergeDeletions(res[1]);
    }
    
    timerID = setTimeout("checkForEntries()", 1000);
}

The PHP for this is fairly complicated. It has to create the three arrays I mentioned above, check the instracker table for new entries, get all their id numbers, get those posts from the entries table, and put them into their positions in the arrays. It then needs to return the deltracker array.

if (!empty($_POST['last'])) {
    $last = $_POST['last'];
    $totalArr = array();
    $insArr = array();
    $delArr = array();
    
    $trackQuery = "select * from instracker where (entry=".$last.")";
    $trackResult = $db->query($trackQuery) or die ("Error in tracker");
    if ($trackResult->numRows() > 0) {
        $track = $trackResult->fetchObject();
        $tq2 = "select * from instracker where id>".$track->id;
        $tr2 = $db->query($tq2) or die ("Error in tracker");
        if ($tr2->numRows() > 0) {
            while ($obj = $tr2->fetchObject()) {
                $eq = "select * from entries where id=".$obj->entry;
                $er = $db->query($eq) or die ("Error in tracker");
                if ($er->numRows() > 0) {
                    $o2 = $er->fetchObject();
                    $insArr[] = $o2;
                }
            }
        }
    }
    
    $dq = "select * from deltracker";
    $dr = $db->query($dq) or die("Error in deltracker");
    if ($dr->numRows() > 0) {
        while ($obj = $dr->fetchObject()) {
            $delArr[] = $obj->entry;
        }
    }
    
    $totalArr[] = $insArr;
    $totalArr[] = $delArr;
    echo $json->encode($totalArr);

    unset($totalArr);
    unset($insArr);
    unset($delArr);
}

When it gets the data back from the database, the program now has to make sure everything is sync’d. It loops through the new entries and builds each one. It also needs to take the deltracker array (the second item in the returned array) and merge it with the deletions array (global). The mergeDeletions function simply loops through the given array, starting at the index that would be past the end of the deletions array, and removes the entries specified.

function mergeDeletions(del) {
    for (var j = deletions.length; j < del.length; j++) {
        removeEntry(del[j]);
    }
}

We’re almost done now. The only thing left is to write the init function, which will run when we load the page. It needs to initialize the global deletions array, load the existing posts, and then start checking for new ones.

function init() {
    deletions = new Array();
    loadEntries();
    checkForEntries();
}

I mentioned the three global variables we needed as we went along. They are defined here:

//global variable for the timer
var timerID;

//global variable to track the most recent received id
var mostRecent;

//global deletions array, to keep track of all the deletions made
var deletions;

Recap

If you’ve followed along and built this program, it should now be working for you. Clearly, using XMLHttpRequest and JSON, you can do things with web pages that weren’t possible at all just a few short years ago. This Graffiti program is one such page. Previously, if you wanted to do something like this, you would have had to write a Java applet or a Flash program, both of which require clunky plugins (which I personally hate). Now you can do it with Javascript, which ends up being much nicer for the user.

Beyond the application itself, what this tutorial should have shown you is a kind of development process. The process goes from the IDEA phase, where you sit and dream up what you want your program to do, to the DATA phase, where you define your data structures (including the database), to the INTERFACE phase, where you write your HTML and CSS and decide how the program is going to look, to the FUNCTIONALITY phase, where you write your Javascript (and PHP) and give your program the desired behavior.

To Do

Make it look nicer. It’s either a forum or a chat, each of which have their own display requirements.
New posts come in at the bottom of the page. This isn’t always what you want. So update the program so it prepends the new posts (puts them at the top). And make it an option!
Think about users … what will have to be done to this program to create a login system so that only the author of a post can delete it?
Dynamically change the timeout interval based on usage to maximize efficiency.

HTML Escaper

2005-07-24T12:59:00.000-05:00

Yesterday I posted a tutorial, and the code I put up was displayed in images. I was not satisfied with that solution, because images are not searchable and also you won't be able to copy and paste the code. So I needed to find some way to display my code on this blog. It automatically parses HTML, so if I just put the code up, it would render instead of displaying the code. So I decided to write a small program to escape HTML so it can be displayed on this blog.

It's fairly simple, and probably not robust enough at this point for general use. Basically all it does is change < to <, > to >, tabs to four  's, spaces to  , and newlines to <br />. This should be good enough for most code, but if you need it do more things, it's easy enough to update the function.

What I want to do is have a textarea on the page where you can put in your HTML code, and click a button which will escape it. Then it takes the escaped code and gives it to you in a couple of ways. First, obviously, you will need the code in a form that you can copy and paste it into an HTML form and it will display properly (after all, that's the point). To do this, I've made a second textarea that will hold the escaped text. But that escaped code looks like gibberish, really, and it's not easy to tell how it'll look. So this program also renders your text at the bottom of the page, so you can see what it will look like once it's been escaped.

Here's the HTML:

<textarea id='source' rows='20' cols='40'></textarea>
<input type='submit' value='Escape' onclick='do_escape();' />

<textarea id='resultcode' rows='20' cols='40'></textarea>
<div id='result'></div>

(I used the Escaper program to generate that, in case you were wondering if it works. It does not make the border or change the font, I had to do that manually.)

As you can see, the interface code for this program is quite simple, but it works very well. The Javascript is equally straightforward. The do_escape() function calls the html_escape() function once, and puts the same escaped text into both the "result" and "resultcode" fields.

    function do_escape() {
        var esc = html_escape(_$("source").value);
        _$("result").innerHTML = esc;
        _$("resultcode").value = esc;
    }

The html_escape() function does the meat of the work in this program, but is actually very easy to understand, and is even easier to implement. Basically, it takes a string and goes through it character by character. If the character being examined doesn't need to be escaped, it is just passed along. Otherwise, it is replaced by its escaped version. The fully escaped string is returned once it is ready.

    //need to convert the given string into html-escaped form
    function html_escape(string) {
        var rval = "";
        for (var i=0; i < string.length; i++) {
            switch (string[i]) {
                case "<":
                    rval += "&lt;";
                    break;
                case ">":
                    rval += "&gt;";
                    break;
                case "\t":
                    rval += "&nbsp;&nbsp;&nbsp;&nbsp;";
                    break;
                case " ":
                    rval += "&nbsp;";
                    break;
                case "\n":
                    rval += "<br />";
                    break;
                case "&":
                    rval += "&amp;";
                    break;
                default:
                    rval += string[i];
                    break;
            }
        }
        return rval;
    }

You can put this code onto your local server and leave it going in a tab in the background for every time you need it. Next time you want to post code in a forum, you don't have to wonder if the forum will render your code as text or as HTML. If you want it displayed as code, just escape it yourself!

I would have loved to be able to put a demo of it right onto the page, but it seems that that won't work. You'll just have to put it on your own machine and run it from there. Enjoy!

Interest Rate Calculator

2005-07-23T17:34:00.000-05:00

A good program to write when you're just starting out should be simple enough to understand quickly yet demonstrate enough features of the language and environment that you will have a base of knowledge to fall back on and build off of when you start writing other, more complicated, programs. One of the first I typically write is an interest rate calculator. It's nice and simple, you use plenty of input, and the formula uses a couple of features of the math libraries, so you can see what the language can do.

The first thing to do when designing a new program (after you've decided what program to write, which we've just done) is to design the interface. Many times, if you can design a good interface to a program, which should work in an obvious way, the features you need to implement become obvious and the code pretty much falls out of the interface. (That's why Cocoa programs are made by designing your interface, connecting actions and events, and then writing code to handle the interaction with the interface.)

To calculate interest, you need the initial amount, the interest rate, and the time passed. We'll need a text field for each of those, another text field to show the result, and a button to initiate the calculation. Simple enough, I'd say. I usually don't like to do this, but this works pretty well with a table layout. So here's the HTML code for the interface:

If you type that code into your favorite editor, and save it into an .html file, you'll be able to load it in your web browser. It looks exactly the same as your finished program will. So now it's time to implement the functionality!

To save time and code space, I typically include a shortcut function in my projects that allows me to type less when I need to get a reference to an element on the page. "document.getElementById()" will get a reference to an element, but it's an awful lot of typing if you're going to need to do it a lot. Here's my time-saving function:

It's probably a good idea to put this function in a separate .js file that isn't specific to any one project. That way, you can easily take all your personal utility functions along at the beginning of a new project. And if you're doing things right, you'll start to amass a few things in your lib.js file.

Now that we have that in place, we should write the calculate() function. It needs to get the values of the initial amount, the interest rate, and the number of years, then it needs to use those values to calculate the interest earned, and put that final value into the "Total" field.

The formula for calculating interest, compouded yearly: TOTAL = INITIAL * (1 + RATE) ^ YEARS

Once we have that formula, the function is trivial:

The first three lines get the values of the input fields and parse them into numeric values. This ensures that it won't try to calculate a total dollar value if there is, for example, a letter or some other non-numeric character thrown into the works. The initial dollar amount and the interest rate are decimal values, so we use parseFloat(), but the number of years must be a whole number, so here we use parseInt(). Since we want the interest rate to be entered in a human-friendly way (5% interest, rather than 0.05), we will divide the rate value by 100.

Then we calculate the total value based on our formula, and place it into the "total" text field. This is the output of our program, and it immediately shows up on the page. No page refreshes here.

Notice that there are only two decimal places in the total field. This is exactly what we want, of course, because monetary values are displayed with only two decimal places. In order to make sure there are no more than that, we use Javascript's Math.round() function to round the value, multiplied by 100, down to the nearest integer, then divide it by 100 to get the decimals back. (For future reference, this technique works for any number of decimals, just change the magnitude of the multiplier/divisor, which in this case is 100.)

The Math.pow(x,y) function does about what you'd expect it to do: it returns x^y.

Now that program was simple enough, right? And it doesn't really scratch the surface of what you can do in a modern web application. Here are some things to try out on your own, to improve the interest calculator program:

Give it some style! Right now, it's a fairly ugly program. It uses the system default font and font size. Create a CSS file and get to work on making it a bit prettier.
Have you noticed what happens when the number of cents in the total field is divisible by ten? And what if there are no cents at all? Try to update the calculate() function so that it will always give the proper number of trailing zeroes.

Now this was a fairly simple little program, but I hope it helped get your feet wet. The next one will be a bit more complicated and interesting. It's coming soon, so don't go far!

Introduction

2005-07-23T17:23:00.000-05:00

Hi there. I'm Sean, and I do some programming. A lot of the things I learned have been from the web, and many of those were in some random guy's blog about some cool/stupid thing they did. So I've decided to be one of those random internet people, and help out people new to programming with some of the cool/stupid things I've done.

But on thinking about it, I don't really consider much I do to be particularly cool, and it's not even that stupid. So why not dedicate to something else? I have decided to make this not so much a showcase for things I've done as a place for people to learn. I'll try to publish reasonable tutorials on various things ... for the most part they will be on modern web applications (or AJAX, if you're into the term), as that's what I've been spending most of my time on lately.

And on my larger projects, I'll try to post updates on my progress, just so that anybody who finds themselves interested in them (for whatever reason) can follow along and make fun of my slow rate of progress or lack of glitzy features. Or something.

The first real post should be up shortly. Don't hold your breath (it's not good for you).