Programming is a wonderful mix of art and science; source code is both a poem and a math problem. It should be as simple and elegant as it is functional and fast. This blog is about that (along with whatever else I feel like writing about).

Tuesday, June 06, 2006

Installing TurboGears on Ubuntu

I recently came across a new web framework called TurboGears that I think seems pretty interesting.

The basic concept is to integrate several different elements of a framework into one simple, easy install. It uses Python's SQLObject to interact with the database (SQLite, MySQL, PostgreSQL, and Firebird are supported), Kid for templating, the MochiKit Javascript library, and CherryPy for the actual server functionality. They attempt to blend front-end and back-end development in a way that seems like it would allow a developer to be much more productive. Anyhow, it looked good to me and I decided to install it.

The installation is supposed to be easy, and for the most part it is. However, at least if you're using Ubuntu Linux (I'm running the newly released Dapper Drake), there are a few things you need to make sure you have installed.

First, you need python2.4, and python2.4-dev, and libc6-dev. These can all be gotten from the apt repositories.

sudo apt-get install python2.4-dev libc6-dev

should do it for you.

Then you simply follow the instructions here, which I will summarize:

1. Download the ez_setup.py script.
2. Run sudo python ez_setup.py -f http://www.turbogears.org/download/index.html --script-dir /usr/local/bin TurboGears


After that, it should just blow through the installation and you'll be ready to rock. Now I'm off to see what I can do with it!

Friday, May 19, 2006

My Own JSON/PHP Web Services: SSWebService

Web Services. It's a new buzzword. Everyone needs them, because they'll make everything better, right?

Well, nuts to that.

What web services really amount to is nothing more than remote procedure calls that you can make from a web browser. That's cool enough. You make a request, and get a response. Instead of sending HTML that you render, the web service just returns the data. Most web services today use XML to package that data, though a few places -- like Yahoo -- are starting to use JSON instead.

JSON has a few major advantages over XML. First, it is smaller: less bandwidth is wasted by 1.0 crapola. Second, it is very easy to parse it in Javascript. In fact, JSON stands for "Javascript Object Notation," and is native to Javascript. Parsing a JSON-encoded object takes very little time. I've written Javascript applications that need to parse an XML file, and let me tell you, parsing XML in Javascript is not trivial and is a major performance bottleneck.

For an upcoming project, I wanted to develop a web service. There will be PHP interacting with the database, and I want to make simple calls from the browser. The frontend will be a Javascript application, and it is not page-based at all. So I needed a way to pass data back and forth between the browser and the server simply and quickly.

I investigated using SOAP, but was underwhelmed. I decided to design my own web service in PHP, which would use JSON for all its data passing.

The first thing I needed was to be able to register functions with the service, such that only functions that are actively made public are available. I quickly put together a PHP class with a "methods" instance variable and a "register" instance method.

Each function will take a single parameter, but this parameter is expected to be an arbitrary JSON object, which can hold any amount of data. So rather than calling a function like sum(1,2), you would call the function sum($par), where $par = {one:1, two:2}. Then you parse that parameter object in the sum() function and handle it appropriately. It's an extra step and a bit more to remember, but I feel that it adds plenty of flexibility. For example, I wrote a sum function using this brand of parameter passing, and it was utterly trivial to make it work for an arbitrary number of values.

function sum($par) {
    $sum = 0;
    foreach ($par as $key=>$val) {
        $sum += $val;
    }
    return array("sum"=>$sum);
}


Oh yes. I forgot to mention how the return values work. Rather than simply returning a single value, I will always be returning an associative array/object (the PHP associative array type is most closely related to the Javascript object type). This way I know what the return value is (it's called the "sum"), and for more complicated functions I can return as much data as I want.

In order to serve a request, which should be a POST with two parameters ("method" and "param"), I need to decode the param parameter from JSON into a PHP-native object, check that the method is registered, and then call the method with the parameter. I then take the function's return value, JSON-encode it, and return it to the browser.

Here is the code for the web service (sswebservice.php):
<?php

require_once("JSON.php");

class SSWebService {
    var $methods;
    var $json;
   
    function initialize() {
        $this->json = new Services_JSON(SERVICES_JSON_LOOSE_TYPE);
    }
   
    /*
    add a function to the server, so it can be accessed
    */
    function register($name) {
        $this->methods[$name] = true;
    }
   
    /*
    set a registered function to be inaccessible
    */
    function deregister($name) {
        $this->methods[$name] = false;
    }
   
    /*
    execute the given method, passing its single parameter
    JSON-encodes the return value, which should be an object or associative array
    */
    function call($name, $param) {
        if ($this->methods[$name] == true) {
            $evalstring = $name."(\$param);";
            eval("\$rval=".$evalstring.";");
            return $this->json->encode($rval);
        }
    }
   
    /*
    decode the JSON param into a native object, and call the given method
    return the JSON-encoded object to the browser via echo
    */
    function serve($method, $param) {
        $obj = $this->json->decode(stripslashes($param));
       
        if ($this->methods[$method] == true) {
            $res = $this->call($method, $obj);
        } else {
            $res = $this->json->encode("Not a registered function.");
        }
       
        echo $res;
    }
}

?>


And here is a demonstration web service that uses it (jsontest.php):
<?php

require_once('sswebservice.php');

$server = new SSWebService;

$server->initialize();

function helloWorld($par) {
    return array("string"=>"Hello, world!");
}

function hello($par) {
    return array("string"=>"Hello, ".$par["name"]."!");
}

function sum($par) {
    $sum = 0;
    foreach($par as $key=>$val) {
        $sum += $val;
    }
    return array("sum"=>$sum);
}

$server->register("sum");
$server->register("helloWorld");
$server->register("hello");

$method = $_POST["method"];
$param = $_POST["param"];

$server->serve($method, $param);

?>


As you can see, this web service has three available functions: sum, hello, and helloWorld. Now I just need a way to call them from the browser.

I tend to use the Prototype/Scriptaculous libraries for my web development, so I first whipped something up using Prototype's Ajax.Request object, but that turns out to be quite a bit of code every time I want to make a call to my web service. So I created a very simple class with nothing in it but a class method (ssclient.js):

function SSClient() {

}

SSClient.call = function(url, method, obj, callback) {
    var param = "method=" + method + "&param=" + obj.toJSONString();
    new Ajax.Request(url, {
        parameters: param,
        onSuccess: function(req) {
            var rval = req.responseText.parseJSON();
            callback(rval);
        },
        onFailure: function(req) {
            alert("Call failed.");
        }
    });
}


So when I want to make a call to my web service, I simply need to supply the URL, the method I want to call, the Javascript object I want to send, and a callback function to execute when it returns. The callback function should expect to receive a JS-native object, since I parse the JSON-encoded object before calling the callback function.

Here is an example of the code to call the web service:

function button1() {
    SSClient.call("jsontest.php", "sum", {
        one: 1,
        two: 6,
        three: 9
    }, sum_callback);
}

function sum_callback(r) {
    alert(r.sum);
}


As you can see, the SSClient.call() function is very easy to use, and I can create an anonymous object to pass along. In this case, the callback function simply pops up the return value. Note that it is accessed by r.sum, which is because of how I set up my return values in jsontest.php. Neat.

This requires Prototype, json.js from http://www.json.org/json.js, and JSON.php from http://mike.teczno.com/JSON/JSON.phps.

And now I'm off to build an application with my new web service!

Tuesday, May 02, 2006

Intel's Cheap Laptop Proposition

By now we've all heard quite a bit about MIT's One Laptop Per Child initiative, and its sub-$100 laptop. In order to reach such a low cost, the machine uses a slow and inexpensive AMD Geode processor, and runs Linux. Naturally, this has led Microsoft and Intel executives to call it "a gadget" and to insist that it is inadequate for children in developing nations.

Now Intel has backed up their statements with a plan to produce their own version of a cheap laptop computer for students in developing nations, called Edu-Wise. They claim that it will cost $400 or less, and will run some version of Microsoft Windows on an Intel processor faster than the AMD in MIT's inexpensive offering.

Another key differentiator between the two machines is that the MIT laptop uses 802.11 b/g wireless mesh networking to provide connectivity between machines, whereas the Intel laptop will include a WiMAX chip, allowing it to connect from much farther away from any access points and removing the requirement that it be used in a classroom setting.

While I understand Intel's claim that "that Wi-Max deployments will leapfrog stages of development in the nonindustrialized world," I find it somewhat dubious that the nonindustrialized world will gain access to WiMAX deployments any time soon. After all, there have only been minimal deployments of the technology even in industrialized areas, and widespread adoption does not appear to be imminent. The inclusion of WiMAX in an ultra-cheap device before it is even an option on high end equipment is an indication that this is little more than a stunt to keep developing nations from signing on with MIT's proposal, which happens to actually exist.

Another question is the actual cost. MIT promised a sub-$100 notebook, and have reported that it currently costs them $135 to build one. They haven't made their goal yet, but they're pretty close. On the other hand, Intel is starting from the $400 price point (which, by the way, is already what cheap laptops cost on the US market today), and their recent record with hitting price targets is not good. For example, the recent UMPC was projected to cost around $500, but is actually available for sale in the $1100 range. Engadget expects that Intel's new machines will cost close to $750, rather than $400. So the questions Intel has to answer are:
  1. How is this machine an improvement over those that already exist at the same price point?
  2. Is the announced price point remotely possible?
  3. Is $400 affordable for students in developing nations?
  4. Do students in developing nations really need more processing power than the AMD Geode can offer?
  5. Do students in developing nations really need the expensive Microsoft Windows installed on their computers, or would they be better served by a Linux-based solution which would drive costs down?
At this time, none of these questions have been answered. The closest Microsoft or Intel will come to addressing these issues is to publish statements such as
"We don't think you cross the digital divide with old technology," he said. "It doesn't need exotic technology and it runs real applications."
made by Paul Otellini, which implies that they want to somehow use new (expensive) technology to address the demands of this new (poor) market.

This venture is doomed to fail. It is nothing but another scare tactic from Intel, designed to cripple its competitors without having to produce a competing product. They did the same thing with the Itanium, and it worked. Now they are doing it with Edu-Wise. A real product may or may come of this. If it does, it probably won't meet the announced price point. And even if it somehow does, it will still be too costly for the market they are trying to address. The real test of whether this is a successful move by Intel will be whether or not developing nations delay their purchases of the OLPC in favor of waiting for Intel's notebook. If that happens, Intel and Microsoft have already won (and they know this better than anyone).

Monday, May 01, 2006

Democracy - Television for the Internet Age

Last week I came across a rather new video player with an interesting twist: it is meant to be a platform for distributing internet-based television. Democracy's goal is to be a democratization of the media, allowing anyone with a video camera and an internet connection to put together their own channel, producing art, entertainment, or news.

Rather than simply playing videos downloaded elsewhere (this is, in fact, not possible in the current player, though I have been told that it is planned for the next release in mid-May), in Democracy you subscribe to channels and videos are downloaded automatically as they are made available. It is the work of a moment to begin watching in a window or in full screen mode. And as an added bonus, all downloaded videos play one after the other so your television experience is not interrupted by having to start the next video.

In an effort to curb hard drive use, Democracy has implemented an expiration system based on Tivo's, in that downloaded videos expire in 6 days (by default, though that number can be changed), unless they are explicitly saved for longer. And the website reminds you that you can always download expired videos again if you want to see them.

I have only used the program to a somewhat limited extent thus far, and have not fully explored the channel options. However, the content I have found is mostly good, and in one case is better than the options on television (Channel Frederator). I am certainly looking forward to the creation of new and interesting video content, and the distribution opportunities being made available by standards such as RSS and innovative new players such as Democracy.

While I don't think it has the potential to replace the networks, the ability for anyone to produce their own news should force the news networks to remain diligent and the ability for anyone to produce and publish their own shows should force the studios to remain creative. This democratization of the media, by putting the tools into the hands of the individual, can only be a good thing.

Democracy is open source, released under the GPL, and is available for Windows, Mac OS X, and Linux.

Google Complains About IE7's Default Settings

Just saw this article about Google officially complaining that Microsoft will set IE7 to use MSN search by default. The claim is that it is unfair to limit choice and drive users to MSN rather than Google.

This is a specious argument coming from Google, considering the fact that Google search is the default in Firefox, Opera, and Safari. The only difference would be that Microsoft owns both the browser and the search engine ... but if MS owns both then they should be allowed to link them as they see fit, yes? It certainly wouldn't make any sense to require Google search to be the default on IE7, when MSN search is the only option Microsoft has if they want to default to a non-competitor.
"The market favors open choice for search, and companies should compete for users based on the quality of their search services," Marissa Mayer, the vice president for search products at Google, told the Times. "We don't think it's right for Microsoft to just set the default to MSN. We believe users should choose."
Yes, users should be able to choose. But you still need a default, and for once Google isn't going to be that default. Perhaps that's what's gotten under their skin.

For the record, Microsoft did what they could not to come off well either:
Microsoft told the Times that giving users of the new version of IE an open-ended choice could add complexity and confusion to the browser set-up process, while offering a few options would be arbitrarily limiting.
Aside from once again demonstrating their lack of respect for their own users, how arbitrarily limiting can offering a few choices possibly be? The options would probably be:
  1. MSN Search
  2. Google Search
  3. Yahoo Search
  4. Add another search engine...
Perhaps all Google was trying to do was to raise awareness of the issue, and to force Microsoft to offer options. However, they should avoid whining like this, lest they lose their darling status. Google should get back to their roots: winning by being better, rather than by having better lawyers and journalist shills.

Friday, February 17, 2006

Netflix Online Video Distribution

Netflix has a problem, in that it's getting harder and harder for them
to make a profit. They break even on a renter if he/she gets 5 DVDs
per month. So, they need to find some way to augment their service
somehow.

One possible way they could do this is to start getting into the
online distribution game. They have all these DVDs, and they're a
known commodity for the studios. They would need to license the movies
for online distribution, but the fact that they also distribute DVDs
should help in those negotiations. Then, once they can offer the
movies online, they have a few options as to how to get them to their
customers.

One possibility is to have each movie streamable from the Netflix
website as long as the customer has the DVD checked out. So when they
ship you a movie, they also make available a stream so you can watch
it online. The problem with this is that they then incur the cost of
shipping and of online distribution, making it somewhat non-viable.

A second possibility is to offer an added level of service that
includes streaming movies, allowing people to watch a movie instantly,
without having to wait for the disc to come in the mail. This would
free Netflix from having to ship the movie, saving them some money.

A third possibility is to offer the movies for download, for a fixed
cost per movie. This way, Netflix would be able to recoup the cost of
the transfer in the purchase fee. However, they would have to
implement some kind of DRM scheme.

They would want to build the DRM system so that it's as fair as
Apple's FairPlay. Since the best codec for distributing video online
at the moment is H.264, and they would want their files playable on
the iPod anyway, they should try to make a deal with Apple. Apple lets
the iPod play Netflix's DRM'd video files and possibly shares
bandwidth, licensing and advertising costs, while Netflix only
supplies video in this format, helping to ensure iPod dominance. It
would be a win for both companies, while allowing them both to do what
fits within their business model.

For the client side, Netflix would need an application to play the
streamed/downloaded video. There are a few possibilities here.
1) They could roll their own client which plays H.264 files and
decodes the Netflix DRM. They would probably need to make it talk to
the iPod somehow, but this would presumably be part of the deal with
Apple. Making it cross platform is important.
2) They could do some kind of bundling deal with Apple for Quicktime.
This would give them the benefit of getting advertisement through
Quicktime even for non-subscribers, but would be difficult to do and
would get into the thorny full screen issue (do you get Quicktime Pro
when you subscribe to Netflix?).
3) They could update an open source client, such as VLC. VLC already
has support for H.264, and for streaming video over the network, as
well as being very cross platform. There are two issues here, which
are related. The client needs to be able to decode the DRM, and VLC is
an open source program. The way I see it, DRM doesn't really work when
the code is open. Even if the technical aspect were completely figured
out and the DRM was uncrackable (dubious, considering that all DRM has
been cracked), the studios would never agree to it. Say what you will
about open source (I know I like it), it doesn't instill confidence in
the major content owners.

A solution could be developing a program that opens the
downloaded/streamed file, decodes the DRM, and passes the video along
to VLC, or any other video player. This greatly simplifies the
software from Netflix's perspective and gets them away from having to
open source the decoder. However, there could be a problem with the
player receiving the unencrypted file, with no way to prevent the user
from saving the stream to the hard drive. (Of course there will be
people who figure out how to do it, but the point of DRM is to make it
at least somewhat difficult, so the process doesn't enter the
mainstream.) Because of this issue, developing their own player seems
to be the best option.

Netflix needs to do something to help the bottom line, other than
attempting to screw their customers. Withholding discs from their
happiest customers is not a good way to keep those customers happy.
This would be a good way to make more money by offering more services,
rather than fewer. And partnering with Apple would been a boon for
both companies; in fact, it's difficult to tell who would benefit
more.

Obviously, I have no inside information about this. I say it because
it's what I would do if I were in a position of authority at either
Netflix or Apple. And as a user of both, I'd certainly like to see it
happen.

Friday, February 10, 2006

Apple Making Intel Uncomfortable?

Just saw a story over on Ars about Apple's new ads making Intel uncomfortable. Basically, the premise is that some analysts are worried that Apple's ad (which insults Intel's other partners like Dell & HP by calling them "dull little boxes") will alienate Intel from their other, larger customers.

They cite Intel's "public love-fest" with its newest customer, the 1000 employees dedicated to The Switch, and the fact that Apple got the first Core Duo chips as other reasons Intel may be in trouble with its other customers.

But I'd like to point out another problem with their claim. Intel's in the driver's seat in these relationships. Dell only uses Intel chips. And they can't even threaten to go to AMD, because Intel can pull their sweet prices out from under them, or even stop selling them chips entirely, leaving Dell to find some way to get 10 million processors per quarter from AMD. I'd be willing to venture that AMD isn't capable of supplying that many processors. And the same goes for HP, Lenovo, and everyone else. If you want to offer AMD chips, then you can, but you have to stick with Intel for the same reason Apple chose Intel: manufacturing capability. Intel is the only place to go if you want a guaranteed supply.

So I'd say that Intel has nothing to worry about. In fact, they're probably excited that Apple is lighting a fire under the other computer makers. The CPU-comparison is now gone, so the difference maker is not processing power any more. That puts an onus on the designers of the whole system to put together a better offering ... and whether it's Apple's Front Row or Dell/HP/etc VIIV, Intel stands to benefit. A lot.

And I think everybody knows it. That's why none of the computer makers are saying they're insulted. Dell's spokesperson had this to say:

"As far as their marketing goes, that's marketing," said Dell spokesman Jess Blackburn. "We continue to have a very solid relationship with Intel and nothing has changed in that respect. We probably are their largest customer; we shipped 10 million systems in the fourth quarter. I don't think any supplier is going to ignore a customer that is building that many systems and using that many of their parts in them."

My guess is that Intel is quietly encouraging Apple to be as insulting as they want. If Dell and HP lose customers to Apple, it's probably better for Intel. They'd prefer their chips to be part of the high end, high margin space that Apple aims for, because it means more money for them. But this entire issue is just meaningless controversy invented by analysts. I guess they have to justify their paychecks somehow.