For FreeBSD Users Only

FreeBSD byobu snack module fix:
portmaster -m WITH_PYTHON=yes devel/newt

Explanation: somebody decided that compiling python-newt by default was wrong, and broke the byobu package in the process. I’m a new FreeBSD user, so this was pretty annoying to fix. It kept bothering me that byobu-config was broken, and I eventually went into the ports collection makefile to find that there was a WITH_PYTHON option in there all along.

Of course, all the linux-based system monitors found in byobu, xmobar, etc. are broken in FreeBSD.

BTW, knowing the answer makes it much easier to find. :) (already found it twice)

An Associative Database

About a year and a half ago, I wanted a metadata database (or metabase), where I could collect all my notes about songs I’ve listened to and pictures I’ve taken.

The idea is that the photo and its associated notes are both data (metadata is also data). It’s important not to loose any of them. The hard part – for which I don’t know of an existing solution – is how to save the associations.

For multiple reasons, it is not a good idea to specify the association of datum by placing them in the same file. If I had original and cropped versions of the same photograph, I wouldn’t like to “glue” the two versions together by placing them in the same file. Also, computer programs aren’t good at guessing which files are related to which, so placing several mp3 files together in the same directory may not automatically create a mixtape. (If you’ve ever used an iPod you know what I’m talking about.)

Thus, the original idea of a metabase is now not as much about storing metadata, as it is about storing associations.
Such a database would store pairs of associated GUIDs in one table, and assorted external identifiers (filenames, hashes) for each GUID in another.

Of course, somebody has already thought of this.

Twitter Updates on Google Reader

I’m one of the unlucky people living in far-away places that don’t get Twitter updates via SMS. Recently, Twitter activity has increased among some of my friends, so I wanted to get updates without having to remember to visit the site (I’m lazy that way).

Long story short, the “friends_timeline” RSS feed that twitter offers is password protected, whereas Google Reader doesn’t support passwords (and even if it did, I wouldn’t want to put my Twitter password there). So my idea was to create a non-password protected proxy page that takes care of entering the password and returning the contents of the feed. Below is a simple PHP script that does just that. You can put it on your web server and subscribe to it using your choice of RSS reader, no passwords required.

<?php
// A simple proxy to fetch password-protected pages on services that don't
// support it. Example: twitter feed on google reader
//
// based on:
// http://developer.yahoo.com/javascript/samples/proxy/php_proxy_simple.txt
//
// Author: Jason Levitt
// December 7th, 2005
//
// Modifications: Ehud Meiri, Feb 2009

$url = 'http://USERNAME:PASSWORD@twitter.com/statuses/friends_timeline/NUMBER.rss';

// Open the Curl session
$session = curl_init($url);

// Don't return HTTP headers. Do return the contents of the call
curl_setopt($session, CURLOPT_HEADER, false);
curl_setopt($session, CURLOPT_RETURNTRANSFER, true);

// Make the call
$xml = curl_exec($session);

echo $xml;
curl_close($session);

?>

To use it for your account, change the “$url” line to match your details (you can find your NUMBER in the “RSS feed” link here).

Note that this solution is insecure (esp. on shared machines), as the password is stored in plaintext in the script and anybody who can read the script source can know your password. Be careful.

Wacom Bamboo and Debian

I’ve just purchased a new toy: the Wacom Bamboo tablet. Without any modifications, the tablet behaves in X like a (very slow) touchpad.
Luckily, kernel and xorg drivers are available, though support is not mature yet (I was expecting some gnome configuration tool).

Installing a Wacom Bamboo on Debian testing:
A. Install the following packages:
Packages that should be already on your system:
1. xserver-xorg-input-wacom (version 0.7.9.3-2)
2. linux-image-2.6.24-1-686 (or possibly 2.6.23, 2.6.22, YMMV)

Packages that you probably don’t have yet:
3. wacom-tools (version 0.7.9.3-2)

B. Tweak your /etc/X11/xorg.conf:
1. Add the following sections.

Section "InputDevice"
        Identifier      "stylus"
        Driver          "wacom"
        Option          "Device"                "/dev/input/wacom"
        Option          "Type"                  "stylus"
        Option          "USB"                   "on"
EndSection

Section "InputDevice"
        Identifier      "eraser"
        Driver          "wacom"
        Option          "Device"                "/dev/input/wacom"
        Option          "Type"                  "eraser"
        Option          "USB"                   "on"
EndSection

Section "InputDevice"
        Identifier      "cursor"
        Driver          "wacom"
        Option          "Device"                "/dev/input/wacom"
        Option          "Type"                  "cursor"
        Option          "USB"                   "on"
EndSection

Section "InputDevice"
        Identifier      "pad"
        Driver          "wacom"
        Option          "Device"                "/dev/input/wacom"
        Option          "Type"                  "pad"
        Option          "USB"                   "on"
EndSection

2. Change the following section to look something like this:

Section "ServerLayout"
        Identifier      "Default Layout"
        Screen          "Default Screen"
        InputDevice     "Generic Keyboard"
        InputDevice     "Configured Mouse"
        InputDevice     "stylus"                "SendCoreEvents"
        InputDevice     "eraser"                "SendCoreEvents"
        InputDevice     "cursor"                "SendCoreEvents"
        InputDevice     "pad"                   # no core events
EndSection

C. Configure the touch ring to act like a mouse wheel (i.e. scrolling).
1. I put the configuration in the in file /etc/X11/Xsession.d/80wacom-config_scrollpad, for lack of a better place.
Its contents:

xsetwacom set pad AbsWDn "5"
xsetwacom set pad AbsWUp "4"

D. That’s it.

More documentation and options here.

The Sentient Teeples (or Tin Peoples)

Robert X. Cringley writes:

We’re talking about tin whiskers, single crystals that mysteriously grow from pure tin joints but not generally from tin-lead solder joints. Nobody knows how or why these whiskers grow and nobody knows how to stop them, except through the use of lead solder. Whiskers can start growing in a decade or a year or a day after manufacture. They can grow at up to nine millimeters per year. They grow in any atmosphere including a pure vacuum. They grow in any humidity condition. They just grow. And when they get long enough they either touch another joint, shorting out one or more connections, or they vaporize in a flash, creating a little plasma cloud that can carry for an instant hundreds of amps and literally blow your device to pieces.

Too Many Bookmarks

TMB (Too Many Bookmarks) is my homepage script. It is intended to give quick access to my most frequently used bookmarks, from any computer I may be using. I used to have a wiki page that did this, but now TMB allows me to easily add bookmarks, assign their category, and (roughly) order them.
It is very very basic, but it works, it’s fast, and I love it.

Requirements: PHP 5.2 with SQLite 3

Download it at the TMB page.

BTW Instapaper is a great tool for bookmarking pages that you want to read later. You can flag pages as read/unread, add pages to your list with a press of a button (I might add this to TMB), and the registration is really simple.

Metadata and Tagging

We used to write important information on the back of photographs. Today they are digital, but we would still like to remember, years from now, where who and what is in those photos we took.

Wrestling the information into the filename might be a good idea. It assures us that the metadata (descriptive info) is copied along with the data (the file). However, it’s not an elegant solution for most common filesystems, since they place restrictions on the length of filenames and character sets used. Ideally, a block of descriptive metadata should replace filenames altogether.

Most modern file formats include an internal metadata format (JPEG has Exif, MP3 has ID3), and maybe the solution is to use several metadata implementations (de facto standards). But what I need is to be able to take a piece of text, an image, or a web page snapshot, and associate it to an arbitrary file. I also want the associated data to accessible from the context of that file, be it that of a text viewer, media player, or file manager.

Is there a metadata standard, or is that just an oxymoron? I mean, metadata can be, by definition, all types of data. So writing such a standard would mean to anticipate all future kinds of data.

Is metadata necessary? You can argue that there is no end to metadata and that it’s pointless. Yet we still find ourselves appending notes to images, tagging cats, and blogging about other people’s creations.

Cult movie I want to see: The Room

The Room (imdb) is a movie in the “so bad it’s good” category. Not having seen it yet, just reading the reviews in imdb and seeing the clips on Mahalo Daily got me giddy with excitement. I love how the interviewer managed to keep a straight face throughout the interview, except at the very end.

Comment excerpts:

So Independent it’s Cinematic Masturbation!
– Mroussele

The centerpiece of this filmic holocaust is Mr. Tommy Wiseau himself. Without him, it would still be the worst movie ever made, but with him it is the greatest worst movie ever made. Tommy has been described as a Cajun, a Croatian cyborg, possibly from Belgium, clearly a product of Denmark, or maybe even not from this world or dimension. All of these things are true at any one moment. He is a tantalizing mystery stuffed inside an enigma wrapped in bacon and smothered in cheese.
– Brickyard Jimmy

Bookmooch’s Search Engine

[Illustration credit Andrice Arp, courtesy of BookMooch.com]

Regarding my previous post about Twitter’s database architecture and scaling, here’s a post about bookmooch.com’s scaling troubles (bookmooch is a book swapping site).

Bookmooch had a search engine for books, using a single table that indexed each word in a book’s title/author/etc. This system was very efficient for searching (one query per word). The problem was that adding new books to the index was very costly (the more popular the word was, the longer it took to update). It was up to the point where the disk writes took as much as 20 seconds out of their alloted minute.

In short, the solution was to have two tables for the index instead of one. One table was optimized for queries (faster searches), while the other for updates (adding new books quickly). These tables would be synchronized periodically.

IMHO, the guy in charge did nothing wrong with the initial design of the search engine. There are certain characteristics of a system that are hard to predict and even then, one needs a lot of experience to correctly optimize things in advance (like the problem in the second paragraph). As in the Twitter story, profiling saved the day, but don’t forget to monitor your logs!

Funny names

Out of boredom:

1. Name for a site (only Hebrew speakers will get this): vayehi.org
2. Name for a girl: lolcat