Other

a reason to travel

From my roommate Tim’s New Zealand travelogue, that I recently discovered:

The whole cost? Twenty-five dollars (and those are New Zealand dollars, it was $13 American).

For the curious ones among you, he has strained his lateral tendons and may have inflamed some of the nerves in his arm. If it’s not better in six weeks, we’ll find another clinic and he’ll be taking care of for twelve dollars. Amazing.

The woman we were staying with described the wonders of socialized medicine and reasonable pension plans and all things New Zealand. All I could think about was how much it all cost and how it was paid for. She seemed to anticipate my question and explained “Of course it helps when you’re not paying to build nuclear weapons and fight illegal wars.” There it is. As a country, the United States has chosen war and destruction over education, medicine, science, and infrastructure. What a choice.

This is why travel is important. It allows you to see other systems, meet other people, and experience the roads not taken. It makes you a better citizen of the world and of your community. The world is a big place and you should always try to see more of it than you have. And there the lesson ends.

|

purpose

From Eyewriter:

Tony was diagnosed with ALS in 2003, a disease which has left him almost completely physically paralyzed… except for his eyes. This international team is working together to create a low-cost, open source eye-tracking system that will allow ALS patients to draw using just their eyes. The long-term goal is to create a professional/social network of software developers, hardware hackers, urban projection artists and ALS patients from around the world who are using local materials and open source research to creatively connect and make eye art.

Also see Dasher.

|

RIP, Asheem Chakravarty

Indian Ocean’s percussionist and singer, Asheem Chakravarty passed away yesterday after a heart attack. This is truly sad news; Indian Ocean is one of my favorite bands and I have been to many of their concerts. Losing a member of a band as tight-knit and flawless as Indian Ocean is quite a big blow. My heart goes out to Asheem’s family and friends, Sushmit, Rahul, and Amit.

Asheem’s percussion was the backbone of the band, and if you listen carefully you’ll notice that it only stops for one thing — his vocal solos (This is why having 2 percussionists in a band is awesome). My most distinctive memory of Asheem is his solo singing of a sloka that opens a song from the Kandisa album, Khajuraho:
Brahmanandam parama sukhadam kevalam jnanamurtim dvandvaateetam gagana sadrusam tatvamasyadi lakshyam ekam nityam vimala machalam sarvadheesaakshibuutam bhaavaateetam triguna rahitam sadgurum tam namaami.

(Rough translation from here — “I salute to that true teacher who is the source of eternal bliss, supreme happiness, who has true wisdom who is beyond the dualities, who is infinite, whose attention is always on the divine, who is unique, eternal, pure, steady, and who sees with the eyes of wisdom who is beyond thoughts and beyond three faculties.”)

I am thankful for the wonderful rendition of slokas that are thousands of years old into your songs and making this accessible to our “pop-rock” generation. You may not be with us any more, but your voice and your tabla will echo in the universe forever. Rest in peace.

My year in cities, 2009

To continue the tradition of two years (2007, and 2008), here is a list of cities I have been in 2009. It seems quieter than the last few, but I realized some of them were multiple trips and extended stays. I hope 2010 is a longer list!

  • San Francisco, CA
  • Bay Area, CA (Sunnyvale, Mountain View, etc)
  • Assilomar, CA
  • New York, NY
  • Providence, RI
  • Paris, France
  • Lyon, France
  • Boston, MA
  • Detroit, MI
  • Ann Arbor, MI

Getting middle-click to work for tab browsing in Mac OS X

If you’re a recent convert to Mac OS X (Tiger / Leopard / Snow Leopard / etc) or someone who uses multiple operating systems at the same time, the differences in mouse and keyword shortcuts get confusing, even irritating sometimes. One of the most irritating ones for me is the difference in what happens when you middle-click on the mouse.

In Windows / Linux, middle clicking in browsers is used to open and close tabs. In OS X, this doesn’t work because middle click is used to trigger the Dashboard. Every time I would want to open or close a tab, the dashboard would show up! To disable this, all you have to do is go to System Preferences > Exposé & Spaces and set the mouse shortcut to “-”.

For the newbies, here’s a screenshot guide. First select “System Preferences”:

Then click on the “Exposé & Spaces” button:

Set the “Dashboard” mouse shortcut to “—” :

So that it looks like this:

And that’s it! You will now be able to middle click to open and close tabs in Mozilla Firefox and Google Chrome. For Safari, you can open tabs, but closing tabs don’t work.

And there you have it, middle click tabs on Mac OS X!

Chasing Money

Philip Greenspun’s article on money has an interesting opening parable:

“There are three ways to make money. You can inherit it. You can marry it. You can steal it.”
— conventional wisdom in Italy

A young man asked an old rich man how he made his money. The old guy fingered his worsted wool vest and said, “Well, son, it was 1932. The depth of the Great Depression. I was down to my last nickel. I invested that nickel in an apple. I spent the entire day polishing the apple and, at the end of the day, I sold the apple for ten cents. The next morning, I invested those ten cents in two apples. I spent the entire day polishing them and sold them at 5 pm for 20 cents. I continued this system for a month, by the end of which I’d accumulated a fortune of $1.37. Then my wife’s father died and left us two million dollars.”

| |

Barack Obama Nobel Prize Sound Bites

From the Wikipedia :

On October 9, 2009, U.S. President Barack Obama was awarded the Nobel Peace Prize less than one year after his taking office (in fact, the nominations closed on February 1, about 11 days after Obama took office). While the committee praised his ambitious foreign policy agenda, it acknowledged that he had not yet actually achieved many of the goals that he had set out to accomplish. Former Polish President Lech Wałęsa, a 1983 Nobel Peace laureate, commented: “So soon? Too early. He has no contribution so far. He is still at an early stage. He is only beginning to act.”

This is pretty amazing news. My Facebook, News and IM streams are flooded with one-liners. I though I’d collect them all:

  • “I too would like a Nobel Peace Prize for the thesis I am about to write in the future.” — me
  • “it’s a pretty swell booby prize for losing out on the Olympics” – n.d.
  • “Surely preventing Sarah Palin from taking over the free world deserves a prize… even if it is a Nobel?” — v.b.
  • ““NASA bombs moon”; “Obama wins Nobel Prize” — is today Onion News Day?” — me
  • “Barack Obama linked to terrorist Yasser Arafat” — fark via a.a.
  • “The Nobel? Really? I mean, cool…but it seems like we have our cart on the wrong side of the horse. Not that it isn’t a very nice cart.” — c.m.
  • “…thinks they might as well have given him the Nobel Prize for Literature, Chemistry (we’ve all seen the shirtless photos), Physics and Economics as well. Oh and made him a Knight Commander of the Order of the Bath” — r.d.
  • “Nobel Committee Rewards Obama For Not Being Bush” — f.n.
  • “I just want to point out that the Nobel Committee made its decision BEFORE Miley Cyrus quit Twitter.” — j.h.
  • “Obama will win a second Nobel next year if he can restrain himself from reacting to the snark generated by this one.” — m.w.
  • “Pretty sure Obama will just trade in his Nobel for a Google Wave invite.” — t.b.
  • “The news of Obama’s Nobel Peace Prize spreads. Across the miles I can almost HEAR my dad’s eyes rolling.” — p.g.
  • “Obama wins Nobel Peace Prize? About time Rakhi Sawant wins an Oscar, then.” — s
  • “If you don’t think Obama deserves that Nobel, then you’ve never seen Sasha and Malia fight.” — a.e.
  • “Apparently Arizona State has a higher standard than the Nobel Committee. Good thing I never tried to apply there.” — r.m.
  • Business Insider has some more.

Tony Rosenthal, 1914 -- 2009

Michigan Today has a slideshow tribute to Tony Rosenthal, abstract artist and sculptor, who passed away this July. A Michigan alum, I know of him primarily from his Rosenthal Cubes, a pair of identical 15-foot cubes called Endover and Alamo. Endover is located near on Central Campus in Ann Arbor, while Alamo is located at Astor place in Manhattan, New York.

I think the difference between the two is that the New York cube has a platform and is a little harder to spin (yes, I’ve spun both!). I still find it amazing that the 41-year old sculptures are fully functional despite being exposed to the elements for so long.

A Quick Fix for Yahoo branding on Flickr

Techcrunch continued their usual Yahoo-bashing with this story today:

It appears that a few days ago there was a slight change to Flickr’s logo: an addition of a small Yahoo logo to the right side so it reads “Flickr from Yahoo.” In response, many Flickr users have taken to the photo-sharing site’s forums to express their horror at the Yahoo’s new branding of Flickr.

There is definitely some truth to the community backlash, but what I see as more aggravating is a great missed branding opportunity for Yahoo!.

Flickr and Delicious have both been adamant opponents to Yahoo! branding. Even though Yahoo! owns it, the Delicious frontpage doesn’t contain a single mention of Yahoo. Both sites’ communities are predominantly “indie” brand lovers; and don’t want “the man” to infringe their beloved service (even if the man is running it).

What’s crazy is that Yahoo recently launched a $100 million campaign called “Y!ou and Yahoo!”. What’s also interesting is that Flickr actually had a branding that said “Flickr loves you” (in place of Flickr BETA), which reflected Flickr’s personality and branding. People got used to it, and some even thought it was cute.

The last thing you want to do is force a new logo on to the community in an ungraceful manner. Here’s a convenient solution: to morph the “loves you” logo into the “Y!ou and Yahoo!” campaign and do a “flickr loves Y!ou” logo, killing two birds with one stone. The community sees a subtle evolution of the existing logo, and the “Y!ou” campaign is placed on a huge community”.

| |

BaconSnake: Inlined Python UDFs for Pig

I was at SIGMOD last week, and had a great time learning about new research, discussing various research problems, meeting up with old friends and making new ones. I don't recall exactly, but at one point I got into a discussion with someone about how I'm probably one of the few people who've actually had the privilege of using three of the major distributed scripting languages in production: Google's Sawzall, Microsoft's SCOPE and Yahoo's Pig. The obvious question then came up -- Which one do I like best? I thought for a bit, and my answer surprised me -- it was SCOPE, for the sole reason that it allowed inline UDFs, i.e. User Defined Functions defined in the same code file as the script.

I'm not aware if Sawzall allows UDFs, and Pig allows you to link any .jar files and call them from the language. But the Microsoft SCOPE implementation is extremely usable: the SQL forms the framework of your MapReduce chains, while the Mapper, Reducer and Combiner definitions can be written out in C# right under the SQL -- no pre-compiling / including necessary.

Here's how simple SCOPE is. Note the #CS / #ENDCS codeblock that contains the C#:

R1 = SELECT A+C AS ac, B.Trim() AS B1 FROM R WHERE StringOccurs(C, “xyz”) > 2 

#CS 
public static int StringOccurs(string str, string ptrn) {
   int cnt=0; 
   int pos=-1; 
   while (pos+1 < str.Length) {
        pos = str.IndexOf(ptrn, pos+1) ;
        if (pos < 0) break; cnt++; 
   } return cnt;
}
#ENDCS

Since I'm working at Yahoo! Research this summer, and I missed this feature so much, I thought -- why not scratch this itch and fix the problem for Pig? Also, while we're at it, maybe we can use a cleaner language than Java to write the UDFs?

Enter BaconSnake (available here), which lets you write your Pig UDFs in Python! Here's an example:

-- Script calculates average length of queries at each hour of the day

raw = LOAD 'data/excite-small.log' USING PigStorage('\t')
           AS (user:chararray, time:chararray, query:chararray);

houred = FOREACH raw GENERATE user, baconsnake.ExtractHour(time) as hour, query;

hour_group = GROUP houred BY hour;

hour_frequency = FOREACH hour_group 
                           GENERATE group as hour,
                                    baconsnake.AvgLength($1.query) as count;

DUMP hour_frequency;

-- The excite query log timestamp format is YYMMDDHHMMSS
-- This function extracts the hour, HH
def ExtractHour(timestamp):
	return timestamp[6:8]

-- Returns average length of query in a bag
def AvgLength(grp):
	sum = 0
	for item in grp:
		if len(item) > 0:
			sum = sum + len(item[0])	
	return str(sum / len(grp))

Everything in this file in normal Pig, except the highlighted parts -- they're Python definitions and calls.

It's pretty simple under the hood actually. BaconSnake creates a wrapper function using the Pig UDFs, that takes python source as input along with the parameter. Jython 2.5 is used to embed the Python runtime into Pig and call the functions.

Using this is easy, you basically convert the nice-looking "baconsnake" file above ( the .bs file :P ) and run it like so:

cat scripts/histogram.bs | python scripts/bs2pig.py > scripts/histogram.pig
java -jar lib/pig-0.3.0-core.jar -x local scripts/histogram.pig

Behind the scenes, the BaconSnake python preprocessor script includes the jython runtime and baconsnake's wrappers and emits valid Pig Latin which can then be run on Hadoop or locally.

Important Notes: Note that this is PURELY a proof-of-concept written only for entertainment purposes. It is meant only to demonstrate the ease of use of inline functions in a simple scripting language. Only simple String-to-String (Mappers) and DataBag-to-String (Reducers) functions are supported -- you're welcome to extend this to support other datatypes, or even write Algebraic UDFs that will work as Reducers / Combiners. Just drop me a line if you're interested and would like to extend it!

Go checkout BaconSnake at Google Code!

Update: My roommate Eytan convinced me to waste another hour of my time and include support for Databags, which are exposed as Python lists. I've updated the relevant text and code.

Update (Apr 2010): Looks like BaconSnake's spirit is slowly slithering into Pig Core! Also some attention from the Hive parallel universe.