|
Oct 12th, 2005
What am I reading?
I read a bunch of super cool books recently.
On
Intelligence by Jeff Hawkins and Sandra Blakeslee is a
fascinating theory of the neocortex - the seat of human
intelligence. Jeff and Sandra present a detailed and credible
account of the mechanics of the neocortex and show how (and
argue why) their predictive memory model results in
intelligence. Their theory is elegant and, even though far from
complete, I am thoroughly convinced that it is a decent
approximation of what goes on in the brain. Biological machinery
is extremely simple, it only seems fantastically complex due to
properties that emerge at scale. Authors build on the premise
that every cell in the neocortex runs the same algorithm and
intelligence is an emergent product of this neocortical
algorithm. Read this book - it will no doubt be
considered one of the most important works of our time. If you
have a machine learning background, check out Dileep
George's paper
on invariant representations in the visual cortex. Dileep is
a colleague of Jeff's (and they recently founded a company together called
Numenta) who is already
converting insights from the research into efficient algorithms.
The Wisdom of Crowds by James Surowiecki is
another kick-ass book. The thesis of this book is why (and in
what situations) collective decision making surpasses expert
knowledge. Since my work on spam filtration has been an
exercise in collective decision making, I really appreciated
the depth of analysis and ingenuity in formalization of the
topic. The book is extremely content rich (and well written to
boot!); I learned all sorts of stuff from it - design of Iowa Electronic Markets,
how TV show ratings and advertisement pricing is computed,
different kind of decision biases - cooperator bias and
confirmation
bias, and the many nuances of group dynamics.
Schild's Ladder by Greg Egan is brilliant. I also
read through most of Joel on Software and I violently agreed with half of it and mildly disagreed with the rest. Maus is memorable in a haunting way and Damnation Alley was a perfect companion on a three-hour, mega turbulent flight.
April 26th, 2005
The cycle of complex elegance to inelegant complexity (and back)
The grand unified
theory of the universe is a search for elegance. Mastering
the art of winning a combinatorial game is a search for
elegance. Discovering the attitude and philosophy of everyday
living is a search for elegance. Good software design is a
search for elegance. In the last 10 years of doing software
design, the part I've come to enjoy most is achieving that
palpable state of elegance. I can feel an elegant design
soon as it emerges and what feels elegant at outset, tends to
withstand the deep logical scrutiny that invariably follows.
Elegance scales. Elegance embraces newness; newness that wasn't
considered or even known at the time of original contemplation.
That, in fact, turns out to be the only objective test for
elegance: how well it embraces and adapts to additions in
specifications, to new information.
If you design a system over the long-term (years), incrementally
adapt it to growing needs, there comes a time when the elegance
breaks down. To retain the semblance of balance you expend
tremendous amounts of energy, rethink everything, restructure,
rewrite, redesign just to meet new requirements in a consistent
manner.
However, another piece of information challenges this new design
and you give in and accept the unweidlyness of evolution, the
kludgeness of forms plastered onto an elegant system, turning it
into a complexity that, while powerful, is aesthetically
unpleasant. The system works, but it leaves you on shaky ground,
specially with respect to the future.
There are umpteen accessible examples of this in the process of
"designing" theoretical frameworks of natural phenomena.
Physicists, biologists, chemists and geologists have tried,
time and again, to fit new observations to existing theories,
by proposing extensions that explain new observations in the
framework of the existing theory.
Apollonius_of_Perga's
epicycles are probably the most famous example. In order to
accommodate troubling observations of varying planetary
brightness and retrograde motion into the geocentric model of
the solar system, Apollonius came up with idea of epicycles:
planets did not circle earth in their concentric orbits, rather
they were attached to circles (epicycles) within the concentric
orbit. This model held for a while, but fell to further
observations. Ptolemy, then, proposed the idea of epicycles
within epicycles to "solve" for new observational data.
Eventually, it all came tumbling down when Copernicus's
observation of the star Aldebaran could
not be solved with any permutations of the epicycle equipped
geocentric model. Copernicus proposed the heliocentric model
and Kepler
refined it to remove epicycles altogether. Elegance reigned
once again.
Much like models (designs) from Aristotle (complex elegance) to
Ptolemy (inelegant complexity) to Kepler (complex elegance),
software design goes through the cycle as new requirements
surface. We've all witnessed our beautiful software turn into a
dangerous concoction, and then resolve into a higher level of
elegance that continues to scale effortlessly (for a while).
The basic problem is that of a limited granularity of
perception. We design for the requirements (or capture
reality in an elegant model) based on the understanding of the
world as it stands today. We use our predictive faculties to
adapt our designs and models to what might appear tomorrow. But
we can only see so far in the future. The universe is larger
than our brains. This implies, any process of modeling or
design will inevitably go through the cycle of "complex
elegance to inelegant complexity" and, hopefully, back. If we
formalize this notion, we can learn to discern when it occurs
and modify our design processes to accommodate it. For instance
one strategy that emerges from this knowledge is that if a new
requirement doesn't fit our design, we should not waste effort
trying to fit it in the current model, we should just special
case it and integrate the special case into an elegant whole
when new requirements show way to a higher level of elegance. As
Brook's pointed out in Mythical Man-Month, "Plan to
throw one away, because you will anyway". Another strategy is to
accumulate multiple requirements before hitting the design
board, as multiple requirements are more likely to expose holes
in the current design and hint at a better one.
Not all designs are elegant. If new requirements break a design,
it is not necessarily an honest mistake. There are multiple ways
to skin a cat, and it's quite possible that you happened to pick
the wrong one. How is one to know if a design is the best
possible representation of the requirements? One way to examine
a design is to see how tightly it is coupled to the
requirements. The tighter the coupling, the poorer the design.
If the design was created only to solve the particular
requirements, it is more likely that it will break when new
requirements arise. A design that is based on abstraction of the
requirements tends to be more robust. In fact, the designers
goal should be to abstract as much as possible from the
requirements, and then design to the final abstractions. Is
there a danger of being too abstract? Yes and no. This danger
can be thought of as a trade-off with the time of
implementation. A more abstract design will require the
implementer to first create the abstraction framework and then
specialize it to meet specific requirements. The advantage of an
abstract design is that it leaves "space" to elegantly fit
future requirements. It's time expended upfront, but saved in
future. We usually make this trade-off based on the importance,
longevity, and other constraints.
Another way to assess the elegance of design is when it is
replaced with a newer design. If major elements of of the
original design were carried over into the subsequent one, than
the original design could be thought of as elegant. The
percentage of design elements borrowed is a good barometer of
elegance. In case of the Solar System models, the element that
was carried over from Aristotle's model to Kepler's was the idea
of bodies circling each other in a concentric fashion. The only
two mistakes was the choice of the body and shape of orbits.
From that sense, Aristotle's design was quite elegant. When
Relativity and QM replaced Newtonian Mechanics it continued to
hold true, it just became a special case of the overall picture.
This, in my mind, is the best case scenario. If a design is
retained as is, as a special case of the newer level of complex
elegance, I believe the original to be a perfectly elegant
solution for the time and circumstances of its creation.
December 7th, 2003
RS500
The RollingStone Magazine compiled
a list of top 500 albums of all time. It contains a lot of albums that I
haven't heard, but I am not surprised given Britney Spears was one of the
editors. :-)
Here's a link to the RS list.
I don't like ordering my favorite music, but I am in a particularly
ordinal mood tonight, so here goes:
1. The Doors (The Doors) (RS 42)
2. The Soft Parade (The Doors)
3. Cyberpunk (Billy Idol)
4. Crash! Boom! Bang! (Roxette)
5. Money For Nothing (Dire Straits)
6. Painkiller (Judas Priest)
7. Heavy Horses (Jethro Tull)
8. The Marshall Mathers LP (Eminem) (RS 302)
9. The Violin Player (Vanessa May)
10. Coming Up (Suede)
11. Balance (Van Halen)
12. In Search Of... (N.E.R.D)
13. Discovery (Daft Punk)
December 6th, 2003
The World's Worst Server Rooms
I keep losing the links to this year-old story, so I am cataloging 'em
here. Here's the original,
the second followup
and the final one.
If you manage a hosted service or happen to have a ton of computers
at home, the photographs in this story will stir up something deep inside
you, and make you crack up so hard that you'll be on your floor, enmeshed
in the wires going from your machine to the wall, trying very very hard to
forget what you just saw.
Enjoy. And, oh, register++.
December 3rd, 2003
Self-Referential Puzzle
While I was holed up in bed for more than a week due to a nasty strep
throat, a friend sent me the following puzzle to help me bind my time.
It's really cool... give it a shot; estimated time to solve is about 60
minutes.
Here's the puzzle.
November 20th, 2003
Skating Away On The Thin Ice Of A New Day
I heard this song again after many many years. Ian Anderson is such an
incredible poet; Jethro Tull continues to blow me away till this
day.
Meanwhile back in the year one
When you belonged to no one
You didn't stand a chance son
If your pants were undone
'Cause you were bred for humanity
And sold to society
One day you'll wake up in the present day
A million generations removed from expectations
Of being who you really want to be
Skating away, skating away, skating away
On the thin ice of the new day
So as you push off from the shore
Won't you turn your head once more
And make your peace with everyone
For those who choose to stay
Will live just one more day
To do the things they should have done
And as you cross the wilderness
A-spinning in your emptiness
You feel you have to pray
Looking for a sign that the universal mind
Has written you into the passion play
Skating away, skating away, skating away
On the thin ice of the new day
And as you cross the circle line
Well, the ice-wall creaks behind
You're a rabbit on the run
And the silver splinters fly
In the corner of your eye
Shining in the setting sun
Well, do you ever get the feeling
That the story's too damn real
And in the present tense
Or that everybody's on the stage
And it seems like you're the only
Person sitting in the audience
Skating away, skating away, skating away
October 22nd, 2003
Thought of the Week
On a recent trip to Hot Springs, NC, this thought kept ricocheting in my
head:
Don't plan for future, let the future plan for you.
I have this theory about defeating
Murphy's Agents that work against
The Intentional Stance. The theory is to plan for future in quantum
superpositions of mutually contradictory goals and disentangle at the
very last moment, giving Murphy no time to resist. In other words, the
trick is to accept multiple outcomes till it becomes critical to pick a
particular one. The hard part is to actually believe in the multiple
outcomes. You can't take an intentional stance and disguise it behind a
superposition; Murphy is not fooled so easily.
Caveat: This thought is not to be taken too seriously.
October 14th, 2003
Foo Camp
Tim O'Reilly invited some 200 wireless, P2P, open-source people to camp
out at the O'Reilly Campus in Sebastapol. It was an experiment in self
organization, and from what I could tell, it was spectacularly successful.
I spent only half a day there (due to other commitments), but it was the
most wonderful half day I ever spent at any conference. Some unforgeable
moments:
Playing Table Tennis with Roger Dingledine
late into the night while discussing anonymity, cryptography and Tor as zenned-out Meng Wong, wrapped in a
shocking-blue electroluminescent wire, stood watching to the
background of festive strumming and singing from an adjacent
room.
Eating dinner with
Tom Christiansen, without knowing that Tom Christiansen was on
the table. For those who don't know him, Tom is a co-author of
Programming Perl and known for his scathing rants on #perl,
like
this one.
Discussing spam and reputation systems with
Paul Vixie,
Bram Cohen,
Meng, and
Fyodor, and discovering,
yet again, that there's more than one way to do it.
Hanging out with my good friend, Gene Boggs.
Meeting Ben Trott and
Sudhakar Chandra, two people I've known for a
long time, but had never met in person before.
October 5th, 2003
What was I reading?
Down and Out in the Magic Kingdom, by Cory Doctorow, is
a story of a man in the "Bitchun Society" where death has been conquered,
the only scarce resource is reputation, and Disneyland is open source. A
simple story that doesn't de-construct the technology or culture in great
detail, rather chooses to portray the basic human condition in a very
different (but very believable) reality. It reminded me a little of the
Hitchhiker Guide to the Galaxy and Philip K Dick's short stories. A fun
read, highly recommended.
The Free Lunch,
by Spider Robinson, another theme park story (I believe
Amazon recommended this to me when I bought Down and Out) that is
so vivid that I'll always remember it and has a cool time-travel twist.
Perl 6 Essentials by Allison Randal, Dan Sugalski,
Leopold Totsch, provides a whirlwind tour of Perl 6 and Parrot (the
virtual machine that Perl 6 will compile down to). Perl 6 is incredibly
exciting, and quite possibly the most significant work in programming
language design underway. I can't wait for the first beta (which is
probably another year away). Perl 6 Essentials does an excellent job of
introducing language semantics as well as being a hands-on guide to Parrot
assembly. (Yes, Parrot is available today and you can write programs for
it!) It's hard to write a decent programming book, and Perl 6 Essentials is
stellar. It sits next to K&R; and SICP in my bookshelf.
September 30th, 2003
MIT Emerging Technologies Conference
Flatteringly enough, MIT Technology
Review named me one of the Top 100 Young Innovators In the
World. The TR100 award ceremony was held at The MIT Emerging Technologies Conference,
which was a complete blast. I was pleasently surprised to find out that
most of the TR100, including me, were motivated by the "coolness" of their
ideas. It was fun to meet so many smart people working on stuff as diverse
as printing nanocircuits, combining molecules to make more effective
drugs, using neural nets to simulate human body motions, or tensor algebra
to recognize faces. I also hung out with Ian Clarke (of FreeNet) and Vinay
Gidwani (of Control-F1) who are supercool.
July 23rd, 2003
Don't break email to save it
I wrote an
article for the MIT Tech Review that argues tractability of the
anti-spam problem in the current email architecuture.
July 18th, 2003
Books, Music, Movies
I just finished reading
Midnight's Children by Salman Rushdie. This is the
first Rushdie book I've read, and I am completely blown away. An
autobiographic account of an unlikely protagonist whose life is
intertwined with the destiny of his country, Midnight's Children is an
all-encompassing account of post-independence India, its people, cultures
and politics. An eclectic cast of gifted children, army generals, impotent
husbands, religious zealots, movie actresses play out their extraordinary
parts in the tranquil valley of Kashmir, "magical" slums of Old Delhi, an
oppulent British estate in Bombay, the enchanted jungles of Sundarban and
military encampments in Pakistan, connecting dots that bring out a picture
which is astounding and surreal even to a native like me. Highly
recommended.
I also read 25th Hour by David Benioff and saw Spike Lee's movie based on
the book (with Edward Norton as protagonist). This is one of the very rare
instances where the movie is as good and powerful as the book.
After being completely put off by the deadening cynicism of Generation X,
I approached Coupland's Microserfs with heavy skepticism, but was
pleasantly surprised by its keen depiction of geek life (and lack thereof)
at Microsoft and Silicon Valley in early 90s. I especially liked author's
use of the term "isotope" to describe similar people, and "exothermic
reaction" to describe sucessful business processes.
Some really impressive movies I saw recently: Equilibrium, The Red Violin,
Barry Lyndon, Mulholland Drive, Twin Peaks, and The Animatrix.
July 14th, 2003
Matrix Table Tennis
Unfortunately, you can't be told what this is; you have
to see it for yourself.
Equally amazing is the new Honda Accord
advertisement.
March 27th, 2003
rover, precog, gentoo
I dropped rover, my Sony Vaio R505JL, from a height of about 4 feet, which
resulted in the most spectacular LCD screen crash I've ever seen (a .JPG
coming soon). Thanks to the sturdy VAIO design, the harddisk and all other
components survived the drop. I acquired an IBM T30 (precog) to replace
rover and decided to install gentoo
linux on it. gentoo is incredible. I don't think I've been so happy
with a linux distribution in a long time. It's exactly what I wanted, a
source distribution with source based dependency resolution that can be
optimized for your processor. If you are planning to install linux on one
of your machines, do check gentoo out!
February 26th, 2003
Lebanese Blonde
Too low to find my way
Too high to wonder why
I've touched this place before
Somewhere in another time
Now I can hear the sun
The clouds drifting through the blinds
A half a million thoughts
Are flowing through my mind
Every once in a while, I hear a song that reflects my state of mind so
perfectly, that I play it in a continuous loop for many many days. This is
rather upsetting to my co-workers and friends who happen to be in audible
range, but my right brain refuses to listen to anything else. "Lebanese
Blonde" by Thievery Corporation from The Mirror Conspiracy album is the
song I am currently obsessing with. It's beautiful, chill, edgy, perfect
and it speaks to me. Here's the MP3.
February 25th, 2003
Solar Sails
"Hundreds of years ago, early discoverers used the Sun as a compass.
Turns out the light of the Sun can do more than just guide us; it can
actually propel us farther and faster into the vast realm of space
than we've ever been able to go. With new solar sail technology,
scientists are finding a way to convert light energy from the Sun into
a lightweight, propellant-free source of propulsion for spacecraft."
Here's a link to the rest of the JPL
article. The Planetary Society is building a privately funded
solar sail propelled spacecraft; more details here,
including a section on building your own solar
sail!
This is good stuff.
February 10th, 2003
Amsterdam, Berkeley, Cambridge, Hawaii
Last few weeks have been hectic, relaxing, fun, and uncertain... a
rather typical segment of my paradoxical existence.
I went to the Spam
Conference. It was a one day conference held in room 26-100 of MIT
on January 17th. The turnout was roughly 600 people, instead of the
expected ~50. There was a good mix of academic and industry
participation, and the mood was generally optimistic. Bulk of the
presented material was about statistical methods (Bayesian classifiers
in particular), but topics as diverse as impact of spam on ISPs, legal
recourse for the spammed, and social engineering attacks to track down
spammers were presented. It was great to see so many smart people
applying themselves to the problem. I look forward to Spam Conference
2.
In other news, my co-founder at Cloudmark, Jordan Ritter, and his
girlfriend Jessie
Garrehy got married in great style in Maui, Hawaii.
Congratulations to them both! We had a blast in Maui; 4 days is
definitely too little time to spend in such a beautiful place. Oh, and
Hawaiian is cool. In the ATA in-flight magazine I learnt that Hawaiian
had only 12 characters, and I remarked to a friend that this would
mean that Hawaiian had a lot of really long words. And sure enough,
couple of hours later we were eating at
Humuhumunukunukuapua'a.
My friends were just as surprised as me at my decision to move to
Berkeley two weeks ago. I rented an apartment in downtown Berkeley,
but it was way too loud, and its 1920s charm didn't quite offset the
lack of polish and comfort. The recurring noise of heavy metallic
objects being dumped into a truck outside my apartment at 5 AM every
alternate morning was the last straw. I am moving back to South of
Market soon as I can find a more hospitable domicile. Berkeley is a
really cool place but it's not quite where I want to live.
Spent a night wandering around Amsterdam. Dammit. It's all true. I am
going back there even if I have to fly KLM.
January 13th, 2003
San Francisco Bound
My vacation in New Delhi has come to an end, and I am leaving for San
Francisco tomorrow morning. I was supposed to leave tonight, but KLM
canceled flights due to heavy fog. Apparently, KLM has been delaying
flights for the last week, but "erroneously" informed me that flights
were on time when I inquired three days ago. This truly sucks, I am
going to miss several important engagements and critical work and my
schedule will be all messed up. This will also be the last time I'll
fly KLM. On my previous KLM trip, my luggage arrived 4 days after I
did. A quick look on the web suggests I am not the only person who has
vowed never to fly
KLM again.
Now, if it were Singapore
Airlines, a beautiful (and scantily clad) lady would have showed
up at my house a week before the departure date to apologize profusely
and to inform me in person of the probability, slight as it might have
been, of cancellation of my flight and the resulting inconvenience it
would entail.
Note to self: Don't fly any thing other than Singapore or Thai
Airlines if you can help it.
January 6th, 2003
SpamNet designers nominated for Wired Rave Awards
January 3rd, 2003
What Should I Do With My Life?
Po Bronson is exceptionally
skilled at capturing Zeitgeists. His book Nudist On The Late
Shift transports you to the Silicon Valley of late 90's and
puts you right in the middle of all the action and exuberance. He
recently set out to meet people who have found meaning in life,
and discovered that more and more people are focusing on what they
like to do (instead of rushing for the metaphorical gold) and this
makes them happy people. Here's the link to the
article.
It is interesting to me that a downward trend in the economy has
forced people to make the right choices. I am often asked about
"software scene in the valley" and in my personal experience, I've
noticed that people who are still around are those who really love
their work. They might be overworked, underpaid, gone freelance, or
working on open source projects, but they'd rather not do anything
else. Writing code or participating in another dimension of software
production is what they have identified to be their calling. It's
merely a happy coincidence that they get paid for it.
December 27, 2002
What I am I reading?
|
|
|
|