A Brief History of Dangerous Ideas (Brief because (a) I really don’t know that much history, and (b) there’s really only one idea here with a few examples, and once you get the point, we’re done and can move to the outro.)

Astronomy and Celestial Mechanics were dangerous ideas because they undermined the most powerful organization of their day, The Church. That’s why people have been banned, tortured, and burned at the stake for talking about these ideas.

Crossbows were a dangerous idea because they allowed an untrained peasant to kill a knight. Longbows were not a dangerous idea, because only trained archers can kill a knight with a longbow, and the nobility were the only people who could compel peasants to practise yeomanry.

Cryptography is not a dangerous idea, because we really don’t know if any of our algorithms and protocols are resistant to the NSA. This was hi-lighted when researchers “discovered” differential cryptanalysis. When they looked at the DES algorithm IBM has been promoting since the 1976, they found that it had been specifically tuned to resist differential cryptanalysis. IBM ‘fessed up: the US government had told them to tune things that way without explaining why, leading us to conclude they had known about this attack for decades before it became public knowledge. Today, we have no idea whether what we think is strong is actually strong or whether it has vulnerabilities and back doors governments can exploit.


Zed and Reg at Rubyfringe exchanging dangerous ideas, photo by Libin Pan
Miles Davis was a walking, talking, trumpet-playing dangerous idea. Not because he reinvented Jazz five different times (Dear Steve: Apple II, Macintosh, iPod/iPhone, Pixar. One more for the tie, two for the record.) Miles was an egocentric, venal man who worked the system, not undermined the system. But he was still dangerous because he got white people directly interested in black music. There was no Elvis or Vanilla Ice or anyone else between his music and the mainstream audience. For a government intent on keeping America’s two dominant cultures divided through fear, anything uniting them was a threat.

People say Miles’ legacy is his music. To me, Miles’ lasting legacy is people like me, people with one parent from each culture who grew up dancing to the same music together. People who, incidentally, do not vote for governments that take a divide and conquer approach to culture.

And on to tech. Microcomputers were not a dangerous idea. But personal computers were dangerous. It took decades for IT departments to regain control over people bringing their own computers to work. They can thank Microsoft for helping them get back into the driver’s seat.

(This, incidentally, is why I really dislike Microsoft’s policies: it has nothing to do with their lack of taste, it has to do with their mission to make the computer on my desk belong to my IT department or the record label or the movie studio or—I suspect—the government.)

Web applications are dangerous. Never mind the fact that they make desktop applications obsolete. The people who built desktop applications just go and get jobs writing web applications. Same people, different shit. But as Giles Bowkett pointed out, web applications just might make venture capital obsolete! When you don’t need hundreds of programmers and distribution channels and all the other friction-managing elements of a company that ships old-school software, you need a lot less money to start a business.

And on to media. You know that the web is busy putting newspapers out of business. My wife and I watched YouTube last Saturday Night. I’m not talking about the advertising business: I think we would have been happy to watch ads to watch our Mitch Hedberg and Billy Connoly comedy clips. But the web lets us choose what we want to watch, when we want to watch it. The network can’t put their up-and-coming show on right after their hit to give it a boost. The new show has to compete on its own merits. That puts users in control, and that’s dangerous.

Joel Spolsky said a similar thing about pricing all music at 99 cents a track: it means the labels can’t kill an artist by sticking their CD in the $3.99 crapola bin. Users choose what they want to listen to. That’s dangerous, again because users are in control.

Okay, that’s enough. Dangerous ideas are the ones that subvert the existing hierarchy of control, not just the ones that shuffle people around in the same old chairs. Apple Macintosh with a GUI replacing a PC with a command line? Not dangerous. Apple Macintosh with a Laserwriter and Aldus Pagemaker allowing someone to launch a magazine in their basement that competes with a company employing dozens of layout artists? That’s dangerous, and that’s interesting.

Dangerous equals subversive equals interesting.

outro

Deep breath. Okay, the next thing is not particularly dangerous for the world at large, but it is for me. I am retiring from blogging and retiring from hacking on Ruby. Maybe I’ll un-retire one day. I don’t know, the future is not set.

Miles Davis wasn’t afraid to move on when the time was right, even if what he was doing seemed to make people happy. Respecting his legacy means seeking what he sought.

Remember how I said that microcomputers were not dangerous, but personal computers were? Right now, I would say this: Ruby is not dangerous, but Rails is dangerous and Merb is dangerous and Sinatra is dangerous. Rewriting for Ruby is interesting. I believe it is useful. But is it dangerous? No.

Likewise, I can say with a clear conscience that while writing is gratifying, and trying to write out and explain ideas has helped me understand things, the writing I’ve been doing is not dangerous. It doesn’t subvert.

So while hacking away on Ruby the language and blogging about software development is gratifying and useful, they are not dangerous activities. They are microcomputers, but they are not personal computers.

I am going on vacation from August 2nd through 10th. During that time I plan to do absolutely no thinking about computers. For what it’s worth, I will be engaging in activities with faux danger: wreck diving and sport climbing. (p.s. these are not solitary pursuits: if you want to try some of the world’s greatest wreck diving and sport climbing, get in touch).

When I return, I will give things some serious thought and hopefully, discover a way I can help make our world a more dangerous place.

Thank you ever so much for your support and interest and feedback. Especial thanks to my fellow bloggers like Joel, Joey, Giles, Damien, Obie and so many others who exchanged ideas with me and kept the debate alive. And Reddit? And Hacker News? You rock, you are the future. I can’t wait to see how your communities and technologies evolve.

Warmest regards,


Reginald Braithwaite  ¶ 10:04 AM

  That not is not the not I meant During my presentation at RubyFringe, I put this slide up while talking about “Adverbs:”



Judging by the fun we had discussing this point later, I really needed a better example or a new idea. But for thse who are a little curious about where I was trying to go with this…

more on why not is not the same as not

I saw the #not method somewhere, and the point was that the author felt it read better than writing “!blitz.blank?” As someone pointed out, Ruby also gives us a not keyword, so you could write “not blitz.blank?” if you prefer the word “not” to the punctuation “!”

So the argument is that:

blitz.not.blank? == !blitz.blank? # and also == (not blitz.blank?)

But there’s another point we’re missing comparing the word to the punctuation. Let’s switch to English and you’ll see what I mean. The sentence “It is not the case that blitz is blank’ does not mean the same thing as “blitz has the property of not being blank.”

Whaaaa?

The key here is to think about where the parentheses go when we parse the two ways to express ourselves. Imagine “blank?” and “not” are functions instead of methods or keyword. Then two forms would look like this:

not(blank?)(blitz) # == blitz.not.blank?
not(blank?(blitz)) # == not blitz.blank?

Very different. The first expression passes the blank? function to the not function. The result is another function, the inverse of blank? itself, and that’s what we use. The second form is our old not keyword, where we pass blitz to out blank function, which presumably returns true or false, and we pass the result to our not function which inverses the boolean.

So in one case we are taking the inverse of a function, in the other we are taking the inverse of a boolean. This is what I was feeling out with my digression into wondering why Ruby reads from right to left. We have ways of applying functions to functions, but we don’t have easy ways of writing methods that alter other methods directly. And we certainly don’t have an easy way of sending a message that invokes such a method on an arbitrary object.

I guess I wonder whether we could ever write this blitz.not(blank?) to express this idea, or perhaps blitz.(not blank?), or perhaps some other syntax. But the point would be that we want the inverse of a method, not the inverse of a boolean.

speaking of jazz, so what?

It seems very abstract when we look at something like #not.blank?, because the result of an inverted #blank? method would usually be identical to the result of inverting its result.

But let’s consider the case where blitz is actually a proxy for a value in a SQL database column. So you might have something like “my_model.blitz.not.blank?” Fair enough?

Now if you know SQL, I think you will agree that there are times when the following is true:

  (model1.blitz != model2.blitz) != (not model1.blitz == model2.blitz)

Although it looks like x.not.eql?(y) should always have the same semantics as not x.eql?(y), SQL has special semantics for testing NULLs. Specifically, NULL is neither = nor <> any other value. So if you carefully map your proxies to the underlying SQL behaviour, you want the above to have this behaviour.

(For those who are blessed with a lack of SQL in their lives, NULL is a special value that is not the same thing as a Ruby nil. For example NULL = NULL is false in SQL. So if we map SQL values to Ruby objects, the following is the case: NULL != anything is false, NULL == anything is also false, therefore not NULL == anything is true, and therefore (NULL != anything) != (not NULL == anything).

If this digression into SQL NULLs now makes you regard your current ORM framework with deep suspicion, congratulations. They all suck, the trick to choosing an ORM is finding one that sucks in ways that aren’t fatal to your project.)

Essentially, Ruby already has a not adverb as a special case for #not==. #!= is there because sometimes (not foo == bar) is not equal to (foo != bar). However, like many things in Ruby this is a one-off thing. There isn’t a bang-method construct, #== and #!= are two completely arbitrary symbols that we happen to associate with equality and its inverse.

But maybe if we had a way to derive #!= from #== using a not adverb, maybe if we could write #== and then write a #not for the method, we would be able to do interesting things like replicate SQL’s trivalent logic more accurately. We would know that foo.not < 5 is not the same thing as not foo < 5, because if foo is a SQL null, foo < 5 is false, as is foo.not < 5. And neither throws an exception!

So getting back to the #not method as presented, I don’t know if it’s necessary, and if it does nothing more than replicated the exact semantics of the bang punctuation or not keyword it is clearly a matter of taste.

But I do feel that there is a use case for “adverbs,” for a way to talk about modifying methods themselves. Unfortunately, right now the mechanisms for working directly with methods are somewhat arcane and thus we don’t naturally consider them. So discussions like this feel very fringe and “out there.”

But at least now I feel like I have properly explained the not I meant, and not the not I didn’t mean.  ¶ 5:00 AM

  "L" is not a code smell During my presentation at RubyFringe, I shared a question that has been swirling around in my brain for a while: Are IDE features really language smells?

I don’t think it’s an original thought. If nothing else, it’s a corollary to what I believe to be true about many of the GoF design patterns: Many of them are workarounds, ways to Greenspun missing language features. Now, this is probably not the case for all IDE features, and in truth it may be that there are some features which could be implemented in either the language or the IDE, but the IDE may be the best place to put them.

But there is a fairly large class of IDE features that strike me as language workarounds. One of them is definitely the ability to spit out a lot of boilerplate. If you need a lot of code written, you ought to be able to get your programming language to do it for you, not your IDE.

There is room for people to disagree about this. There are some who feel (Strawman alert!) that programs consisting of large numbers of simple elements are easier to understand than programs consisting of a small number of highly abstract elements. Those folks feel an IDE gives you the ease of writing a program quickly plus the ease of reading that same program quickly. They feel that abstractions make the program easier to write but harder to read.

I happen to disagree with this, and if you have been reading this weblog for more than a couple of days you have already read why my experience leads me down a different path. Although in deference to my colleagues with different views, I offer this quote:

All problems can be solved by adding another layer of abstraction, except the problem of having too many layers of abstraction.

Anyone who has dealt with an hammer factory will agree.

So back to “L.”

Two speakers before me, Giles Bowkett gave his excellent Archaeopteryx—um—presentation. I hesitated over that word, because I could just as easily say performance. Performances are terrific entertainment, but they sometimes obscure the message behind them. I want to say outright that while this is true of many other subjects, I felt it worked for Giles because the subject of his presentation was software development as a how rather than a what, and for Giles the “what” is performance.

(Giles got a standing “O,” and many people might be tempted to rush out and make their presentations just as stimulating (400+ in-your-face slides punctuated with loud, driving drum and bass). Be sure that your material matches your presentation style! If not, people may walk away saying “Wow, amazing, but what exactly did she say?” I think it worked for Giles and that’s quite an accomplishment.)

Now really, back to “L.”

Giles is one of the people using closures in Ruby. Meaning, he is passing functions around and storing functions in objects. I am not going to try to say exactly what Archaeopteryx does, so I will describe this style of programming using an imaginary companion program that creates walking bass lines. I will call it Troody.

Let’s simplify things greatly and say that Troody will only ever play in perfect 4-4 time and further that Troody only ever play one of the eight notes in a particular chord’s standard scale. The probability of playing each of those notes on any one “beta” could be represented as an array with eight elements, like this: [.35, .05, .1, .05, .25, .05, .1, .05 ]. You can imagine passing arrays like this around in Troody.

For example, we can pass this array to an object that actually plucks the strings: Plucker.new.start_plucking([ 0.35, 0.05, 0.1, 0.05, 0.25, 0.05, 0.1, 0.05 ]).

Let’s try writing a naïve Troody Plucker:

class Plucker
    def start_plucking(probs)
        while (self.tune.playing)
            if (Metronone.on_the_beat)
                r = rand
                cumulative_probs = probs.inject([]) { |cum, element| 
                    cum + [ cum[-1] && (cum[-1] + element) || element ] 
                }
                notes_to_cumulative_probs = (1..8).zip(cumulative_probs)
                note_to_play = notes_to_cumulative_probs.detect { |note, prob| prob >= r }
                self.pluck(note_to_play)
            end
        end
    end
end

You pass it a set of probabilities, it produces bass notes. But stop, that’s so procedural. Let’s learn from a flying creature, let’s learn from Archaeopteryx. Instead of passing arrays, let’s pass lambdas, like this: lambda { [.35, .05, .1, .05, .25, .05, .1, .05 ] }. Now whenever Troody needs the probability of something, we call the function with .call or Ruby’s [] alternative syntax. So now we write Plucker.new.start_plucking(lambda { [.35, .05, .1, .05, .25, .05, .1, .05 ] })

Our new Plucker code is the same as the old, except we write:

cumulative_probs = probs.call.inject([]) { |cum, element| 
    cum + [ cum[-1] && (cum[-1] + element) || element ] 
}

We now call the probs lambda when we need a note. That’s it, we’ve added a .call call. What does that get us? Well, here’s one thing: If we want the probability to change over time, our function can do that, and we don’t have to rewrite our start_plucking method to handle the idea.

For example, here’s a probability lambda that usually plays the same way but from time to time decides it ought to play pedal notes (refactoring to OO is an optional exercise):

probs = lambda { |bars_of_pedal, beat|
    lambda {
        if beat == 0
            if bars_of_pedal == 0
                bars_of_pedal = 1 if rand < .05
            elsif bars_of_pedal == 5
                if rand < .25
                    bars_of_pedal = 0 
                else
                    bars_of_pedal += 1
                end
            elsif bars_of_pedal == 9
                if rand < .5
                    bars_of_pedal = 0 
                else
                    bars_of_pedal += 1
                end
            elsif bars_of_pedal == 13
                bars_of_pedal = 0
            end
        end
        beat = (beat + 1) % 4
        if bars_of_pedal == 0
            [ 0.35, 0.05, 0.1, 0.05, 0.25, 0.05, 0.1, 0.05 ]
        else
            [ 1.0, 0, 0, 0, 0, 0, 0, 0 ]
        end
    }

}.call(0, 0)

Thanks to the way we’ve separated the probabilities from the plucking, we do not need to subclass Plucker to try a different playing style in Troody.

As Giles pointed out, this is the Strategy Pattern. We are making different kinds of pluckers by encapsulating the logic of what to pluck in something we pass to a plucker. Archaeopteryx appears to do this everywhere. There are lambdas paramaterized by lambdas, lambdas that return lambdas…

This creates a problem. Imagine a programing language where all the keywords are in upper case: IF foo THEN bar ELSE bizzat. Try reading such a program aloud, and you end up shouting the punctuation but speaking the words. This is wrong! We should be shouting the words and whispering the punctuation!

And the problem with Ruby’s lambdas is that if you use a lot of them the word lambda really starts to stand out. So Giles fixed this by aliasing it to L: and using [] instead of .call():

alias :L :lambda

L{ |a| a + a}[5].
    => 10

Much nicer, and as Giles pointed out, this is an example of Ruby’s strength. If you have a program that rarely uses lambdas, you probably want lambdas to stand out when you use them, so you don’t alias lambdas to L and you use the call method, not the square brackets. But if you use lambdas a lot, it’s a win to abbreviate things.

Okay, we’re talking about “L.” Good.

Now in my talk, I said that abbreviating lambda to L was a code smell. I was wrong! Giles, my bad!!

What I actually think is that needing to abbreviate lambda to L is a language smell. Very different. If you show me a Java program and you show me Strategy Pattern, I shouldn’t say it’s a code smell. I should say too bad for you that you need all that boilerplate when Ruby lets you do that with the word lambda and a pair of curly braces.

So now to “L:” Giles, if lambdas are integral to Archaeopteryx, if they are so woven into the fabric of what Archaeopteryx does that you want the keyword “lambda” to fade away, I honestly think this is a place where the language could help you.

For example, what if Ruby had call-by-name semantics? You could write:

Plucker.new.start_plucking([ 0.35, 0.05, 0.1, 0.05, 0.25, 0.05, 0.1, 0.05 ])

# or...

bars_of_pedal, beat = 0, 0
Plucker.new.start_plucking(
    if beat == 0
        if bars_of_pedal == 0
            bars_of_pedal = 1 if rand < .05
        elsif bars_of_pedal == 5
            if rand < .25
                bars_of_pedal = 0 
            else
                bars_of_pedal += 1
            end
        elsif bars_of_pedal == 9
            if rand < .5
                bars_of_pedal = 0 
            else
                bars_of_pedal += 1
            end
        elsif bars_of_pedal == 13
            bars_of_pedal = 0
        end
    end
    beat = (beat + 1) % 4
    if bars_of_pedal == 0
        [ 0.35, 0.05, 0.1, 0.05, 0.25, 0.05, 0.1, 0.05 ]
    else
        [ 1.0, 0, 0, 0, 0, 0, 0, 0 ]
    end
)

And you would get the same behaviour as if you were using lambdas.

That’s it, that’s what I should have said on stage: any time you are working around your language—whether in your IDE, or by modifying open classes, or by abbreviating things—that’s a place where we should step back, where we should ask if our language is missing something.

The answer may very well be “no.” But we ought to at least ask the question.  ¶ 12:43 PM

  Raganwald on the Fringe of a Nervous Breakdown Thanks to Joel, I now have a format for Sunday’s presentation at RubyFringe: Twenty slides, each on the screen for exactly twenty seconds.  ¶ 11:40 AM