Axiomatic perspective on fairness, and the power of discussion

Sorelle Friedler, Carlos Scheidegger and I just posted a new paper on the arxiv where we try to lay out a framework for talking about fairness in a more precise way. 

The blog post I link to says more about the paper itself. But I wanted to comment here on the tortuous process that led to this paper. In some form or another, we've been thinking about this problem for two years: how do we "mathematize" discussions of fairness and bias? 

It's been a long and difficult slog, more than any other topic in "CS meets X" that I've ever bumped into. The problem here is that the words are extremely slippery, and refract completely different meanings in different contexts. So coming up with notions that aren't overly complicated, and yet appear to capture the different ways in which people think about fairness was excruciatingly difficult. 

We had a basic framework in mind a while ago, but then we started "testing" it in the wild, on papers, and on people. The more conversations we had, the more we refined and adapted our ideas, to the point where the paper we have today owes deep debts to many people that we spoke with and who provided nontrivial conceptual challenges that we had to address. 

I still have no idea whether what we've written is any good. Time will tell. But it feels good to finally put out something, however half-baked. Because now hopefully the community can engage with it. 

Congrats to John Moeller, Ph.D

My student +John Moeller (moeller.fyi) just defended his Ph.D thesis today! and yes, there was a (rubber) snake-fighting element to the defense.

John's dissertation work is in machine learning, but his publications span a wider range. He started off with a rather hard problem: attempting to formulate a natural notion of range spaces in a negatively-curved space. And as if dealing with Riemannian geometry wasn't bad enough, he was also involved in finding approximate near neighbors in Bregman spaces. He's also been instrumental in my more recent work in algorithmic fairness.

But John's true interests lie in machine learning, specifically kernels. He came up with a nice geometric formulation of kernel learning by way of the multiplicative weight update method. He then took this formulation and extended it to localized kernel learning (where you don't need each kernel to work with all points - think of it like a soft clustering of kernels).

Most recently, he's also explored the interface between kernels and neural nets, as part of a larger effort to understand neural nets. This is also a way of doing kernel learning, in a "smoother" way via Bochner's theorem.

It's a great body of work that required mastery of a range of different mathematical constructs and algorithmic techniques.  Congratulations, John!

FOCS workshops and travel grants.

Some opportunities relating to the upcoming FOCS.

• Alex Andoni and Alex Madry are organizing the workshops/tutorials day at FOCS, and are looking for submissions. The deadline is August 31, so get those submissions ready!
• Aravind Srinivasan and Rafi Ostrovsky are managing travel and registration grants for students to go to FOCS. The deadline for applications is September 12. 

Of course FOCS itself has an early registration deadline of September 17, which is also the cutoff for hotel registrations. 

Pokégyms at Dagstuhl

Yes, you read that correctly. The whole of Dagstuhl is now a Pokégym and there are Pokémon wandering the streets of Wadern (conveniently close to the ice cream shop that has excellent ice cream!)

Given this latest advancement, I was reminded of Lance Fortnow's post about Dagstuhl from back in 2008 where he wistfully mourned the fact that Wifi now meant that people don't hang out together.

Times change. I am happy to note that everything else about Dagstuhl hasn't changed that much: we still have the book of handwritten abstracts for one thing.

Carl Zimmer's series on exploring his genome

If you haven't yet read Carl Zimmer's series of articles (one, two, three), you should go out and read it now!

Because after all, it's Carl Zimmer, one of the best science writers around, especially when it comes to biology.

But even more so because when you read the story of his personal quest to understand his genetic story in all its multifaceted glory, you understand the terrifying opportunities and dangers in the use of genetic information for predictive and diagnostic medicine. You also realize the intricate way that computation is woven into this discovery, and how sequences of seemingly arbitrary choices lead to actual conclusions about your genome that you now have to evaluate for risk and likelihood.

In a sense, this is the tale of the use of all computational approaches right now, whether it be in science, engineering, the social sciences, or yes, even algorithmic fairness. Zimmer uses the analogy with telescopes to describe his attempts to look at his genome, and this explanation is right on the money:

Early telescopes weren’t terribly accurate, either, and yet they still allowed astronomers to discover new planets, galaxies, and even the expansion of the universe. But if your life depended on your telescope — if, for example, you wanted to spot every asteroid heading straight for Earth — that kind of fuzziness wouldn’t be acceptable.

And this quote from Robert Green, one of the geneticists who was helping Zimmer map out his genome:

Ultimately, the more you know, the more frustrating an exercise it is. What seemed to be so technologically clear and deterministic, you realize is going through a variety of filters — some of which are judgments, some of which are flawed databases, some of which are assumptions about frequencies, to get to a best guess.

 In this is a message for all of us doing any kind of data mining.

NIPS Reviewing

This year, NIPS received over 2400 submissions. That's -- well --- a lot!

As a reviewer, I have been assigned 7 papers (note that this number will be utterly incomprehensible to theoryCS PC members who think that 30 papers is a refreshingly low load).

But none of that is as interesting as what NIPS is trying this year. From the PC Chairs:

New this year, we ask you to give multiple scores for technical quality, novelty, impact, clarity, etc. instead of a single global score. In the text boxes, please justify clearly all these scores: your explanations are essential to the ACs to render and substantiate their decision and to the authors to improve their papers.

Specifically, the categories are:

• Technical quality
• Novelty/originality
• Impact/usefulness
• Clarity and presentation

and there are also a few qualitative categories (including the actual report). Each of the numerical categories are on a 1-5 scale, with 3 being "good enough".

I've long felt that asking individual reviewers to make an accept/reject judgement is a little pointless because we lack the perspective to make what is really a zero-sum holistic judgement (at least outside the top few and the long tail). Introducing this multidimensional score might make things a little more interesting.

But I pity the fate of the poor area chairs :).

The Man Who Knew Infinity

I generally avoid movies about mathematicians, or mathematics.

I didn't watch Beautiful Mind, or even the Imitation game. Often, popular depiction of mathematics and mathematicians runs as far away from the actual mathematics as possible, and concocts all kinds of strange stories to make a human-relatable tale.

Not that there's anything wrong with that, but it defeats the point of talking about mathematics in the first place, by signalling it as something less interesting.

So I was very worried about going to see The Man Who Knew Infinity, about Srinivas Ramanujan and his collaboration with G. H. Hardy. In addition to all of the above, watching movies about India in the time of the British Raj still sets my teeth on edge.

To cut a long story short, I was happy to be proven completely wrong. TMWKI is a powerful and moving depiction of someone who actually deserves the title of genius. The movie focuses mostly on the time that Ramanujan spent at Cambridge during World War I working with Hardy. There are long conversations about the nature of intuition and proof that any mathematician will find exquisitely familar, and even an attempt to explain the partition function. The math on the boards is not hokey at all (I noticed that Manjul Bhargava was an advisor on the show).

You get a sense of a real tussle between minds: even though the actual discussions of math were only hinted at, the way Hardy and Ramaujan (and Littlewood) interact is very realistic. The larger context of the war, the insular environment of Cambridge, and the overt racism of the British during that period are all signalled without being overbearing, and the focus remains on the almost mystical nature of Ramanujan's insights and the struggle of a lifelong atheist who finally discovers something to believe in.

It took my breath away. Literally. Well done.

Google Recruiter Survey: Tell us what you think!

(ed: I can't believe it's been almost three months since I posted. Maintaining two blogs and twitter is more work than one might think)

Hello,
  Thank you for applying to recruit me to Google. I'm continuously working to provide a great experience to my recruiters throughout the hiring process, so I greatly value any feedback you’re willing to share about your experience—both what’s going well and what needs work.

Please share your feedback through my recruiter experience survey, which will be open from now through Monday, May 44. The survey should take less than 15 seconds to complete, and you can skip over any questions you prefer not to answer. Please keep in mind that your responses are not confidential, and will be used for humor improvements—not in decisions as to who I choose to allow to recruit me.

I absolutely do not love chatting with recruiters, though I sometimes receive more emails than I can respond to and have to prioritize questions regarding technical difficulties. Below are a few of my most frequently asked questions (FAQs) that may provide the answer you need.

Thank you for your time and have a great day,

Suresh, Suresh Venkatasubramanian Recruitment Experience Team.

FAQs
I never received feedback on my recruitment email - can you give me feedback?

I'm pretty limited on what I have access to within your recruiter profile (to ensure that my brain doesn't melt). I don't particularly mind you feeling confused about the outcome. Therefore, I suggest reaching out to your friends and family  if you have specific questions regarding your attempt to  recruit me at Google.


I’d like to provide more input on your process - should I email that over?
Please use the open ended comments at the end of the survey to leave any anecdotal feedback or additional thoughts. It's in the handy box titled /dev/null. This ensures your thoughts are not saved as I ignore aggregate feedback to share with my internet following.