So You Want To Be A Seed AI Programmer

From The Transhumanist Wiki

Written by Eliezer Yudkowsky.

The first thing to remember is...

Not everyone needs to be a seed AI programmer. SIAI will probably need at least 20 regular donors per working programmer, perhaps more. There is nothing wrong with being one of those donors. It is just as necessary.

Please bear this in mind. Sometimes it seems like everyone wants to be a seed AI programmer; and that, of course, is simply not going to work. Too many would-be cooks, not enough ingredients. You may not be able to become a seed AI programmer. In fact it is extremely likely that you can't. This should not break your heart. You can learn seed AI after the Singularity, if we live through it and we're all still human-sized minds - learn the art of creating a mind when it's safe, when you can relax and have fun learning, stopping to smell the roses whenever you feel like it, instead of needing to push as hard as possible.

Meanwhile, before the Singularity, consider becoming a regular donor instead. It will be a lot less stressful, and right now we have many fewer regular donors than people who have wandered up and expressed an interest in being seed AI programmers.

Seed AI. It sounds like a cool idea, something that would look good on your resume, so you fire off a letter to the Singularity Institute indicating that you may be vaguely interested in coming on board.

Stop. Hold on. Think for a second about what you're involving yourself in. We're talking about the superintelligent transition here. The closest analogy would be, not the rise of the human species, but the rise of life on Earth.

We're talking about the end of the era of evolution and the beginning of the era of Recursive Self-Improvement. This is an event that happens only once in the history of an entire... "species" isn't the right term, nor "civilization"; whatever you call the entire sequence of events arising from Earth-originating intelligent life. We are not talking about a minor event like the Apollo moon landing or the invention of the printing press.

This is not something you put on your resume. This is not something you do for kicks. This is not something you do because it sounds cool. This is not something you do because you want your name in the history books. This is not, even, something that you do because it strikes you as beautiful and terrifying and the most important thing in the world, and you want to be a part of it, you want to have been there. These are understandable feelings, but they are, in the end, selfish. This is something you should only try to do if you feel that it is the best thing you can do, out of all the things you might do; and that you are the best one to do it, out of all the people who might apply for a limited number of positions. The best. Your personal feelings are not a consideration. Is this the best thing for Earth?

You may not get paid very well. If there are any dangerous things that happen to seed AI programming teams, you will be directly exposed to them. At any time the AI project might fail or run out of funding and you will be left with nothing for your work and sacrifice. If the AI project moves to Outer Mongolia you will be expected to move there too. Your commitment is permanent; once you become an expert on a piece of the AI, on a thing that doesn't exist anywhere else on Earth, there will be no one who can step into your shoes without substantial delay. Ask yourself whether you would still want to be a seed AI programmer if you knew, for certain, that you would die as a result - the project would still have a reasonable (but not certain) chance of success; but, win or lose, you would die before seeing the other side of the Singularity.

Do you, having fully felt the weight of those thoughts, but balancing them against the importance of the goal, want to be an AI programmer anyway? If so, you still aren't thinking clearly. The previous paragraphs are not actually relevant to the decision, because they describe personal considerations which, whatever their relative emotional weight, are not major Existential Risks. You should be balancing the risks to Earth, not balancing the weight of your emotions - you cannot count on your brain to do your math homework. If you reacted emotionally rather than strategically, you may not have the right frame of mind to act calmly and professionally through a Hard Takeoff. That too is a job requirement. I have to ask myself: "Will this person panic when s/he realizes that it's all really happening and it's not just a fancy story?"

Certain people are reading this and thinking: "Oh, but AI is so hard; everyone fails at AI; you should prove that you can build AI before you're allowed to talk about people panicking when it all starts coming true." They probably aren't considering becoming AI programmers. But for the record: It doesn't matter how much macho hacker testosterone has become attached to the problem of AI, it makes no sense to even try to build a seed AI unless you expect that final success would be a good thing, meaning that it is your responsibility to care about the complete trajectory from start to finish, including any problem that might pop up along the way. It doesn't matter how far off the possibility is; the integrity of the entire future project trajectory must be considered and preserved at all points. If someone would panic during a Hard Takeoff, you can't hire them; you're setting yourself up to eventually fail even if you do everything right.

The tasks that need to be implemented to create a seed AI can be divided into three types:

First, there are tasks that can be easily modularized away from deep AI issues; any decent True Hacker should be able to understand what is needed and do it. Depending on how many such tasks there are, there may be a limited number of slots for nongeniuses. Expect the competition for these slots to be very tight.

Second are tasks that require a deep understanding of the AI theory in order to comprehend the problem.

There's a tradeoff between the depth of AI theory, the amount of time it takes to implement the project, the number of people required, and how smart those people need to be. The AI theory we're planning to use - not LOGI, LOGI's successor - will save time and it means that the project may be able to get by with fewer people. But those few people will have to be brilliant.

What kind of people might be capable of learning the AI theory and applying it?

Aside from anything else, they need to be very smart people with plenty of raw computational horsepower. That intelligence is prerequisite for anything else because it's what would power any more complex abilities or skills.

Within that requirement, what's needed are people who are very quick on the uptake; people who can successfully complete entire patterns given only a few hints. The key word here is "successfully" - thanks to the way the human brain is wired, everyone tries to complete entire patterns based on only a few hints. Most people get things wildly wrong, don't realize they have it wildly wrong, and resist correction.

But there are a few people I know, just a few, who will get a few hints, complete the pattern, and get it all right on the first try. People who are always jumping ahead of the explanation and turning out to have actually gotten it right. People who, on the very rare occasions they get something wrong, require only a hint to snap back into place. (My guess is that some people, in completing the pattern, "force link" pieces that don't belong together, overriding any disharmony, except perhaps for a lingering uneasy feeling, soon lost in the excitement of argument. My guess is that the rare people with a talent for leaping to correct conclusions don't try to build patterns where there are any subtle doubts, or that they more rapidly draw the implications that would prevent the pattern from being built.)

Take, for example, people hearing about Evolutionary Psychology for the first time. Most people I've seen immediately rush off and begin making common mistakes, such as postulating group selection, seeing evolution as a benevolent overarching improvement force, confusing evolutionary causation with cognitive motivation, and so on. When this happens I have to lecture on the need to become acquainted with the massive body of evidence and accumulated inference built up about which kinds of evolutionary reasoning work, and which don't - the need to hammer one's intuitions into shape. When I give this lecture I feel like a hypocrite, because I, personally, needed only a hint for the picture to snap into place. I've also seen a few other people do this as well - immediately get it right the first time. They rush to conclusions just as fast, but they rush to correct conclusions. They don't need to have the massive body of accumulated evidence hammered into them before they let go of their precious mistakes; they just never postulate group selection or evolutionary benevolence to begin with. No doubt many people reading this will have leapt to a right conclusion on at least one occasion and are now selectively remembering that occasion, but on the other hand the very rare people who genuinely do have very high firepower in this facet of intelligence should recognize themselves in this description.

If I were to try quantifying the level of brainpower necessary, my guess is that it's around the 10,000:1 or 100,000:1 level. This doesn't mean that everyone with a 160 IQ or 1600 SAT will fit the job, nor that anyone without a 160 IQ or 1600 SAT is disqualified. Standardized tests don't necessarily do a very good job of directly measuring the kind of horsepower we're looking for. On the other hand, it isn't very likely that the person we're looking for will have a 120 IQ or a 1400 on the SAT.

I sometimes run into people who want to work on a seed AI project and who casually speak of "also hiring so-and-so who's a senior systems architect at the place I work; he's incredibly smart". The "incredibly smart" senior systems architect is probably at around the 1,000:1 level; the person who used the phrase "incredibly smart" to describe that level of intelligence is probably around the 100:1 level. That's not good enough to be an AI programmer; even leaving aside the ethical requirements, this is simply out of range for positions that can be filled by hiring friends of friends. Maybe if you went looking on the Extropians mailing list, or a Foresight Gathering, you would find a few potential AI programmers. To expect to find an AI programmer at your workplace... it's not going to happen. Too much of a coincidence.

The theory of AI is a lot easier than the practice, so if you can learn the practice at all, you should be able to pick up the theory on pretty much the first try. The current theory of AI I'm using is considerably deeper than what's currently online in Levels of Organization in General Intelligence - so if you'll be able to master the new theory at all, you shouldn't have had trouble with LOGI. I know people who did comprehend LOGI on the first try; who can complete patterns and jump ahead in explanations and get everything right, who can rapidly fill in gaps from just a few hints, who still don't have the level of ability needed to work on an AI project. You need to pick up the theory very rapidly and intuitively, so that you can learn the practice in less than a lifetime.

To be blunt: If you're not brilliant, you are basically out of luck on being an AI programmer. You can't put in extra work to make up for being nonbrilliant; on this project the brilliant will be putting in extra work to make up for being human. You can't take more time to do what others do easily, and you can't have someone supervise you until you get it right, because if the simple things have to be hammered in, you will probably never learn the complex things at all.

So you'll learn AI programming after the Singularity and probably get more real enjoyment out of it than we did, because you won't be rushed.

Very few people are qualified to be AI programmers. That's how it goes.

(I'm sorry if you got the point a while back and I've just been hammering it in since then, but it takes a lot of effort to pry some people loose from the idea.)

In the third category are tasks that can only be accomplished by someone capable of independently originating and extending AI theory. We will have to hope that all of these jobs can be done by one person, because one person is all that we are likely to have.

I distinguish between programmers, AI programmers, and Friendly AI programmers, corresponding to the three task types.

Some people are reading this and thinking: "Well, Eliezer, brilliant people are hard to find; you may have to compromise." You'll be glad to know that I did compromise. I compromised away from requiring that an AI programmer be capable of independently extending the AI theory. Unfortunately I think that's as far as the compromise can go and retain what seems like a realistic probability of success. I still worry that being brilliant enough to grasp AI theory is not enough, and that it may turn out a Friendly AI project cannot in fact succeed without many people of the caliber needed to invent AI theory. But it seems like people of that level might be impossibly hard to find, and I see a realistic chance of succeeding with the merely brilliant - people who fully comprehend the theory, even if they would not have been capable of inventing it and cannot take it further, as long as they have enough hackerish creativity to apply the theory, translate it into a systems design, without a Friendly AI programmer needing to hover over them constantly.

The inherent difficulty of the problem is logically unrelated to what seems like a "reasonable demand", or what's easy to accomplish with resources we already have. The worker bees of an AI project would be queens in any other hive, people who would ordinarily be inventing their own brilliantly original ideas and translating them into designs. Finding that class of people will not be easy. But the idea that I am supposed to personally exert enough design energy to translate an entire AI into a drone-codable specification is hopeless. I would burn out like dry grass. I will count it a success if I can push the project over the impassable part of the AI hump, so that the rest of the work can be done by people with enough creative energy that they would ordinarily be starting their own original complex projects from scratch.

Hopefully this leaves some margin for error, so that there is enough free creative energy in the project to handle unexpected obstacles. The other problem with "compromising" on people who are less than genuinely brilliant is that the project would always be struggling - no safety margin. Such a project will fail the first time something pokes it.

"What should I study in order to join the team?"

Are you looking to fill one of the nongenius slots? If so, the primary prerequisite will be programming ability, experience, and sustained reliable output. We will probably, but not definitely, end up working in Java. Advance knowledge of some of the basics of Cognitive Science, as described below, may also prove very helpful. Mostly, we'll just be looking for the best True Hackers we can find.

But bear in mind that other requirements are universal - we can't "just hire" programmers, even if you know some really smart guy at your workplace, etc. See the later comments about ethics, and the earlier comments about people who might panic in a crisis.

"What should I study to become an AI programmer?"

At minimum you will need to grasp the elementary ideas of information, entropy, Bayesian reasoning, and Bayesian decision theory; the things that bind together the physical specification of a system and its cognitive content. You should see the "information content" of a pattern as its improbability or its Kolmogorov complexity, rather than ones and zeroes on a hard drive; you should be capable of explaining how the scientific method and conversational argument and the visual cortex are all really cleverly disguised manifestations of Bayes' Theorem; you should distinguish between subgoals and supergoals on sight and without needing to think about it.

You should be familiar with the Design Signature of natural selection - optimization by the incremental recruitment of fortunate accidents, following pathways in fitness gradients which are adaptive at each intermediate point and which are directed in the maximally adaptive direction at each intermediate point. Not because we're going to use evolution, of course, but because you need to know what a nonhuman design process looks like and how it behaves nonanthropomorphically. Evolution is the only current example of this around.

You should be familiar with Evolutionary Psychology and game theory, not because we're planning to build an imperfectly deceptive social agent or make it play zero-sum games, but so that you can have applied this knowledge to debugging your own mind, and so that you can detach your knowledge of morality from the human domain.

You should be familiar with at least a few specific examples of the intricacy of biological neural circuitry and computing in single neurons and functional neuroanatomy, not because we're going to be imitating human neurology, but so that you don't expect cognition to be simple. You should be familiar with enough cognitive psychology of humans that you know human reasoning is not Aristotelian logic - for example, Gigerenzer's fast and frugal heuristics, Tversky and Kahneman, and so on; being able to interpret all of this as an imperfect approximation of Bayes' Theorem is another plus.

You should have read through Levels of Organization in General Intelligence and understood it fully. The AI theory we will actually be using is deeper and less humanlike than the theory found in LOGI, but LOGI will still help you prepare for encountering it.

The four major food groups for an AI programmer:

• Cognitive Science
• Evolutionary Psychology
• Information theory
• Computer programming

Breaking it down:

• Cognitive Science
  • Functional neuroanatomy
    • Functional neuroimaging studies
    • Neuropathology; studies of lesions and deficits
    • Tracing functional pathways for complete systems
  • Computational neuroscience
    • Suggestions: Take a look at the cerebellum, and the visual cortex
  • Computing in single neurons
  • Cognitive psychology
    • Cognitive psychology of categories - Lakoff and Johnson
    • Cognitive psychology of reasoning - Tversky and Kahneman
  • Sensory modalities
    • Human visual neurology. Big, complicated, very instructive; knock yourself out.
  • Linguistics
  • Note: Some computer scientists think "Cognitive Science" is about Aristotelian logic, programs written in Prolog, semantic networks, philosophy of "semantics", and so on. This is not useful except as a history of error. What we call "Cognitive Science" they call "brain science". I mention this in case you try to take a "Cognitive Science" course in college - be sure what you're getting into.
• Evolutionary Psychology
  • Popular evolutionary psychology; dating and mating; Robert Wright and Matt Ridley
  • Formal evolutionary psychology; neo-darwinian population genetics and complex adaptation; Tooby and Cosmides
  • Game theory; nonzero-sum and zero-sum games
  • Evolutionarily stable strategies for social organisms
  • Tit-for-tat, the evolution of cooperation, the evolution of cognitive altruism
  • Evolutionary psychology of human "significantly more general" intelligence
    • Mostly this means reading LOGI; there's not much else out there.
    • But see also Lawrence Barsalou and Terrence Deacon.
  • Evolutionary Biology
    • Incrementally adaptive pathways; levels of organization; etc.
  • Biology (a complex system not designed by humans)
  • Genetics
  • Gene regulatory networks (another good look at evolution's bizarre signature, and also a look at the way humans actually get constructed)
  • Quantitative genetics
  • Anthropology - the good old days
• Information theory
  • Shannon communication theory
    • Shannon entropy
    • Shannon information content
    • Shannon mutual information
  • Kolmogorov complexity
    • Solomonoff induction
  • Bayesian statistics
    • Interpretation of human thought as Bayesian inference - see Jaynes.
  • Any other kind of statistics
  • Utilitarian Bayesian decisionmaking
    • Decision theory is not classically part of "information theory", but does, in fact, belong together with the other items in this category
    • Child goals, parent goals, "supergoals" (intrinsic desirability)
    • Actions and desirability - read Creating Friendly AI as a preliminary to the longer story.
• Computer programming
  • Knowledge of many languages
  • Java programming (that's probably what we'll end up doing it in)
  • Being an excellent programmer
  • Parallelism
    • Multithreading
    • Clustered and distributed computing - we may not need this for a while, but then again, we may
  • Any kind of experience working with complicated dynamic data patterns controlled by compact mathematical algorithms - some of the interior of the AI may end up looking like this
• Other stuff
  • Computer security (experience with defensive caution; not that it's sufficient for Friendly AI or even a good attitude, but it's a start)
  • Physics
  • Thermodynamics
    • The second law of thermodynamics
      • Noncompressibility of phase space
      • The arrow of time and the development of complex structure
  • Traditional AI methods
    • History of error; don't repeat past mistakes
    • We might reuse one or two design patterns at some point
  • Transhumanism or transhumanist SF - sufficient exposure to have a very high future shock tolerance; helps to "take it all in stride"
  • Mathematics

Obviously we are not requiring a doctorate, in Cognitive Science or any other field, because no doctorate is going to tell you one-fifth of what you need to know. (I am tempted to say that a doctorate in AI would be negatively useful, but I am not one to hold someone's reckless youth against them - just because you acquired a doctorate in AI doesn't mean you should be permanently disqualified.)

What you need is not an academic specialization in any one of these fields, but an interested amateur's grasp of as many of them as possible. An AI programmer doesn't need to independently pull together a single consilient explanation from this mess of separate disciplines. What is useful is if you understand the basics of these separate disciplines as they are usually understood, so that a Friendly AI programmer who does know the consilient explanation has a shared language in which to describe specific cognitive processes.

If you are to be able to implement cognitive processes on your own without constant supervision, you will need to be very fast on the uptake, fill in patterns from a few hints and get them right the first time, learn new skills rapidly and easily, discriminate subtle distinctions without needing them hammered into you, and have a great deal of plain good-old-fashioned mental horsepower. Even so, much of your first days on the project will still consist of having your suggestions shot down over and over again, until you pick up the knack. For every AI problem there are a million plausible-sounding methods that do nothing; a thousand methods that are sort of right or almost right or work a little but then break down; and a small handful of manifestations of the right way. It's going to take a while to catch the rhythm of how this works, and learn to distinguish solutions that accomplish something from the ones that spin their wheels and go nowhere.

"Gee, y'know, I think I'll start my own AI project, in like my spare time while I'm finishing high school. How much effort does it take to become a Friendly AI programmer?"

If you are not willing to devote your entire life to doing absolutely nothing else, you are not taking the problem seriously. By that I mean your entire life. I don't mean that you do it as a hobby. I don't mean that you do it as your day job. I don't mean that you give it all your time and energy. I mean that you allocate your entire self to be sculpted by the problem into the shape of a Friendly AI programmer.

There is nothing unusual about that. Consider human history, and how many people have sacrificed so much more for so much less. I don't consider myself at all remarkable by comparison with millions of anonymous people throughout human history who willingly walked into much more inconvenience, pain, and death than I ever have experienced or expect to experience. Total dedication is something that plenty of people can and do undertake, and it isn't anywhere near as painful as commonly thought.

That is what it takes to start your own AI project, as a necessary but nowhere near sufficient condition. You will also need to be as smart as it gets. In terms of raw horsepower, there are probably tens or hundreds of anonymous world-class geniuses around for each one that makes it into the history books. It also takes luck, perseverance, the right place at the right time, a field in chaos so that it can be set in order, etc. But it is nonetheless true that only a world-class genius has any hope at all.

"How much effort does it take to become an AI programmer?"

If you want to be an AI programmer you should be willing to devote most of your life and yourself to AI. All of yourself would be better, but is not strictly necessary, and I am aware that it is considered hard.

"What does it take to get a security clearance?"

There are two approaches to ethics. The first is to try and set up a set of tests to try and keep out noticeably bad people. That is, you start with an "allow all" policy and then add denials.

The second approach, which makes me feel a bit ashamed because it seems exclusionary and unfair, is to say: "Even if someone passes every formal test I can think of, I'm not going to hire someone unless they strike me as unusually trustworthy, because anyone else is just too much of a risk."

Having considered this at some length, I think that only the second option is safe. If something feels wrong, but you can't really think of a "justification" for avoiding it, avoid it anyway. Doing something that makes you feel a little bit uneasy, without really being able to say why, can be much more dangerous than taking a known calculated risk. Often it means you know so little that you don't have any idea how much danger you're in or how large a risk you're taking.

I know some people whose ethics are such that I would actually feel good about adding them to a Friendly AI project. The thought of adding anyone else makes me feel a little bit uneasy.

You've probably read "The Lord of the Rings", right? Don't think of this as a programming project. Think of it as being something like the Fellowship of the Ring - the Fellowship of the AI, as it were. We're not searching for programmers for some random corporate inventory-tracking project; we're searching for people to fill out the Fellowship of the AI. Far less important projects get hundreds of millions of dollars and thousands of programmers. What is our substitute? Knowing exactly what we're doing. Having exceptional people on the team. Otherwise it simply isn't realistic. We are gathering for the last defense of Earth. You can't fill those positions by running a classified ad.

From the standpoint of personnel management, the original Fellowship of the Ring was a disaster. The only real heavyweights were Gandalf and Aragorn; two out of nine. Legolas and Gimli were added to fill minority quotas. Boromir was allowed onto the team despite being blatantly unreliable. No fewer than three useless hobbits shoved themselves onto the team, and nobody screwed up the courage to kick them off. The Fellowship splintered almost immediately after it was assembled. In real life, the Fellowship would have been strawberry jam on toast. We can, and must, do better.

This is Earth's finest hour, and Earth's finest should meet it.

Much of what I have written above is for the express purpose of scaring people away. Not that it's false; it's true to the best of my knowledge. But much of it is also obvious to anyone with a sharp sense of Singularity ethics. The people who will end up being hired didn't need to read this whole page; for them a hint was enough to fill in the rest of the pattern.

Of course you would have to dedicate your life to a Friendly AI project. Of course you have to be brilliant. It's a Friendly AI, for goodness's sake! It's asking enough of human flesh and brain that we build one at all, let alone that we do it with less than our full effort.

The standards are set terribly high because this is a terribly high thing to attempt. We cannot be "reasonable" about the requirements because Nature, herself, has no tendency to be "reasonable" about what it takes to build a Friendly AI. This is and always was an unreasonable problem.

No one who might actually be hired will be scared off by the thought of an extremely difficult job or harsh competition. Even if they are brilliant people who haven't yet become comfortable admitting their brilliance to themselves, they will, I expect, say something to themselves along the lines of "If Eliezer hasn't gotten funding in a couple of years, he might have taken the design down to the point where it could be implemented by nongeniuses, or the Singularity Institute might need to start hiring and I, heaven help them, might end up being one of the best people available at that point. And if not, well, I was planning to spend the first hundred years after the Singularity studying Cognitive Science anyway, so I'll get a head start." Or whatever. If someone is brilliant enough to have any realistic chance of becoming an AI programmer, and ethical enough to be accepted, nothing I could possibly say would scare them away from applying. They are not thinking in terms of "scared/not scared"; they are thinking in Singularity ethics.

If that describes you, please subscribe to the mailing list seedAIwannabes@singinst.org by sending mail to majordomo@singinst.org. Plenty of people have expressed vague interest, but SIAI has very few serious candidates. Right now we do not have enough AI programmers to put a team together, and that is a blocker problem even if we had a million dollars tomorrow. Subscribe even if you're not sure you're totally brilliant; if we eventually accumulate a bunch of totally brilliant people and it's clear you're outclassed, you can always withdraw yourself from consideration at that point. Meanwhile it may prove useful to have gathered a group of the smartest people available at any given point.

If that doesn't describe you, then again I emphasize that not everyone has to be an AI programmer! You don't have to be an AI programmer to help! It may be frustrating to see something exciting, like the superintelligent transition, and not be able to run off at it immediately, but even getting the project started is a serious strategic problem with many simultaneous dependencies. Don't try to solve the entire problem in a single step. See if you can figure out how to make one definite contribution. Then you can go on to consider expanding it.

A revision has rendered obsolete my previous comment. Also, it has addressed a primary concern, which is we don't have time to build Seed AI programmers, we must find them. --Justin Corwin

Ironic Humor Dept.: "We assume that ... Liouville's theorem holds. This assumption is known as adiabatic incompressibility of phase space, or AI for short." -- Mitchell Porter

This bit of ironic humor was rendered inapplicable by a revision. Sorry, Mitchell. -- Eliezer Yudkowsky

Legolas and Gimli were useful team members, they just couldn't independently extend ring theory. -- Starglider

I have to wonder about the psychological effects of this essay on potential programmers. What if they're not overly enthusiastic about the idea, but they fulfill the requirements, and are in complete rational agreement with the essay, and would be willing to do that sort of work? What if they simply don't have the self-confidence necessary to overcome the intertia of living life? What if your strict requirements invoke someone's sense of modesty? The most capable people are often the most modest.

It would require a huge jolt to get someone who agreed with the Sing. Inst.'s values to put in all the extra study required to round out their knowledge on the chance that they'd be accepted as a member of the team, especially if that meant getting deep into one or two fields that as of yet have no experience in. Are you going to exclude people who don't already know almost everything they need to know to apply immediately for the position? --Chris Capel

It doesn't require a 'huge jolt'. It requires being a moral utilitarian. If there is a significant chance that devoting the rest of your life to studying seed AI will save six billion lives and enable a vastly richer future, you should do it. If you understand the issues deeply enough to consider this in the first place, the only question is 'would the increase in the chance of being a seed AI programmer be worth the drop in the level of donnations I could make to the SIAI, if I focused on that'. Even that's not necessarily a problem; in my experience focusing on learning seed AI has actually increased my expected financial contribution to the SIAI. -- Starglider

My opinion: Everyone who subscribed to the list with the words 'AI WANNABE' in it should immediately be eliminated from consideration. -- Marc Geddes

Why? Are you kidding? --Some Anonymous Guy

So if someone want to get power over the world he must try to get into your teem - Alexei Turchin

No, all you need to do is steal the source code for the AI, at the right time. No need to go through the effort of pretending to be a dedicated altruist, to get accepted into the team. --PeerInfinity 00:34, 4 October 2009 (UTC)

But how to steal the code? To be a cleaner in the room? Good story for a novel. Another question: If one filantrope suggest 1 billion dollars in exchage for small additition in the rules of friendly AI - e.g. that this AI must create colony of red ants on the Moon - should this suggestion be accepted? - [Alexei Turchin]--92.243.167.227 10:19, 5 October 2009 (UTC)

I found that quote I was looking for: "Because of SIAI’s strict ethics, no donor can have an impact on our thinking about AI friendliness unless that donor is genuinely intellectually persuasive and knowledgeable." --PeerInfinity 13:31, 19 October 2009 (UTC)

I regularly read this page because I enjoy the tone of sheer arrogance.