Can we get a Challenge historical wrapup/tally?

[Sherman Bay]

If this data is posted elsewhere, forgive me and point me in the right direction. This is not about Kramer's log of recent apps, which is great as far as it goes.

How about a historical summary of The Challenge? I would be curious to know:

1. When did The Challenge first take its present form? I know there was a $10,000 version previous, but when did it become $1Mil?
2. How many notarized applications have been received to date?
3. How many preliminary tests have been agreed to?
4. How many actually took place? (Dates, and locations would be nice, but I'll wait for the book for that if it's too much work to post here.)
5. How many pending?
6. How many passed?
7. What would you say is the "best" test, and by that, I mean the one with the highest result tally, where the applicant came closest to his goal? And was this still within the bell curve of chance?

It might also be interesting to see the boiled-down statistics on individual tests. I realize each one is different, and is scored differently, but here's an example: The claimant said he could guess right 5 times out of 10. Chance says he would score 1 of 10. Results: 20 trials, highest score: 4, lowest score: 0. Statistical expectation: (whatever, IANAStatistician).

To cap it off and inject a little levity, how about summarizing the reason for failure according to the applicant?

Okay, okay, I know it's a lot of work. But it could be really useful for reference when the Challenge gets attacked. And I'll even volunteer to do some of the work, if that is possible.

How about it?

[kookbreaker]

Originally posted by Sherman Bay
If this data is posted elsewhere, forgive me and point me in the right direction. This is not about Kramer's log of recent apps, which is great as far as it goes.

How about a historical summary of The Challenge? I would be curious to know:

1. When did The Challenge first take its present form? I know there was a $10,000 version previous, but when did it become $1Mil?
   

1. Mid 90's, IIRC It was first backed just by pledges by various skeptics (the $1000 club). Later it was the JREF operating fund.
   
   

   How many notarized applications have been received to date?
   

   
   I have no idea.
   
   

   How many passed?
   

   
   Actually, in its older incarnation (during the days of snail mail communication) there were some who passed the preliminary test. (see Flim-Flam for details). These days, I don't think anyone has passed the prelim. For the most recent test, look up the Yellow-Bamboo thread. The videos are a hoot!
   
   

   What would you say is the "best" test, and by that, I mean the one with the highest result tally, where the applicant came closest to his goal? And was this still within the bell curve of chance?

Often quoted here by a certain Randi-basher was the Australian Dowsing test:

http://www.skeptics.com.au/journal/divining.htm

Overall the tests showed dowsing failing at chance, but the water test in that case was about 22%, which is about double chance, but still well within the bell curve for chance. It is still a failure, since it is about 64% lower than the lowest rate the dowsers claimed to be able to do. Some have harped on the water test as some kind of success, but it simply is not so. Especially in view of the overall failures in dowsing tests.

It might also be interesting to see the boiled-down statistics on individual tests. I realize each one is different, and is scored differently, but here's an example: The claimant said he could guess right 5 times out of 10. Chance says he would score 1 of 10. Results: 20 trials, highest score: 4, lowest score: 0. Statistical expectation: (whatever, IANAStatistician).

This comes up a lot, but the fact is that the JREF just does not have enough time and resources to get all the stuff on line. KRAMER is working with challenge applicants pretty much full time and when there is a respite he works on back entries and such.

You are correct, it is a lot of work. Keep in mind that the JREF is Randi, Linda, KRAMER, and maybe an intern. They all have things that take up their time.

[Gr8wight]

I recently exchanged PMs with Kramer about this topic. I here attach the reply he sent me.

(My questions appear in bold. Kramer's responces in italicised bold.)

Re: Challenge stats

Gr8wight wrote on 02-01-2005 09:38 AM:
Hello Kramer,
You may heve seen my post in the forums on this topic. I am interested in knowing if there are any basic statistics available regarding past challenge applications. For example:

1) How many properly executed challenge applications have been received?

According to my files, 352, of which 298 are CLOSED, and 54 OPEN. Within 8 weeks, however, at least 20 of the OPEN files will be closed, for lack of activity, cessation of correspondence, or other reasons. The application remains open for one year. If within that time the applicant vanishes or refuses to agree to an acceptable test protocol, the file is CLOSED, and they may re-apply in 12 months time.

FYI, we do NOT simply close a file if the applicant is maiing best efforts to work out a protocol. No file has EVER been closed during protocol negotiations. They must come to a complete status quo before we will consider closing the file on an applicant.



2) Of those, how many applicants have proceded to the preliminary testing stage?

During my tenure (since 3-2004), only one.

3) I know the answer to this one: No applicants have ever
passed a preliminary test. (correct?)

Correct.

4) Of those preliminary tests, do we know the breakdown of how many were claimed dowsers, psychics, etc.?

No, we don't have an exact figure on that, but the great majority of applicants who actually agreed to a protocol (and were tested) were dowsers. I'd say about 50% if not more, with "remote readers", psychics and ESP claims making up the remainder.

I should really do a head count in March, once I've been here a full year. I'd like an exact figure on this, too. But this would mean opening and reading every file, and that's a big job.

-Kramer, JREF

__________________
KRAMER,
JREF Paranormal Claims Dept.

In susbsequent PMs Kramer clarified that the 352 properly executed challenge applications went back to 1997, prior to which the challenge was only $10,000. He was not able to tell me how many preliminary tests were done prior to his joining the JREF last March, just that only one test has taken place since then. Although, the recent Yellow Bamboo test would probably be the second.

I hope I have not stepped out of line by posting this. If I have, let me know Kramer, and I will dutifully extend my hand for the requisite slappage.

[Sherman Bay]

This may answer part of my question:

OUR CHALLENGE APPLICATIONS FILES...The hardworking Kramer, volunteering his services here at the JREF, has announced that the total list of applicants at this date [June 2004] amounts to 323. They are:

65 "open" files
5 currently being negotiated
253 "closed" cases — including many that were without an official application form, were in a foreign language, arrived without return addresses, were textually incomprehensible, or were so badly handwritten that we could not read them.

it doesn't say how many tests have been performed.

[Sherman Bay]

Originally posted by Gr8wight
[Kramer] was not able to tell me how many preliminary tests were done prior to his joining the JREF last March, just that only one test has taken place since then. Although, the recent Yellow Bamboo test would probably be the second.

Thanks for the info, Gr8wight; your figures are more recent than the commentary page I quoted. I must really learn to type faster. And maybe keep fewer threads open on fewer boards simultaneously!

A capsule wrapup of each test might be just what the doctor ordered, like this:

• Date app received
• Date test performed
• Location of test
• Claim of applicant
• Protocol
• Raw results
• Statistical analysis
• Excuse(s) given by applicant for non-performance

I include the statistical analysis only to placate those who are not familiar with the laws of chance, and to show how close or how far from the original claim the results were.

Hey, scan or xerox the files and send me the scans or dupes -- I'll do the tally!

[Gr8wight]

Originally posted by Sherman Bay
Thanks for the info, Gr8wight; your figures are more recent than the commentary page I quoted. I must really learn to type faster. And maybe keep fewer threads open on fewer boards simultaneously!

A capsule wrapup of each test might be just what the doctor ordered, like this:

• Date app received
• Date test performed
• Location of test
• Claim of applicant
• Protocol
• Raw results
• Statistical analysis
• Excuse(s) given by applicant for non-performance

I include the statistical analysis only to placate those who are not familiar with the laws of chance, and to show how close or how far from the original claim the results were.

Hey, scan or xerox the files and send me the scans or dupes -- I'll do the tally!

Kramer displayed a marked reluctance to go digging into the older files in order to coallate this kind of data. The impression I got is that they are, perhaps, not in a state of general orderliness.

[KRAMER]

Sorry but I cannot answer many of these questions, such as "Who performed best at testing". I've only been here a year, and answering a question such as this would take weeks of research deep into the files.

However, my understanding is that most applicants fail rather gloriously, performing far below CHANCE. According to everything I have been told, no one has performed well, or demonstrated anything that even came close to warranting further inquiry.

Presently, there are 305 CLOSED files, and 50 OPEN, and about 20 of those are presently "pending" with ongoing protocol negotiations. Of those 50, about 30 will be closed by the end of March, because files remain open for 12 months before being closed due to inactivity. We never close a file on an applicant with whom we are actively corresponding.

About 15 applicants are presently negotiating the specifics of the terms of their test with skeptics groups and independent investigators around the globe. Many of these negotiations go on for months before the applicant finally disappears. VERY few tests take place, as very few applicants ever agree to an acceptable test protocol. There have been two tests since my arrival here one year ago. My understanding is that there has never been more than a couple of tests per year.

No one has ever passed preliminary testing, and hence, no final/formal test has ever taken place. That's common knowledge by now, I would hope.

I regret that I am unable to give you more finite data at this point in time.

[jmercer]

Originally posted by KRAMER
However, my understanding is that most applicants fail rather gloriously, performing far below CHANCE.

That's interesting...

[CFLarsen]

Originally posted by KRAMER
However, my understanding is that most applicants fail rather gloriously, performing far below CHANCE.

Playing the Devil's Advocate here, but...what if my claim was that I would always perform far below chance? That I had negative PSI?

That's gotta be paranormal, right?

[jmercer]

You read my mind, CF. Quick, apply!

Even though I'm taking Kramer's statement as simply an opinion (instead of a real statistic), if it were true it could mean any number of highy interesting things.

[Metullus]

Perform below Chance!

I am in the money.

1. I will always be in the slowest moving lane of traffic.
2. If I change lanes it will become, in mere seconds, the slowest lane of traffic.
3. I Always live in an area whose local NFL team finishes last or second to last for the season. Even when I move, the new team instantly starts to loose.
4. I recently lost 63 straight coin tosses. Out of 63. Fair coin. I called 31, my daughter called 32. I was trying to teach her about probability. She thinks I am retarded.

[jmercer]

Originally posted by Metullus
4. I recently lost 63 straight coin tosses. Out of 63. Fair coin. I called 31, my daughter called 32. I was trying to teach her about probability. She thinks I am retarded.

If your daughter is between the ages of 10 and 19, the last part has nothing to do with the coin toss.

[Metullus]

Originally posted by jmercer
If your daughter is between the ages of 10 and 19, the last part has nothing to do with the coin toss.

Nah, she just turned 9. But a very mature 9. I am not looking forward to the next several years.

Does anybody know of a convent that will take a 9 year old for the next 12 or so years?

[jmercer]

Nope, sorry. But if it helps, it only hurts until you lose your sanity entirely.

[Metullus]

I am learning to drink. Think that'll help?

[jmercer]

Only if you do it alone.

[Metullus]

You are right. Its more efficient that way.

[Beady]

Originally posted by kookbreaker
Keep in mind that the JREF is Randi, Linda, KRAMER, and maybe an intern. They all have things that take up their time.

True, but what about a co-op, or even a just-plain volunteer? I'm not due for retirement for another 10 years or so, but I'd love an excuse to get down there and root around in the files.

[T'ai Chi]

Originally posted by KRAMER

However, my understanding is that most applicants fail rather gloriously, performing far below CHANCE. According to everything I have been told, no one has performed well, or demonstrated anything that even came close to warranting further inquiry.

Instead of "understandings" and things you've "been told", hopefully some day we'll see actual prelim statistics (without having to fly down and sort through files) made freely available for interested parties to examine.

[kookbreaker]

Originally posted by jzs


Instead of "understandings" and things you've "been told", hopefully some day we'll see actual prelim statistics (without having to fly down and sort through files) made freely available for interested parties to examine. [/b]

Then you'll have to find something else to complain about.

Several challenge results have been posted in various media over the years.

You have deliberately ignored that.

[T'ai Chi]

Originally posted by kookbreaker
Then you'll have to find something else to complain about.

Only seeing recollections and etc. and not actual data is a valid complaint, I would say. If you prefer recollections and a piece of data here, a piece of data there, that may work for you.

Several challenge results have been posted in various media over the years.

Actual data is appreciated. "various media" doesn't really cut it, nor make it easily accessible to interested parties, wouldn't you say?

[T'ai Chi]

Originally posted by jzs

Only seeing recollections and etc. and not actual data is a valid complaint, I would say. If you prefer recollections and a piece of data here, a piece of data there, that may work for you.
[/b]

For example, kookbreaker, when KRAMER says

"However, my understanding is that most applicants fail rather gloriously, performing far below CHANCE."

What does "performing far below CHANCE" mean specifically? How far below? What are the probabilites and the sample sizes? Is the event of most applicants falling far below chance itself going against what chance would expect?

We have no idea from such statements. I'm curious, however, and I suspect many people, skeptics and "woo's" alike.

[CFLarsen]

Originally posted by jzs
Instead of "understandings" and things you've "been told", hopefully some day we'll see actual prelim statistics (without having to fly down and sort through files) made freely available for interested parties to examine.

Instead of making snide remarks on an Internet board, hopefully some day you'll get your head out of your ass and fly down to JREF to actually do some work of your own.

You are not the boss of JREF. Or anybody else.

[kookbreaker]

Originally posted by jzs
For example, kookbreaker, when KRAMER says

"However, my understanding is that most applicants fail rather gloriously, performing far below CHANCE."

What does "performing far below CHANCE" mean specifically? How far below? What are the probabilites and the sample sizes? Is the event of most applicants falling far below chance itself going against what chance would expect?

We have no idea from such statements. I'm curious, however, and I suspect many people, skeptics and "woo's" alike.

Then I suggest you start reading the material you can get ahold of. There are several accounts of tests in Randi's weekly, a few more online, a few more described on USENET, and several in Flim-Flam. The base material is in FLA. If you want a more thorough answer.

Get working.

[T'ai Chi]

Originally posted by CFLarsen
Instead of making snide remarks on an Internet board, hopefully some day you'll get your head out of your ass and fly down to JREF to actually do some work of your own.

My comments are valid. What does "performing far below CHANCE" mean specifically? How far below? What are the probabilites and the sample sizes? Is the event of most applicants falling far below chance itself going against what chance would expect?

Can you answer me, Claus?

You are not the boss of JREF. Or anybody else.

That is why I am asking, Claus. See the pretty question marks above?

[T'ai Chi]

Originally posted by kookbreaker
Then I suggest you start reading the material you can get ahold of. There are several accounts of tests in Randi's weekly, a few more online, a few more described on USENET, and several in Flim-Flam. The base material is in FLA. If you want a more thorough answer.

Get working.

More general appeals to 'go look' for the data.

The challenge has been in some form since the 60's, but no records or summaries of the prelims that are easily available (not taking time off from the job, paying for planes, a hotel room, and spending many, many full days there searching archives, etc.) to interested parties? That seems pretty important to me.

[kookbreaker]

Originally posted by jzs


My comments are valid. What does "performing far below CHANCE" mean specifically? How far below? What are the probabilites and the sample sizes? Is the event of most applicants falling far below chance itself going against what chance would expect?

Can you answer me, Claus?

That is why I am asking, Claus. See the pretty question marks above? [/b]

You'd better get to work if you want specific answers to your question.

[T'ai Chi]

Originally posted by kookbreaker
You'd better get to work if you want specific answers to your question.

The point is that if a JREF employee can't provide those details, how could I be expected to?

[kookbreaker]

Originally posted by jzs
The point is that if a JREF employee can't provide those details, how could I be expected to?

By going and doing the work that yields the results that you, exlcusively, are interested in.

[T'ai Chi]

Originally posted by kookbreaker
By going and doing the work that yields the results that you, exlcusively, are interested in.

Tons of Skeptics and non-skeptics are interested in those results, kookbreaker, not just me exclusively, as you claim. This thread, for example, was started because someone was interested in them. Right?

JREF's bread and butter is the challenge, yet no one seems to know much of the statistics on them, and a JREF employee believes it would take too much time to do.

That confuses me somewhat, as it seems a very important and desirable thing that would have been thought of. Oh well. Like I said, "hopefully someday".

[kookbreaker]

Originally posted by jzs
Tons of Skeptics and non-skeptics are interested in those results, kookbreaker, not just me exclusively, as you claim. This thread, for example, was started because someone was interested in them. Right?

No. The closest you get is:


"What would you say is the "best" test, and by that, I mean the one with the highest result tally, where the applicant came closest to his goal? And was this still within the bell curve of chance?"

JREF's bread and butter is the challenge, yet no one seems to know much of the statistics on them, and a JREF employee believes it would take too much time to do.

As does Randi.

That confuses me somewhat, as it seems a very important and desirable thing that would have been thought of. Oh well. Like I said, "hopefully someday".

Its funny how you expect everyone to do your work for you, and how you set the priorities for the JREF.

It might be of interest, but it really isn't that important. If you want to see it, you'll have to do it yourself. Whining like a spoiled child is not going to do much.

[T'ai Chi]

Originally posted by kookbreaker
No. The closest you get is:

The details and results of a test can be expressed through statistics, kookbreaker.

Additionally, Sherman added "statistical analysis". You left that out, kookbreaker.

Its funny how you expect everyone to do your work for you, and how you set the priorities for the JREF.

I haven't set any priorities. I do think it would be a good idea, however. If you believe differently, that is fine. People ask about the statistics of the challenge, so it seems important.

Whining like a spoiled child is not going to do much.

Making a sensible suggestion is not "whining", kookbreaker. Considering the tests are gauged by statistics, the suggestion is sensible, kookbreaker.

[CFLarsen]

Originally posted by kookbreaker
It might be of interest, but it really isn't that important. If you want to see it, you'll have to do it yourself. Whining like a spoiled child is not going to do much.

What he said.

[T'ai Chi]

Originally posted by CFLarsen
What he said.

If you want to ignore the actual data, be my un-skeptical guest.

[CFLarsen]

Originally posted by jzs
If you want to ignore the actual data, be my un-skeptical guest.

I don't find it particularly interesting, so why should I bother?

It's this "Justin Likes To Order The World Around" symptom we see again and again. You are a one trick pony.

[T'ai Chi]

Originally posted by CFLarsen
I don't find it particularly interesting, so why should I bother?

You do know you just admitted to not finding the actual data interesting, right?

Definitely saved for posterity.

It's this "Justin Likes To Order The World Around" symptom we see again and again. You are a one trick pony.

Am I a pony or fly excrement? You keep changing your mind.

In regards to the actual topic you desparetly wish to avoid by getting into personal attacks, Randi responded, and it sure is interesting. I'm emailing him now for permission to fwd the email or post it here.

[CFLarsen]

Originally posted by jzs
You do know you just admitted to not finding the actual data interesting, right?

No. I don't find it particularly interesting. There's a difference. Of course, you had to leave that out.

Originally posted by jzs
Definitely saved for posterity.

Please do. Just be sure to save it all.

[T'ai Chi]

Originally posted by CFLarsen
No. I don't find it particularly interesting. There's a difference. Of course, you had to leave that out.

Oh, "particularly", as if that changes the content or meaning of what you said in any way. So you admitted to not finding the actual data particularly interesting. It seems to me, that that is probably *the* most interesting thing, because that is the only way to judge success or failure and other important things, and put them in an objective numerical form.

Apparently one can get more data from the GCP than from the JREF challenge.