Natasha Demkina, |
The Girl with Very Normal Eyes
Photo Caption: Natasha Demkina examines the seven test subjects in hopes of correctly matching six target medical conditions to the correct subjects. As she normally works, it takes her about 10 minutes to give a complete reading of her patient's entire body, from top to toe. Although she was told what target abnormalities to look for, and where in the body to look, she spent four hours examining the subjects. In the end, she missed finding the target conditions in three of the seven subjects, and then wrongly identified the conditions in three subjects, which is a total of six misdiagnoses vs. only four correct "diagnoses." Natasha Demkina's supporters claim she has never made a misdiagnosis in 6 years of diagnosing patients with her special vision.
The subject with a metal plate in his forehead is the second man from the left. Although the plate was in full "view" of the "girl with x-ray vision" for most of four hours, she failed to "see" it. Instead, she stated she saw a metal plate in the man who is the second subject from the right, who has no such plate, but is missing his appendix. (Two subjects are out of view on the left. Walking in the back is Prof. Richard Wiseman, one of the three principal investigators who designed the CSMMH-CSICOP test.)
Believers in Natasha Demkina's 'X-ray Vision' Falsely Accuse CSMMH-CSICOP Investigators of 'Cheating' and 'Ambushing' the Young Russian Psychic.
Although the Discovery Channel documentary, The Girl with X-ray Eyes, has yet to be shown in the United States and Canada, its broadcast in Europe and Asia has already generated a lot of criticisms and protests about CSMMH-CSICOP's investigation of Natasha Demkina, the 17-year-old "medical psychic" from Saransk, Russia. Almost all of the criticism involves groundless accusations or mistakes and miscommunications that mostly were beyond the control of the investigators. Here are our answers to some of these charges:
Accusations: The debunking investigators deliberately produced a documentary for the purpose of humiliating Natasha Demkina and destroying her brilliant international reputation as a gifted psychic.
Answer: Neither the Commission for Scientific Medicine and Mental Health nor CSICOP had anything to do with the production of the Discovery Channel documentary. The program was produced by Monica Garnsey of Shine, Ltd. in the United Kingdom. Neither Monica or Shine, Ltd. or Discovery Channel have any relationship with CSMMH or CSICOP, whose investigators designed and conducted the test.
The Commission and CSICOP first heard of Shine, Ltd. earlier this year when the producer/director contacted CSICOP to ask its help in conducting a scientific test of Natasha Demkina's claims. The Commission's and CSICOP's only role in the documentary was to design and conduct the test. They had no input, influence over, or involvement in any other part of the production. As Monica Garnsey stated in email sent to two of our critics:
"I should say that in my opinion CSICOP's public service motivation is utterly sincere and, unlike the other people who made the programme possible, CSICOP did not ask for or receive payment except travel expenses. "
The rest of the accusation is similarly false.
Accusations: The investigation had been set up with a view to ensuring that Natasha Demkina would fail it.
Answer: This common accusation of believers in the paranormal shows a lack of knowledge about how science works. Failling to find evidence of abilities or forces previously unknown to science rarely if ever makes a researcher famous. We remember and honor Wilhelm Roentgen for discovering the powerful and mysterious form of light he called "X-rays" -- for which he was presented the first Nobel Prize in physics ever awarded. Few people today remember the names of other physicists of his day, who discovered nothing so remarkable. Says Prof. Hyman, "I, for one, would have found my time and efforts much more rewarding if she had passed the test."
Contrary to the accusations, the test was set up to assure that anyone who truly can see abnormalities inside of people's bodies would easily pass. All aspects of the test -- including what score would warrant further study -- were agreed to by the program's producer/director and by Natasha Demkina and her agent five days before the day of the test. The test was set up to see whether Natasha's claimed psychic abilities warrant a more carefully controlled study. It was designed so that anyone with x-ray-like vision, as Natasha claims, should be able to pass by matching all seven medical conditions to the correct subjects. To give Natasha some latitude for errors, we lowered the required number of correct matches to only five.
What we asked Natasha to do was easier than what she does during her normal readings. For each reading, she examines a person from head to toe and describes all abnormalities she sees. In our test, Natasha didn't have to decide what medical problems are present -- we told her. We asked her to look for six different medical conditions that should be easy to detect with x-rays. We clearly explained what the conditions look like and described exactly where to look. It should have been easy for her compared to her usual readings.
Accusations: The debunking skeptics never bothered to find out what Natasha actually claims to be able to do.
Answer: The CSMMH-CSICOP test design was based on a variety of reports of Natasha's healings plus a report from the producer/director Monica Garnsey's of her interview with Natasha Garnsey on April 6, 2004, during which Natasha described how she does her readings and what she is able to see. The purpose of this interview and report was to help us design the test.
Accusations: Natasha never claimed she can see at a cellular level. The investigators are misrepresenting her claims.
Answer: When the producer/director Monica Garnsey questioned Natasha for information to help the investigators design their test, Natasha claimed she "can see right down to the cellular level if she concentrates," Monica reported.
Accusations: The test was invalid because it required Natasha to find healed medical conditions that were no longer causing health problems. Natasha only diagnoses current medical problems, and the investigators knew that.
Answer: Natasha's fame has grown in part from news reports of how she has correctly identified long-healed bone fractures, a removed kidney, implanted metal pins, and other medical conditions that are no-longer causing health problems. Furthermore, the test design and rules were presented to Natasha and her agent five days before the test. They didn't object to the test or indicate in any way that Natasha can only see current health problems.
One of Natasha's supporters is the Russian journalist Igor Monichev, who reported how she convinced him of her abilities by seeing the well-healed evidence of a broken wrist that he fractured as a child 37 years ago.
Accusations: The investigators ambushed Natasha by changing the rules at the last minute and not allowing her to work the way she normally works with patients.
Answer: Natasha and her representatives were informed in writing five days before the day of the test exactly how the test would be conducted and what would be required for her to pass. They agreed to the test rules. None of the test conditions or requirements were changed on the test day -- except for several that Natasha requested to make her more comfortable.
Accusations: The experimenters didn't allow Natasha to examine the subjects while they are standing, as she usually does in her readings.
Answer: Natasha viewed the subjects standing up, from their front, back, and sides.
Accusations: One of the test persons had a metal plate covering part of his brain, and Natasha failed to identify this. But why should she? The man was not suffering from a medical condition. Whatever he had suffered had been treated, and Natasha did not claim to be a metal detector.
Answer: Natasha has on a number of widely reported occasions identified metal screws, plates, and other surgically implanted metal devices. In addition to failing to see the metal plate, she also failed to see that the subject's skull has a large hole, which had been made to remove a large brain tumor.
Accusations: The investigators raised the bar so that Natasha would fail. The odds of Natasha getting four or more correct matches out of seven was 1 in 50 (or 2 percent). Those are odds that are statistically significant and widely regarded in science as passing.
Answer: Those odds are not considered statistically significant for testing extremely unlikely events. As Prof. Hyman points out, "We used accepted conventions that even parapsychologists recognize. The idea is that extraordinary claims require extraordinary proof. For at least 75 years the convention has been that paranormal claims have to be tested at the .01 [probability] level or even smaller. Indeed, the convention is typically the .001 level. Using this convention, the criterion we set up in advance, used the more lenient criterion of .01 rather than .001. She would have had to get at least 5 correct matches to have surpassed the .01 level. "
More importantly, the 1-in-50 odds of getting four or more out of seven correct matches would be for someone blindly guessing. But Natasha was not blindly guessing. She had access to a great number of normal sensory clues that could have helped increase her number of correct matches.
We were not able to design a truly blind test. We had wanted to test Natasha with subjects behind a screen of opaque cloth -- to blind her to normal sensory clues. However, we were told that, for unknown reasons, Natasha can see through cloth when it is worn by a person but not when the cloth is hanging in front of the subject.
Therefore, any test we could devise had to allow Natasha the opportunity to study her subjects and pick up clues about their health problems through normal senses. While we could not prevent this, we could try to reduce the problem by choosing subjects who were of similar age and appearance.
Unfortunately, we did not have the time to recruit ideal subjects for the test; the subjects we were able to recruit were not demographically similar and several of their differences may have provided important clues. For example, the subject who had had open heart surgery was the oldest and was male, and the majority of patients who undergo open heart surgery are elderly men. The subject who had none of the target conditions was the youngest and looked athletic and in good health. Indeed, no one who knows anything about hip transplants would have picked him as the subject with an artificial hip!
In addition, several violations of the test protocols could also have helped Natasha increase her score. Contrary to the test rules, Natasha and her mother, sister, friend, and agent were able to watch at least two of the test subjects climb a flight of steps to enter the test building. Also, at one point in the test, Natasha's friend, who was acting as interpreter, named out loud the condition that Natasha was looking for, which could have caused the subject to react unintentionally and give herself away. (Even dumb horses are able to react to people's subtle, unintentional responses -- commonly called the "Clever Hans" phenomenon, named after the horse that was once famous for its "arithmetic" skills.)
There is no way the investigators could estimate how such factors may have improved Natasha's chances of getting four or more correct matches. But one thing is almost certain: it was very likely greater than 1 in 50 and statistically not significant.
Accusations: Although the investigators acknowledge that their test was only a preliminary exam to see if Natasha Demkina's abilities warranted a more carefully controlled study, they misrepresented it in the documentary as "an experiment specially designed to rule our any ambiguity."
Answer: We have been deliberately clear about the preliminary nature of the test from its very beginning. Indeed, the CSMMH-CSICOP test protocols state:
"It is imperative that the Test Proctor be allowed to explain in the Discovery Channel program that the CSICOP/CSMMH test is not in any way a definitive test. It is too simple and brief to determine the truth of Natasha's claims with comfortable certainty. It can only help decide whether further study of Natasha's claimed abilities are warranted."
Unfortunately, these important points were not provided in the documentary, against our expressed wishes. In a recent email, the producer/director Monica Garnsey explained:
"...though we appreciated this was an important point, we had a lot of competing priorities. In a television programme you have to balance editorial concerns against boring but inescapable practical points such as duration, shot size, background noise, pace, etc. ... Unfortunately the decision was made not to include this 'provisional' clause in the commentary. You as CSICOP certainly made it clear."
Accusations: Natasha's testers stated during the test that it was necessary to place Natasha under stress. Why on earth would you want to place a sensitive psychic under stress during an experiment of this kind?
Answer: The CSMMH-CSICOP investigators made no such statement and did not do anything to purposefully increase her stress. The investigators worked hard to meet Natasha's needs and make her as stress-free as possible during the test. This is supported by a recent comment from the documentary's director/producer, Monica Garnsey:
"We remain incredibly grateful for all the work CSICOP put into the Natasha test. I'm very sorry that you are coming in for abuse on your website about the way the test was designed and conducted. I thought it was very well thought through, that Natasha was treated with courtesy through-out, and that any difficulties were probably caused by the presence of the film crew, and the fact that we'd insisted the test be in new york, and that we'd only got together the night before. I said as much to your critics."
As Prof. Hyman states: "As the head proctor, I went out of my way to make the test as comfortable and nonstressful as possible. I understand that the situation could be stressful for Natasha. She and her supporters had a lot riding on the outcome. I gave her all the time she wanted. She took an hour to make her first match. At no time did I put any pressure on her. Indeed, I repeatedly asked her if everything was going O.K. I kept reminding her that she could call a break at any time she wanted to. If any thing came up that bothered her, I reminded her that she need only inform me and we would retreat into the briefing room where she could express her concerns freely. Only after the test was over, did I hear that she was bothered by the fact that one of the subjects seemed to be laughing or snickering. She never brought this to my attention during the test."
It should be understood that the subjects too were under considerable stress. These seven men and women had to sit still in their chairs without speaking for more than four hours, their vision blocked by opaque sunglasses, and often enduring excruciatingly long periods of silence. It's hardly surprising that one or more subjects may have giggled at something to relieve their stress. The subjects were remarkably patient and tolerant of these uncomfortable conditions; unlike Natasha, they were not allowed to take a break when they wanted. CSMMH and CSICOP are very grateful for their generous help with this investigation.
Accusations: The CSMMH and CSICOP investigators have unethically used the media for propaganda purposes.
Answer: The CSMMH-CSICOP investigators were contacted by the documentary's director/producer Monica Garnsey, on behalf of Natasha Demkina, who wanted to be tested by skeptical scientists -- so the charge is clearly false. In addition, the researchers designed and conducted the test with careful consideration of all ethical considerations, whether in regard to Natasha. the volunteer subjects, documentary production company, or the general public.
Accusations: The investigators had used their economic power over Natasha coercively, insidiously, unfairly, and inequitably.
Answer: CSMMH and CSICOP had absolutely no economic power over Natasha. They had no financial involvement either in testing Natasha Demkina or in the documentary program. Nor did they offer or provide Natasha any financial support or compensation. The Commission and CSICOP were invited by the producer to test Natasha's claims and in the interest of advancing science, public health, and public education, they agreed and did so. In a recent statement to critics, the program's producer/director wrote:
"I should say that in my opinion CSICOP's public service motivation is utterly sincere and, unlike the other people who made the programme possible, CSICOP [and CSMMH] did not ask for or receive payment except travel expenses."
Accusations: The test was much too rigorous.
Answer: What the test required of Natasha was much easier than what she normally does for her regular readings. Instead of having to scan the entire subject from head to toe and describe all the abnormalities she sees, the test only required her to look at seven people and indicate which of them had six different specified medical conditions. These target conditions are easily seen on an X-ray or CT-scan. She didn't have to figure out what is and is not normal. She was told exactly what to look for, what each condition looks like, and exactly where to look.
For example, one of the subjects had had a large brain tumor removed and a large metal plate now covers the hole in his skull. If Natasha's claims are true, then it should have taken her only a few minutes to look at the heads of the seven subjects to see the big hole and metal plate. This was a lot easier -- not more rigorous -- than what she normally does during her readings.
Accusations: The experimenters were unfair in not giving Natasha a chance to rest after her 6000 mile flight from Russia. They required her to take the test even though she had no time to recover from jet lag or to adjust to the 8-hour time difference.
Answer: We were not in anyway involved in planning Natasha's travels. The timing of her travel was decided by Natasha, her family and supporters, and the program's producer. Natasha arrived in New York two days before the test. The day before the test, she gave six people full readings, which most described as very impressive. So both Natasha and her claimed abilities appeared to have been in good form the day before the test. And immediately after the test, Natasha went on to give individual readings, which strongly suggests that she was not overly taxed or tired.
Accusations: Natasha had told the investigators at the beginning of the test that appendixes are one of the organs she has trouble seeing, yet they still insisted that she find the subject without an appendix.
Answer: Natasha told the investigators no such thing. She complained only that her vision was being blocked by surgical scar tissue and also complained that appendixes can sometimes grow back after an appendectomy. Which of course, they cannot.
Furthermore, such post hoc explanations are inconsistent with claims of Natasha's abilities that were widely reported before the test. For example, one of Natasha's supporters is the Russian journalist Igor Monichev, who reported that Natasha told him, "I see everything that happens inside the human body." (Igor Monichev's report)
Accusations: Natasha got five of the matches correct, not four, and therefore passed the test. Natasha was told to find the subject who had a scar on her tummy from an appendix operation. Because she considered the scar on a woman's tummy from a gynecological operation more important, she pointed out that scar instead, but was not given any credit for that correct answer.
Answer: Natasha was not asked to find tummy scars or any scars to correctly identify the subject without an appendix. She was instructed to look for the subject who didn't have an appendix. She was shown pictures of an appendix. In addition, the test briefer held up his pinky and told her, "You should look for the appendix, which is about the size and shape of the person's pinky. And the way to find it is to look at the spot where the large intestine joins the small intestine. You'll find the pinky-shaped appendix right near there -- or you won't, if the appendix was removed." She was not asked to look for scars on the tummy or anywhere else.
Accusations: Despite the fact that Natasha told the investigators at the beginning of the test that finding a missing appendix and a surgically shortened esophagus was outside of her abilities, the investigators insisted that she find them.
Answer: Natasha did not tell the investigators that finding any missing organ or parts of these or any other organ was outside of her abilities. Natasha did offer explanations why she was having difficulty finding the subjects with these two conditions. She complained about "post-operative scar tissue" confusing her and she said that she thought the surgically removed appendix may have grown back, which of course, it cannot.
The scar tissue explanation was not plausible, because even if the subject with the missing appendix had post-operative scar tissue blocking her view from every angle, she still would have been able to identify him by eliminating the subjects who still had their appendix. In addition, the presence of post-operative scar tissue around the site of the appendix should have been an powerful clue that the subject had had an appendectomy!
Also, her excuses are self-contradictory: If she can see post-operative scar tissue blocking the appendix area, then she should not have had any trouble seeing the post-operative scar tissue around the surgically shortened esophagus -- the very clue that she was instructed to look for to find the resected esophagus!
Accusations: Contrary to what the investigators led us to believe, Natasha never claimed she never makes a misdiagnosis.
Answer: A number of news reports have reported she never makes a misdiagnosis. In addition, the documentary shows Natasha's mother saying, "Natasha never makes a mistake. In six years, she hasn't made a single mistake." The documentary also shows Natasha explaining, "If you did it [the test] my way, I would probably have guessed not five but seven of them [all the conditions of the seven subjects]."
Accusations: The CSMMH and CSICOP investigators subjected Natasha to a highly stressful situation by changing the protocol and refusing to allow her mother to stay with Natasha during the test.
Answer: The investigators only changed protocols Natasha requested in order to make her as comfortable as possible. They introduced no changes to the test rules that she didn't request. This false accusation is based upon an incorrect statement in the Discovery Channel documentary.
At the beginning of the test, a couple of the investigators misread the rules and created confusion about who was allowed to stay in the test room. That unfortunate confusion was settled when Natasha's representative, Will Stewart, asked Andrew Skolnick to explain the rule. Skolnick confirmed that Natasha's family members and friends were allowed to stay. Nevertheless, Natasha's mother decided to wait outside the test room with her younger daughter, so that the young girl could move about and talk. The investigators did not bar them from the room and when Natasha asked for her mother, she was immediately brought in.
Accusations: The investigators also tried to change the rules to disrupt Natasha's concentration by substituting their own interpreter.
Answer: The investigators did not provide their own interpreter. There were two interpreters, one hired by the producer and Natasha's teenage friend. Natasha was allowed to use her friend as an interpreter thoughout the test.
Accusations: One of the investigators (Andrew Skolnick) misrepresented his credentials by claiming to be a "medical doctor" when he is not.
Answer: Andrew Skolnick never made such a claim. He is the executive director of the Commission for Scientific Medicine and Mental Health. He is a medical journalist, who worked nearly a decade as an associate editor at the Journal of the American Medical Association, but he is not a physician. That error was made by the documentary script writer.
The program's producer/director, Monica Garnsey, regrets the mistake and says it will be corrected in future broadcasts.
Accusations: The documentary clearly shows evidence of cheating by the investigators. At one point near the start of the test, when the camera is aimed at the man who had a metal plate in his head, you can see him hm raise his eyebrow, which shows that he could see through the supposedly opaque glasses and was not truly blinded.
Answer: The documentary shows CSICOP research fellow Joe Nickell preparing the sunglasses immediately prior to the test. They were made opaque simply by covering the lenses with duct tape. This allowed the subjects to be able to see to their sides and above and below them -- so they would not get disoriented and dizzy -- but not straight ahead.
The opaque glasses prevented them from seeing when Natasha was looking directly at them and, more importantly, it prevented Natasha and others from seeing their eye movements -- which might provide unintentional clues. The subject's eyebrow movement was more likely a reaction to something he heard than to what he saw going on in front of him, since it was impossible for him to see straight ahead.
Accusations: The skeptical investigators went after Natasha Demkina to discredit her as a phony.
Answer: The CSMMH and CSICOP investigators did not go after Natasha Demkina. Earlier in 2004, the investigators were contacted by the program's producer/director and were asked to scientifically test Natasha's claimed abilities. The investigators agreed.
Accusations: The investigators took advantage of an underage girl by inviting her to come to New York for a test where they ambushed and tricked her.
Answer: CSMMH and CSICOP did not invite Ms. Demkina to come to New York. The investigators were invited to come to New York to test Ms. Demkina by the program's producer/director. Although 17 years of age, Ms. Demkina was accompanied and represented by her mother and an English journalist acting as her agent. The test, which was straightforward and fair, was agreed to by all parties prior to meeting in New York for the examination.
The Investigators' Most Noteworthy Mistake|
There is one actual mistake in the test that few if any critics are complaining about. Although the test design required no more than one subject to have any of the target medical conditions, the investigators discovered after the test was over that the man who had metal plate in his head following brain tumor surgery also had had an appendectomy. He had forgotten to mention this when he had been recruited but brought it up after the test because a missing appendix had been one of the test target conditions.
The reason Natasha Demkina's supporters may not be complaining about this error is that it hurts rather than supports their case. It means that Natasha had twice the chance of correctly guessing which subject was missing his or her appendix -- roughly 28% rather than 14%. Despite this doubling of odds, she incorrectly chose a woman who still has her appendix.
Natasha also missed a golden opportunity to convince the researchers of her claimed abilities, and perhaps most other skeptics in the world. During the test, she had an opportunity to say, "Something is wrong here! There are two people who are missing their appendix. But you told me there's only supposed to be one." The investigators would have been embarrassed by the unintentional mistake. But they also would have been convinced.
"Natasha, the Girl with Very Normal Eyes"
© 2004 CSMMH All Rights Reserved