34. Response to Skeptics’ Guide Host, Dr. Steven Novella
Skeptiko host, Alex Tsakiris, responds to Skeptics Guide to the Universe’s, Dr. Steven Novella. In a point-by-point response, Tsakiris claims Novella’s recent broadcast provides further evidence that prominent Skeptics are often severely misinformed about parapsychology and psi research. Tsakiris also details why allegations of research improprieties against Dr. Dean Radin are untrue, and calls upon Dr. Novella to help correct the situation: “…Steve got on me for calling Skeptics hypocritical, but I want you to imagine for a minute how Dr. Novella would feel if he spent five years of his life doing meticulous, careful research, publishing in peer-reviewed journals, only to have some hack come along and make baseless, reckless claims that call into question his integrity. He wouldn’t stand for it, and he shouldn’t stand for it in this case either.”
Play It:
[audio:http://content.blubrry.com/skeptiko/skeptiko-2007-12-07-62055.mp3]Read It:
Alex: Welcome to skeptiko, where we explore controversial science with leading researchers, thinkers, and their critics. I’m your host, Alex Tsakaris.
If you recall, on the last episode of skeptiko, I took a break from the interview format that we generally follow, and took the time to look back on the more than 30 skeptiko interviews that I’ve done. I’ve learned the takeaway for me – if you recall – was not very favorable with regard to the skeptical community.
Well, that last episode generated quite a bit of discussion and dialogue, and as many of you have probably heard by now, Dr. Steven Novella, host of Skeptics’ Guide to the Universe, took a good portion of his show a couple of episodes ago to respond to the points that I had made. I’d add that Dr. Novella has invited on to the Skeptics’ Guide to the Universe show, and I am looking forward to doing that. It will be recorded on December 12 and then will air a few days after that.
Before then, I really thought I should take the opportunity to respond, respond to some of the claims that he has made, so that we can advance the dialogue. I thought originally that I’d just sit back and we’ll have that opportunity to go over things. Then I thought it might be better to lay my cards on the table, give Dr. Novella and the rest of the crew at the Skeptics’ Guide to the Universe show a chance to really digest where I’m coming from. Then I think we can have a more productive dialogue.
So, the first thing I wanted to comment on is the fact that Dr. Novella did respond so completely to the points that I had made, and I think that’s great. Now, it did get a little more personal at points, but in general, in all, it was an attempt at a substantive conversation about the issues, and that’s what I’ve always wanted with skeptiko: to bridge this gap between skeptics and those who are more receptive to controversial science research. Now that we have that out of the way, I want to go ahead and dissect the show, Steve’s show, and give my response.
If you recall, in Steve’s broadcast, he started it out by saying that he really, really, really didn’t like the skeptiko show and that it was terrible. Then he went on to summarize for his listeners what the show was about. Here we go:
He makes three basic claims against skeptics. One, that we do not do research in the paranormal. Two, that we don’t read the research, and the third is that we don’t listen to the skeptical research, that we don’t abide by its findings.
Okay, so those of you who listen to skeptiko on a regular basis might recall that those weren’t really the three points that I made. The three points that I made were that skeptics don’t support research, skeptics don’t read the research, and skeptics don’t do research. I can certainly understand that Steve was just trying to get it right. It’s not a huge mistake on his part. I do listen to the Skeptics’ Guide show, but like him, I probably have my finger on the fast forward button some of the time, so those things happen – no problem.
But I did want to go back and talk about the supporting of research, because I think it’s a really, really important point, and I hope that we can have a dialogue between Steve and myself and his crew on that point. You recall that what I offered as evidence that skeptics don’t support research was two clips: one from Steve, but the most poignant one I really think, was from Richard Wiseman. Here it is:
If time and again you went searching for unicorns, and you didn’t find them, just how many adventures, how many journeys do you go on before you say, “Well, they’re probably not there, and I’m not going to waste any more time?”
You know, no matter how many times I listen to that clip, I can’t get over it – Richard Wiseman equating advances in consciousness research with chasing unicorns. It just seems a little bit bizarre to me. I guess to further sharpen that point, I lay out the situation as it exists in near-death experience research, something I’ve talked about on skeptiko several times because it’s very interesting to me.
Near-death experience research – you’ll recall from discussions we had with Dr. Raymond Moody, who is really the first researcher to really delve into this area, who invented if you will, or coined the term “near-death experience” – really began in the 70s, so it’s a new phenomena.
Along with that, advances in resuscitation, resuscitating people after they die from cardiac arrest, let’s say, have advanced greatly in the last 30 years. The net result of this is that we now have literally millions of people that have been brought back from death. That’s not the right term because if you’re dead, you’re dead, but you get my point. The best studies that we have suggest that probably 10% of those people are having an experience that we’re calling a near-death experience. Now, I’m not saying what theory is behind that, what it means. I’m just saying those are the hard facts.
Let’s switch over and say, “What are the potential implications of near-death experience research?” The implications for us coming to understand whether or not consciousness survives death are enormous. They affect every aspect of science that we can imagine. Again, looking at it from this perspective, we have a bunch of new tools and technologies to explore this that we never had before. So, I juxtapose that with the clip from Richard Wiseman that we’re chasing unicorns. I’m just dumbfounded.
So, my first point, that Steve didn’t get a chance to respond to is that skeptics should be supporting research into these controversial areas, and supporting doesn’t mean advocating a particular outcome. It just means acknowledging that these are central, fundamental issues that science should be looking at. Even if you’re skeptical and you think that we’re going to prove that none of these first person accounts are true, or prove any of the materialistic principles that you have in place, that’s fine, but we still need to do the research. And it’s not getting done. That’s point one.
Let’s move on to – there’s really no nice way to say this, but move on to – the stuff Steve really got wrong. It’s really ironic, because the next point in Steve’s show is challenging my assertion that skeptics don’t read the research. Let me back up and say when I say skeptics don’t read the research, obviously I’m not saying all skeptics never read the research. I’m just saying that in general, skeptics have a tendency – prominent skeptics in the skeptical community have a tendency – to not have a firm grip of the facts.
The first example I cited as evidence that skeptics don’t read the research is this controversy that arose between Ray Hyman and Dean Radin. For those of you that don’t recall, I’m going ahead and play the original clip from Ray Hyman’s appearance on Skeptics’ Guide to the Universe. It’s a little long, but I really want you to listen to the just disparaging things that he has to say about Radin’s work:
The bad thing about Radin is that he comes through as being very sophisticated, and his whole career has been a lie. I’ve gone back and looked at a lot of stuff he’s done in the ESP field, and everything he does…he’s got some very clever, new, novel application of technology and the latest in computer sophistication and so on, and yet he gets results, and he never repeats that. That’s dropped and he goes on to other stuff.
As a cynic, you wonder, “What’s happened here?” Why doesn’t he ever have any follow-up? Why doesn’t he follow up his own great work, right?
Then I found out some other things, and I went through his presentiment work and I found – it took me a while to find it – that he’s done three experiments, each replicated the other. But when I go down to the fancy way that he was correcting for baselines and stuff like that, the correction was one way in one experiment. The second experiment was a different correction. The third experiment was a different correction.
Then I realized…I did some simulations, and the corrections from the first experiment and the ones from the second experiment would cancel each other out. It would give a different result. And why was he always changing his corrections? [laughter] And I’m realizing that this is a clever kind of maybe unconscious, not conscious so much, way of making sure you’re going to get what you want from the data.
Now, let’s step back for a minute, and look at what Ray Hyman said. This is just about the most serious allegation you can level against a scientist – that he’s fooling with the data, that he’s reporting results that aren’t real. Even though he has this old grandfatherly charm and chuckles about things, he’s making the most serious of allegations that he can make.
You may recall that I followed up this by talking to Dr. Radin:
It’s interesting. His first comment is I do a lot of studies and don’t repeat them, and the very next thing is saying I repeated the presentiment experiment a number of times.
The issue on doing lots of experiments and quote, never repeating them – I’ve heard this kind of criticism before. I think it usually comes out of people in a tenured academic position where they may never have worked outside of academia. They’ve had tenure so long they don’t remember oftentimes you don’t get a chance to decide what you’re going to work on. You work on what the available grants will allow you to work on.
So that addresses the replication issue, and I would just add that those experiments have been further replicated at least nine times by at least three different labs around the world. Some of Dr. Radin’s colleagues have recently published another experiment on presentiment where they actually did MRI work where they’re hoping to identify the neural correlates that would more precisely isolate where this effect is taking place, so this is ongoing research.
Let’s go on to the second and more important point that Dr. Radin has to say about this allegation that was made by Ray Hyman:
He does have a point, though. When I’ve done different experiments, I’ve thought of new and improved ways of creating the statistics overall. It is also true that when you want to create a mini meta-analysis of your own results, it becomes very important to see whether you can take the most recent analytical too and apply it to all of the data that you’ve collected uniformly, so that you are no longer mixing and matching different analyses, but you’re using the same method.
In 2004, when I published all of the presentiment studies that I’d done to that point, I used the same statistical method for every single trial. I don’t remember exactly how many trials, a couple of thousand trials, and overall you end up with a very, very significant result. So, that counteracts his complaint about use of different methods. I used the most recent method, which I think is the best, and I applied it uniformly to all of the data.
So, why go through this in such detail? Here’s the reason. When Steve responded to my show, he first got the presentiment part of it wrong. He went and looked up a meta-analysis paper that Radin had published on staring, so he just found the wrong paper. I understand how that happens, but it’s still very, very unfortunate when it was clear to anyone who would look at this that presentiment is really the work that we’re talking about.
More importantly, in his show, Steve goes to great lengths to talk about the effect size that Radin achieved, and that was not the issue. The issue wasn’t whether his results were meaningful, or a theory behind them – any of that. I, for one, am not particularly all that excited about presentiment work. It’s just not my thing. The issue is whether a skeptic of the reputation of a Ray Hyman, who claims publicly to have read more parapsychology papers than most parapsychologists, should make such a reckless, disparaging claim against Radin.
I mean, think about it! He said that Radin was a fraud, that Radin had published this work and fiddled with the data to come out a certain way. As the quote from Radin points out, that’s just not true, and that Radin published this meta-analysis in 2004. Hyman appeared on Steve Novella’s show in 2006, two years after this research had been published.
Now, Steve got on me for calling skeptics hypocritical, but I want you to imagine for a minute how Dr. Novella would feel if he spent three, four, maybe five years of his life doing meticulous, careful research, publishing in peer review journals, only to have some hack come along and make baseless, reckless claims that call into question his integrity. He wouldn’t stand for it. And Dr. Novella shouldn’t stand for it in this case. He should publicly admonish Hyman and call for him to make a public apology to Dean Radin for making the claims that he did.
You know, too often skeptics have sloughed off these kind of slights as, “Hey, that’s just how the game is played.” Well, you know what? That’s NOT how the game is played. That’s NOT how science is played, and that’s not how life is played. It’s about integrity. It’s about telling the truth. If that sounds naive, another claim that Dr. Novella has made about my broadcast, then so be it.
Okay, let’s move on. In the next segment of Dr. Novella’s show, he takes on the Dogs That Know experiment, the work originally done by Dr. Rupert Sheldrake and supposedly debunked by Dr. Richard Wiseman. This research, you may recall, was designed to determine if dogs are psychic – that is, whether some dogs know when their owners are coming home. This is a phenomenon that many dog owners have reported for years, but no scientist had looked at before Dr. Sheldrake. The implications for this research are enormous. They have the potential of redefining our definition of human consciousness, animal consciousness, and starting to get our hands around that special connection that we call “love,” that binds us all together.
Now, Steve had a lot to say about my analysis of that research, and when he came to the end, he got to the two reasons why he thinks that the research isn’t valid and shouldn’t be taken seriously, and I suppose the reasons why he thinks Sheldrake is a terrible scientist – something he said on the show. Here’s Steve’s clip:
One is because we have no idea that Sheldrake wasn’t just retro-dicting, you know, looking for patterns and then declaring that a positive. Wiseman set the criteria up ahead of time and even revised them to make them more fair, and it still was negative. The other thing that Wiseman brought up – a very, very good point – that if you hypothesize that when the owner leaves, the longer the owner is away, the more anxious the dog is going to get for the owner’s return, so the dog will go to the porch more and more frequently and spend more and more time there until the owner returns. Of course, the owner returning ends the cycle, so the dog will have spent the most time at the porch right before the owner comes home.
So, from the first case I cited with Dean Radin, Steve didn’t read the research. In this instance, he fell for the other fatal flaw that skeptics often fall for: he only read one half of it.
Let’s look at the two points that he’s making. First, that there is a post hoc analysis of the data. He’s saying that the only reason Sheldrake got any statistically significant results is because he waited until he gathered all his data up and then he kind of form fitted it to the result that he wanted. This is completely baseless, and I’ll give you four reasons why.
The first is that Sheldrake had already been analyzing the data for months with the same methods that he uses throughout, when Wiseman enters the project. Sheldrake’s doing the work in ’94. Wiseman comes in in ’95. Number two: Sheldrake replicated this research a couple years later with another dog in another location. Going into that research beforehand, he uses the same method of analyzing the data. So, certainly on the replication there can’t be any accusation of post hoc analysis. Number three: Consider that Wiseman only did four trials. I mean, Sheldrake set up all the video equipment, and Wiseman just came in and watched the dog a couple times and did his four trials…versus Sheldrake did 200 trials over a period of years and published his work in peer review journals.
The forth reason, and I think the most dumbfounding one when you really look at this, is that they had discussed this beforehand, and Wiseman is the one that suggested the data be plotted on ten minute intervals. Now, to understand this point, you have to understand just how silly the method that Wiseman settles on really is. Here we are – we’re analyzing whether a dog anticipates when their owner is coming home. Really simple, right? You take a video camera. You aim it at the dog. You send the owner out. You take the percentage of time the dog spends looking out the window when the owner is gone, and you compare that to the time that they spend when the owner is on the return trip home. Compare those two. You have your data, right?
No, not Richard Wiseman. He chose this arbitrary, whacky way of looking at the dog’s first trip to the window as measuring their performance. If the first time the dog goes to the window isn’t during the owner’s return trip home, Wiseman calls it a failure. Not only does this go against the pre-experiment discussions that he had with Sheldrake, but it’s just kind of a dumb way of doing things. It’s kind of evaluating quarterback Tom Brady’s performance based on the first pass that he throws. If it’s incomplete, he gets a zero pass rating for the game.
Look, this whole idea of post hoc analysis is just a bunch of bunk when it comes to this case. I don’t think even Wiseman still holds to that, and that’s what think is the nature of this concession on the skeptiko show, that he now acknowledges his data fits Sheldrake’s pattern. I think he would have said if he still thought it was post hoc analysis.
You know, the other interesting think that came up in the Wiseman interview that I did is I think Wiseman regrets that he came into this experiment as a debunker. Now, Wiseman’s a serious researcher, and he does serious research, but in this case, he kind of got sucked into the debunking role. A television show was doing a report on this. They always like two sides of a controversy. Richard Wiseman was called in, and instead of backing off and saying, “Wait a minute. This guy is doing long-term research. Let’s let him finish the research and then I can look at it and figure out what to do,” he just kind of jumped into the role of being on the other side. That set him up, set him up to be the debunker, the one that finds the reason why this claim is false. I think a lot of the so-called controversy that’s sprung up since then is really result of that debunking part.
Now let’s look at the second reason Steve had for dismissing this research, and that is that the longer the owner’s away, the more the dog is going to anticipate, so that’s just going to create this statistical anomaly that kind of totally invalidates the data.
Well, you know the first point with the post hoc analysis might have required reading a couple of papers and really digging into it. But this second point really doesn’t. It’s just obvious. You want to guess what the percentage of time the dog’s been at the window when the owner was coming home versus when they weren’t coming home?
Let’s take a look at this second experiment that he did with this dog named Cane. There’s a picture of this online. You see this dog, and he’s kind of propped up – you know how dogs are – he’s propped up, his two paws are up on this piece of furniture so he can look out the window. Now that’s his anticipating behavior. We all know that’s dogs anticipating us coming home, right? That’s clear. He spent 26% of his time in that position when the owner was coming home. He spent 1% of his time when the owner was not coming home – 1% versus 26%.
You can slice and dice that 1% any way you want. It’s not going to affect the data. This idea that the dog is anticipating more has been addressed. Sheldrake addressed it directly. He did very long trials. He did trials where the owner never even came home at all, stayed out all night. There was no increased anticipating behavior by the dog on these longer trips. They even took out the first hour that the owner was gone and re-analyzed the data that way. It was even more statistically significant.
So, this second point that he’s making is just completely baseless, and he really should have known this. A simple reading of any of the summaries on this would have given him that point if he just would have read the other side and not depended solely on the skeptic party line that gets rehashed over and over again without ever going back and looking at the original data.
So we’ve re-examined how skeptics don’t support research into controversial areas. We’ve re-examined how skeptics don’t read the data. So, the third point that Steve responded to from my broadcast is my assertion that skeptics don’t do research. Here’s what Steve had to say:
But he was concluding from that that we’re unwilling to do the research, that we’re unwilling to even do personal explorations into this kind of stuff, and that is a completely unfair, completely, patently, factually false claim.
Okay, I can finally say it. He’s right. He’s right by virtue of the fact that he’s invited me to come on the show and he’s invited me to participate with the Skeptics’ Guide to the Universe to do some informal but well-controlled medium research. I applaud him for that.
You know, let me take a minute now. I’ve been hard on Steve, but I think I’ve been factual in this show and have stuck to substantive points. I don’t have anything against Steve Novella. I spoken with him a couple of times, had email exchanged with him. I like him. I like his show. He’s intelligent. He’s personable, and he’s interested in a lot of the same things I’m interested in. We just happen to see the issues on different sides of it, but we still have enough common ground to collaborate and do some research together and see if we can get a little bit closer to finding some answers to these questions that we both care about.
Now, having said that, I did want to make a couple of comments about the Skeptics’ Guide episode on the psychic fair, because I think it’s a real stretch to call that research. I think it was debunking, and I think it was interesting debunking and worthwhile debunking. Again, I think debunking is fine, and to go to a psychic fair and expose people as being illegitimate is terrific. I wish they would have given the names, and I wish they would have had them do the “cleansing” or whatever it is. That kind of behavior is despicable and shouldn’t be allowed in any circumstances. It’s a hustle. It’s somebody trying to make some money off of the fear and misery of other people.
It’s like when Rebecca Watson took a picture of Perry DiAngelis, who recently passed away, up to one of the so-called mediums, and they did this thing and said, “Oh gosh, he’s going to be really financially successful in the next six months.” I don’t care how skeptical you are. That has to hurt a little bit and it has to make you angry. I know it would me. A lot of people, when they come to psychics or mediums, are vulnerable. There are a lot of people who take advantage of that vulnerability. At the same time, that’s not how everyone is.
I just have to relate really quickly the experience with my first readings and what lead me to the protocol that I’ve suggested and that Steve has basically agreed to in terms of going forward with this informal medium research. The first thing I did was look for really good mediums. I started by going to Gary Schwartz’s website, the guy who’s done more medium research than anybody. I looked on his website to see the mediums he had used.
The first medium was there in Tucson, Arizona. I called her up and it was quite expensive. I think it was $200. I sent her the check in advance – standard procedure with mediums – and then I went ahead with the reading. And the reading went nowhere. It was just nothing. No, no, no, no – she wasn’t getting anything right. So, about ten minutes in, she goes, “You know, I don’t know why, but for whatever reason, this reading just isn’t working. It isn’t working for you and it isn’t working for me. I’ll tell you what I’d like to do. I’d like to send you your money back.” Well, I was surprised. She already had my money. The check was cashed. She actually decided to write me a check and send it back to me.
So, I proceeded on my journey and I found another medium. This medium I think I found through James Van Praagh’s website, someone he recommended – similarly $200. Okay, send away the money, call them up. Again, the reading does not go well. We get about 15 minutes into it and he goes, “Alex, I don’t understand it. This rarely ever happens to me, but I’m just not doing a good reading for you. I’ll tell you what I’d like to do. Let’s wait six weeks, call me back up again, and we’ll do another reading and see if that works.” I did. I waited six weeks. I called him back up. The second reading doesn’t go well.
Meanwhile, I’ve chatted with this guy, a super nice guy. We had a very nice exchange. We must have spent an hour on the phone together. At the end of it, he says, “I’d like to send you back your money. The reading wasn’t successful.” So, the end of my story is I found another medium through James Van Praagh’s website and had an extremely successful reading, one that I evaluated on the criteria that Gary Schwartz has used, and my informal analysis was that it was very significant. I relayed all that to show you that not all mediums are out to scam people, and also to emphasize just how important it is to get good mediums going into these things. I’m sure we’ll do that with the work that I do with the Skeptics’ Guide. I’ll select the mediums. I’ll pay for the mediums – I’ve already offered to do that. Then I’m going to suggest that Steve picks the person that we read for, but we’ll hash all that out, I’m sure, in the weeks to come.
There you have it. I guess we could call this episode “Bashing the Skeptics 2, The Sequel” I do sincerely hope that this discussion will generate a productive dialogue with the folks at Skeptics’ Guide to the Universe and with skeptics in general. I think that really dealing with the issues and digging into them and seriously addressing them is something that is so, so lacking in this area of scientific research.
That’s going to about do it for today. Stay tuned for my upcoming interview with Steve, on the Skeptics Guide to the Universe, and, of course, stay tuned – or should I say, stay subscribed – to skeptiko for upcoming episodes. Remember you can always check out all the past episodes of skeptiko at the skeptiko website. That’s skeptiko.com. That’s going to do it for today. I really think I need to give my voice a rest, here. I’ve been doing this and I’m a little bit under the weather. I’m sorry for the sound quality. Hopefully, I’ll sound a little bit better next time. Bye for now.
Podcast: Play in new window | Download
Subscribe: RSS