Description: Dr. Steven Novella from the Skeptics Guide to the Universe discusses his participation in an upcoming psychic medium demonstration. The panel also discusses the protocol used to avoid “cold readings”, and accurately score results.
Download MP3 (56:42min, 26MB)[audio: http://skeptiko.com/older-shows/skeptiko-2008-05-21-57182.mp3]
Announcer: On this episode of skeptiko, Dr. Steven Novella of The Skeptics Guide to the Universe on medium research.
Dr. Steven Novella: …starting hypothesis, is that there is no paranormal psychic phenomenon going on here. Certainly, it’s where…it’s certainly on record that that is our current position based upon the evidence that we’ve seen so far. The whole point of designing a protocol is that the bias as we go into it should be rendered irrelevant…
Announcer: Stay with us for skeptiko.
Alex: Welcome to skeptiko where we explore controversial science with leading researchers, thinkers, and their critics. I’m your host, Alex Tsakiris, and on today…wait…whoa…take that over again. You know, I always say, “Welcome to skeptiko where we explore controversial science.” I really have to stop saying that. I mean, that’s just playing right into this skeptical mindset that, when we talk about things like “psy” or when we talk about things like medium research, we’re talking about controversial science.
Well, it’s controversial, sure, but doesn’t that kind of miss the point? You know, one of the most frustrating and, at the same time, exciting things about what we’re trying to do onskeptiko is the vast unexplored territory that is human consciousness. Why is this not [laughs] being researched? We’ve started the work with the Dogs That Know experiment and it’s generated a little bit of interest but why is that not explored more? And, today, we’re going to venture into the second experiment that we’re doing on mediumship research, that is can a psychic connect with someone’s deceased relatives and access information that would otherwise be impossible to know? Now, can you imagine an area of research that is more directly connected to who we are, where we’re going, and what the world is really all about? And, yet, the amount of real research that’s done in this area is so miniscule that it can be summarized with one person’s name, Gary Schwartz. Let me correct that…two people’s names, Gary Schwartz and his research assistant and now researcher on her own at the Windbridge Institute, Julie Beischel.
Well, I’m very happy to say that we here at skeptiko are going to add just a tiny little bit to this scant bit of research that’s being done by conducting a demonstration in cooperation with The Skeptic’s Guide to the Universe and Dr. Steven Novella. Now, I have to say it took a little bit of prodding but I’m very very happy that they are joining us and I think a collaborative effort between a group that’s clearly very skeptical of this phenomena is the way that everyone agrees is the best way forward in coming to some greater understanding and generating further awareness and, hopefully, more interest in researching this very important topic.
So, on today’s show, we’re going to kick off this little project that we’re doing and I’m going to share with you the recording that Steve’s group and I had last week regarding the protocol that we’re going to use for this demonstration. It’s about an hour long. There’s a lot of detail in there. There’s not a lot of juicy nuggets, a lot of just normal kind of back-and-forth discussion about the protocol, but I think it’s useful in terms of documenting where we’re coming from and how we’re approaching this research.
So, without any more introduction than that, let’s go to this conference call with Steven Novella and the crew from The Skeptics Guide to the Universe on the medium research protocol.
[theme music playing]
Alex: Hey, Steve, I’m just going to say…
Alex: …before we jump into this, I just want to make sure, are you guys comfortable that we’ll record this? And then, you know, I know your schedule’s pretty busy but I’m fine in terms of putting it out just as we record it as a podcast because I think one of the things beside, you know, you’re trying to prove one thing and I’m trying to prove another…I think one of the things we’re trying to do is just expose the process to people as much as we can.
Steve: Sure. Yeah, I’m fine with that. I mean, I’ll be recording at my end as well.
Alex: Right. So, I am too and we’ll…as last time when we talked, we’ll either agree to some agreed edited version or we’ll just agree to just publish the whole thing.
[caller entrance sounds play]
Steve: Actually, Jay’s coming in now.
Steve: Hi, Jay.
Jay: Hi, guys.
Steve: So, we’ve got Alex with us. We’re just talking about recording the conversation, you know, so that we could…either side can use it for our podcasts which I…which we…I agreed to. You guys have no problem with that, right?
Alex: Nope. Nope.
Jay: This conversation?
Steve: This conversation, yeah.
Jay: To document the process of the discussion.
Steve: And, Jay, do I need to chat you the most recent version of the protocol or do you have what I sent you earlier today? I’ll just send it out again.
Alex: It might be useful to go over the protocol now although we really don’t have many points of disagreement on it so, you know, we can also just publish that and maybe talk about the points that we do have a disagreement.
Alex: The other thing I thought would be interesting to cover from the perspective of both recording where you guys are coming at and where I’m coming at are some of the questions that you asked me and then some of the questions that I’d ask you back in terms of where we see this thing going. You asked me, “Hey, what is going on? What do you think is going on?” And I’d like to ask those questions…and I think that’s what people would like to hear too in terms of framing this…
Steve: um hmm…
Alex: …this demonstration. We can’t really call it a research project but it’s a demonstration.
Steve: I agree, you know, calling it a demonstration is fine. You know, I would just…in terms of our overall interest in this, you know, we’d like to, you know, have the process be…at least sufficiently controlled that we could do some objective scoring of how the medium performs and avoid, as much as possible, any, you know, subjective interpretation of what happened. Now, there’s, you know…there’s almost no limit on how detailed you can get, you know, in terms of making everything quantifiable and blinded, etc. and, again, we just don’t have, I think, the time and resources to do something to that level. We’re just hoping to do a reasonable demonstration of, you know, these are the things that we were concerned about, you know, accounting for all the things that you and the medium are concerned about so that everyone’s happy and then we’ll just see how it plays out.
Alex: Right. I think that’s the spirit of things. And, you know, to that end, I have to say…I mean…we’ll see how this plays out but, based on your earlier podcast and the opinions you’ve expressed…I mean…let’s establish…you believe that there’s absolutely nothing happening here. Nothing telepathic or spiritual is, even in a remote way, possible. So, the bar in that sense is pretty low and we can quibble over scoring and statistics and all those kind of things but you guys aren’t coming at this from the perspective of how accurate it is because the last time you did this at the Psychic Fair, you said there was…it was pretty obvious that nothing was going on. So…
Steve: Yeah, although that would…
Female Voice: Yeah…I think you’re…I think you’re very wrong in that this bar is set very high because we…what you’re saying is that you’re expecting some extraordinary psychic powers to be demonstrated and so we’re going to ask for extraordinary results.
Steve: Well, I agree with that. What I would say is, yes, certainly our starting point, our starting hypothesis, is that there is no paranormal psychic phenomenon going on here. Certainly, it’s where…it’s certainly on record that that is our current position based upon the evidence that we’ve seen so far. The whole point of designing a protocol is that the bias as we go into it should be rendered irrelevant…that it’s going to be carried out in an objective enough way that, whatever we think going into it, it won’t..it shouldn’t affect the results. That’s the whole point of agreeing to a protocol. And, in terms of where to set the bar, that…the…where we’re coming from is it has to be set high enough and the protocol has to be at least tight enough that we can rule out what are hypotheses…which is that this is a cold reading..that mediums produce the results that they do through cold readings.
So, what I anticipate is that there’s going to be, you know, three broad ranges of possible results. One is that there is going to be a fairly dramatic actual effect taking place that is not well-accounted for by a cold reading and that would, therefore, seem to support it. There was some other actual process going on. Some, you know, psychic medium or something else. Some anomalous process going on. There’ll be a clearly negative…could be a clearly negative result where what we’re seeing is just purely chance guessing and completely consistent with cold reading but there’s clearly also going to be a broad fuzzy range in between where it’s going to be hard to say based upon these results.
So, I think we have to be thinking in those terms that, you know, at least broadly define where these three ranges of results are…
Steve: …results that we would agree show something anomalous where you would agree clearly does not show something anomalous and yet is going to be this fuzzy, you know, borderland in between.
Alex: I agree with that and I think it’s also, as you’re alluding to…and I like the way you stake that out, there…we could very well fall into that fuzzy ground but we could very well fall into those…either one of those categories where I just don’t think we’d have much disagreement from the way you described your experience in the Psychic Fair. I would not, you know, argue or kind of interject some other hypothesis about what might have been going on. It was a failure. I don’t doubt that those happen and that there’s really no way to explain those away.
I’d also add that, in terms of the protocol, and I don’t think there’s a large point of disagreement here based on the exchange we’ve had about the protocol. You know, you do a couple simple things like we’ve done and you’ve basically nailed down the biggest parts of the protocol. You have a phone reading, number one, where people can’t see each other. You limit the amount of interaction that people are having in terms of the answers back and forth. And then, third, you control any chance that the sitter and the reader know each other or have any information before and there’s not a lot else. So, that pretty much rules out a cold reading. I don’t think you would have any disagreement with that, right?
Steve: Well, it limits a cold reading. A cold reading partly is a process of making guesses that are high probability about anybody. So that does not require any feedback or prior knowledge. That’s just making statements that sound specific but, actually, are likely to be endorsed as positive by anybody which is why, in the protocol, we have to have a designation of…that this is a vague statement, right? And one of the ways in which cold readings produce the impression of an actual phenomenon going on is that, you know, these vague statements are produced in order to give a high probability hit and then that’s later interpreted as a specific hit.
So, just to be clear. A cold reading does not depend entirely on feedback or prior knowledge. It..there are some techniques that are independent of that. And then we’re going to have to…that’s…again, that’s where I see the fuzzy area most likely to be. If she says…if the medium says something about the sitter that is true but that’s just high probability, you know, what are we going to make of that? She says, “Oh, I see that you’ve…you know…you have financial concerns.”
Steve: Okay. You know, that’s the kind of thing that, sure, you know, 80% of people are going to say yes to that.
Alex: Right. Right.
Jay: Well, let’s…let’s take something as an example like…ummm…like…I…let me just ask you a question. Maybe this will help, Alex. Do you think it’s just as likely that a psychic would be able to divine the name of the sitter’s mother as the psychic would be able to divine another name of somebody else in that person’s life that would be, you know, an uncommon person to ask about…like an ex-boyfriend or girlfriend or something a lot more vague? You know, if you were…if you had psychic ability, would you be able to pull out that information just as readily as, you know, something very common or easy to get to?
Alex: I don’t know, Jay. I mean that presupposes so…so many [laughs] so many things. I mean one thing that…that…it touches on a couple of points though. One, as you know from the emails I’ve sent you, I’ve enlisted the support or the help of Julie Beischel who is the person who is…who has done these experiments and has published in peer review journals the results of them from her work with Gary Schwartz at the University of Arizona. She is highly qualified, well-trained. Whether you like her or whether you like her research, she brings a degree of professionalism to this that I’m relying on to make sure that we do things as…as close to a true scientific protocol as we can.
Now, the reason I bring that up, Jay, is because one of the points that she had in her kind of tips for us to look at is to try and remove, as much as we can, any kind of preconceptions we might have about how this phenomena works. And I think we have to. What we’re…we’re demonstrating here is potentially that there is a phenomena to be explored or there isn’t a phenomena to be explored. We don’t want to jump into the business of guessing or imagining how that, you know, phenomena might manifest itself.
Steve: That’s fair enough but…ummm…unfortunately, when you’re trying to do a scientific protocol, you do have to make assumptions about the properties of the phenomenon you’re looking at. Otherwise, you’re just anomaly hunting.
So, you have to be willing to commit yourself to at least some working hypothesis. Otherwise, you just can’t test it. And, for example, we could go through her tips one by one and I could tell you why I…I actually…I think I pretty much disagree with all of them right down the line. They’re actually designed to maximize a cold reading and they’re designed to minimize any scientific utility of the information that we get or the scoring process.
Alex: I think you’re…I think you might be…I saw some of the points that I anticipate that you’re jumping into there and I don’t even want…
Alex: …to go there because we’re going to be able to control that ourselves…
Alex: …your side and our side, in terms of how we do that. I think, you know, the other way of reading that though, Steve, is we don’t want to go in with such a bias that we’re trying to, you know, rip the heart out of the experiment and not…
Alex: …provide it. Because, one point that was mentioned…and now I forget…maybe it was on your point, Jay, or maybe it was yours, Steve, and that’s that, you know, the flip side of that kind of fishing for general information…you might be having financial problems…is a phenomena that I have experienced in the past with one of the readings that I had and I’ve, you know, documented on the skeptiko show that I’ve had several readings and that many…I’ve had maybe four readings and three of them were almost total flops and I had one that was really significant. And part of the most…the most significant part of that experience for me…and this has been kind of affirmed by other people I’ve talked to…I’ve spoken with about their reading experience…is that there’ll be one or two things that are just…seem…if you were to just pass them on to someone else would seem almost meaningless but are highly significant to the individual. Like, maybe a song. Someone will say, you know, “Does this song mean anything to you?” and the person will just, you know, break down in tears and go, “You know, that’s the song…” doing that…and I think that the scoring system has to account for that too. That, you know, the fact that someone could pick a song that has such deep meaning for a person is more than just a…well, check that off. That’s a 50/50 chance. So, that’s what I mean when I say we don’t want to rip the heart out of the scoring system either in order to provide some kind of external notion that we have going in of what a proper control is.
Steve: Yeah, I mean there’s…you know…in this kind of a reading that we’re going to be scoring, there are some things that are going to be fairly objective, you know, and we’re going to give the greatest weight to those kind of things. If the medium, for example, says, “Your mother’s name is Margaret.” and the sitter’s name is Margaret, well, that’s a direct specific hit. There’s no arguing about that. If the medium says, you know, “I’m hearing this song.” And she gives whatever the name or the tune of the song, and then the sitter says, “Yes. That specific song has significance to me and here it is.” Okay, that’s a hit too. I mean I think we would give that a hit. It’s not as specific as, “Your mother’s name is Margaret.” Because, I mean, how many songs might strike a chord with somebody? So, it’s just…it’s certainly is different. But, we would, you know, absolutely, I think that…we’re not going…we’re definitely not going into this with a bias or we don’t want to skew the test so that…to make it negative. We just want, again…the…I think it’s…we have to keep crystal clear that the protocol is worthless if it can’t distinguish cold reading from a genuine psychic or medium reading, right? I mean, that’s…
Alex: I agree.
Steve: …that’s the bottom line. If it doesn’t make that distinction, then it’s not going to serve any purpose for anybody. We’ll both come out of it with…that the…with the same preconceived notion that we went into it with. It just wouldn’t be a useful protocol if it doesn’t do that. So…
Alex: I agree.
Steve: So, ummm…so, like, just for example…and I don’t know if you meant to incorporate these tips specifically into the protocol but like, for example, one of the tips is “Numbers conveyed during a reading may mean a variety of things. For example, four may mean one of four children, the fourth child, a birth, death, anniversary, etc., in April or on the 4th, 14th, or 24th day of a month.” I mean, this is like right out of a cold reading handbook, right? That kind of…maximizing the possible number of hits. I would not count that kind of thing. If somebody said, “four” and it turned out to be one of twenty different possible interpretations of the number four, that’s not a hit. That’s…that’s cold reading.
Alex: Except, Steve…
Steve: Do you agree with that?
Alex: Well, you have to remember the context in which it’s given and that’s that she has the tightest controls on the other side of it in terms of communication. If you look at [laughs] her published research on the last experiment that she did, they had no exchange of information between the reader and the sitter. So, ummm…you know…what she’s providing there are tips for the reader to…ummm…let’s see…I’m sorry…for the sitter to interpret results. Now, I don’t know if those are after the fact or in it but, I can see where…what she’s trying to do is say don’t immediately try and take everything in a completely concrete sense. And, let me tell you how I think it would play out in our scenario. I can see…
Steve: Just be clear. Just be clear. The…uhhh…this is tips on scoring a reading so that’s…
Alex: Scoring a reading.
Steve: So that’s…
Alex: Right. So…
Steve: …that’s what she’s talking about here, right?
Steve: That’s what we’re talking about, how to score the reading.
Alex: Right. And…so I think here’s how it can play out, right? Your…your scenario is…I would agree with you. Okay? I would agree with you. If someone is fishing and goes, “Does four mean anything to you?” and, you know, they go, “Yes.” And say…and then, if they follow up…so then it’s really the followup, right? Because they only can provide a yes/no answer. “Does four mean anything to you?” “Yes.” “Does it mean anything with your family?” “Yes.” “Does it have to do with your kids?” “Yes.” “Is it the number of children you have?” “Yes.”
Well, I…now, I think that’s significant. I don’t think you could…it’s not super significant. I wouldn’t give it like a…
Alex: …a super high probability, but it could be…we have to be open to the possibility of…
Alex: …her first point there that this is not a phone conversation between a deceased person in some kind of spiritual form and this medium who is in the middle. We do not understand the process of communication. So, we have to be as flexible as possible and, at the same time, provide the kind of controls that you’re talking about. I’m not against that.
Jay: Okay. Well, and just again…to make our position clear…because this is sort of the dilemma that we’re in a little bit, is that…it is…it is my hypothesis that the form that a medium reading takes has evolved over time to conform to an actual cold reading. It’s not a coincidence that the way that mediums think the spirit communication works actually also happens to work just like a cold reading. That’s not a coincidence. That happened over time because that’s what they were doing whether they were consciously doing it or subconsciously doing it.
And, just like what you described, you know, there’s a magician’s trick, you know, a mentalist’s trick called “The Magician’s Choice” where they ask a series of yes or no questions and they can get somebody to a predetermined choice or you can also use that to make it seem as if you knew what the answer was at the beginning when you didn’t until the end.
So, we…we can’t let it take the form of, you know, actual mentalist cold reading tricks that have evolved over time.
Alex: Right. But, you know, there is a flip side…
Jay: Do you understand what I’m saying?
Alex: I do. I do. But, you know, the flip side to your hypothesis…that’s your hypothesis and fair enough.
Alex: My hypothesis is exactly the opposite, that the cold reading trick has evolved in a way to simulate and take advantage of the real medium exchange that people are familiar with and then add to it the tricks of the trade that a magician or mentalist knows to make it work for them. So, there’s no way of knowing the origin of these things or why they happen the way they do. All we can do is what we’re doing, try and apply the best controls we can and try and be as forthright and open as possible about evaluating the results.
Jay: What about putting in a control of some kind or…you know…one thing I…while we were discussing this over the past couple of months…I kept saying to everyone was, you know, it would be interesting to see someone do a cold reading and see if the cold reader could beat or be statistically better than what the medium comes up with.
Steve: Would love to see you do it. Would love to see you do it. Would love to see…because I think that’s…that’s one thing that really is aggravating to me is so many skeptics have kind of thrown out this cold reading idea and I’ve never seen a cold reader operate with any kind of controls close to this challenge out there. Anyone do it…would do it in this experiment…wonderful….a cold reader to sit down on a phone reading with the other person only being able to say “yes,” “no,” “I don’t know,” and get statistical results…would love to see it.
Jay: Hey, well, that’s…that is definitely a challenge that we will take up because I have seen, you know, mentalist cold readers perform better by an order of magnitude than anyone proclaiming to be a psychic or a medium in my opinion…but let’s quantify it.
Steve: Uh huh.
Jay: So, we could take this…whatever protocol we’ve…we come up with as Phase II. Once we do the mediums that you select for us, we’ll be happy to select, you know, cold readers to do the same protocol and compare the two.
Steve: Let’s put that as Phase II once we get a protocol. So, why don’t we go through…so, again…
Jay: Hey Steve, don’t…that’s cool and I’m really happy to hear that both of you guys are interested in that but do you think it would help our research here if we did have some type of baseline or something to compare it to?
Alex: Ummm…I don’t think that’s necessary…let me just interject. I don’t think that’s necessary because I think that this is all going to come down to…unless the results are really obvious and we’ll know that right away…it’s all going to come down to the scoring and I think we’re just going to have an open discussion about that. If it’s in that middle ground, it’s going to be…I don’t know that we can control ourselves into falling into one of those categories that Steve outlined at the beginning of either definitely positive, definitely negative.
Steve: Okay. Right. And, well, you know, I think that it’s either going to be definitely negative or gray zone. You know, you obviously think it’s either going to be either…if, you know, the medium is genuine, it should be either gray zone or definitely positive.
Alex: Well, I’ll go on record as saying…what I think is going to happen…I think we’re going to have readings…I think we’re going to have readings that are definitively positive…
Steve: Um hummm…
Alex: …I think we’re going to have readings that are definitely negative and I think we’re going to have readings that are in the gray zone.
Steve: Right. But, ummm…you know…this is…again, because this is a demonstration, this is not actually a scientific experiment because we don’t have things like a baseline. We don’t have any kind of validated tests that we’re using…you know…we don’t have all that background information that’s necessary to know how to interpret your data. But it’s not a bad starting point. In fact, Jay, this kind of is a baseline. This is a baseline doing mediums and then we can compare it to experienced purposeful cold readers and then see if…how they sort out. You know, if one group does consistently better than the other, then that would tend to support one hypothesis or the other. Ummm…so let’s finalize the actual details of this protocol.
So, the version that I have in writing here is the last one that I sent you although I did correct those…the…uh…grammatical stuff that was pointed out…so that’s all corrected…but…so why don’t you tell me which items you would like to discuss.
Jay: I think we should read every single one of them. Like, let’s do it right and go through the whole protocol together.
Steve: Do you want to do that on…sure, if you want to do that live, we can do that…
Jay: You can edit it out if it’s too long or, you know, we’re all going to take snippets from this anyway so…I think that, if we’re trying to conduct a good study, let’s go through the whole thing together one last time.
Alex: The only thing I’d say is I have about forty-five minutes before I have to resume my [laughs] resume my duties…
Alex: …at home here. And, I guess the other thing I’d say is, you know, if that…was that Jay who just said that?
Alex: Jay, what I’d do is focus in on the ones that you think are going to be most controversial because I only have one or two points. Other than that, I don’t see anything controversial and we can definitely publish that whole thing but I’m open…if you want to go through it quickly…
Steve: Yeah, that’s fine, Jay. I…you know…I don’t know if you joined us late but, yeah, we’ll just publish the written protocol and let’s just talk about the points of contention here. So, again…
Steve: …what I sent you, I’m happy with every…that was my final version or my latest version. So, I’m happy with it. So, you tell me, Alex, which points you would like to discuss.
Alex: The only point I wanted to discuss, and it’s not even a major point, was the one brought up by Julie Beischel and that’s that…well, there’s two minor points. One, on 3D, the allowed answers “yes,” “no,” “I don’t know,” fine. She suggested we add a “maybe,” or “I’m not sure”…you know…some flexibility in how their…how they answer those in that way. So…uh…
Steve: So, we have…should we have “yes,” “no,” and “I don’t know”…
Alex: So, I guess we want to add “maybe” to that?
Steve: Ummm…I’m not adverse to that. I mean I think that kind of “not sure” I think is the same as “I don’t know” functionally. Basically, anytime the sitter cannot answer “yes” or “no,” there’s…what’s the default answer? You know. I don’t know if we need to parse that out to “maybe” vs. “not sure” vs. “I don’t know.” I mean, is that…I think that’s unnecessary complexity.
Female Voice: I think it should probably just be “can’t answer”…”yes,” “no,” or “can’t answer.” A “maybe” isn’t a good idea because a “maybe” can lead to other sorts of inferences on the part of the cold reader. So, I think that a “can’t answer” would probably suffice.
Steve: “Yes,” “no,” “I don’t know,” or “can’t answer?”
Alex: …uhhh, I don’t know.
Female Voice: Maybe just “yes,” “no,” or “can’t answer.”
Steve: “unable to answer?”
Alex: We’ll see…let’s just see how it goes and…we don’t want to get too…I think…anal about it. We want to…
Female Voice: This is…this is kind of important.
Steve: Yeah. Because…absolutely, feedback is…it is a major component of cold reading so, how…you know…ummm…controlling the feedback is a critical point.
Alex: Okay. Well, we’ll…
Steve: I do think that this…I do think that this, you know, whether or not we allow “maybe”vs. “I don’t know” vs. “can’t answer” is kind of splitting hairs. It’s basically “yes,” “no,” and then “I can’t give a yes or no” answer for whatever reason. Ummm…
Alex: Okay. I’m sure…I’m sure we can work that out. The last point that I had was your point about any information given about people other than…
Alex: …the sitter should be disqualified and I think Julie’s point was…and I think it’s a good point…is that we shouldn’t automatically disqualify that information. Where I saw you going was you don’t want a fishing expedition of “Is there anyone out there [laughs] with the name of Mike?”
Steve: Yeah. That’s exactly right. So, that was just vague on my point. I meant what she said which is that the medium…we’re only going to score hits that relate to the sitter or people that relate directly to the sitter. So, it’s okay if they’re talking about the sitter’s dead uncle. That’s fine. It just can’t be about, you know, her friend’s dead uncle that the sitter never heard of or never saw before but just, you know, there has to be some limitations set on how much we’re going to expand the possible field of hits. Because that’s the other mechanism that is used to generate false positives.
And, again, just to put things into…further into, you know, scientific perspective, what we’re really talking about here is balancing false positives vs. false negatives. Right? And…meaning…scoring a hit that’s actually not psychic, which would be a false positive, or scoring a miss, something that actually was psychic but should be a false negative. We’re obviously more concerned about limiting false positives and you’re concerned about limiting false negatives but, in all scientific studies, you…it’s almost like it’s a zero-sum game. You have to strike a balance between the two and any…and the protocol is always going to be sacrificed in one at the expense of the other. So, it’s just a matter of finding the balance.
Steve: So, yeah, we may be excluding hits. Maybe the psychic did get information about the sitter’s friend’s, you know, third cousin’s uncle and we won’t count it because it doesn’t directly related to someone that’s connected more directly to the sitter but, if we do allow that kind of thing, then it becomes impossible to score because, you know, if you allow second- and third- and fourth-generation connections to the sitter, then the number of possible targets for the hit becomes so open that hits become meaningless. Right?
Alex: Agreed. And, you know, we’re back to the same thing we keep touching on and it’ll be an interesting part of this whole procedure and that’s scoring. So, we’ll have some…
Alex: …we’ll have some fun doing that.
Jay: So, what number was that about relating to the sitter…?
Jay: Oh, that was the yes/no/maybe. Ummm…
Alex: I think it’s maybe 4E…4…
Jay: Right, 4E. And so it should be hits should only apply to the sitter themselves…okay…and we’re going to stay with that?
Steve: We’re going to say, “…the sitter themselves or persons directly connected to the sitter.”
Alex: Sure. Fine.
Steve: Right. Is that fine?
Steve: That’s what I…that’s what I meant…and so that’s fine to clarify that. And then, we also have in there, 4E, “Future predictions will also not be scored as hits.” So, in other words…
Steve: …turning a past or present prediction to say well, this is something that’s going to happen in the future…
Steve: We’re just going to exclude any statements about the future because that’s not when…we’re not…they’re unscoreable…right?
Steve: We agree? Okay, fine. So, was that it? Were those the only two points?
Alex: Yes. They were. Those were the only two points.
Jay: So, what’s our…what’s our next step guys? When do you want to do it? How are we going to orchestrate the actual day of the event and…uh…how’re we going to record it? All that stuff.
Steve: So, actually we we’re discussing whether or not it was going to be in person, over the phone, Alex, and you wanted to do it over the phone which I think is fine.
Alex: Well, I think over the phone does a couple of really nice things. One, it…it certainly gets us away from some of the cold reading problems that we run into when people are…are in close proximity to each other. And…uh…the second thing that it does is it just frees us up and makes it a lot easier geographically to hook up people who are far apart, so…
Steve: Yeah…yeah…and then…
Female Voice: And, also, if we can do it over Skype, we could have the…whoever’s being read responding just via text and that would limit the amount of cold reading that the reported psychic could do just based on tone of voice, male/female, that kind of thing.
Jay: That’s a cool idea.
Alex: Yeah. But I don’t want to…I don’t want to get into that…
Steve: If the medium’s okay with that. I mean, that obviously would be an extra layer of protection against cold reading but we also want to make sure that the…we’re not…you know…this is where the balance thing comes in. I want…I want the medium to say, “I can do a reading under these conditions.” Right? If they don’t…if they think that the conditions are not amenable to a reading then, again…
Jay: …the whole thing becomes pointless.
Female Voice: Well, obviously, that goes for everything.
Steve: Yeah. Right. And the other thing I would add to the…doing it over Skype or over the phone is that, in that case, we can’t use the sitter’s real name. Because then, we are not in control of the environment. We are not in control of the medium and we can’t know that they’re not gaining information by ordinary means because they’re not going to be physically present with us.
Alex: And you’re saying we could if it’s over the phone?
Steve: And so, if we were in the same room with the medium, I know she’s not Googling the sitter. Right?
Alex: Oh, right. Right. Right.
Steve: What we can…just use a first name…
Jay: Over the phone…
Steve: We’re just going to use a first name, right?
Jay: Yeah. Yeah. So not the person’s full name.
Steve: Right. Right.
Jay: We’re not going to give the medium any information that they could use to identify that person individually.
Alex: Correct. Correct.
Steve: …and then…right…okay, that’s fine.
Jay: Hey, Alex…and I totally will go with that but I also think it might be interesting if…ummm…we didn’t give the name and see if the medium could actually come up with the name. I mean, if there’s any piece of information out there, that would be one you’d think…that…
Female Voice: Yeah, I think that it’s…and if we can do it over Skype and just have the person respond via text then…each…the sitter should just get a number and be referred to only by that number.
Alex: Let’s just try and keep it as natural as possible and I think…so, a first name I think is a natural way people kind of talk to each other. That’s fine. That’s one bit of information that they can’t…uh…
Female Voice: But what does natural have to do with it? I’m not sure I understand.
Alex: Ummm…you know…you’re talking about…
Female Voice: Like you could call a person, you know, like “A” for instance.
Alex: Is that how you normally talk to people? [laughs]
Jay: You know, the medium is probably going to want something to chew on, you know…So, I mean…
Female Voice: Exactly. But we…[laughs] the whole point of this is to not give them anything, you know, to chew on…and…
Steve: Well..but a first name is benign. You know, yes, it does give a little bit of demographic information about the person, you know.
Jay: It just would have been cool to see if they could come up with that though.
Female Voice: Well, it could…it could give an awful lot of information. It can give, you know, an idea of age…and nationality and things like that. Ummm…and so, yeah, if there’s no name given then that allows us to choose, you know, a larger breadth of people and not have to worry about that and, also, if the sitter isn’t using their voice to respond, “yes,” “no,” “I don’t know,” then that’s another layer because then, you know, a voice can give away an awful lot in terms of gender, in terms of age, and, you know…
Steve: Like, “Are your parents still alive?” [very dramatically]“No…ahhh….” [laughs]
Female Voice: Right. [laughs]
Steve: [laughs] It also gives away education level and…it does…
Female Voice: Yes.
Steve: It does give away a lot of demographic information but…
Jay: What do you think, Alex?
Alex: I think we should, again, we’re talking about scoring and I think we’ll have fun and, if you want to bring up…we’ll see how those points stand up to scoring the results. But I don’t think we want to…
Female Voice: but what does that…what does that mean? I mean we’re talking about the actual protocol of…
Alex: Right. And I’m saying…
Female Voice: …the test.
Alex: …I think that the protocol we have is fine and I think we’ll look at the results and…if you’re still making those points at the end when we’re scoring the results, then that will be a separate conversation. I think introducing…
Jay: Okay, but…
Jay: Go ahead. First name, though, we’re going to do…
Female Voice: But no, you know, if those points affect the actual test though, then the whole thing is kind of for nothing. Do you see what I’m saying?
Jay: Let’s just…let’s just block out if we’re doing first names or not.
Steve: Alright, here’s the thing. Uhhh…if the…the tighter the protocol is upfront, then the more liberal we can be in the scoring. The looser it is upfront, then the tighter we have to be in the scoring. Right, Alex? So, if we give the patient’s actual first name or, if they’re speaking with their voice, then any information that the medium could glean from that, we…we’ll be more strict about scoring as a hit. Whereas, if they’re an ageless, you know, gender-nonspecific number or a pseudonym that does not give any information about the individual, and the sitter…I mean the medium is able to get that information, then, you know, we would, they would score higher. You know what I mean? So, either way, we’ll make up for any looseness on the actual reading in how we score it. Because you have to understand that, you know, that’s how it’s going to work. But I would say is, let’s find out from each medium what they feel is necessary in order to feel comfortable and to perform a reading. If they really insist on a first name, I’d be okay with allowing that but then we…that gets taken into consideration when we score. Uhhh…if they really insist on hearing the person’s voice, I would be okay with that but then we have to take that into consideration when we score. The…the less contact the medium will accept upfront, then the more generous we can be in the scoring. Does that make sense?
Alex: I…I agree. And that’s why I say it keeps coming back to scoring. And I think it will all kind of come out in the wash. Now, if someone has a very unique name, has a very unique voice, we’ll all hear that and we’ll all be able to kind of figure that stuff out. And I’m not against your idea too. We can certainly ask the medium if they’re comfortable going without those things or if they’re comfortable, you know, working on a keyboard at the same time that they’re doing it and all those things.
Jay: And, also, keep in mind, guys, if…if we go through this process and we find, oh boy, you know what, we overlooked a couple of glaring things or things that we thought, you know, compromise…
Jay: …what we do, we could do it again.
Steve: Oh, yeah. Absolutely. Ummm…well, you know, we’re not going in this completely inexperienced. We’ve all read, you know, this type of research and have at least monitored these kind of studies. We have some idea how these things work but this is the first time we’re doing this together. If we decide after we get through it that…ummm…we can improve the protocol and do it again, then, we’ll…hopefully, it will be a learning curve and we’ll improve things as we go along. No one gets…no one gets research protocols perfect the first time through, you know…it’s always a work in progress.
Alex: Yeah, that’s a good point.
Steve: But, we’ll see. Again, we may get really dramatic results, you know, and then we’ll have to go from there.
Jay: So, when do you want to do it?
Alex: Well, you know, one thing I thought would free us up a little bit is if we did it, you know, we don’t try and do it all in one day or even two days because we’re going to have scheduling issues with both sitters and readers.
Alex: You know, one thing I thought we might talk about is how we are going to select the sitters and I was going to pass along that…I think craigslist would be a really good tool to run an ad in craigslist and then we’re truly picking, I think, from a, you know, pretty random sample there. I mean, it’s…it’s…there’s some demographic issues but I think…that was my suggestion but I’m open to hearing any other ideas you have, running ads in newspapers or whatever.
Steve: Yeah. I think that, again, using some kind of public forum where we can pick from a large pool of somebody who does not have any prior connection to us or to you or to the medium obviously is the way we’ll do it but, again, it has to be done in a way that, you know, you and the medium don’t know who it is, right? So…
Alex: Right. So, I mean, I’m happy to help you with that but…
Steve: If only we know…
Alex: …I mean, if you control the craigslist ads, then you would control all that communication. If you don’t have the time to do it…
Alex: …I’m happy to help but I would think, you know…
Alex: …that’d be something that you’d want to do.
Steve: So, yeah, we’re…we’re…we will do that. We will do that part and…ummm…what we will do is disclose in exact detail how…the process that we went through to find the sitter. And it will be somebody that we do not know ahead of time that does not know…that does not know you…that does not…you know…we’ll have some, you know, criteria. Like, obviously, they shouldn’t be a psychic themselves or whatever. You know. Some obvious criteria. Somebody that would be, you know, a good generic sitter and all and that would be anonymous basically.
Jay: Alright. So we’re going to need to come up with that criteria then and pass that back and forth to each other.
Steve: But, yeah. We’ll do…this shouldn’t be difficult. We’ll, you know, we’re just going to advertise for somebody to partake in this type of experiment without really giving them too many details and then we…it’s also in the protocol that we will…ummm…coach them so that they know what kind of feedback is permissible and what not to do…what kind of feedback is not permissible. And…uh…again, we’ll document all of that.
Alex: And, I just had some points in there as well in terms of we want someone who’s open to the idea of spirit communication…ummm…we want someone who hasn’t…isn’t in grief…didn’t have someone close to them just pass in the last six months and they’re really grieving. That wouldn’t be good for either proponents or more skeptical people. And, just in general, I think we want people who are…that know basically what the experiment’s about and want to participate in it.
Jay: Yeah. I think that’s completely fair. You know, we definitely don’t want to get like a hardcore skeptic in there who is going to unconsciously be biased or consciously be biased. We do want someone that’s going to have an open mind just to…it’s going to be better results that way.
Steve: Sure. That’s all reasonable and, again, they should be sort of open and neutral both ways.
Jay: Yeah. And I also think that…ummm…what do you guys think about…after we go through it…we do…I don’t know how many people we’re going to end up doing but, let’s say, you know, it’s a handful, three to five or whatever. If those people, at that point, want to make themselves publically known to anyone who’s interested, like, in other words, Alex, if you wanted to talk to them afterwards, I think that is 100% reasonable…ummm…if you wanted to do…did you say the no post interview, Steve, or what was the protocol on that?
Steve: Well, what we didn’t want to have happen is have the sitter and the medium chat with each other about the reading afterwards because that would contaminate the scoring, right?
Jay: But what about Alex? If he wants to talk to them?
Steve: Once it’s…listen…once it’s scored, I think that, you know, the medium and the sitter have their very confined interaction as determined by the protocol with the feedback as we outlined it. We get all of the reading, right? We record the whole thing so that we have the reading. Once that’s done, then we score it.
Steve: Once it’s scored and done…all the data has been extracted from that experiment…then, who cares? You know, then, you know, we can reveal who the actual person was. Alex can talk to her. Whatever. You can do whatever you want. But we have the scoring. It’s done. But we…I think it’s important to limit any interaction prior to scoring because we don’t want there to be any, you know, coaching or contamination of how to score it, right?
Alex: I agree with all that. The only thing I’d add is I think we should really err on the side of confidentiality and, to that end, I would not…
Steve: It’s up to that individual. It’s up to them. Once we’re done with them.
Alex: The only thing I was going to add is I think we should try and push people towards being private in their information and not in enthusiasm saying, “Yeah, I want to go on and I want to reveal myself.” and, then, six months later, they’re feeling like, “Gee, that was a bad decision.”
Steve: Yeah, that’s a good point. Yep, that’s a good point.
Jay: Actually…that’s actually a really good point. If we leave any real, you know, no first name/last name, revealing. Even if the…I think, even if the medium guesses their full name, we should maybe still leave it up to the sitter if they want any of that information made public.
Jay: Right. You know what I mean?
Steve: Well, uh…
Jay: If they name like…if they name…what if the psychic says something very very embarrassing or whatever that happens to be true? Yeah. I wouldn’t want my name attached to that information…
Jay: …with thousands of people reading it. Like, we’re conducting, you know, we are the ones that are in control of the data.
Steve: Well, you know, if we want to be maximally paranoid, I mean, this is…we’re not doing an actual official protocol and, you know, I want to do everything that we can at this stage for this not to be considered actual research on a human subject because we don’t want to go there. Right? That’s…
Steve: That’s not what we’re doing. But, if you want to be maximally paranoid, there are rules in place to protect the privacy of other…of people who do get involved with studies like this. So, even though this is just a demonstration, we want to be clear about that. I think, you know, ummm…complying with those rules just to be extra careful and ethical is a good thing to do. So that means no personal identifying information gets published publically. Right?
Jay: Okay, but I think, behind the scenes, if Alex wants to talk to them and interview them afterwards, I think that’s totally legit.
Steve: They’re two private citizens once we’re done with the protocol. They can do whatever they want.
Jay: Okay. Okay.
Steve: But it will be up to the sitter to make any contact or to make anything known about them because we won’t be passing any information on that would identify the individual. That’s…
Steve: …that’s the standard…that’s the ethical standard in research.
Jay: So, let’s talk dates and how to guys.
Steve: Well, Alex, the ball’s in your court in terms of finding a medium, right? Or do you already have…
Alex: We have the mediums lined up and we’re using the mediums that Julie Beischel has used in her research which gives us a certain leg up in terms of they understand the process and should be pretty comfortable with, you know, explaining what’s going on.
Steve: So do you want to get like just a preliminary agreement that we’ll record this like on a Thursday evening because this is actually a good night for us in terms of working around our podcast.
Alex: Okay, great.
Steve: And then…when…so then, the limiting factor at this point is we will find a sitter and…and then just come up with a Thursday night where we’re all available and then push that out to you guys.
Alex: Okay, great. And then, I guess we can deal with the next…the actual procedure of doing it whether we have the sitter go through a sequence of sittings that night or whether we, you know, shuffle or have the medium read various sitters. We’ll just have to figure that out and space it out as best we can because we could have…do you see what I’m saying? Because we could have a sitter go and sit with three different mediums, one after another, or we could have the medium read for three different people, one after the another.
Steve: I think it would be complicated to have one sitter and multiple mediums because then each reading will not necessarily be independent or uncontaminated. So, why don’t we just do one sitter…that each sitter gets read once. And, if we can set up multiple sitters in an evening, then we’ll do that but we’re only going to have each sitter be…have one reading. Does that make sense?
Jay: You know, but…in Phase II, if we ever get to that, Steve, it would be really really interesting to have a medium do a reading on the same sitter that a cold reader does it so we could compare the two side by side and watch the process unfold and see what information each one gets out of the same person. That would be cool.
Steve: Yeah, that would be interesting although that…the other layer of complexity when you do that is that we have to make sure that whoever is doing the reading second gets no information from the first reading.
Jay: Oh, right.
Alex: So, let me make sure I understand, Steve, are you thinking then that for, the people who are going to get the readings, the sitters…how many readings are they going to get? Just one?
Steve: Each sitter will get one reading, yeah.
Alex: I think it’d be useful…
Steve: Again, just…
Alex: …to have them…it’s totally independent in terms of one reading to the next. I think it would be…even if we spread it out over several days or nights, to have them get at least a couple of readings from uh…
Steve: But that adds an extra layer of complexity in that…then it’s up to us to make sure that there’s no contamination from one medium to the next and that’s a layer of complexity I don’t want…
Jay: I see what you’re…I see what you’re getting at Steve.
Alex: Uh huh. Uh huh.
Steve: How do we know that…how do we know that medium 1 isn’t giving the feedback that they got to on…passing it on to medium 2?
Jay: That’s a good one, Steve.
Steve: Yeah. So, I just…I don’t want to deal with that complexity. We can just completely eliminate that possibility by just having each sitter only be read once.
Steve: I mean, honestly, there’s no reason why we need to recycle the same sitter. I mean, we’ll probably get so many people in that we…we…we’ll find ten people if we need to without a hitch. Right? Right?
Jay: Well, we’ll see. I don’t know. I have no idea how easy or hard it’s going to be.
Alex: I think you’re right. I think you’ll find…I think you’ll find a ton of people, a ton of people on craigslist.
Steve: Yeah. In which case, it shouldn’t be a problem to have multiple sitters, you know, so we can have one sitter with each medium you have available that evening. And, you know, just the fatigue factor…we’re probably not going to want to do one or…more than one or two in a night I am guessing.
Jay: How long is the reading going to last? I mean, are we going to put a thirty-minute time limit, an hour? Or what do you want to do?
Steve: That’s actually a good point. I don’t think we set a time limit on it.
Alex: You know, you had suggested…
Steve: Actually, I did. Thirty minutes. I said thirty minutes.
Alex: I think that’s perfect and I think what we ought to shoot for then is getting two readings in on a night. And then…
Steve: That’s fine.
Alex: …and then maybe we do it…
Jay: Okay, then, for setup and the post stuff that we’ll do, you know, it’ll be like an hour and a half to a two-hour thing. Now, is there going to be a moderator? Who’s going to be the moderator and what…how’s that going to be handled?
Steve: Well, I think where we each…like Alex and one of us will monitor the sitting.
Steve: So…because we have to…and we’re going to be recording it. So, then we’ll all be able to listen to it afterwards.
Steve: Does that sound reasonable?
Alex: Yeah, that sounds good. And, again, Skype is the ideal tool here because we can Skype out…
Steve: We can each record it.
Alex: Right, we can each record. We can Skype out to people that have the phones and we can do all the stuff we want to do in the background.
Jay: Look, guys, I’m also, you know, I think it would be good if…like, yeah…so, you’ll end up with four people in the chat room and then, Steve, you and Alex can, you know, if something happens that’s unpredictable, right? Like a protocol is breached or something, you guys are going to have to chime in and say hey, you know, we would prefer not…that you can’t ask questions like this or that or whatever…like we…you guys are going to have to…ummm…like…
Steve: Yeah, if we say…if the sitter inadvertently reveals inappropriate information to the media, that will not disqualify the sitter or the reading but that information will be considered when scoring the results, right? But to any…any hits that flow from the information just won’t count.
Jay: Do you think there’s going to be a situation where you’re actually going to need to break into the reading and say…
Steve: You never…you never know…but we might have to. We might have to remind the sitter of the coaching that we did because they’re only allowed to say “yes,” “no,” or “I can’t answer that question.” So, if they say anything else, we’ll have to say, “You’re only supposed to, you know, give “yes,” “no,” or “I don’t know” answers.” Ummm…and if, you know, because that…that…you know…just decreases the amount of information we’re going to get out of that sitting. Whoever then…
Jay: You and Alex need to be on and be able to talk to each other because you might have to make an on-the-fly decision.
Steve: Yeah. And, again, once we see how it goes, you know, we may want to throw some extra protocols in there just to correct any errors that are cropping up that we didn’t anticipate.
Steve: I think…I think what will happen is that, you know, if we properly coach the sitter, that would minimize the chance of any problem cropping up like that, but…
Steve: If it does, then we’ll deal with it on the fly and, again, it will all be recorded and we…just…it’s already agreed upon in the protocol that any contamination that comes from that will factor out in the scoring process.
Jay: Okay. Alright. So, now, once again, guys, dates?
Steve: Jay, we have to find a sitter. That’s the next step.
Jay: Yeah, but do you want to put some parameter on it, like, you know, I’ll start looking immediately, Steve. Like, I can start…
Steve: As soon as we get people…as soon as we have someone lined up, then we say, alright, what’s the next Thursday we’re all available and that’s when we’ll do it.
Jay: Okay, fine. Get somebody first.
Steve: And, hopefully, we can get two. We’ll try to get two people lined up for the first session.
Alex: Great. No, that’s a good…that’s a good plan and then…
Alex: ..and that also gives me the flexibility. If we have three different readers, I’m sure I’ll be able to get one of them to sit in for, you know, the night that we pick.
Steve: Good. That sounds great. So, I think we have a plan.
Jay: Alright. This is good, guys. We’re going to have a lot of fun doing this.
Steve: Yeah, this is always fun.
Alex: It’ll be good. It’ll be good.
Jay: Okay, so…so then the next communication will be…Alex, we’ll contact you when we have a few people that we think are ready to move to Stage II.
Alex: Right. You just might want to touch base with me on how exactly you’re going out and what information you’re giving folks in recruiting them but I don’t…I’m not too much of a stickler on that but I’d like to have a heads up on that.
Steve: Yeah, we’ll keep you in the loop.
Alex: Alright, guys. Great.
Steve: Good talking…
Alex: I look forward to doing this.
Steve: Alright, so…alright, Alex. We’ll be talking with you soon.
Alex: Okay, bye.
Jay: Goodnight, Alex.
Alex: Well, thanks again to Steven Novella and the group over there at The Skeptics Guide; one, for participating in this demonstration and, two, for agreeing to publish this protocol discussion that we have and share that with everybody.
Now, next week, I’m going to continue delving into this medium research. Hopefully, I’ll have an interview up with Julie Beischel who’s going to be assisting us and helping to put together this demonstration and carry it out in the best way possible. So, look forward to that. I also have a couple of other interviews coming up that I think will be of great interest to everyone out there. In the meantime, if you’re interested in any of our past shows, be sure to check out the website at skeptiko.com. You’ll also find links to our email address, previous episodes, transcripts…I’m trying to get more and more transcripts of the shows up there and you’ll find those there. So, check out the website. I also have a blog going where I publish some additional information and, in this case, I’ve published the full text of the protocol that we’re using on the blog. So, check that out. I’ll have a link there in the show notes.
Well, that’s going to about do it for today. Until next time, bye for now.