Wouter J. Hanegraaff : Esotericism and the Academy: Rejected Knowledge in Western Culture
Gerhard Kienle - Leben und Werk: Bd. 1: Eine Biographie / Bd. 2: Ausgewählte Aufsätze und Vorträge: 2 Bde. [Gebundene Ausgabe]
Peter Selg (Autor)
Rudolf Steiner 1861 - 1925. Lebens- und Werkgeschichte. 3 Bände im Schuber
Peter Selg (Autor) - Gebundene Ausgabe: 2148 Seiten
Verlag: Ita Wegman Institut (6. Dezember 2012)
ISBN-10: 3905919273 - ISBN-13: 978-3905919271
Flawed Experiments being passed off as Failed ExperimentsMany sceptics claim that astrology has consistently failed over many years in ‘thousands of scientific tests’. This popular myth that has been duplicated in many websites. I have asked numerous critics of astrology including Dr Geoffrey Dean to cite their best single test. (For a review of what are considered the 'best' attempts.). Here we look at the extensive work of Dean - who is widely cited by sceptical websites.
Geoffrey Dean - dedicated and prolific researcher and writer
Dean's most famous experiments - good science or smoke and mirrors?
The 'Unpublished' Unaspected Planets StudyIn Recent Advances (Dean 1977), Dean outlined his own promising two year study of 20-40 cases for each of ten unaspected planets compared with controls. Meanwhile another independent analysis of 'several dozen examples of each unaspected planet' was conducted by E.A. Moore in New Jersey. (Moore 1976) According to Dean "In general their findings are in good agreement. Both workers found that the planetary principle is unchanged by lack of major aspects...". However, despite the great potential, Dean's original study has remained in the file drawer has yet to be published by a peer reviewed scientific journal. (Dean 1975)
Dean's Phantom Time-Twin Study:Dean's study involving 2,101 people born in London between 3-9 May 1958 appears to have great potential. Though he refers to it in his paper on PSI (Dean 2003 p.188)  (Dean, forthcoming), Dean is yet to publish his results and will not share this government data. As I write, Dean has sat on this data for seven years prompting some to wonder if there are unreported significant patterns in the time twin data. Here is yet another study confined to the file drawer.
Dean's GIGO Meta Analysis:Multiple comparisons of experiments is one of the best ways to identify trends and review a number of related tests with small samples. However, the key to a meta-analysis is the selection of the tests, since the choice is open to bias. To take an extreme example, testing whether werewolves come out at the full moon is not a valid test of astrology. Now the problem with Dean's analysis is that it is not possible to check or verify his selection. When the results were published on-line, it ended with a worrying caveat: "References: A list of the studies meta-analysed above is in preparation." This remained unchanged on the web for many years until 2011 when the page was taken down. However, for research purposes, I resurrected a copy using the waybackmachine.
Now, even in the social sciences, sources must always be provided. Without supporting sources, these 'scientific' analyses are no better than fiction for entertainment purposes only.
One possible reason that the list of tests was never published is that an undue number of the sceptical tests selected were inappropriate or unreliable - which would have undermined the study. A few unreliable tests would have been ironed out in the full analysis. However, when I asked Dr Dean in 2009 by email to cite the most persuasive tests that astrology was no better than chance, he could only come up with prize challenges (similar to the Randi Million Dollar Challenge), Sun Sign tests and tests that have been shown to have an obvious fatal flaw. Now that the Carlson test(Carlson 1985) has been shown to favour astrology, rather than re-analyse the appropriate Meta-Analyses, Dean considers that this test to be outdated. How does this square up with his inclusion of tests from the mid 1980s and before?
Meta-Analyses suffer from publication bias. While some astrologers are strongly motivated to produce positive results, very few do any systematic studies that would be acceptable for publication in a journal. Many were educated in the humanities rather than science and there is no funding for astrological research. Yet, scientists who produce results that favour astrology are disinclined to report their results. A case in point is Professor Cajochen who sat on his study that showed a correlation between sleep and lunar phase for four years for fear of being considered a 'lunatic'. Dean himself has contributed to this file drawer effect by not publishing at least two of his own studies.
This witholding of results or reluctance to publish is partly down to a taboo against fringe subjects and partly because scientists are warned that "the more extraordinary a claim, the heavier is the burden of proof demanded." ~ Marcello Truzzi This failure of research and the "file drawer effect" can distort meta-analysis.
So given the background to the selection process by a professional sceptic, it is not unreasonable to assume that the Meta-Analyses suffer from GIGO (Garbage In, Garbage Out) or the fallacy that Many Wrongs do not make a Right. Yet, there are still sceptical sites proclaiming this as their best evidence against astrology.
Test of Reported Intuition: Why testing only outliers can be misleading:Dean claims to have been unable to detect any significance in 45 astrologers' assessment of neuroticism and extraversion in a birth chart. This experiment entitled "Can astrology predict E and N?" published in Correlation(Dean 1985) used an impressive original sample size 1,198 test subjects. However, "to make each direction clear-cut" Dean reduced the sample size to 40 in each of 4 categories (High v Low Extraversion and High v Low Neuroticism) by taking the 6.66..% most extreme cases. (Dean 1986) Why did this happen? It would be like testing a wine expert on the dregs from the bottom of an old bottle of wine rather than on the wine.
Normal practice in psychology would be to take the extreme 33.3..% on either side of the continuum. This would avoid confining the test only to the 'outliers'. Now if Dean had reviewed the Eysenck Personality Questionnaire [EPQ](Eysenck 1975) results and chose the extremes knowing that the data was not typical, the data has been cherry-picked. If, as is more likely, it was an arbitrary choice or done to keep the sample size manageable, then there is a problem with sampling bias and exclusion bias in particular) since typical or average examples of Neuroticism and Extraversion have been removed by taking only extreme cases. Either way this sampling error calls into question the value of the conclusions that he drew from the experiment:-
- Outliers are often the result of a measurement error or in a personality test, psychopathy or a hoax response where spoof answers are given to convey a ridiculous extreme personality. In additon when Dean broke up the large sample (~1.2k) into small samples (40), he enabled these rogue responses to have an exaggerated impact on the results. By doing this, Dean also made it easier for him to dismiss small effects as being random and slight effects can be disguised in apparently neutral results. The Gauquelin and other studies of astrology have shown that large samples (10k +) are required to overcome artifacts and to demonstrate the effects of astrology.
- In Jungian and other branches of psychology, exteme self-definitions of personality (the outliers) are often a cover for the opposite traits. This results in what is known as the shadow personality. Here's an example of a shadow personality that has been documented by research that shows that homophobic individuals are aroused by male homosexual stimuli despite being in denial or unaware of this. So the self-reporting descriptions of their character would be the polar opposite of their nature.
- A related problem is the accuracy of self-compiled psychological tests. In this test the Eysenck Personality Questionnaire [EPQ] (Eysenck 1975) is assumed to be the objective standard against which astrology is measured. However, the study may have revealed more about the accuracy of the EPQ. Experience with theCarlson test shows that subjects were unable to identify their own self-reporting test results suggesting that they may not be 'fit for purpose'. [for more] I have tried the EPQ. Despite the apparent lie detector, it is easy to skew the results. Those who dislike these tests or hate to be labelled take a delight in providing perverse information. There's also an element of fantasy - rather like asking a woman to select her weight or a man his earnings. Everyone ticks honesty, sincerity and modesty. The problem is EPQ lacks the objectivity and incisiveness of astrology.
The Carlson Double Blind Astrology Test (1985)- which now supports astrologyDr Dean happened to be in California when Shawn
Carlson performed his Astrology Test in the early 1980s - some eight thousand miles from home in Perth, Australia. Dean was able to provide advice at the time and later by mail. The test was overseen, sponsored and published by members of CSICOP - a group considered by some scientists to be dedicated to debunking metaphysical claims. The tests have since been criticised by three professors including Hans Eysenck (Eysenck 1986) for faulty conclusion (Type II error) and design (Data Selection bias, Sampling bias and Mathematical Error).(Currey 2011) Professor Suitbert Ertel has shown that where astrologers were able to rate their performance, they performed to a statistically significant level that cannot be dismissed as chance. (p=.037) (Ertel 2009)
Why are so many astrology tests flawed?In the field of astrology, there are considerably more fatally flawed tests than real evidence. There are many reasons for this. There is no budget for testing astrology and most astrologers are more motivated by the study and application of astrology than in addressing the challenge of providing and defending scientific proof. So most tests are run by sceptics with budgets in fields like psychology who design quantitative tests when the data requires qualitative analysis that would be better addressed by those who understand astrology. There are also real procedural hurdles to jump.
- Carlson, Shawn (1985) A Double Blind Test of Astrology. Nature, December 1985 Vol.318, pp.418-425.
- Currey, Robert (2011): U-turn in Carlson's Astrology test, Correlation. Vol.27 (2), July 2011
- Dean, Geoffrey (1975) Unaspektierte Planeten, Kosmobiologisches Jahrbuch 1977, Ebertin Verlag, Aalen, pp.111-133
- Dean, G.A. & Mather, Arthur (1977) Recent Advances in Natal Astrology. A Critical Review 1900-1976. Analogic, Subiaco, Australia
- Dean, G. (1985), ‘Can astrology predict E and N? 2: the whole chart’,Correlation, 5 (2), pp.2–24.
- Dean, G. (1986), ‘Can astrology predict E and N? 3: discussion and further research’, Correlation, 6 (2), pp. 7–52. Includes meta-analyses of astrological studies.
- Dean, G., Mather, A. & Kelly, I.W. (1996), ‘Astrology’, in The Encyclopedia of the Paranormal, ed. G. Stein (Amherst, NY: Prometheus Books), pp. 47–99. Includes meta-analyses, effect size comparisons and artifacts.
- Dean, G. & Mather, A. (2000) Patron of Research, Tribute to Charles Harvey Astrological Journal 42(6), pp.45-46, Nov/Dec 2000
- Dean, G. & Kelly, Ivan (2001) ‘Does astrology work? Astrology and skepticism 1975–2000’, in Skeptical Odysseys, ed.
- Dean, G. & Kelly, I.W. (2003) Is Astrology relevant to Consciousness and Psi? The Journal of Consciousness Studies, 10, No. 6–7, 2003, pp.175–198
- Dickersin, K. (1990) The existence of publication bias and risk factors for its occurrence JAMA 263 (10) pp.1385–1359. March 1990
- Ertel, Suitbert (2009) Appraisal of Shawn Carlson's Renowned Astrology Tests, Journal of Scientific Exploration, Vol.23, #2.pp.125-137
- Eysenck, Hans Jürgen & Eysenck,Sybil B. G. (1975) Manual of the Eysenck Personality Questionnaire. London: Hodder and Stoughton
- Eysenck, H. & Nias, D.K.B. (1982) Astrology, Science or Superstition St Martins Press, London
- Eysenck, H. (1983) Methodological Errors by Critics of Astrological Claims. Opening lecture at 3rd Institute of Psychiatry Conference, May 1983.
- Eysenck, H. (1986) Critique of “A Double-Blind Test of Astrology”, Astro-Psychological Problems, Vol.4 (1), January 1986. Eysenck wrote “The conclusion does not follow from the data”.
- Kruskal, W. (1960) Some notes on wild observations, Technometrics, University of Chicago.
- Moore, E.A. (1976) From personal communication with Dean based on printed material used in the Moore School of Astrology on unaspected planets. New Jersey 1974.
- Ripley, Brian D. 2004. Robust statistics
- Robert Rosenthal (1979) "The file drawer problem and tolerance for null results" Psychological Bulletin 86 (3): pp.638–641
- Dean G. & Kelly, I. W. (2003). "Is Astrology Relevant to Consciousness and Psi?". Journal of Consciousness Studies 10 (6–7): p 188 - 190.
From personal correspondence between Robert Currey 19 July 2008 to 8 September 2008: Dr Dean claimed that he is still working on the paper. In 2008, I requested permission to publish these exchanges but Dean has yet to reply.
- Choi, Charles (2013) Bad Sleep? Blame the Moon Livescience. July 25, 2013 Professor Cajochen "It took me more than four years until I decided to publish the results, because I did not believe it myself,I was really skeptical about the finding, and I would love to see a replication."
- The Meta-Analysis published on-line (~2008) and presumably (though it does not say) an update compiled from previous meta-analyses. Dean, G. (1986), Dean, G., Mather, A. & Kelly, I.W. (1996), Dean, G. & Kelly, I.W. (2001)
- The Shadow is an unconscious aspect of the personality which is not recognised by the conscious ego and can be rejected."Everyone carries a shadow,and the less it is embodied in the individual's conscious life, the blacker and denser it is." ~ Jung, C.G. (1938). "Psychology and Religion." Psychology and Religion: West and East. P.131.
- Is homophobia associated with homosexual arousal? (1996) Journal of Abnormal Psychology Psychol. 1996 Aug;105(3):440-5. Adams HE, Wright LW Jr, Lohr BA. Department of Psychology, University of Georgia, Athens 30602-3013, USA."Homophobia is apparently associated with homosexual arousal that the homophobic individual is either unaware of or denies."
- The Eysenck Personality Questionnaire for Introversion/Extraversion has different parameters. I would consider the EPQ to be an inferior measure compared with astrology. For example, practicality is ranked against reflectiveness. Which is the extraverted quality? As an astrologer, I would not equate practicality or reflectiveness with extraversion.
- Dean (2010). In personal email correspondence between Robert Currey and Geoffrey Dean 15 July 2010, Dean commented: "I was in California at the time Carlson was doing his experiment, was able to discuss it with him in person and subsequently by mail, and was able to meet some of the astrologers involved in it."
- Sheldrake, Rupert (2004) Distorted Visions. Letter published in the Times Higher Education Supplement. December 17 2004. "CSICOP is an ideologically motivated debunking organisation..."
- American Heritage® Medical Dictionary (2004) Houghton Mifflin Company.
- GIGO stands for Garbage In, Garbage Out. The expression originated to explain how invalid data entered into a computer program results in invalid output.
- Cherry picking, suppressing evidence, or the fallacy of incomplete evidence is the act of pointing to individual cases or data that seem to confirm a particular position, while ignoring a significant portion of related cases or data that may contradict that position. Argument By Selective Observation and in this instance, counting the misses and forgetting the hits.
- Sampling bias in statistics, is where a sample is collected in such a way that some members of the intended population are less likely to be included than others. In the Carlson experiment, the sample was from a homogenous population which hindered the task of differentiation. Sampling bias can also take the form of the selection of studies to support a particular hypothesis as appears to have occurred in Dean's Meta-Analyses.
- Data Selection bias can occur in several ways:
- When data is partitioned with knowledge of the contents of the partitions, and then analysed with tests designed for blindly chosen partitions. In the Carlson test and CSICOP's replication of the Gauquelin studies, samples were subdivided into smaller samples so that significant results were dampened down.
- When "bad" data is rejected on arbitrary grounds, instead following previously stated or generally agreed criteria.
- When outliers are either rejected or selected exclusively. Rejection of or selection of only outliers on statistical grounds can also result in biased results.
- Type II Error is the error of failing to reject a false null hypothesis or wrongly accepting a false null hypothesis.
- Exclusion bias results from exclusion of particular groups from the sample.
- Publication bias is the practice of investigators, reviewers, and editors to submit or accept manuscripts for publication based on the direction or strength of the study findings. One aspect of this is the file drawer effect where studies are conducted but either abandoned or not reported because the results do not conform to the experimenter's expectations or intentions. A small number of papers left in a researcher's drawer can result in a significant bias. (Rosenthal 1979)
- Outliers are observations that are numerically distant from the rest of the data and can indicate faulty data, erroneous procedures, or areas where a certain theory might not be valid. "They (Outliers) can play havoc with standard statistical methods." (Ripley 2004) "It is a dangerous oversimplification to discuss apparently wild observations in terms of inclusion in, or exclusion from, a more or less conventional formal analysis. An apparently wild (or otherwise anomalous) observation is a signal that says: 'Here is something from which we may learn a lesson, perhaps of a kind not anticipated beforehand, and perhaps more important than the main object of the study.'" (Kruskal 1960)
- Fallacy Of Composition is assuming that a whole has the same simplicity as its constituent parts. In fact, a great deal of science is the study of emergent properties.