Evidence-based teaching?

Evidence-based medicine is all the rage at the moment. While not lacking in critics and almost certainly leading to the dismissal of some brilliant mavericks and their ideas alongside the control of crackpots and incompetents, all the evidence seems to suggest that the process of asking doctors and hospitals to put data before instinct leads to huge statistical improvements in health. So, can we imagine a sometime in the future world in which everything we do in the classroom will have to be backed up by research? It’s possible, but TEFL will as ever be at least twenty years behind mainstream education, and there’ll have to be a huge increase in research funding. So, at the moment we should be in the clear position that we all know that we don’t know anything and all accept that virtually anything in the classroom can be backed up by quotes from books that are no more or less credible than the books saying the exact opposite. Right?


The position we are actually in is a complete fudge, most clearly illustrated by the influence of research on the Baby Bentley of TEFL qualifications, the Cambridge DELTA. In order to pass the DELTA you will need, it is true, to read and quote loads of books. Nowhere in the criteria does it say, however, that any of those quotes and books be based on actual research. A couple of opinions by the two opinionated Michaels (Swan and Lewis) plus at least one more, and one of your essays for the lesson observations is done. When I was doing the DELTA myself I launched into a ten minute rant during one input session on being endlessly told about and being told to reproduce who said what about what when, with not a jot of data behind any of those statements. The DELTA trainer’s reaction was a tired looking “Well, that was interesting” and back to another photocopy from 1982, and that seems to sum up Cambridge’s attitude also- including to the whole concept of outdated and up to date sources that would be important in virtually any other field.

You could say that all that is fair enough, because if there were any conclusions to draw on, they could tell candidates “Please, no more lessons on Cuisenaire Rods. They have been conclusively proved to be useless for learning” or at least “Candidates will be lose marks for mainly quoting references in support of their arguments when the consensus of research is quite clearly against”. Obviously none of us are in the position to say that, but what then is the use of quoting old issues of ELTJ- even if you do (and you do) have to compare that to your own experience? Is it simply to prove that you know what the ELTJ is in the same way that you must define terms that you already know that the person who is reading knows??

More on science and TEFL in the two posts below, and will give some links to interesting pieces elsewhere when I wrap it up with the next post.

This entry was posted in Cambridge Delta, Cambridge ESOL, TEFL Heroes- Michael Lewis, TEFL heroes- Michael Swan. Bookmark the permalink.

54 Responses to Evidence-based teaching?

  1. Andy Mallory says:

    This is indeed a very good point. I did the Trinity Diploma and while I learned a lot and my ability to teach owes a lot to what I learned or perhaps became aware of on that course – I can’t offer much evidence about why it works or should do so.

    In fact – a lot of it doesn’t seem to work all that well – and I’m gradually giving up on those bits.

    The essence of these higher TEFL qualification is – I think – that we as teachers should be trying to use class and homework time productively and reflect on that use constantly. There is perhaps a tacit acknowledgment from those running Dips that they don’t have any clear and definitive answers – especially since their newly Dip – qualified teachers might go anywhere in the world and teach anything in this incedibly varied industry. It is thus up to us to do our own operational research and not rely on an assigned coursebook or gimmick methodology.

    When I’m having trouble with a class or see another teacher struggling it is usually possible to find something from my diploma input sessions – often something very basic – that will help. It might not be a complete cure but it will help.

    I think 90% of our difficulties (excepting issues to do with class size or mixed levels etc) come from failing to do the basics well. All the little stuff like drilling, eliciting, varied interaction patterns, error feedback, planning, having interesting and well produced materials and so on. Many lessons fall foul of too much TTT or too much use of class fronted teacher led activities.

    I don’t think teaching (TEFL or the more mainstream kind) will ever be run by evidence based decision making. I admit it ‘could’ be but it’s far easier and simpler for the education Czar to simply pick their own pet theory and impose that on schools than to fund any research…literacy hour anyone??

    I can’t ever see a body powerful enough in TEFL to impose anything on schools they do not actually own.

    Education (mainstream or TEFL) is not about learning – it has other agendas that are not well served by evidence based decision making. Say evidence proved that schools should only run in the mornings when kids were best able to concentrate?? Unlikely to sit well with the need for mothers to work full time. TEFL is based on giving the customers what they want – regardless of the evidence. Where is the evidence that native teachers have to be white? But it is harder for black or Asian teachers to find work than for whites.

  2. Sputnik says:

    This kind of hand-wringing is endemic in all university departments that aren’t pure science. Every discipline seems to want to mimic the empiricism of the sciences, but, this seems a little strange given that even the most ideal form of scientific research attempts to occult its own position of enunciation and thus becomes something it is not.
    Having said that, I would like to wire some of my students up to electrodes – purely to find out what, if anything, is going on inside.

  3. Jeremy says:

    Hi Alex

    Great post, as always. A couple of random reactions:
    1. Hate to be pedantic, but I’ve just finished reading an amazing book (Gut instincts, Gerd Gigerenzer, http://www.powells.com/biblio/2-9780670038633-7), which demonstrates in a series of very impressive experiments that healthcare decisions based on mountains of evidence tend to be worse than informed gut instincts. I have no opinion on this myself, never having conducted any medical research, but the claims in the book suggest you’re at least partly wrong when you say “all the evidence seems to suggest that the process of asking doctors and hospitals to put data before instinct leads to huge statistical improvements in health.”

    2. I remember thinking the same thoughts as you (not Gerd) around the time of my Diploma – it seemed to be based on fashions and great ideas rather than evidence. One minute, the communicative approach or the lexical approach or whatever is the bee’s knees, next minute it’s an embarrassingly unfashionable relic from yesteryear. But all that seemed to change was the fashion, not the evidence.

    I remember submitting such a question to the Ask a Linguist section of Linguist List many years ago. The reply I got was that language learning has so many variables based on the teacher’s and student’s individual talents, motivations, needs, etc that it’s basically impossible to set up controlled experiments to test hypotheses. And even if you could find some trends in the statistics, there’s no guarantee those trends would be useful in deciding how to teach a particular student in a particular situation.

    Anyway, plenty to think about


  4. Mark Bain says:

    I’ve recently finished reading Ben Goldacre’s Bad Science, and, aside from being a great read, it gave me a lot to think about regarding my own teaching.
    Basically, I have no idea if what I’m doing in the classroom works or not. I’ve just taken someone else’s word for it. As Goldacre points out, this can often be a very bad idea.
    Consequently, I’ve come to the precipitous conclusion that evidence-based practice should become the basis for what we do in the classroom.
    How could this come about?
    I think it’s time we started being a bit less accepting of nonsense. We could start with the easy targets, like learning styles and multiple intelligences.
    And, before it’s mentioned, I don’t think pseudoscience is harmless – it muddies the waters and leads to the idea that it doesn’t really matter which conflicting opinion you “believe”, that science can be used to prove anything. At worst, it leads to national governments spending $85m on bomb detectors which don’t work, thus leading to countless deaths.

  5. Mark Bain says:


    How does Gigerenzer suggest our instincts become ‘informed’?
    Isn’t part of that process weighing up the available evidence?
    And do you really think language learning really too complicated to research?

  6. jeremy says:

    Hi Mark

    Ah, it’s complicated, and I don’t have the book with me to check the details. It involves making decisions (e.g. whether to send a patient to the coronary unit) based on a small number of key variables, often only one variable, rather than plugging all available data into huge and complex formulas. The challenge is to identify which variable is the key one, which of course is based on evidence/statistics.

    In test after test, this simpler approach outperformed both the computer-generated decision and the pure gut reaction decision. It’s illogical, but it works.

    The point is that for many years doctors were expected to input all the variables to come up with a decision, calculated by a computer, but often their gut instincts told them the computer decision was wrong. Gigerenzer’s technique helped sort the good gut instincts from the bad.

    Anyway, I’m not an advocate or expert on this, I just read a book on it. I enjoyed the book and strongly recommend it. I have no idea if these findings are relevant to ELT …

    And in answer to your second question, no, absolutely not. The idea that “language learning is too complicated to research” is just what i was told by the experts, but like you and Alex, I find it hard to accept that our work can’t be based more on evidence.

    I’ve remembered something else I was told in reply to my question to the experts – it also all depends on what we mean by ‘successful learning’ – if technique A leads to better fluency and technique B leads to better accuracy, which is ‘better’. And so on and so on. It’s not like assigning someone to a coronary unit, which can be evaluated on whether they go on to have a heart attack.

    Again, these aren’t my opinions, just reporting/interpreting what I was told.

    My humble opinio: it’d be quite tricky to set up a controlled experiment. Tricky, but perhaps not impossible.


  7. Mark Bain says:

    Coincidentally, I just came across a paper I downloaded a while back by Gigerenzer (et al) called “Helping Doctors and Patients Helping Doctors and Patients Make Sense of Health Statistics”. If you’re interested, you can get it here:

    Click to access pspi_8_2_article.pdf

    Same source as Harold Pasher, et al, “Learning Styles: Concepts and Evidence” – a pretty damning meta-analysis of… well, the name says it all.

    Click to access PSPI_9_3.pdf

    In this paper, Pasher and his colleagues lay out the sort of evidence that would be needed to indicate the existence of learning styles. And they make it very clear that, at the moment, no evidence exists (nevermind evidence that teachers can tailor their lessons to a particular style). A must-read for any teacher interested in evidence-based practice.
    Don’t want to come across as too extreme – instinct is important, but so is evidence. For example, there’s been a lot of research done recently on meta-cognition which suggests that we are not good at judging how we best learn. We tend to focus on short-term (and therefore easily visible) improvements, which don’t tend to last. Have a look at the paper for references.

  8. Alex Case says:

    From the second description, it seems to me that Jeremy’s book is an example of the good use of research rather than a debunking of the importance of it. In TEFL we don’t know which of the thousands of things we look out for are important, and only research and new theories to go with them can possibly tell us. I agree that it will be difficult to design repeatable experiments with widely useable results to tell us what to do minute by minute, but it should at least be possible to make useful generalisations in public school English classes, e.g. to work out whether the trend to introduce EFL from primary has any positive effects at all.

  9. Diarmuid says:

    Not sure how relevant Ben Goldacre is here (altho the book is a masterpiece). There are so many variables in teaching and learning that I suspect that it doesn’t lend itself to scientific exploration. Hence, despite thousands of years of introspection, the teaching profession cannot describe how to teach perfectly. [I have just read everything above and see that this argument has already been wheeled out. Apologies, but I am glad to hear that I am in agreement with “the experts”…and am letting this stand in case I can convince anyone that the experts are in agreement with me.]

    If you want a good scientific approach to teaching, allow me to recommend Diane Larsen Freeman’s and Lynne Cameron’s seminal “Complex Systems and Applied Linguistics”. In fact, I have just taken it off the bookshelf and am going to go through it again. If you can’t afford to buy it, can’t find it to steal it or can’t be arsed to go the the library to borrow it, DLF and LC put forward the metaphor of complex systems and chaos theory to describe language acquisition. With the reminder that “chaos” is a mathematical term describing a situation where behaviour is entirely unpredictable but not random, the authors write (p.57):

    It is the sensitivity to small changes that makes the behavior of chaotic systems unpredictable. To be able to predict behavior, we would need to know absolutely accurately every small detail of the starting state, called its ‘initial conditions’. Any small error in describing the initial conditions will be multiplied and amplified as the system changes, leading to large variations in actual behavior. The behavior of a system in a chaotic attractor [that is, a chaotic state] is unpredictable, although it is still technically deterministic, in that its current state determines its future states. The problem is that we cannot ever know the current state sufficiently accurately to be able to predict future behavior (Cohen and Stewart 1994:189).
    from Larsen-Freeman, D. and L.Cameron 2008 Complex Systems and Applied Linguistics. Oxford:OUP

    As for evidence based learning, I am all in favour!

  10. Alex Case says:

    Here are a couple of pieces of science that could well be applicable to TEFL:



    Again, they aren’t going to tell you what to do when planning lessons- let alone in real time in the classroom- anytime in the near future, but this kind of research that can have influence on policy makers seems a good first step

  11. Mark Bain says:

    Another book for my Amazon wishlist!
    Regarding variables, any experiment involving people is subject to a wide range of variables, isn’t it? That’s why you make the subject group as large as possible. And surely, interventions such as correction should have a noticable impact, regardless of other variables. Otherwise, are they worth doing?
    And, as complex as this may be, what are the alternatives? Basing practice on anecdotes, hearsay and misconceptions?
    I mentioned Ben Goldacre because of his championing of evidence-based practice, something lacking in TEFL. Some of the cases he discusses impact on education – the fish oil trials, for example.

  12. Alex Case says:

    You can have my copy if you write me a review for TEFL.net Mark, as I found it to be one of the most pointless TEFL books I have ever read. Can’t remember what my objections were now until I reread the enraged scribblings I put in the margins, but I seem to remember the main one was that chaos is basically a mathematical theory and there wasn’t a single equation in the book. The things they talk about are just metaphors to explain chaos theory in the same way as the billiard balls on a trampoline have very little actual connection to general relativity, and yet they then use those metaphors to make more metaphors and somehow call what they do based in science (with the usual meaningless academic caveats). Having said that, wasn’t nearly as bad as the attempts by Michael Lewis to use chaos theory as a justification in one of his books.

  13. Alex Case says:

    Do still have a next post or two in the series, just got overtaken by events…

  14. Mark Bain says:

    No equations? So it’s a “popular science” book, then, Nothing wrong with that. I mean, books are entitled to have friends, too…
    Not wanting to rob Larsen Freeman and Cameron of income, but a free book in exchange for several hours of sweating out sentences over a laptop sound great. Send me more details, please!

  15. Diarmuid says:

    Perhaps Alex wouldn’t have been so angry if he had taken the time to read about the book before buying it! Even a cursory skim of the preface should have made that clear. It’s a book – and most certainly not deserving of being labelled “popular science”-which is hard going, but illuminative. That is, it threw some light on what language learning might be about. That’s pretty much all a book can hope to do, I guess. That said, I don’t pretend to be a mathematical genius and I was thrown out of nearly every science class I ever attended, so perhaps I’m not the best person to judge. But for us people of lesser intellect, the book was far from pointless.

    Regarding variables, Mark, my suspicion is that extending the size of the subject group is not enough in itself. The subjects are only one element of a number of key variables. I can’t see how hard science has come up with a methodology to deal with what is essentially a sociocultural phenomenon.

    I liked your argument that correction “should” have an effect – although I chose to read it plaintively! I am happy to be corrected, but I think that the jury is still out on that one. Correction is not believed to have a direct impact on long term learning – I think some intrepid researchers have even tried to demonstrate this. Are they worth doing? Well – they can’t hurt and, in most cases, they are exactly what students want and expect. In that case, it might be damaging NOT to correct because you risk losing any validity that you may have. The fact that it is intuitive to learning is also not one to be dismissed. I don’t necessarily think it is the correction per se that results in language development, but perhaps this isn’t the time or the place for this.

    What are the alternatives? The alternatives to what? To a scientific methodology that still refuses to give us clear indicators or answers? From this perspective, the idea of actually going with making modifications to what you do in the classroom, based on what you actually see happening in front of you is not all that ludicrous. Talking to colleagues, observing colleagues, trying out things that might result in abject failure, reading widely and trying to extrapolate, going to conferences, arguing with colleagues, forming your own individual understanding and then putting it out there for others to discuss…they seem like workable alternatives.

    I hope you don’t think I was being critical of your mentioning of Ben Goldacre. His book is wholly enjoyable and also very illuminative – very much at home in the “popular science” category (am assuming that even popular science can have equations!). I’m just not convinced that teaching needs to shoulder its way into the world of hard science [yet]. It’s a social science and the ‘soft’ sciences have their own methodologies that have served them well. Linguistics may be more of a medium-boiled science and I can see how it could be served by equations and inclusion of half of the Greek alphabet. But when it comes down to people working with people within cultures within cultures within cultures, I think humble observation and documentation is all that we need (NB observation was missing from your list of alternatives).

  16. Alex Case says:

    Free copy from OUP (may I plug reviewing for TEFL.net yet again at this point?), so it wasn’t financial pain! I’m a huge fan of metaphors as a way of learning teaching, think I even had the start of a series on them on this blog at some point. What I am not a fan of is trying to back up those metaphors by linking them to an unconnected piece of science. Now must find my copy of that book to see if that really was my objection…

  17. Mark Bain says:

    Popular or unpopular, I’m still keen to read it. Alex, email me a copy, OK? I’ll write you a review.
    In response to Diarmuid,

    Clarification: I don’t think correction “should” have a impact; I meant to say that, for it to be worthwhile, any intervention needs to have a noticeable effect on all learners, regardless of motivation, culture and expectations.

    Other points…

    “…what is essentially a sociocultural phenomenon.”

    Don’t misunderstand me – I’m not a hard-science extremist. Indeed, I come from a very soft social science background. But even sociocultural phenomena can be studied with some rigor. We’re discussing evidence-based practice. I’m not suggesting there needs to be a mathematical equation for everything.
    “…a scientific methodology that still refuses to give us clear indicators or answers”.

    Over the centuries, science has provided us with answers to many important questions. In our own field, there is nothing unclear about, say, the research into learning styles. There is no good evidence that learning styles exist, end of story. Oh, there are lots of reports, articles, papers and books which claim to show evidence of learning styles. Loads of them. But not one is based on good research. By good research I mean a fair trial reported fully and honestly.

    “…they seem like workable alternatives.”

    Depends what you mean by “workable”. Sounds like a pleasant way of passing a career, but as a way of getting closer to the truth of the matter, it leaves a lot to be desired. Observation is fine for some things, but no good for others. Simply put, we can’t always believe our own eyes. Hence, the scientific method. And, without any scientific basis, all the rest is just chatter.

    I assume that, when you refer to “trying out things that might result in abject failure”, you are exaggerating. A certain amount of experimentation on a very small scale might be justifiable. But students aren’t guinea-pigs, and teachers shouldn’t treat them as such. I don’t think “abject failure” is an outcome any more acceptable in the field of EFL than it is in mainstream education.

    Oh, and your final point – that individual observation and documentation is enough – starts sounding an awful lot like science, especially when coupled with public discussion. Maybe we agree more that it seems?

  18. Alex Case says:

    Will do Mark, email me your address via the Contact me link top right on this page

    For me, the real loss if we dismiss the possible impact of science on TEFL is that most of the most important scientific theories are counter-intuitive, but most teacher training, e.g. the Cambridge DELTA, seems to be asking you to use your common sense (or other writers’ judgements that are based on their own common sense) as the deciding factor

  19. Mark Bain says:

    Have done so.
    Agree fully. If our common sense is such an effective guide, why would so many people buy lottery tickets? Or smoke? There are countless examples of situations where “common sense” doesn’t lead to the right answer.
    Take the famous “Monty Hall” paradox. I know the right answer but, try as I might, I can’t “feel” that it’s right. I’ve just had a look at Wikipedia, and, apparently, when the problem and solution were published in Parade magazine, 10,000 people wrote to complain that the solution was wrong! I would have been one of them. That’s why I look to science to help me out with my reasoning.

  20. Mark Bain says:

    For anyone who, when presented with two contradictory scientific claims, thinks “I don’t know what to think”, here’s a useful guide to good scientific research at Sense About Science (Registered Charity No. 1101114):

  21. Mark Bain says:

    I’m not in the habit of agreeing with Howard Gardner, but I’ll make an exception:
    “In the middle of the nineteenth century, a serious proposal was made to close the U.S. Patent office because all inventions of significance had been made. In light of the subsequent appearance of the telegraph, telephone, radio, television, airplanes, and computers, we now laugh at the naiveté of this proposal. A few years ago, an American journalist named John Horgan wrote a book entitled The End of Science (Horgan 1996). In this book, he speculated that the important questions about the nature of matter and life had been answered, and that most remaining questions about nature and mind were not susceptible to scientific answer. A century from now, the suggestion that science was at an end in the 1990s will seem equally ill informed.”
    Source: H Gardner, “The Ethical Responsibilities of Scientists”, http://www.howardgardner.com/Papers/documents/Eth%20Resp%20of%20Sci_Feb-02_HG%20pdf.pdf

  22. Mark Bain says:

    Quick pub quiz question:

    Which idiot recently said, “I don’t care… (w)hether it’s magic or scientific, what I care about is it detects bombs.”

    My recent blog post on “Why evidence matters”: http://markcbain.com/?p=501

  23. Diarmuid says:

    I suggest we probably DO agree more than disagree. My real beef is with those who would claim that if the science says it’s right, then that’s the way it gets done. I don’t think that such a thing can be said of teaching or learning. My views on observations are that they allow people to make up their own minds – and I prefer this localised deduction process to the more global deduction process that science might typically lead us down.

    Learning styles? Like you, I splutter when I hear grown ups talking about them – but some people took to them and used them and swore by them and got their students to swear by them and -quite possibly- became better teachers because of them. Goldacre would be fascinated by the placebo effect of these. And rightly so. My suspicion is that scientific enquiry also can only ever have a placebo effect in the language classroom.

    I don’t want to misrepresent myself. Despite having been a particularly bad science student, I am fascinated by the idea that there is a discipline that proposes a methodology by which we can uncover the workings of forces far far greater than we could ever hope to be. And there is a lot of science that is beautifully poetic. I have really reached the point where I can think back and think, “If only I’d paid attention…” I am also aware that people would have once thought it impossible to walk on the moon, unravel the secrets of DNA etc. So, I am not dismissive of science.

    But, for the time being, I am attracted by the complex systems metaphor. It captures the difficulty of detailing language acquisition nicely for me. We may be able to gaze and uncover the secrets of a particular aspect of language acquisition, but we have no way of knowing what effects this particular knowledge might have on any given class in any given classroom.

    btw Abject failure: yes, a certain amount of hyperbole was there. I wasn’t advocating the treatment of students as labrats; I intended to convey the message that I think it is important for teachers to experiment (as in ‘try new things’) and not worry about complete failure. Scientific certainty makes such experimentation more difficult in the classroom. I have had lessons that bombed’ rather than beat myself up about it and label myself ‘a bad teacher’, I think I benefitted from the insight that 100% success was not a requirement and that sometimes things would go very badly. Of course, it is also true that scientific methods might be the basis for post-disaster analysis…so maybe we ARE singing from the same songsheet.

    Will definitely tootle off to your blog now. My resistance to science in teaching is also part of the socratic dialogue (wherein I am the idiot in the marketplace). I suspect that I can learn a lot from arguing with you!

  24. Alex Case says:

    Here is my objection (as I remember the book) in more detail.

    There is only one way to know if something is really a complex system (in the technical meaning of the word), and that is to create a mathematical formula that reproduces the results of that system but does not help you predict its future actions due to a tiny change in the starting variables making a huge difference in the final result (metaphor completely unrelated to real maths- the butterfly’s wing in Los Angeles causing a leather mini skirt in Newcastle on a Friday night to show even more of what you don’t want to see due to a gust of wind). If you cannot successfully do that mathmatical modelling, you cannot say if something is a complex system. Whatsmore, you can only learn something about a complex system by creating the right mathmatical formula.

    In conclusion, complex systems as a metaphor have no more basis in science than seeing your life as trying to cross a stream with strong currents- both nice metaphors, though.

  25. Diarmuid says:

    Have I got news for you! Faced with ANY definition that challenges MY understanding of the world, I turn to the Undergraduate’s Friend, Wikipedia. And there I learn that

    “In the study of complex systems that are less usefully represented with equations various other kinds of narratives and methods for identifying, boundaries, exploring, designing and interacting with complex systems are used…A consensus regarding a single universal definition of complex system does not yet exist.”

    As for “you can only learn sth about a complex system by creating the right mathematical formula”; I throw back at you, “[Hayek] believed that economics and the sciences of complex phenomena in general, which in his view included biology, psychology, and so on, could not be modeled after the sciences that deal with essentially simple phenomena like physics…[he] would notably explain that complex phenomena, through modeling, can only allow pattern predictions, compared with the precise predictions that can be made out of non-complex phenomena.”

    I suspect that you are not going to take up the Larsen-Freeman/Cameron cause again. But in recompense, I suggest that you throw my way any other books that you are sent to review that make you incensed.

  26. Alex Case says:

    Quite enjoy being incensed by books, better than being bored.

    Reading between the lines, it sounds like lots of my fellow physicists share my view, and that is why these economists and what have you have decided to try and refute it. Unless the science has changed since I graduated, the results of a complex system and the results of a non-complex system plus lots of random noise or lots of non-complex systems acting on each other appear basically the same. The way you tell the difference is through mathematical modelling. There may be things that in other fields are called complex systems that do not involve maths, but those things can have no more than a metaphorical relationship to the original mathematical concepts and therefore those mathematical concepts cannot be used to back up the validity of those other things. The way I remember the book, they used every thing called complex systems they could get their hands on to back up their arguments, then tried to take ammunition away from people like me by claiming it was mainly metaphor. So why even mention the science, unless you were trying to live off its glory without needing to be testable under scientific criteria?? The only other reason I can think of is padding to make it book length…

  27. Mark Bain says:

    There, then, is a valid target for wrath: Those who make scientific claims, but, when their methods are questioned, they claim that their particular field isn’t subject to the same rules.

  28. Diarmuid says:

    Possibly…although I think there are many more deserving objects for wrath. But I don’t think either author of the book in mention made any scientific claims as such.

  29. Diarmuid says:

    Incidentally, I started reading through the book again last night and the amount of space it devotes to the need for computer modelling in SLA would seem to belie some of Alex’s comments.

    And are you really saying that another field cannot take a concept that has been established in something like physics and make modifications to it so that the insights it offers can be of use to something like, say, anthropology? Do anthropologists REALLY have to subscribe to the same methodology as physicists and submit themselves to the same rules? It sounds somewhat misguided to me!

  30. Mark Bain says:

    I have enough wrath to go around. In fact, my wife says I’m turning into a prematurely grumpy old man. The other day I got mad – really mad – at a poster for a charity (it was plastered over every available surface in the center of town, including businesses, churches, and private homes). The day before it was an advert for pizza (racist).

  31. Alex Case says:

    I am indeed not really saying that. Not only can anthropology learn a lot from physics, but physics can and has learnt a lot from hindu mythology. However, as the relationship is one of metaphor, proof of one “branch” can never be used as evidence to back up the other. As far as I remember (the book is either one filling up my in laws house in Japan or has been nicked by another teacher), the book goes to great pains to say that the relationship here is mainly metaphorical. Even having gone through the motions of dropping in those typical Academic English caveats, however, there is a danger that such a book could try to gain credibility by living off the reflected glory of a basically unconnected science. I believe that I remember that I believed that they stepped well over that line.

    To put it another way, if anyone invented Quantum Anthropology, I hardly think they would dedicate the first few chapters of their book to Quantum Physics just because it was that that set off their chain of thought. I still think the book would have made a great ETP article, in the “inspired by…” style that many great articles over the years have had.

  32. Sara Hannam says:

    I so want to get involved, but I am afraid : ) Hee hee (last time you and I discussed stuff of this nature it went on and on as it is doing now so this is obviously a hot topic). I think maybe there is a tendency to separate the branches too much and not see the whole tree. Knowledge is falsely divided by academia after all and this puts us in the position of arguing whichever corner of it we happen to know most about and not always seeing the panoramic possibilities it has to offer? But knowledge itself is a totality and each part influences the other and overlaps. Now I haven’t read that book (so perhaps that disqualifies my opinion).

  33. Mark Bain says:

    Hi Sarah
    I remember the discussion well – MI theory, wasn’t it? Great stuff. I’ve just had another look. I made a few comments, but not as many as I would have liked, because there was just too much to read!

  34. I feel you’re being a bit harsh when you say, “and that seems to sum up Cambridge’s attitude also- including to the whole concept of outdated and up to date sources that would be important in virtually any other field”.

    What you were reacting against was your tutor’s uncritical regurgitation of uncritical methodology. Good for you! I think we’ve all come up against a bit of that in our time. However, I don’t think it’s fair to describe this as Cambridge’s attitude.

    Here is some evidence that critical reflection on practice and reference to evidence-based research are highly-valued in the Cambridge Delta scheme.

    1) Module 2 requires teachers to implement and reflect on an action-research project based on their own teaching context

    2) Critical engagement with theory is written into the Delta syllabus. For instance, to fulfil Unit1 of the syllabus for module 1, candidates need to show that they can “identify, explain, compare and evaluate theories of first and second language acquisition” and “relate the influence of such theories to specified approaches and methodologies”. For Unit2, they have to be able to discuss current developments in ELT (ie not those of the 1970’s!) and critically discuss assess and evaluate the practical effectiveness of methodologies in particular contexts. And so it goes on.

    3) Examiners’ reports show that this is taken seriously in assessment. The June 2009 reports on module 2 and 3, for instance, strongly emphasise the need for candidates to refer critically to relevant sources. For instance, the module 3 report highlights this lack of engagement with relevant sources as one of the major issues for centres and candidates to address: “Weaker assignments failed to refer explicitly to the literature on needs analysis, diagnostic testing, course design, types of syllabus, principles of assessment and types of assessment. Some candidates failed to consider the theory, while others mentioned it without direct reference to key sources. Some candidates summarised the views of an author but failed to include any criticality and did not indicate whether they agreed or disagreed with the views expressed.”

    4) Critical engagement with relevant, current sources is clearly associated with higher grades in my own experience. I am part of the marking team for Delta module 3, which is based on a 4500 report on a specialised area and programme within ELT. I also work as an external moderator for Delta module 2 – which includes observed lessons backed up by 2500 word reports on the area of language or skills and approaches, materials that are available for teachers in this area. I can confirm that what’s in the syllabus and assessment criteria really does matter. Uncritical fools are not suffered gladly.

    5) Not all centres take such a lame approach to current literature as Alex reported above. The centre where I worked on Delta courses in 2009 already had a pretty impressive library of ELT books – but was pretty willing to buy any additional books which we recommended by ploughing methodically through the 2009 catalogues of all the major UK and US publishers selecting recent books on key areas. The centre also compiled several folders of research articles to which they subscribed. In addition, the candidates also had access to all the physical and digital resources of the local university library, including interlibrary loans from overseas, where required. Our handouts for input sessions on major areas such as discourse, grammar teaching etc included fairly substantial annotated bibliographies. On the topic of learning styles, for instance, rather than just tread out the well-worn VAK model, we also referred students to the comprehensive review on this topic carried out for the UK Ministry of Education a few years ago (Coffield et al., 2004), which finds scant evidence for the concept of learning styles overall and is particularly scathing about the VAK model. I don’t know enough about other centres to say how typical this is. What I do know is that the approach we followed in this centre does align very well with the points I made above regarding the syllabus and assessment – the centre had a 100% pass rate overall in 2009, with every one of the candidates for module 3 (where research is particularly relevant) gaining either a merit or distinction.

  35. Alex Case says:

    Surely being harsh is what blogs are for??

    More seriously good points, but I wonder how many times you could say “unfortunately, this quote wasn’t based on any research either” in one essay?

  36. Diarmuid says:

    Hmm…I think your criticism is ironically unscientific. You are really criticising your memory of the bok as opposed to the book itself. And Mark seems to be placing a great deal of credence in what you have to say. Not what I would expect from two esteemed boffins. And there is a lot of subjectivity in your interpretation that the writers “went to great pains to say”. If we let the scientific discourse analysts go to work on your post above, then we would undoubtedly have to mourn the passing of scientific objectivity.

    btw if I was planning to read a book about quantum anthropology and I was just a humble little anthropologist, I think I WOULD expect some kind of introduction to the theories that lay behind the book.

  37. Alex Case says:

    What can I say? Would pick up the book again if I could, but the laws of physics prevent me! And there’s always my favourite defence- it’s just a blog

  38. Diarmuid says:

    The laws of physics are pretty immutable. So perhaps it’s time to move on. I like the defence, “It’s just a blog/a book,” but suspect that it is a defence that LF/Cameron would be better employing against the Wrath of Mark and the Incense (?) of Alex.

    So, we’re agreed then? Language learning is very similar to a complex system and the calls for a scientific approach to teaching are absolute ballix. There you go, with a bit of discussion, everything becomes clearer.

  39. Mark Bain says:

    I fear I haven’t made myself absolutely clear, which is often the case.
    While it seems that Alex and I agree on certain things – the importance of evidence-based practice – I cannot concur with his assessment of the LF/Cameron book, as I haven’t read it.
    Having hear what he Alex has to say, I won’t deny I have misgivings about the book, and I suspect that, if I ever get the time and opportunity to read it, I may well form a similar opinion. Those of a scientific bent sometimes do form a similar opinion, and not because they are members of a worldwide conspiracy.
    I certainly shan’t start saying that the book is a load of BS because Mr Alexander Case says so.
    And, although I risk being labelled a Case fanboy, I think the “it’s only a blog defence” is a important one. Imagine every post had to be a fully-researched academic paper! Blogging would grind to a halt. A book, clearly, is something else entirely.

  40. Alex Case says:

    Actually, I wasn’t calling for a more scientific approach to teaching- what would a scientific book review look like?? Hopefully not like this:


    I’m just thinking aloud about the situation we are in where trusting the minimal science we have or rejecting it out of hand both seem inadequate- and living off the glory of only vaguely related scientific theories (if that is indeed what they were doing) seems worst of all. Hence the quotes I chose for my latest post. In answer to mine and Sara’s long disagreement which I don’t think was so much so (http://sixthings.net/2009/10/16/six-things-about-multiple-intelligences-that-you-might-not-know/) I have great respect for the methodologies of good anthropology etc, and that is precisely why they should not try to live off the reflected “glory” of pure sciences or be tempted into borrowing their often unsuitable research methods. Sara seemed to agree with me on that, but think that sociology etc would lose their penis envy for Physics if they were also defined as science. As a scientist by background, I suppose I should be happy to swallow as many university departments as we can, but I actually think it would have the opposite effect. Sara refused to define science, but her definition of good research seemed to include literary criticism. Does anyone think that is a science??

    As ELTers, we live in a grey area where “scientific” methodologies (meaning here ones similar to those of the pure sciences) could be useful in some things and totally unsuitable in others, such is also the case in economics and maybe psychology.

    Not sure anything ever becomes clearer with a bit of conversation, let alone a bit of blogging and let alone with a topic like this! If we can identify where the cloudy bits are perhaps we’ve achieved something. Alternatively, if we’ve kept ourselves entertained we most certainly have done

  41. Sara Hannam says:

    Alex your last contribution made me revisit our dialogue – the whole of the 6 things post was fascinating. The thing is that I didn’t refuse to define science (ahhhhhhh – hollow screaming felt as it reverberates around space!). I simply asked if it was OK not to define it within your terms of reference. So you are right, we don’t disagree per se, but we are coming from really different positions and sort of missing each other in the central meaning section of the conversation. You seem to be saying that anthropology, sociology etc would be better served if they just accepted they were not sciences, whereas I say is that we should deconstruct the power base of academia which prioritises so called ‘scientific’ understanding of the world along narrow (and I would argue often conservative) lines. I believe the latter to be true for all disciplines. So for me the problem does not lie in the word, but the way the disciplines are understood, which are represented by the word ‘science’ in our understanding. I still see this as a disagreement that you and I have. Here is how I defined science in the 6 things post:

    BTW, what do you mean by ‘literary criticism’ – please define.

    If its not OK to quote something I said which did define science in a more encompassing way on 6 things here – take it down.

    “There are two issues here – ’science’ as a term and what it signifies, and the way science is represented institutionally in society. I think “science” (as a term) should include all the variations of different styles of research, both pure and otherwise (and I don’t like the way it is currently categorised TBH as I am more than happy to accept both), which is what I was trying to argue from the start. It is all science because it all aims to answer a particular question using a rigorous framework of analysis – whether quantitative (as you describe) or qualitative (as I am involved in). I don’t really get why it is so hard for you to accept social science as “science” – we are obviously dealing with different concepts right? In relation to science as an insitution, as you point out, it has become intertwined with political and academic power to such an extent, that its uses, results, funding etc are tied up completely with those power structures. I said before its about the institutions, not the individuals (who you rightly say are often powerless). I do not agree with you that research has to fulfil the premise you outline in physics I am afraid, as I think human enquiry of all kinds should be represented in a working definition of science, repeatable or not (in psychology a lot of claims made about repeatabilty are spurios anyway). None of this would matter if research was carried out with the aim of finding the best means to answer the question rather than being a scrabble to prove allegiance to a narrow definition of “science” because it is considered superior”

  42. Sara Hannam says:

    To be clear – I did refuse to define “good” and “bad” science as this implies a value judgment. Sorry I hit send before the answer was finished : )

  43. Alex Case says:

    Hi Sara

    Am always happy to revisit this very interesting conversation…

    First of all, I am using the word “science” as it is popularly understood, and how it is understood by most pure scientists, or at least was when I studied the philosophy of science during my degree (coincidentally, chaos theory and phil of sci were the only things I took any interest in in three years of Physics, so the amount I remember about complex systems is probably more than my knowledge of the applied linguistics part of the book). To revisit an example I tried to use on Sixthings but didn’t really come across, if you tell someone at someone at a house party you are a scientist and they ask you which kind to which you answer “Anthropologist”, Joe or Joeanna Bloggs is going to produce a thought bubble saying “Is anthropology a kind of science?” If you tell them that you are a physicist (if they haven’t already guessed from your heavy metal T shirt and greasy hair), they will instantly find someone else to talk to at the party, but you won’t have the thought bubble problem.

    As my knowledge of the world outside TEFL is rapidly shrinking, there is still the chance that people’s definition of science has moved on since my time. If not, you basically seem to be saying that the meaning “should” be different. I don’t think you’ve entirely made that case. For one thing, I still ask you if there is any difference between your definitions of “science” and “research”.

  44. Sara Hannam says:


    Thanks for your response. LOL at your party description (I hear your pain as a physics student!) but as you well know having sprung from those ranks yourself, these are just stereotypes. I mean look at you 🙂 If anything its another illustration of the power of the word ‘science’ to conjour up images right down to the appearance of those interested in it. It is something we relate to cos its the way its been presented to us.

    I really struggle with the idea of trying to second guess what the “general populace” thinks about anything (even humorously) as my own research background has taught me that people are nothing if not complex, full of contradictions and ever changing. None of us ever recognises our own contradictions which seems to be part of the human condition, so for me seeing how any one issue is perceived is a way of understanding the possible range of views on any given topic. So a uniform response is very unusual when you dig a bit deeper. Were I at your party I would want to ask a few more questions and find out what science means to people. Yes that would make me very popular I am sure and perhaps I might only have an audience of your fictional physicists, or maybe not, if people are asked what they think rather than assuming their responses. Most people if given the chance and made to feel that the person they are talking to is interested in that response, do have a lot to say on a lot of things.

    I imagine (if we are talking about a momentary thought about what science is) that there would be a range of responses. Some would see white coats and nuclear scientists in their mind’s eye, others would feel and smell the memory of the test tubes at school, others would imagine brainy boffins, others people on the news with lots of academic titles and views on important matters, from all parts of the academy. The sociologist asked about football violence would still be seen as a scientist by some. But what unites all of this – the creation, extension, circulation and critique of knowledge……research in other words. I also think there are some who would see anthropology as a science – because it lives in the academy where all knowledge is protected and removed from general consumption, if only by virtue of the language used to describe it, and the assumption of preknowledge when it is discussed. Many outside the university setting would perhaps feel this ‘distance’ and lack of inclusion.

    If people didn’t imagine anthropology as a science, it could also be because physics is a much longer established ‘discipline’ whereas anthropology is relatively recent . Ethnography (main research approach in anthropology) as a method emerged around the time of colonisation and was used for very dubious purposes of documenting the ‘natives’ and their behaviour – all the better to dominate them. They were studied like specimens in the very test tubes of the scientific laboratory that exist in people’s imaginations. The critical form of anthropology (which is the one we are alluding to) is a much more recent development and ironically is on the downcrease – in many universities it is a dept likely to close due to lack of income generation, apparently it has no future job prospects and it is no longer relevant to study people and their culture in the way it once was. The answers have all been found as the world is now globalised. Paradoxically critical ethnography is needed now more than ever. These issues have a major effect on how aware people might be of a particular field. Additionally, it has never been given the coverage in public discourse that physics has which is taken for granted as a complete and unquestioned form of knowledge. So perhaps people thinking anthropology is not a science is a subliminal awareness of its own departure from the natural sciences as a means of distancing itself from its colonial inheritence?? They may not see it in those terms, but the ‘idea’ exists in society never the less.

    So, you asked me are my definitions of research and science the same? When it is seen as knowledge that is at stake rather than academic discipline then I suppose yes they are. I wonder what you think about that? Is there something else important that I am missing which causes you to separate them so?

    Would still like to know what you mean by literary criticism????

  45. Alex Case says:

    No idea if I got the right term but what I meant, and you seem to be confirming, is that someone doing properly sourced, referenced, evaluated and whathaveyou research on full stops in the works of Franz Kafka would be classified as a scientist under your system. Ditto historians. Oh, and art historians. Where exactly would science end?? Would future universities consist of just “Science”, “Fine arts” and “Drama”? As I believe I have stated several times before, if “science” means exactly the same as “research”, it loses all meaning because we already have a perfectly good name for research. Can anyone guess what word beginning with R and ending with CH I am thinking of?

  46. Sara Hannam says:

    No Alex, I think the ideal universities would be interdisciplinary. I know its hard to imagine but I would hope if they ever do change in the way I would like, those divisions you feel are so necessary might cease to be relevant and we wouldn’t feel so bothered by where the boundaries would be drawn. We would see knowledge as a more total entity perhaps? But that is a long way into a possibly never to be fufilled future – its important to keep it in mind so as not to get too lost in the false divisions of knowledge today.

    So within the existing way things are organised, both the examples you give are furthering knowledge of one kind or another, providing they are being done rigorously (and I would argue critically). All research has to be assessed on its own merits. Under the existing system which unfortunately we have to live with, rather than excluding areas that you have termed literary criticism (which might include comparitive literature or history for example) as less important, they need to be accepted as equal but different ways of carrying out research to further knowledge. Comparitive literature is very compatible with other social sciences as it provides further understanding of different ways of looking at and transforming the world around us. The intrinsic value of each type of approach needs to be valued.

    In order for that to happen, ‘science’ as a term needs to be deconstructed from its ivory tower. Humantities are currently being decimated across the academy for reasons that are linked to whether they are relevant with much the same reasoning and logic that you express above. I would be very guarded about sharing the same platform or educational agenda as those who seek to remove humanities from the curriculum unless being clear about the reasons for doing so.

    I don’t really mind if the word remains TBH as the word itself is not the problem for me, its the assumption and force behind the word that I am questioning.

    Are we any closer?

  47. Alex Case says:

    ‘Fraid not. For one thing, because you are reading things into my use of the word science that are not there. To start with, well over half of the non TEFL books on the bookshelf next to me now are history books, so I hardly think you can say I am saying it is less important because it is not science. My objection is and remains purely linguistic. First of all, no one uses the word science to include anything to do with literature, therefore your definition of science is simply wrong. Whether such a definition would be nice is entirely another matter. I also disagree on that- when one word is changed to mean exactly the same as another (e.g. my bugbear confusions of “blog” and “blog post”, and “blog” and “comment”), the word loses all value and communication becomes more difficult.

    We do agree that many people have a respect for science that makes them trust men in white coats far more than they should, but putting historians in white coats (metaphorically!) is hardly the answer. To start with, you’d probably find that people still had the same attitude to the words “physics” or maybe “particle accelerator”. Maybe we should then change the meaning of physics to include all knowledge and particle accelerator to include all sources of it??

    There is a big difference between deconstructing a word and choosing your own definition of it. Are you perhaps suggesting that the next edition of the Oxford Advanced Learners’ Dictionary has definitions based on how much those uses of the words are going to make the world a better place rather than on how people use them?

  48. Sara Hannam says:

    Sorry Alex. I didn’t intend to read anything that wasn’t there into your posting. I guess that is always a risk when coming at a subject from different angles and exploring the vast idea that is ‘science’. I know that you place great value on socio-historical sources too as we have often discussed them. At the same time, I am not trying to reinvent the dictionary as you suggest (though its true I do see dictionaries as rather limiting our understanding of concepts!).

    I thought about this a lot last night to try and pin down where the different takes on this come from as we seem to be finding it hard to find a common ground closer to the centre of the discussion, though we do agree about other sub-issues. I think it might be how we see the linguistic meaning of it all. As you say above “My objection is and remains purely linguistic” whereas I say in a previous comment “I don’t really mind if the word remains TBH as the word itself is not the problem for me, its the assumption and force behind the word that I am questioning”. So I thought that might be a fresh place to start unravelling the issue.

    It seems that what you want is to pin down a coherent definition of ‘science’ and ‘research’ so as to work out how to fit other areas that have not traditionally been thought of as ‘science’ into your understanding. Is that fair? To think more about how ELT engages with scienctific understandings and makes claims about various issues in the field. I am also interested in this but what I am also doing, on the other hand, is saying that it will not be possible to find a coherent definition of science, because concepts like that are too wide to pin down, as they tend to leak into the words that surround them like ‘research’ or ‘academic’ or ‘validity’ or whatever. I don’t think that any of these terms are discrete and complete on their own as they carry with them a sort of signification chain which is not self-evident or complete (I am nicking this idea from Derrida by the way, who was influenced by Saussure, and there is plenty to google which explains it simply and quickly). So perhaps our point of departure from one another is the belief in finite and coherent definitions?

    This basic difference may seem small, but infact it influences the flow of the discussion. For me, I am much more interested in the issues that lie behind the term, such as the rise of the power of the academy and the use of ‘science’ as a way of discrediting other disciplines and approaches. At a deeper level I am also in favour of multi-disciplinarity and think the false divides between subjects in the academy is harmful for the greater good so naturally am seeking to increase the category to incorporate a more holistic view of knowledge. Hence, the terms themsleves are simply representations of these issues for me. I know that you agree with some of the above and have seen it first hand in your days as a physics student. However, I guess where we are different is that you still want to try and define ‘science’ and its relationship to other disciplines using those same categories. I am saying those categories are too steeped in the academic structures, and too limiting in their conceptual understanding of knowledge, to be up for the job. Does that make sense?

    Now that is the ideal, but we are still living the world where the term ‘science’ dominates so we have to use it. Where does that leave anyone who is in a discipline which operates outside the traditional category or ‘science’ as it is currently understood? The here and now part of the discussion. Well……basically those who work in areas such as social science or the humanities are increasingly under pressure to justify how their work measures up within the quantiative paradigm. Not just in terms of numbers of publications, but approaches and applications to industry and commerce. They are ‘forced’ if you like to apply their discipline to a structure that whilst not completely reflective of that which is contained in ‘science’ is nevertheless influenced by it. That is why I said that it is hard to discuss this without falling into the same sort of dialogue as that being had within the academy about the demise of areas like anthropology or history and their usefulness to ‘knowledge’. “We” have no choice but to point out how our disciplines contribute to knowledge. Does that make sense.

    I did find a definition that is closer to how I see things in bit of a morning web trawl (ref at end):

    “The word science comes from the Latin “scientia,” meaning knowledge. How do we define science? According to Webster’s New Collegiate Dictionary, the definition of science is “knowledge attained through study or practice,” or “knowledge covering general truths of the operation of general laws, esp. as obtained and tested through scientific method [and] concerned with the physical world.” What does that really mean? Science refers to a system of acquiring knowledge. This system uses observation and experimentation to describe and explain natural phenomena. The term science also refers to the organized body of knowledge people have gained using that system. Less formally, the word science often describes any systematic field of study or the knowledge gained from it.

    What is the purpose of science? Perhaps the most general description is that the purpose of science is to produce useful models of reality. Most scientific investigations use some form of the scientific method. Science as defined above is sometimes called pure science to differentiate it from applied science, which is the application of research to human needs. Fields of science are commonly classified along two major lines:
    – Natural sciences, the study of the natural world, and
    – Social sciences, the systematic study of human behavior and society.


    Now for me, the definition here of social science should also include comparitive literature which is a reflection of human behaviour. Maybe not everyone would agree with me, but that brings us back to the discussion on the very nature of knowledge itself and how human beings have sought to divide things up to make them digestable? There is also perhaps a difference between how terms are used formally and informally?

    Does this help at all in our discussion do you think?

    Cheers for making me think this through.

  49. Alex Case says:

    Thanks Sara. I really thought there was nothing more to say on the subject, but that is quite thought provoking and I think I have some more questions for you in revenge. Really must refuse to continue this discussion for now so that I don’t distract you from your interview though- which will luckily give me some time to think of more dinner party metaphors

  50. Sara Hannam says:

    OK Alex. Happy to continue whenever you have time. Questions always welcome!

  51. Sara Hannam says:

    I thought this might add another dimension to the discussion too. An article from the Guardian about cuts in universities in the UK with some interesting staff about maths/science -v- humanities http://www.guardian.co.uk/education/2010/feb/07/job-losses-universities-cuts

  52. Alex Case says:

    Oh boy, when it comes to budgets I’m glad to elect some bonehead to throw a dart at a dartboard in place of me, because in present circumstances the question is what would you cut? Not sure losing any artificial distinctions between sciences and humanities would help, because scientists have exactly the same complaints about cutting “pure” research to fund things that are likely to have profitable spin offs.

    What interests me more about defining science, and I would argue would have more long term effects than arguments about budgets, is the ability of a good philosophy of science to allow people to dismiss creationists, new age mystics, astrologers etc as in no way doing the same thing as evolutionists, medical researchers and astronomers. Ditto for the bad science and often fraud of actual scientists behind claims of cloning humans, perpetual motion machines etc, and ditto for social scientists who have used the “scientific method” (mentioned in your quote) when they really should’ve been using another methodology entirely.

  53. ebefl says:


    i started a blog about evidence in efl. check it out if you like.


  54. Russ mayne says:

    Great post Alex!

Leave a comment (link optional and email never shared)

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.