Pluralism vs. Monoculture in Scholarly Communication, Part 2

Calls for a monoculture of scholarly communication keep multiplying. But wouldn’t a continued diversity of models be healthier?

The post Pluralism vs. Monoculture in Scholarly Communication, Part 2 appeared first on The Scholarly Kitchen.

How One Society Is Supporting Peer Review Excellence in their Community: An Interview with Laura Feetham of IOP Publishing

In today’s post, Alice Meadows talks to Laura Feetham of IOP Publishing about their work to improve peer review quality in the physical sciences through their ongoing peer review excellence program.

The post How One Society Is Supporting Peer Review Excellence in their Community: An Interview with Laura Feetham of IOP Publishing appeared first on The Scholarly Kitchen.

Guest Post — Building the Last Mile: A Plan for Bringing the Expanding Universe of Digital Primary Sources into Classrooms

Getting digitized primary source materials into the classroom requires an open dialogue among researchers, teachers, and archivists. A workshop from historians of business shows how.

The post Guest Post — Building the Last Mile: A Plan for Bringing the Expanding Universe of Digital Primary Sources into Classrooms appeared first on The Scholarly Kitchen.

Do You Really Want to Test a Hypothesis?

I’ve uploaded one of my favorite lectures in the my new MOOC “Improving Your Statistical Questions” to YouTube. It asks the question whether you really want to test a hypothesis. A hypothesis is a very specific tool to answer a very specific question. I like hypothesis tests, because in experimental psychology it is common to perform lines of research where you can design a bunch of studies that test simple predictions about the presence or absence of differences on some measure. I think they have a role to play in science. I also think hypothesis testing is widely overused. As we are starting to do hypothesis tests better (e.g., by preregisteringour predictions and controlling our error rates in more severe tests) I predict many people will start to feel a bit squeamish as they become aware that doing hypothesis tests as they were originally designed to be used isn’t really want they want in their research. One of the often overlooked gains in teaching people how to do something well, is that they finally realize that they actually don’t want to do it.
The lecture “Do You Really Want to Test a Hypothesis” aims to explain which question a hypothesis tests asks, and discusses when a hypothesis tests answers a question you are interested in. It is very easy to say what not to do, or to point out what is wrong with statistical tools. Statistical tools are very limited, even under ideal circumstances. It’s more difficult to say what you can do. If you follow my work, you know that this latter question is what I spend my time on. Instead of telling you optional stopping can’t be done because it is p-hacking, I explain how you can do it correctly through sequential analysis. Instead of telling you it is wrong to conclude the absence of an effect from p > 0.05, I explain how to use equivalence testing­­. Instead of telling you p-values are the devil, I explain how they answer a question you might be interested in when used well. Instead of saying preregistration is redundant, I explain from which philosophy of science preregistration has value. And instead of saying we should abandon hypothesis tests, I try to explain in this video how to use them wisely. This is all part of my ongoing #JustifyEverything educational tour. I think it is a reasonable expectation that researchers should be able to answer at least a simple ‘why’ question if you ask why they use a specific tool, or use a tool in a specific manner.
This might help to move beyond the simplistic discussion I often see about these topics. If you ask me if I prefer frequentist of Bayesian statistics, or confirmatory or exploratory research, I am most likely to respond ? (see Wikipedia). It is tempting to think about these topics in a polarized either-or mindset – but then you would miss asking the real questions. When would any approach give you meaningful insights? Just as not every hypothesis test is an answer to a meaningful question, so will not every exploratory study provide interesting insights. The most important question to ask yourself when you plan a study is ‘when will the tools you use lead to interesting insights’? In the second week of my MOOC I discuss when effects in hypothesis tests could be deemed meaningful, but the same question applies to exploratory or descriptive research. Not all exploration is interesting, and we don’t want to simply describe every property of the world. Again, it is easy to dismiss any approach to knowledge generation, but it is so much more interesting to think about which tools willlead to interesting insights. And above all, realize that in most research lines, researchers will have a diverse set of questions that they want to answer given practical limitations, and they will need to rely on a diverse set of tools, limitations and all.
In this lecture I try to explain what the three limitations are of hypothesis tests, and the very specific question they try to answer. If you like to think about how to improve your statistical questions, you might be interested in enrolling in my free MOOC Improving Your Statistical Questions”.

Improving Education about P-values

A recent paper in AMPPS points out that many textbooks for introduction to psychology courses incorrectly explain p-values. There are dozens, if not hundreds, of papers that point out problems in how people understand p-values. If we don’t do anything about it, there will be dozens of articles like this in the next decades as well. So let’s do something about it.
When I made my first MOOC three years ago I spent some time thinking about how to explain what a p-value is clearly (you can see my video here). Some years later I realized that if you want to prevent misunderstandings of p-values, you should also explicitly train people about what p-values are not. Now, I think that training away misconceptions is just as important as explaining the correct interpretation of a p-value. Based on a blog post I made a new assignment for my MOOC. In the last year Arianne Herrera-Bennett (@ariannechb) performed an A/B test in my MOOC ‘Improving Your Statistical Inferences’. Half of the learners received this new assignment, explicitly aimed at training away misconceptions. The results are in her PhD thesis that she will defend on the 27th of September, 2019, but one of the main conclusions in the study is that it is possible to substantially reduce common misconceptions about p-values by educating people about them. This is a hopeful message.
I tried to keep the assignment as short as possible, and therefore it is 20 pages. Let that sink in for a moment. How much space does education about p-values take up in your study material? How much space would you need to prevent misunderstandings? And how often would you need to repeat the same material across the years? If we honestly believe misunderstanding of p-values are a problem, then why don’t we educate people well enough to prevent misunderstandings? The fact that people do not understand p-values is not their mistake – it is ours.
In my own MOOC I needed 7 pages to explain what p-value distributions look like, how they are a function of power, why p-values are uniformly distributed when the null is true, and what Lindley’s paradox is. But when I tried to clearly explain common misconceptions, I needed a lot more words. Before you want to blame that poor p-value, let me tell you that I strongly believe the problem of misconceptions is not limited to p-values: Probability is just not intuitive. It might always take more time to explain ways you can misunderstand something, than to teach the correct way to understand something.
In a recent pre-print I wrote on p-values, I reflect on the bad job we have been doing at teaching others about p-values. I write:
If anyone seriously believes the misunderstanding of p-values lies at the heart of reproducibility issues in science, why are we not investing more effort to make sure misunderstandings of p-values are resolved before young scholars perform their first research project? Although I am sympathetic to statisticians who think all the information researchers need to educate themselves on this topic is already available, as an experimental psychologist who works at a Human-Technology Interaction department this reminds me too much of the engineer who argues all the information to understand the copy machine is available in the user manual. In essence, the problems we have with how p-values are used is a human factors problem (Tryon, 2001). The challenge is to get researchers to improve the way they work.
Looking at the deluge of papers published in the last half century that point out how researchers have consistently misunderstood p-values, I am left to wonder: Where is the innovative coordinated effort to create world class educational materials that can freely be used in statistical training to prevent such misunderstandings? It is nowadays relatively straightforward to create online apps where people can simulate studies and see the behavior of p-values across studies, which can easily be combined with exercises that fit the knowledge level of bachelor and master students. The second point I want to make in this article is that a dedicated attempt to develop evidence based educational material in a cross-disciplinary team of statisticians, educational scientists, cognitive psychologists, and designers seems worth the effort if we really believe young scholars should understand p-values. I do not think that the effort statisticians have made to complain about p-values is matched with a similar effort to improve the way researchers use p-values and hypothesis tests. We really have not tried hard enough.
So how about we get serious about solving this problem? Let’s get together and make a dent in this decade old problem. Let’s try hard enough.
A good place to start might be to take stock of good ways to educate people about p-values that already exist, and then all together see how we can improve them.
I have uploaded my lecture about p-values to YouTube, and my assignment to train away misconceptions is available online as a Google Doc (the answers and feedback is here).
This is just my current approach to teaching p-values. I am sure there are many other approaches (and it might turn out that watching several videos, each explaining p-values in slightly different ways, is an even better way to educate people than having only one video). If anyone wants to improve this material (or replace it by better material) I am willing to open up my online MOOC for anyone who wants to do an A/B test of any good idea, so you can collect data from hundreds of students each year. I’m more than happy to collect best practices in p-value education – if you have anything you think (or have empirically shown) works well, send it my way – and make it openly available. Educators, pedagogists, statisticians, cognitive psychologists, software engineers, and designers interested in improving educational materials should find a place to come together. I know there are organizations that exist to improve statistics education (but have no good information about what they do, or which one would be best to join given my goals), and if you work for such an organization and are interested in taking p-value education to the next level, I’m more than happy to spread this message in my network and work with you.
If we really consider the misinterpretation of p-values to be one of the more serious problems underlying the lack of replicability of scientific findings, we need to seriously reflect on whether we have done enough to prevent misunderstandings. Treating it as a human factors problem might illuminate ways in which statistics education and statistical software can be improved. Let’s beat swords into ploughshares, and turn papers complaining about how people misunderstand p-values into papers that examine how we can improve education about p-values.

Magnifying Power to the People with the Foldscope

The microscope holds a place on the short list of inventions that have changed the world and revolutionized our understanding of science. Microscopes are crucially important public health tools, allowing workers to identify pathogens and correctly diagnose the cause of illnesses. As educational tools, they can excite and engage students, revealing a world invisible to the naked eye. And, as many people who’d love a microscope but don’t have one can tell you, they are also expensive. Millions of doctors, health workers, and patients worldwide lack the resources to benefit from this vital tool, and millions of students have never seen a microscope before. In a dramatic step to address this problem, researchers from Stanford University have designed ultra-low-cost microscopes built from an inexpensive yet durable material: paper. They recently published their designs and data in PLOS ONE.

Foldscope template

Meet the Foldscope. Borrowing from the time-honored tradition of origami, the Foldscope is a multi-functional microscope that can be assembled much like a paper doll. Users cut the pieces from a pattern of cardstock, fold it according to the printed lines, and add the battery, LED, and lens, and?voilà?a microscope. Foldscope schematicClick here to watch a video of how one is assembled. Some of their coolest features are as follows:

  • Foldscopes are highly adaptable and can be configured for bright-field and dark-field microscopy, to hold multiple lenses, or to illuminate fluorescent stains (with a special LED).

Foldscope Configurations

  • They can be designed for low or high powers and are capable of magnifying an image more than 2,000-fold.
  • They accept standard microscope slides, and the viewer can move the lens back and forth across the slide by pushing or pulling on paper tabs.
  • Users can focus the microscope by pushing or pulling paper tabs that change the lens’ position.
  • Foldscopes are compact and light, especially when compared with conventional field microscopes. They also weigh less than 10 grams each, or about the weight of two nickels.
  • They are difficult to break. You can stomp on them without doing much damage, and they can survive harsh field environments and encounters with children.

Stepping on FoldscopeWhat’s the total cost, you ask? According to authors, it’s less than a dollar.  At that price, it’s easy to imagine widespread use of Foldscopes by many who previously could not afford traditional microscopes. In this TED Talk, Manu Prakash demonstrates the Foldscopes and explains his hopes for them. The authors envision mass producing them and distributing different designs optimized for detecting the pathogens that cause specific diseases, such as Leishmaniasis and E. coli.  They could even include simple instructions for how to treat and prepare slides for specific diagnostic tests or provide pathogen identification guides to help health workers in the field make diagnoses.  This is just one way in which the ability to see tiny things could make a huge difference in the world.

Related links:

Low-Cost Mobile Phone Microscopy with a Reversed Mobile Phone Camera Lens

Community Health Workers and Mobile Technology: A Systematic Review of the Literature

Citation: Cybulski JS, Clements J, Prakash M (2014) Foldscope: Origami-Based Paper Microscope. PLoS ONE 9(6): e98781. doi:10.1371/journal.pone.0098781

Images: Images are from Figures 1 and 2 of the published paper

The post Magnifying Power to the People with the Foldscope appeared first on EveryONE.

The Tyranny of Pi day

pigraphicMarch 14th is \(\pi\)-day in the US (and perhaps \(4.\overline{666}\) day in Europe). The idea of a day devoted to celebrating an important irrational number is wonderful — I’d love to see schools celebrate e-day as well, but February 71st isn’t on the calendar. Unfortunately, March 14th has also become the day in which 4th and 5th graders around the US practice for one of the most pointless exercises imaginable – a competition to recite the largest number of digits of \(\pi\).

Memorization of long digit strings is not an exercise that teaches a love of mathematics (or anything else useful about the natural world).  This is solely an exercise in recall, which is perhaps valuable for remembering phone numbers, but not for understanding transcendental constants. For all practical purposes, only the first few digits of \(\pi\) are really necessary – the first 40 digits of \(\pi\) is enough to compute the circumference of the Milky Way galaxy with an error less than the size of an atomic nucleus.

So, because \(\pi\) is a such an accessible entry to mathematics and science, I thought I’d come up with a list of other cool \(\pi\) things that could replace these pointless memory contests:

  • The earliest written approximations of \(\pi\) are found in Egypt and Babylon, and both are within 1 percent of the true value. In Babylon, a clay tablet dated 1900–1600 BC has a geometrical statement that, by implication, treats \(\pi\) as 25/8 = 3.1250. In Egypt, the Rhind Papyrus, dated around 1650 BC, but copied from a document dated to 1850 BC has a formula for the area of a circle that treats \(\pi = \left(\frac{16}{9}\right)^2 \approx 3.1605\).
  • In 220 BC, Archimedes proved that \( \frac{223}{71} < \pi < \frac{22}{7}\).  The mid-point of these fractions is 3.1418.
  • Around 500 AD, the Chinese mathematician Zu Chongzhi  was using a rational approximation for \(\pi \approx 355/113 = 3.14159292\), which is astonishingly accurate.  For most day-to-day uses of \(\pi\) this particular approximation is still sufficient.
  • By 800 AD, the great Persian mathematician, Al-Khwarizmi, was estimating \(\pi \approx 3.1416\)
  • A good mnemonic for the decimal expansion of \(\pi\) is given by the letter count in the words of the sentences: “How I want a drink, alcoholic of course, after the heavy lectures involving quantum mechanics. All of thy geometry, Herr Planck, is fairly hard…”
  • Georges-Louis Leclerc, The Comte de Buffon came up with one of the first “Monte Carlo” methods for computing the value of \(\pi\) in 1777.  This method involves dropping a short needle of length \(\ell\) onto lined paper where the lines are spaced a distance \(d\) apart.  The probability that the needle crosses one of the lines is given by:  \(P = \frac{2 \ell}{\pi d}\).
  • In 1901, the Italian mathematician Mario Lazzarini attempted to compute \(\pi\) using Buffon’s Needle.  Lazzarini spun around and dropped a 2.5 cm needle 3,408 times on a grid of lines spaced 3 cm apart. He got 1,808 crossings and estimated \(\pi = 3.14159292\). This is a remarkably accurate result!   There is now a fair bit of skepticism about Lazzarini’s result, because his estimate reduces to Zu Chongzhi’s rational approximation.  This controversy is covered in great detail in Mathematics Magazine 67, 83 (1994).
  • Another way to estimate \(\pi\) would be to use continued fractions.  Although there are simple continued fractions for \(\pi\), none of them show any obvious patters.  There’s a beautiful (but non-simple) continued fraction for \(\frac{4}{\pi}\):
    \(\frac{4}{\pi} = 1 + \frac{1^2}{2 + \frac{3^2}{2 + \frac{5^2}{2 + \frac{7^2}{2 + …}}}}\)

    Can you spot the pattern?

  • Vi Hart, the wonderful mathemusician, has a persuasive argument that we should instead be celebrating \(\tau\) day on June 28th.   Actually, all of her videos are wonderful.  If my kids spent all day doing nothing but playing with snakes  it would be better than memorizing digits of \(\pi\).Pie Plate Pi
  • Another wonderful way to compute \(\pi\) is to use nested round and square baking dishes (of the correct size) and drop marbles into them randomly from a distance.  Simply count up the number of marbles that land in the circular dish and keep track of the total number of marbles that landed in either the circle or the square. Since the area formulae for squares and circles are related, the value of \(\pi = 4 \frac{N_{circle}}{N_{total}}\).

There are probably 7000 better things to do with \(\pi\) day than digit memory contests. There are lots of creative teachers out there — how are all of you going to celebrate \(\pi\)-day?

The Up-Goer Five Research Challenge

I thought this was silly at first, but after struggling to do it for my own research, I now think it can be a profound exercise that scientists should attempt before writing their NSF broader impact statements. Here’s the challenge: Explain your research using only the 1000 most common English words. Here’s a tool to keep you honest: http://splasho.nfshost.com/upgoer5/  The idea was inspired by Randall Munroe’s wonderful Up Goer Five explanation of the Saturn V moon rocket.

And here’s my attempt:

The things we use every day are made of very tiny bits. When we put lots of those bits together we get matter. Matter changes how it acts when it gets hot or cold, or when you press on it. We want to know what happens when you get some of the matter hot. Do the bits of hot matter move to where the cold matter is? Does the hot matter touch the cold matter and make the cold matter hot? We use a computer to make pretend bits of matter. We use the computer to study how the hot matter makes cold matter hot.

The task is much harder than you think.   Here’s a collection curated by Patrick Donohue (a PhD candidate in lunar petrology right here at Notre Dame):  Common words, uncommon jobs