On Eggs and Citations

Failing to observe a platypus laying eggs is not a demonstration that the platypus does not lay eggs. You have to actually observe the provenance, ab ovo, of the little newborn platypusses, if you want to demonstrate that they are not engendered by egg-laying.

Failing to observe a significant OA citation Advantage after a year (or a year and a half — or longer, as the case may be) with randomized OA does not demonstrate that the many studies that do observe a significant OA citation Advantage with nonrandomized OA are simply reporting self-selection artifacts (i.e., selective provision of OA for the more highly citable articles.)

You first have to replicate the OA citation Advantage with nonrandomized OA (on the same or comparable sample) and then demonstrate that randomized OA (on the same or comparable sample) eliminates the OA citation Advantage (on the same or comparable sample).

Otherwise, you are simply comparing apples and oranges (or eggs and expectations, as the case may be) in reporting a failure to observe a significant OA citation Advantage in a one one-year (or 1.5 year) sample with randomized OA — along with a failure to observe a significant OA citation Advantage for nonrandomized OA for the same sample either (because the nonrandomized OA subsample was too small):

The many reports of the nonrandomized OA Citation Advantage are based on samples that were sufficiently large, and on a sufficiently long time-scale (almost never as short as a year) to detect a significant OA Citation Advantage.

A failure to observe a significant effect with small samples on short time-scales — whether randomized or nonrandomized — is simple that: a failure to observe a significant effect: Keep testing till the size and duration of your sample of randomized and nonrandomized OA is big enough to test your self-selection hypothesis (i.e., comparable with the other studies that have detected the effect).

Meanwhile, note that (as other studies have likewise reported), although a year is too short to observe a significant OA citation Advantage, it was long enough to observe a significant OA download Advantage — and other studies have also reported that early download advantages correlate significantly with later significant citation advantages.

Just as mating more is likely to lead to more progeny for platypusses (by whatever route) than mating less, so accessing and downloading more is likely to lead to more citations than accessing and downloading less.

Stevan Harnad
American Scientist Open Access Forum