Everything that’s wrong with e-book statistics - a comparison of four medical e-book packages

Introduction

Uppsala University has large medical and bio-medical institutions with nursing, pharmaceutical and medical programs, and also a large university hospital. There are two subject libraries for researchers, student and hospital staff, where librarians meet the users. All electronic resources are accessible from campus and from home.

Uppsala university library has four big medical e-book collections; AccessMedicine, Books@Ovid, Medicines Complete and Thieme E-book Library. Digital e-book collections are an important part of a modern academic library collection, and it is important to analyze the use. However, there are many difficulties regarding the usage statistics, for example analyzing, comparing and gathering as described by Cox. (1)

Each year our library gathers usage statistics from the vendors, but so far we haven’t really done much with the statistics.

Objectives

We wanted to see if usage statistics could be helpful in the renewal process. We wanted to use the statistics to establish value for money, by comparing the number of downloads and price per download.

We also wanted to understand the different models of usage statistics, and the implications they had on comparing usage statstistics for our library.

Method

An investigation was done to describe the type of usage statistic delivered from each vendor to understand the underlying mechanisms on how the statistics are gathered. In the case of Thieme and Medicines Complete it was difficult to find detailed information, and we had to contact their customer services.

Usage statistics was gathered from the vendors platforms, and financial information was gathered from the library’s economic department.

Results

There is an existing Counter Code of Practice for e-books, (2) but it includes two different types of book reports, Book Report 1 and Book Report 2. BR1 shall deliver statistics on “title” level, only to be given if the e-book is constituted of one file and the usage cannot be analyzed in more detail. That means that if a user reads from many chapters, it still is only registered as one download.

BR2 shall report use of sections of e-books, on “the first level of subdivision” of a book, which in most cases means chapter level. But this wording also leaves room for different interpretations on what constitutes “the first level of subdivision”, and consequently what constitutes a download. Some vendors count use by page, and others, like ebrary, even includes printing and copying of pages. Using page level as “the first level of subdivision”, of course gives publishers a higher number of downloads.

The comparison of the type of usage statistics of the four packages clearly showed the lack of a common standard for e-book statistics. In this analysis three of the four medical collections follow the Counter standard; Books@Ovid, Thieme (3) and Medicines Complete (4) while AccessMedicine does not. (5) 

Books@Ovid reports at title level (BR1), MedicinesComplete reports at title level (BR2) and Thieme reports at page level (BR2) . Since they use different reports and deliver statistics on different levels, a relevant comparison can’t be done, even though they are all Counter compliant.

AccessMedicine doesn’t follow the Counter standard at all. Instead they report nearly all use of their web content as “downloads”. The statistics include not only book chapters, but also video lectures and self-assessment tests found on their web site. This of course gives AccessMedicine a very high usage.

The number of downloads for 2011 is 2440 for Books@Ovid, 2420 for Thieme, 6998 for MedicinesComplete and 45168 for AccessMedicine. (See table 1).

The analysis also looked at the price per download to see which package was the most cost effective. However, because of the different definitions of “a download” mentioned above, the same problem remains, and no comparison could be done. In the comparison AccessMedicine is the most expensive package, but got the lowest cost per download (0.23 €), simply because of the high number of “downloads”. Thieme, that follow the recommended BR2 definition, got the highest cost per use by far (2.27 €).

Discussion

We found in this study that neither the number of downloads nor the price per download can be compared among these four packages. They are not all Counter compliant and even those who are cannot be accurately compared, because they all have a different definition of “a download”. This means we have no real use of the usage statistics we collect every year. The only thing we can do is compare each package from year to year to see the trends, provided that the package doesn’t differ a lot in content from year to year.

The lack of comparable statistics leaves acquisition librarians confused with no support in the renewal process, because there is no way of knowing how to compare” title” use and ”page” use. It also makes it hard to establish value for money and to justify the purchases to our stakeholders. How can we establish a reasonable cost for an e-book download if there is no way of calculating? Without the detailed knowledge of the differences in definitions of “downloads”, it would be an easy answer to cancel a package because of a high cost per download, when there in fact might not be a real difference. The price examples above from AccessMedicine and Thieme give little guidance regarding what a download can cost. This is also an example of the million-dollar question: What is a reasonable price for a download?

The difficulties in defining “a download” also made us wonder if vendors might “tweak” the statistics to better suite their needs. It is of course more preferable to count pages than chapters, and still be able to be “Counter-compliant”. A new release of the Counter Code of Practice will come in spring 2012, and maybe it will have a solution to this ambiguity.

There is a need of higher awareness among librarians of these difficulties. The experience from our library is that librarians know there are problems with comparing usage statistics, but they do the comparisons anyway. These usage statistics are all we have, so we use them even if they don’t give us the answers we want. It was surprising to notice how hard it was to get information on the type of statistics delivered. This made it extra hard to get all the facts we needed to make an analysis. It would facilitate if vendors would clearly describe their statistics on their website.

This presentation has no easy solution to the problem, but it highlights the importance of a common standard for e-book statistics and higher awareness among librarians.