EBM support

The Institute of Public Health (IPH) (www.ishp.gov.al) is recognized as the leading agency of Ministry of Health of Republic of Albania for protecting  health and safety of Albanian people, being the national focus for developing and applying disease prevention and control, environmental health and health promotion and education activities in specialized departments.

With the help and other support from AIHA and USAID, a LRC was created in IPH after 2 years of preparatory and testing time, for the enhancement of awareness of local specialists and researchers for Internet use, especially for Cochrane and other EBM controlled databases working in a quite wide range of research oriented topics.

The main aim of this LRC is to involve as many specialists and as many projects as possible in EBM practice, transforming it gradually in a national public health information center. In the framework of AIHA project and further, on our initiative, on other projects, many local workshops and individual training was provided to all the specialists of the IPH and also of different regional public health directories in the field of EBM principles using JAMA guidebooks, and they were provided with OVID, HINARI, NHS CASP facilities in order to better select the sources of medical literature to fulfill their specific projects requirements(e.g.HIV-AIDS national  programme of prevention and control, other viral and bacterial diseases projects in the fields of microbiology and epidemiology). This has been done creating documentary profiles, email lists and different subscriptions online, e.g. to Medscape, Dynamed etc. The use of medterms of Medline translated in local language was used for keywords in local language research publications.

Materials obtained by courses I have attended online for EBL ScHARR, University of Sheffield have been used in trying to obtain an initial estimate of indicators for monitoring and evaluating Health information local products and services, according to respective USAID Guide, 2007. Different books regarding bird flu, SARS and other epidemics has been translated and published on license, thus helping in the local establishment of valid structures of prevention and control of possible outbreaks of new and scarcely known epidemics.

We have been constantly helped in this work also by the copious and constant contribution of Ms.Ibraghimova in links provided, and especially those in Russian for Russian educated specialists, and also by EBLIP journal published by University of Alberta, USA.

Introduction

At the University of Liège, an Evidence-Based Medicine (EBM) course intended for 4^th year students in medicine is focused on information search, selection and evaluation. This course is held in the context of a workshop in pharmacology and is divided into three activities of 2 hours each. Given the number of students (about 140 per year), the 3 activities are run four times for groups of 35 students. The first meeting provides a theoretical introduction to EBM and Medline searching. Then the students have a couple of weeks for small group work on a clinical question. They have to apply the principles of EBM in order to find, summarize and comment on a controlled trial comparing the efficacy of different drug classes for the treatment of respiratory diseases (asthma or chronic obstructive pulmonary disease). During the second meeting, each small group returns for a twenty minute oral presentation of arguments for or against the drug class they have studied. The last meeting consists of an advanced course in Medline searching through the online OvidSP portal.

Questions/Objectives

Two questions were asked in the present study: 1) Do EBM classes improve students' information literacy? 2) Are the teaching methods appropriate?

Methods

In 2011-2012, the first question was addressed by running the validated Fresno test (1) both before and three months after the EBM course. The Fresno test is an instrument that evaluates four EBM steps and that has strong psychometric properties (2). The test begins with the presentation of two clinical scenarios and includes 12 questions (open-ended questions, fill-in-the-blank questions and calculations).  Students are requested: 1) to formulate a clinical question for each scenario; 2) to name types or categories of information sources useful to answer the questions, and to describe their strengths and weaknesses; 3) to identify the best research design for one of the two scenarios; 4) to explain the strategy they would use for Medline searching; 5) to cite the criteria to be considered for critical appraisal of an article (relevance, validity and magnitude of effect size). The last five questions are related to mathematical calculations and best evidence for diagnostic and prognostic issues. These questions were pre-tested, but were removed from the post-test because the corresponding topics had not been taught and the students were not expected to have progressed in these areas.      

The relevance of teaching methods was evaluated through a survey administered after the post-test. Students were invited to express their opinions about the course.

Results

109 out of 154 enrolled students responded to both the pre-test and post-test. A sample of 25 students was considered in this paper corresponding to the first 25 names on the alphabetical registration list. A total of 50 pre- and post-tests was scored independently by two reviewers in order to assess inter-rater reliability. The maximum possible score for the first 7 questions of the Fresno test is 168 (24 points for each question).

18 students performed better in the post-test, in comparison with the pre-test, with scores increasing by between 1 and 36 points. 3 students obtained the same final score, and 4 others obtained lower scores in the post-test with a decrease of 3 to 20 points.    
In the present study, for the pre-test, the lowest score was 9 and the highest was 66. For the post-test, the lowest value was 17 and the highest value was 83.

Here, the average score was 41 for the pre-test and 50 for the post-test.

The sum of scores obtained by the 25 students for each question showed some improvement on the following topics: elaboration of a focused clinical question (240 to 266; +26 points), selection of the most appropriate clinical study design (81 to 219; +138 points), strategy for Medline searching (251 to 312; +61 points) and validity assessment (113 to 202; +89 points).

Responses to three questions obtained lower scores in the post-test: strengths and weaknesses of different sources of information (258 to 186; -72 points), relevance (34 to 30; -4 points) and magnitude of effect size (49 to 24; -25 points).

23 students out of the sample of 25 answered the satisfaction survey. All the respondents thought that the training on Medline searching should be scheduled before the oral presentation in order to support team working. The majority considered that the workshop objectives (14/23) and instructions (18/23) were clear as well as the user guide provided to help in the preparation of the oral presentation (19/23). 17 students acknowledged that they had not been motivated to follow these sessions. 21 recognized that the teachers were ready to help but only 8 thought that the teachers made the sessions attractive. 13 students considered that the EBM course would be useful for their studies and 20 that it would be useful for professional life.

Discussion

In answer to the question “Do EBM classes improve students' information literacy?”, we found that two-thirds of the students managed to improve their scoring in the Fresno test, even though their overall performance remained barely satisfactory. Several elements could explain these results: 1) the scores might have been better if the post-test had taken place directly after the last training session instead of three months later, and if the participation of the test were not just optional but taken into account for certification; 2) student progression might have been more striking if the students had been inexperienced at the time of the pre-test; however, they had already completed an introductory course in information retrieval two years earlier; 3) the severity of scoring.

Surprisingly, students were less successful in the post-test on the question related to strengths and weaknesses of different information sources. Perhaps they focused attention on issues related to EBM, and disregarded the question on library resources?

Regarding the severity of scoring, the following elements should be highlighted. For question n°1 (translation of clinical scenarios into answerable questions), no point was attributed if students provided keywords on a PICO grid instead of writing a full sentence.
Writing a complete sentence is certainly more complex than just citing elements. In some cases, students provided both answers: the different ingredients of the PICO grid were good, but the sentence did not match the PICO grid.           
In order for students to answer questions n°5 (relevance), n°6 (validity), and n°7 (magnitude of effect size), we decided that it was mandatory to clearly distinguish between the three criteria. However, in their assessment of the reliability and validity of the Fresno test in EBM, Ramos et al. (1) state that overall justifications made by respondents may be acceptable where they describe, for example, issues of relevance in answers to any of these three questions.       
It was very helpful to have two reviewers sharing opinions in the marking process, since the Fresno test is not easy to score despite having a scoring template provided by the authors. Numerous elements must be considered and raters must also refrain from interpreting students' ambiguous responses. As noticed by Shaneyfelt (2) and by Lewis et al. (3), time and expertise are required to score the test. For all these reasons, we consider that comparing scores assigned by different examiners in different settings should be undertaken with caution.

The survey highlighted an important organizational change that has to be made in the future (moving the Medline training session to before the oral presentation); it also reported students' overall satisfaction with the support offered for team working. Nevertheless, the number of students per group is much too high, meaning that they have to divide the work into parts and share responsibilities. Very few participants involve themselves in the different activities offered by the EBM classes: writing a clinical question, searching on Medline, selecting relevant papers, reading and reviewing a clinical article, giving an opinion and presenting results to others. As teachers, we shall have to adjust our evaluation grid in order to take into account individual participation in the different tasks.

Ideally, much more time should be devoted to the EBM course, in particular to extending the information search in databases other than Medline. But the curriculum of the degree course in medicine is already overwhelming. An alternative would be a transversal integration of EBM activities into other courses in order to improve students' knowledge and skills without increasing the workload.      

As a conclusion, we should remember that 20 out of the 25 students surveyed think that the EBM course will be useful for professional life. This is rewarding for librarians, who consider it a part of their mission to ensure medical students gain the proper tools for EBM practice and lifelong learning.

Setting

The PhD study programme at Palacky University Faculty of Medicine & Dentistry (Olomouc, Czech Republic) is open to any University graduates with the intention to improve their qualifications in theoretical, preclinical and clinical medical specialties. The paper will demonstrate a role of medical librarians in teaching evidence-based medicine and online courseware development.

Perspective

PhD students start with the Essential Introductory Course, consisting of 14 modules/2 credits each. They select topics according to their field of interest to get at least 8 credits. Four modules are related to evidence-based practice, and two of them, B001 - Information retrieval and B006 - Evidence-based medicine have long been taught with the input of medical library staff.

The increasing demand to acquire EBM knowledge has triggered an initiative to innovate the two EBM-related modules.

Intervention

A new design of the modules was laid out by a multiprofessional team (medical teacher, librarian, IT specialist, PhD programme manager) to comply with a blended learning methodology. Four contact hours are supported with online study materials and interactive activities accessible on Moodle at http://biomedicine.upol.cz/moodle. Module B001 is focused on developing search skills to get the best evidence from multiple resources, including library catalogues, deep web, bibliographic databases, and point of care information summaries.

In both courses, online materials consist of text descriptions of the clinical scenario, information resources, EBM history, theory, methods and apps, audio synched across powerpoint, and additional reading links. A bridge over modules B001 and B006 is the clinical question dealing with treatment modalities of monosymptomatic nocturnal enuresis in school children. To complete module B006 the trainees have to critically appraise a randomized controlled trial using an interactive form and submit their solutions before the course.

The main points of the critical appraisal are then summarized by the facilitators during workshop.

Comparison

The blended learning approach has proven practical, more efficient, attractive and time-saving for teaching & learning EBM skills than mere contact hours used before.

Evaluation

The first group of PhD students (n=39) completed the newly designed module in June 2011, all of them passed and got their credits. A survey demonstrated both high perceived value of the course and level of satisfaction.

In 2012 the modules are slated for January and June and the expected total number of survey respondents will be up to 120.

The Terveysportti health portal, started in 2000, is an online service operated by Duodecim Medical Publications Ltd (owned by the Finnish Medical Society Duodecim). At a single Internet address (www.terveysportti.fi), the portal provides key medical contents to support the work of healthcare professionals.

The portal is a comprehensive clinical information source which offers the Physician’s, Nurse’s and Dentist’s Databases, several drug  information databases, patient information database, and the ICD-10 and procedure search engines.

The Finnish national Current Care guidelines together with their patient versions can be searched via this health portal. Also the Cochrane Library, several medical e-textbooks, dictionaries and the major professional journals from Finland and abroad are available in Terveysportti. PubMed can be searched by using Finnish Medical Subject Headings (FinMeSH).

During the year between November 2010 and November 2011 almost 40 million documents were opened from Terveysportti The most popular information source were the drug databases (14 million opened documents), followed by Physician’s database (12 million opened documents), consisting of EBM Guidelines (a point-of-care guideline collection also translated into several other languages) and related databases, and ICD-10 database (5 million opened documents). The use is still growing more than 10% yearly.  The health portal is accessible in all the hospital districts and primary care health care centres and in 700 of the 800 pharmacies in Finland. The portal is used in all the medical libraries in Finland and the librarians teach the medicine students the search skills needed.

Terveysportti is also a kind of patient safety tool in Finland by providing a vast range of quality information sources that are comprehensively available.

Introduction

Weill Cornell Medical College in Qatar (WCMC-Q) is part of Weill Cornell Medical College in New York and shares its mission of dedication to excellence in education, patient care and research.

The Distributed eLibrary  (DeLib) at WCMC-Q is a predominantly electronic “library without walls”, transcending space and place, existing in a distributed environment. One of its key defining features is its disseminated yet interconnected services, resources, and presence that are accessible, flexible, and dynamic.  Its collection consists of about 15,471 ebooks and 7,874 ejournals in addition to databases, images and a small print collection of about 3,950 titles.

Supporting the mission of WCMC-Q, DeLib’s mission is: “Applying our concept of Library as Presence, our mission is to deliver diverse information resources and services that engage students, faculty and staff of WCMC-Q, as well as local, regional, and international communities.“

In June 2008, DeLib launched a strategic plan, Futura, which called for DeLib to support and facilitate professional growth for all DeLib staff, and, promote a continuum of learning; offer relevant support of the pre-medical and medical programs, clinical medicine and research; extend DeLib’s presence by seeking opportunities for collaboration and outreach.

Since the inception of the Medical Program in 2004, WCMC-Q has taught Evidence-Based Medicine (EBM) to its first-year medical students. Initially, DeLib librarians’ participation was limited to providing in-class searching support.  However, in 2008 the librarians were invited to play a greater role in this course, which required them to develop their knowledge of the concepts and teaching methods of EBM.

Objective

Within the framework of DeLib’s mission and strategic plan, describe the EBM training and professional development undertaken by DeLib librarians and the consequent growth and change in their teaching roles.

Methods

DeLib librarians were offered a variety of internal and external professional development opportunities to develop their basic subject knowledge of EBM and expose them to different ways of teaching the subject. To begin with, one librarian, who already possessed some knowledge of EBM, conducted a six-week internal train-the-trainer style workshop to introduce the others to the basic process and principles of the topic. Then, in order to learn the widest possible range of EBM teaching techniques, librarians attended courses at established centers of EBM education (Centre for Evidence-Based Medicine, Oxford University, UK; McMaster University, Hamilton, ONT, Canada; New York Academy of Medicine, New York, NY). Librarians also attended EBM sessions at various library conferences to pick up other teaching tips. Finally, librarians took advantage of opportunities within DeLib to share what they had learned and put their teaching skills into action. “EBM Delight” sessions were held where one librarian would provide an in-depth example of teaching an EBM session to the other librarians, such as how to introduce students to searching for information on a diagnosis question. Librarians also instructed DeLib’s library technicians in how to provide support during EBM workshops.

Results

The professional development and training undertaken by the DeLib librarians over the period of 2008-2011 had several outcomes.

First was the competence and confidence that comes from having an understanding of the concepts of EBM, as a substantial degree of subject familiarity is essential for teaching such a technical topic. A well-developed understanding of the objectives of EBM also allowed DeLib’s librarians to assess what was within their area of expertise (question formulation and searching) and in particular to place that in the context of the EBM process. Exposure to a wide variety of teaching practices allowed the development of a consistent approach in DeLib’s EBM teaching sessions. For example, this included beginning with a case scenario to engage student interaction immediately, before explaining how to optimize searching strategies to suit the scenario. Anecdotal evidence from student and faculty feedback indicates an improvement in students’ enjoyment of EBM training sessions, and in learning outcomes as measured by assignment scores. 

This also led to the DeLib librarians being invited to provide EBM training in other institutions within Qatar where WCMC-Q’s students receive clinical training. In 2009 DeLib librarians began training Hamad Medical Center medical staff (physicians, pharmacists, and occupational therapists) on EBM concepts and in the basics of searching for clinical evidence. In 2011, DeLib also begun providing some EBM-related training at Qatar’s Aspetar hospital, the first specialized orthopedic and sports medicine hospital in the Gulf Region.

Another outcome was an increased role in teaching EBM within WCMC-Q. After becoming an essential part of the EBM section of a first-year medical course, Liaison Librarians to the third-year clerkships successfully introduced an “EBM refresher” in third-year clinical clerkships.

DeLib librarians have also found opportunities to teach further afield in the GCC, having provided an EBM workshop for medical librarians and physicians at the Royal Hospital Medical Library in Muscat, Oman in January 2012. This workshop formed part of a conference accredited by the Oman Medical Specialty Board and was accredited by the Medical Library Association in the US.

Discussion

Investing in professional development in EBM and then sharing the techniques learned have paid off handsomely for DeLib. The librarians gained a valuable professional knowledge base and an improved skill set that met a clear need in WCMC-Q’s medical curriculum. In doing so, they demonstrated DeLib’s capacity to contribute to the education of medical students. By responding to other EBM teaching opportunities that arose, both within and outside WCMC-Q, DeLib librarians earned valuable lessons in dealing with a variety of audiences, particularly students and health professionals with widely differing degrees of familiarity with EBM. As DeLib begins to share its experience with other medical librarians in the Gulf Region, it is clear that a well-targeted investment in professional development, aligned with the institution’s mission and the department’s strategic plan, can pay dividends beyond any library’s walls.