S505 is an introduction to basic and applied research methods for evaluating library sources and services. Special emphasis is on how information professionals generate information that they use to provide better sources and services. Methods covered include: checklists, bibliometric and citation analysis, list checking, circulation and use statistics, interlibrary loan data, document delivery, and availability measures, among others, as well as in-depth interviews, focus groups, and surveys.
GOALS AND OBJECTIVES
Upon the course's completion, students should be able to:
- Describe methods available for evaluating library collections and services;
- Describe the benefits and drawbacks of these methods;
- Determine which methods are most appropriate for a particular evaluation project;
- Design and carry out evaluation projects; and
- Understand the potentials and pitfalls of electronic resources in evaluation projects.
DELIVERY METHODS
Several delivery methods will be used to achieve the goals and objectives of this course, including but not limited to: interactive lecturing, in-class demonstrations of evaluation tools and methods, in-class group work and activities, and the completion of three assignments. We will use the computer lab to support class activities and complete course projects.
EXPECTATIONS
- This course heavily involves student participation which necessitates attendance and completion of readings and assignments prior to class.
- You are expected to attend all sessions in this course. If you must miss a class because of an illness or family emergency, please let the instructor know. Also make arrangements with your classmates to pick up class notes. One grade will be deducted from your final grade for each absence (e.g., from A to A-). If you miss three classes, you will be asked to drop the course.
- Use Chicago Manual of Style for citing and listing sources in your projects.
- You will abide by the Indiana University and SLIS policies on academic dishonesty. Students found to be engaging in plagiarism, cheating, or other types of dishonesty will be reported to the Dean's Office for appropriate action. There is extensive documentation and discussion of the issue of academic dishonesty in the Indiana University "Code of Student Rights, Responsibilities and Conduct." Of particular relevance is the section on plagiarism:
- A grade of Incomplete (I) may be given in this course after discussion with the instructor, but, depending on the circumstances, there will be a penalty applied at the discretion of the instructor.
- The instructor reserves the right to make, with notice, adjustments to the calendar and content of the course syllabus.
- All assignments must be handed in on their due dates. If you cannot deliver an assignment or a project on the date it is due, it is your responsibility to discuss your situation with the instructor, preferably in advance. LATE assignments will automatically lose one-letter grade from the grade they would have gotten had they been turned in on time. In case an absence is inevitable on a day an assignment is due, e-mail the assignment beforehand to the instructor to avoid the penalty.
ASSIGNMENTS
The Final Grade for the course will be made up of the following: (1) Three Projects--25% each; and (2) Class Attendance and Participation--25%. The assignments include:
- Evaluating a book and journal collection
- Evaluating databases
- Research proposal on evaluating a library service
More details on the assignments will be provided in class. Each assignment is designed so you will have opportunities to demonstrate a thorough knowledge of the course materials, demonstrate originality and creativity, and exceed the course expectations.
With regard to participation, it should be noted that there is no straight method for calculating participation points; this portion of your grade will be determined by synthesizing a tally of your attendence and the instructor's own subjective impressions of your enthusiasm for the course material and active and meaningful participation in the discussion of readings and lecture topics in the classroom.
GRADING
Grades will be assigned on an A, B, C, D bases. The following definitions of letter grades have been defined by student and faculty members of the Committee on Improvement of Instruction and have been approved by the faculty (November 11, 1996) as an aid in evaluation of academic performance and to assist students by giving them an understanding of the grading standards at SLIS.
| Grade |
GPA |
MEANING |
| A |
4.0 |
Outstanding achievement. Student performance demonstrates full command of the course materials and evinces a high level of originality and/or creativity that far surpasses course expectations. |
| A- |
3.7 |
Excellent achievement. Student performance demonstrates thorough knowledge of the course materials and exceeds course expectations by completing all requirements in a superior manner. |
| B+ |
3.3 |
Very good work. Student performance demonstrates above-average comprehension of the course materials and exceeds course expectations on all tasks as defined in the course syllabus. |
| B |
3.0 |
Good work. Student performance meets designated course expectations, demonstrates understanding of the course materials and is at an acceptable level. |
| B- |
2.7 |
Marginal work. Student performance demonstrates incomplete understanding of course materials. |
C+
C
|
2.3
2.0 |
Unsatisfactory work. Student performance demonstrates incomplete and inadequate understanding of course materials. |
C-
D+
D
D- |
1.7
1.3
1.0
.07 |
Unacceptable work. Coursework performed at this level will not count toward the MLS or MIS degree. For the course to count towards the degree, the student must repeat the course with a passing grade. |
| F |
0.0 |
Failing. Student may continue in program only with permission of the Dean. |
Please note that all assignments and most readings can be accessed through ERes. Also note that this course demands a HEAVY commitment of your time. It is expected that you will spend an average of 18 hours each week on readings and assignments.
SCHEDULE AND READINGS
May 07: Introduction to Course; Overview of Collection & Service Evaluation Methods
- Eldredge, J. D. (2004). Inventory of research methods for librarianship and informatics. Journal of the Medical Library Association, 92(1), 83-90.
- Johnson, P. (2004). Fundamentals of collection development & management. Chicago: American Library Association. Chapter 9 - Collection Analysis: Evaluation and Assessment (pp. 268-297).
May 09: The Checklist Method
- Dennison, R. F. (2000). Quality assessment of collection development through tiered checklists: Can you prove you are a good collection developer? Collection Building, 19(1), 24-26.
- Dilevko, J., & Gottlieb, L. (2003). The politics of standard selection guides: The case of the Public Library Catalog. Library Quarterly, 73(3), 289-337.
- Lundin, A. H. (1989). List-checking in collection development: An imprecise art. Collection Management, 11(3-4), 103-112.
- Nisonger, T. E. (2008). Use of the checklist method for content evaluation of full-text databases: An investigation of two databases based on citations from two journals. Library Resources & Technical Services, 52(1), 4-17.
- Senkevitch, J. J., & Sweetland, J. H. (1998). Public libraries and Adult fiction: Another look at a core list of 'Classics'. Library Resources & Technical Services, 42(2), 102–112.
May 12: Citation Analysis: Uses, Strengths, and Weaknesses
- Pancheshnikov, Y. (2007). A comparison of literature citations in faculty publications and student theses as indicators of collection use and a background for collection management at a university library. The Journal of Academic Librarianship, 33(6), 674-683.
- Vallmitjana, N., & Sabat, L. G. (2008). Citation analysis of ph.d. dissertation references as a tool for collection management in an academic chemistry library. College & Research Libraries, 69(1), 72-81.
May 14: Journal Evaluation: Bibliometric Methods
- Braun, T., Glanzel, W., & Schubert, A. (2005). A Hirsch-type index for journals. The Scientist, 19(22), 8.
- Chapron, G., & Husté, A. (2006). Open, fair, and free journal ranking for researchers. Bioscience, 56(7), 558-559.
- Choinski, E. M. (2007). Journal use in pharmacy: A citation analysis of faculty publications at a school of pharmacy. Science & Technology Libraries, 27(3), 53-64.
- Chung, H. K. (2007). Evaluating academic journals using impact factor and local citation score. The Journal of Academic Librarianship, 33(3), 393-402.
- Nisonger, T. E. (2004). The benefits and drawbacks of impact factor for journal collection management in libraries. The Serials Librarian, 47(1-2), 57-75.
- Nisonger, T. E. (2007). Journals in the core collection: Definition, identification, and applications. The Serials Librarian, 51(3-4), 51-73.
May 16: Journal Evaluation: Expert Opinion and Other Non-Citation Based Methods
- East, J. W. (2006). Ranking journals in the humanities: An Australian case study. Australian Academic and Research Libraries, 37(1), 3-16.
- Lewis, J. S., & McDonald, J. D. (2002). Defining an undergraduate core journal collection. The Serials Librarian, 43(1), 45-59.
- Nisonger, T. E., & Davis, C. H. (2005). The perception of library and information science journals by LIS education deans and ARL library directors: A replication of the Kohl-Davis study. College & Research Libraries, 66(4), 341-377.
- Walters, W. H. (2004). Criteria for replacing print journals with online journal resources: The importance of sustainable access. Library Resources & Technical Services, 48(4), 300-304.
May 19: ILL, Circulation, In-House Use, Doc. Delivery, Availability Measures (DUE: Project #1)
- Blecic, D. D. (1999). Measurements of journal use: an analysis of the correlations between three methods. Bulletin of the Medical Library Association, 87(1), 20–25.
- Ciliberti, A. C., Radford, M.L., Radford, G.P., & Ballard, T. (1998). Empty handed? A material availability study and transaction log analysis verification. The Journal of Academic Librarianship, 24(4), 282-289.
- Mortimore, J. M. (2006). Access-informed collection development and the academic library: Using holdings, circulation, and ILL data to develop prescient collections. Collection Management, 30(3), 21-37.
- Nisonger, T. E. (2001). Accessing information: The evaluation research. Collection Management, 26(1), 1-23.
- Nisonger, T. E. (2007). A review and analysis of library availability studies. Library Resources & Technical Services, 51(1), 30-49.
- Yang, Z. Y. (2006). Improving turnaround time for document delivery of materials owned but not on the shelf: A case study from an academic library. The Journal of Academic Librarianship, 32(2), 200-204.
May 21: Database Evaluation Methods
- Banks, R. A. (2006). Decision-making factors related to bibliographic database cancellation. Behavioral & Social Sciences Librarian, 25(1), 93-110.
- Blessinger, K., & Olle, M. (2004). Content analysis of the leading general academic databases. Library Collections, Acquisitions, & Technical Services, 28(3), 335-346.
- Schaffer, T. (2001). Databases and political science research. Online Information Review, 25(1), 47-53.
- Tellman, J. (2001). A comparison of the usefulness of IBZ and FRANCIS for historical research. Reference & User Services Quarterly, 41(1), 56-66.
May 23: Technology and Evaluation
May 28: Evaluation in Public and School Libraries (DUE: Project #2)
- Hylen, J. (2004). The top ten reasons a library media specialist is a teacher's best friend. The Clearing House, 77(5), 219-221.
- Lance, K. C., & Russell, B. (2004). Scientifically based research on school libraries and academic achievement: What is it? How much of it do we have? How can we do it better? Knowledge Quest, 32(5), 13-17.
- Penta, M., & McKenzie, P. J. (2005). The big gap remains: Public librarians as authors in LIS journals, 1999-2003. Public Library Quarterly, 24(1), 33-46.
- Puacz, J. H. (2005). Electronic vs. print reference sources in public library collections. The Reference Librarian, 91-92, 39-51.
- Valenza, J. K. (2007). You know you're a 21st-century teacher-librarian if.... Teacher Librarian, 35(1), 18-20.
- Willcoxon, W. O. (2001). Collection evaluation in a Georgia elementary school: A look at the process and resulting change in teachers. Knowledge Quest, 29(5), 23-29.
- Wolfe, G. (2001). How do you measure a library collection? Multimedia Schools, 8(3), 30-33.
- Young, T. E. (2005). Better data...better decisions. Library Media Connection, 23(4), 14-19.
May 30: Interviews and Focus Groups
- Higa-Moore, M. L., et al. (2002). Use of focus groups in a library's strategic planning process. Journal of the Medical Library Association, 90(1), 86-92.
- Ho, J., & Crowley, G. H. (2003). User perceptions of the "Reliability" of library services at Texas A&M University: A focus group study. The Journal of Academic Librarianship, 29(2), 82-87.
- Hughes-Hassell, S., & Bishop, K. (2004). Using focus group interviews to improve library services for youth. Teacher Librarian, 32(1), 8-12.
- Kuruppu, P. U. (2007). Evaluation of reference services--A review. The Journal of Academic Librarianship, 33(3), 368-381.
- Meho, L. I. (2006). E-mail interviewing in qualitative research: A methodological discussion. Journal of the American Society for Information Science and Technology, 57(10), 1284-1295.
June 02: Surveys
- Babbie, E. (2007). The practice of social research. 11th ed. Belmont, CA: Wadsworth/Thomson. Read pages: 179-217, 243-284. [On closed reserve at: Kent Cooper Room / H62 .B2 2007]
- King, D. B., Jr. (2005). User surveys: Libraries ask, "Hey, How Am I Doing?" Law Library Journal, 97(1), 103-115.
- Saunders, S. (2007). The LibQual+ phenomenon: Who judges quality? Reference & User Services Quarterly, 47(1), 21-24.
- Woo, H. (2005). The 2004 user survey at the University of Hong Kong libraries. College & Research Libraries, 66(2), 115-135.
June 04: Presentation of Project #3
ADDITIONAL READINGS
- Alabaster, C. (2002). Developing an outstanding core collection: A guide for libraries. Chicago: American Library Assocuiation.
- Baker, S.L., & Lancaster, F.W. (1991). Measurement and evaluation of library services. 2nd ed. Arlington, VA: Information Resources Press.
- Bertot, J.C., McClure, C.R., & Ryan, J. (2001). Statistics and performance measures for public library networked services. Chicago, American Library Association.
- Evidence Based Library and Information Practice: an open access, peer reviewed journal published quarterly by the University of Alberta Learning Services and supported by an international team of editorial advisors.
- Hernon, P., & Dugan, R.E. (2002). An action plan for outcomes assessment in your library. Chicago: American Library Association.
- Lancaster, F.W. (1993). If you want to evaluate your library... 2nd ed. Champaign: University of Illinois, Graduate School of Library and Information Science.
- Van House, N.A., Weil, B.T., & McClure, C.R. (1990). Measuring academic library performance: A practical approach. Chicago: American Library Association.
- Van House, N.A., Zweizig, D., & Rodger, E.J. (1987). Output measures for public libraries: A manual of standardized procedures. 2nd ed. Chicago: American Library Association.
- Wallace, D.P., & Van Fleet, C., eds. (2001). Library evaluation: A casebook and can-do guide. Englewood, CO: Libraries Unlimited.