Upon the course's completion, students should:

• understand what methodologies are available for evaluating library collections and services;
• understand the benefits and drawbacks of various methodologies;
• be able to identify which methodologies are most appropriate for a particular evaluation project;
• be able to design and carry out an evaluation project; and
• understand the challenges of evaluation in the emerging electronic era.

METHODS OF INSTRUCTION

The following methods of instruction will be used:

• lectures
• classroom discussion
• assigned readings
• student projects

GRADING

The final course grade will be based on two projects. Each will count 50%.

In computing the final grade the following numerical equivalents will be used:

READINGS

(Students should read at least three readings of their choice for each topic. Additional readings may be suggested during the course.)

ITEMS OF GENERAL INTEREST

Baker, Sharon L., and F. Wilfrid Lancaster. Measurement and Evaluation of Library Services . 2d ed. Arlington, Va.: Information Resources Press, 1991.

Bertot, John Carlo, and Denise M. Davis, eds. Planning and Evaluating Library Networked Services and Resources . Westport, CT and London: Libraries Unlimited, 2004.

Bertot, John Carlo, Charles R. McClure, and Joe Ryan. Statistics and Performance Measures for Public Library Networked Services . Chicago, American Library Association, 2001.

Durrance, Joan C. , and Karen E. Fisher. How Libraries and Librarians Help: A Guide to Identifying User-Centered Outcomes. Chicago: American Library Association, 2005.

Everhart, Nancy. Evaluating the School Library Media Center: Analysis Techniques and Research Practices. Englewood, CO: Libraries Unlimited, 1998.

Guide to Performance Evaluation of Serials Vendors ; Association for Library Collections & Technical Services, Serials Section Acquisitions Committee. Chicago: American Library Association, 1997.

Hall, Blaine H. Collection Assessment Manual for College and University Libraries. Phoenix, Ariz.: Oryx Press, 1985.

Hernon, Peter, and Robert E. Dugan. An Action Plan for Outcomes Assessment in Your Library. Chicago: American Library Association, 2002.

Lancaster, F. Wilfrid. If You Want to Evaluate Your Library... 2d ed. Champaign: University of Illinois, Graduate School of Library and Information Science, 1993.

McClure, Charles R., and others. Statistics, Measures and Quality Standards for Assessing Digital Reference Library Services: Guidelines and Procedures. Draft version, July 30, 2002, available on Web at URL: http://quartz.syr.edu/quality/Quality.pdf

Nisonger, Thomas E. Evaluation of Library Collections, Access, and Electronic Resources: A Literature Guide and Annotated Bibliography. Westport, CT and London: Libraries Unlimited (Greenwood Publishing Group), 2003.

Van House, Nancy A., Beth T. Weil, and Charles R. McClure. Measuring Academic Library Performance: a Practical Approach . Chicago: American Library Association, 1990.

Van House, Nancy A., Douglas Zweizig, and Eleanor Jo Rodger. Output Measures for Public Libraries: A Manual of Standardized Procedures. 2d ed. Chicago: American Library Association, 1987.

Wallace, Danny P., and Connie Van Fleet, eds. Library Evaluation: A Casebook and Can-Do Guide . Englewood, CO: Libraries Unlimited, 2001.

Walter, Virginia A. Output Measures and More: Planning and Evaluating Public

Library Services for Young Adults . Chicago : Young Adult Library Services Association, Public Library Association, American Library Association, 1995.

Walter, Virginia A. Output Measures for Public Library Service to Children: A Manual of Standardized Procedures . Chicago: American Library Association, 1992.

White, Andrew, and Eric Djiva Kamal. E-Metrics for Library and Information Professionals: How to Use Data for Managing and Evaluating Electronic Resource Collections. New York: Neal-Schuman Publishers, 2006.

Whitlatch, Jo Bell. Evaluating Reference Services: A Practical Guide . Chicago: American Library Association, 2000.

FOR JANUARY 27: OVERVIEW OF EVALUATION METHODOLOGIES

Baker, Sharon L., and F. Wilfrid Lancaster. "The Evaluation of Library Services: An Introduction." In their Measurement and Evaluation of Library Services , 1-26. 2d ed. Arlington, Va.: Information Resources Press, 1991.

Baker, Sharon L., and F. Wilfrid Lancaster. "The Range and Scope of Library Services." In their Measurement and Evaluation of Library Services , 335-367. 2d ed. Arlington, Va.: Information Resources Press, 1991. (ON RESERVE IN SLIS LIBRARY)

Lancaster, F. Wilfrid. "Introduction." In If You Want to Evaluate Your Library ..., 1-20. Champaign: University of Illinois, Graduate School of Library and Information Science, 1993.

FOR FEBRUARY 3: OVERVIEW OF COLLECTION EVALUATION

American Library Association. Resources and Technical Services Division. Resources Section. Collection Management and Develop­ment Committee. Subcommittee on Guidelines for Collection Development. Guide to the Evaluation of Library Collections. Barbara Lockett, ed. Chicago: American Library Association, 1989. (Collection Management and Development Guides, no. 2).

Baker, Sharon L., and F. Wilfrid Lancaster. "Collection Evalua­tion: Materials-Centered Approaches" and "Collection Evaluation: Use-Centered Approaches." In their Measurement and Evaluation of Library Services , 39-122. 2d ed. Arlington, Va.: Information Resources Press, 1991. (ON RESERVE IN SLIS LIBRARY)

Clayton, Peter, and G. E. Gorman. "Collection Evaluation and Review." In their Managing Information Resources in Libraries: Collection Management in Theory and Practice , 160-86.

London: Library Association Publishing: 2001.

Evans, G. Edward, with the assistance of Margaret R. Zarnosky. "Evaluation." In their Developing Library and Information Center Collections, 314-338 ,5^th ed. Westport, CT and London: Libraries Unlimited, 2005.

Everhart, Nancy. "Collections." In her Evaluating the School Library Media Center: Analysis Techniques and Research Practices, 91-117. Englewood, CO: Libraries Unlimited, 1998.

Hall, Blaine H. Collection Assessment Manual for College and University Libraries . Phoenix, Ariz.: Oryx Press, 1985.

Intner, Sheila S. "Objectifying Subjectivity." Technicalities 21 (January/February 2001): 3-7.

Mosher, Paul. "Quality and Library Collections: New Directions in Research and Practice in Collection Evaluation." In Advances in Librarianship , vol. 13, edited by Wesley Simonton, 211-38. Orlando, Fla.: Academic Press, 1984.

Van Fleet, Connie. "Evaluating Collections." In Library Evaluation: A Casebook and Can-Do Guide , edited by Danny P. Wallace and Connie Van Fleet, 117-28. Englewood, CO: Libraries Unlimited, 2001.

FOR FEBRUARY 10: THE CHECKLIST METHOD

Dennison, Russell F. "Quality Assessment of Collection Development Through Tiered Checklists: Can You Prove You Are a Good Collection Developer?" Collection Building 19, no. 1 (2000): 24-26.

Elzy, Cheryl and F. Wilfrid Lancaster. "Looking at a Collection in Different Ways: A Comparison of Methods of Bibliographical Checking." Collection Management 12, nos. 3/4 (1990): 1-10.

Goldhor, Herbert. "Analysis of an Inductive Method of Evaluat­ing the Book Collection of a Public Library." Libri 23, no. 1 (1973): 6-17.

Halliday, Blane. "Identifying Library Policy Issues with List Checking." In Library Evaluation: A Casebook and Can-Do Guide , edited by Danny P. Wallace and Connie Van Fleet, 140-52. Englewood, CO: Libraries Unlimited, 2001. (ON RESERVE IN SLIS LIBRARY)

Harris, Matthew, and Gregory A. Crawford. "The Ownership of Religious Texts by Academic Libraries." College & Research Libraries 63 (September 2002): 450-58.

Lundin, Anne H. "List-Checking in Collection Development: An Imprecise Art." Collection Management 11, nos. 3/4 (1989): 103-12.

Porta, Maria A., and F. Wilfrid Lancaster."Evaluation of a Scholarly Collection in a Specific Subject Area by Bibliographic Checking: a Comparison of Sources." Libri 38 (June 1988): 131-37.

Schaffer, Thomas. "Psychology Citations Revisited: Behavioral Research in the Age of Electronic Resources." Journal of Academic Librarianship 30 (September 2004): 354-60.

FOR FEBRUARY 17 & FEBRUARY 24: USE OF BIBLIOMETRICS IN EVALUATION

Barkett, Gina R. 'Conducting a Citation Study." In Library Evaluation: A Casebook and Can-Do Guide , edited by Danny P. Wallace and Connie Van Fleet, 155-64. Englewood, CO: Libraries Unlimited, 2001. (ON RESERVE IN SLIS LIBRARY)

Budd, John. "Scholarly productivity of U.S. LIS faculty: an update." Library Quarterly 70 (April 2000): 230-45.

Buzzard, Marion L., and Doris E. New. "An Investigation of Collection Support for Doctoral Research." College & Research Libraries 44 (November 1983): 469-475.

Cameron, Brian D. "Trends in the Usage of ISI Bibliometric Data: Uses, Abuses, and Implications." Portal 5 (January 2005): 105-125.

Devin, Robin B., and Martha Kellogg. "The Serial/Monograph Ratio in Research Libraries: Budgeting in Light of Citation Studies." College & Research Libraries 51 (January 1990): 46-54.

Dilevko, Juris, and Keren Deli. "Improving Collection Development and Reference Services for Interdisciplinary Fields through Analysis of Citation Patterns: An Example Using Tourism Studies." College & Research Libraries 65 (May 2004): 216-41.

Nisonger, Thomas E. "A Test of Two Citation Checking Techniques for Evaluating Political Science Collections in University Librarie­s." Library Resources & Technical Services 27 (April/June 1983): 163-76.

Sinn, Robin N. "A Local Citation Analysis of Mathematical and Statistical Dissertations." Science & Technology Libraries 25, no. 4 (2005): 25-38.

Stelk, Roger Edward, and F. Wilfrid Lancaster. "The Use of Textbooks in Evaluating the Collection of an Undergraduate Library." Library Acquisitions: Practice & Theory 14, no. 2 (1990): 191-193.

FOR MARCH 3: JOURNAL EVALUATION METHODS

Black, Steven. "Using Citation Analysis to Pursue a Core Collection of Journals for Communication Disorders." Library Resources & Technical Services 45 (January 2001): 3-9.

Esteibar, Belen Altuna, and F. W. Lancaster. "Ranking of Journals in Library and Information Science by Research and Teaching Relatedness." Serials Librarian 23, nos. 1/2 (1992): 1-10.

Kuyper-Rushing, Lois. "Identifying Uniform Core Journal Titles for Music Libraries: A Dissertation Citation Study." College & Research Libraries 60 (March 1999): 153-63.

Nisonger, Thomas E. "The Benefits and Drawbacks of Impact Factor for Journal Collection Management in Libraries." Serials Librarian 47, nos. 1-2. (2004): 57-76.

Nisonger, Thomas E. "Use of the Journal Citation Reports for Serials Management in Research Libraries: An Investigation of the Effect of Self-Citation on Journal Rankings in Library and Information Science and Genetics." College & Research Libraries 61 (May 2000): 263-75.

Nisonger, Thomas E. , and Charles H. Davis. "The Perception of Library and Information Science Journals by LIS Education Deans and ARL Library Directors: A Replication of the Kohl-Davis Study." College & Research Libraries 66 (July 2005): 341-77.

Vaughan, Liwen, and Debora Shaw. "Web Citation Data for Impact Assessment: A Comparison of Four Science Disciplines." Journal of the American Society for Information Science and Technology 56 (August 2005): 1075-87.

FOR MARCH 10 & MARCH 24: AILABILITY & DOCUMENT DELIVERY STUDIES

Baker, Sharon L., and F. Wilfrid Lancaster. "Evaluation of Materials Availability." In their Measurement and Evaluation of Library Services , 143-180. 2d ed. Arlington, Va.: Information Resources Press, 1991. (ON RESERVE IN SLIS LIBRARY)

Ciliberti, Anne C., Marie L. Radford, and Gary P. Radford, "Empty Handed? A Material Availability Study and Transaction Log Analysis Verification." Journal of Academic Librarianship 24 (July 1998): 282-89.

Gregory, David J., and Wayne A. Pedersen. "Book Availability Revisited: Turnaround Time for Recalls versus Interlibrary Loans." College & Research Libraries 64 (July 2003): 283-299.

Lancaster, F. Wilfrid. "Shelf Availability." In his If You Want to Evaluate Your Library..., 129-146. Champaign: University of Illinois, Graduate School of Library and Information Science, 1993. (ON RESERVE IN SLIS LIBRARY)

Mansbridge, John. "Availability Studies in Libraries." Library & Information Science Research 8 (October-December 1986): 299-314.

Shaw-Kokot, Julia, and Claire de la Varre. "Using a Journal Availability Study to Improve Access." Bulletin of the Medical Library Association 89 (January 2001): 21-28.

Steynberg, Susan, and S. F. Rossouw. "Testing Orr's Document Delivery Test on Biomedical Journals in South Africa." Bulletin of the Medical Library Association 83 (January 1995): 78-84.

Van House, Nancy A., Beth T. Weil, and Charles R. McClure. "Materials Availability and Use." In their Measuring Academic Library Performance; A Practical Approach , 54-76. Chicago: American Library Association, 1990. (ON RESERVE IN SLIS LIBRARY)

Wall, Terry, and Jan Williams. "Availability, Accessibility, and Demand for Recommended Books in Academic Libraries." Journal of Librarianship & Information Science 31 (September 1999): 145-51.

FOR MARCH 31: EVALUATION OF USE OF LIBRARY MATERIALS

Bertot, John Carlo. "E-metrics and Performance Indicators: Availability and Use." In Planning and Evaluating Library Networked Services and Resources , edited by John Carlo Bertot and Denise M. Davis, 95-126. Westport, CT and London: Libraries Unlimited, 2004.

Black, Steve. "Impact of Full Text on Print Journal Use at a Liberal Arts College." Library Resources & Technical Services 49 (January 2005) p. 19-26, 56 .

Christianson, Marilyn, and Marsha Aucoin, "Electronic or Print Books: Which are Used?" Library Collections, Acquisitions, and Technical Services v. 29 no. 1 (2005): 71-81 .

Bertot, John Carlo, Charles R. McClure, and Joe Ryan. Statistics and Performance Measures for Public Library Networked Services , 6-37. Chicago, American Library Association, 2001. (ON RESERVE IN SLIS LIBRARY)

Blecic, Deborah D., Joan B. Fiscella, and Stephen E. Wiberley, Jr. "The Measurement of Use of Web-based Information Resources: An Early Look at Vendor-Supplied Data." College & Research Libraries 62 (September 2001): 434-53.

Everhart, Nancy. "Usage." In her Evaluating the School Library Media Center: Analysis Techniques and Research Practices , 209-33. Englewood, CO: Libraries Unlimited, 1998. (ON RESERVE IN SLIS LIBRARY)

Ferguson, Anthony W. "Digital Information Use Statistics: Useful?Useless?" In Charleston Conference Proceedings 2002 , edited by Rosann Bazirjian and Vicky Speck. 27-35. Westport, CT: Libraries Unlimited, 2003.

Lancaster, F. Wilfrid. "Evaluation of the Collection: Analysis of Use." In If You Want to Evaluate Your Library ..., 51-75. Champaign: University of Illinois, Graduate School of Library and Information Science, 1993. (ON RESERVE IN SLIS LIBRARY)

Lancaster, F. Wilfrid. "In-House Use." In If You Want to Evaluate Your Library..., 76-86. Champaign: University of Illinois, Graduate School of Library and Information Science, 1993. (ON RESERVE IN SLIS LIBRARY)

Tonta, Yasar, and Yurdagül Ünal. "Scatter of Journals and Literature Obsolescence Reflected in Document Delivery Requests." Journal of the American Society for Information Science and Technology 56 (January 1, 2005): 84-94.

FOR APRIL 7: EVALUATION OF REFERENCE SERVICES: QUESTION ANSWERING

Arnold, Julie, and Neal Kaske. "Evaluating the Quality of a Chat Service." Portal: Libraries and the Academy 5 (April 2005): 177-93.

Baker, Lynda M., and Judith J. Field. "Reference Success: What has Changed over the past Ten Years?" Public Libraries 39 (January/February 2000): 23-30.

Baker, Sharon L., and F. Wilfrid Lancaster. "Evaluation of Reference Services: Question Answering." In their Measurement and Evaluation of Library Services , 229-272. 2d ed. Arlington, Va.: Information Resources Press, 1991. (ON RESERVE IN SLIS LIBRARY)

Bradford, Jane T., Barbara Costello, and Robert Lenholt. "Reference Service in the Digital Age: An Analysis of Sources Used to Answer Reference Questions." Journal of Academic Librarianship 31 (May 2005): 263-72.

Crews, Kenneth D. "The Accuracy of Reference Service: Variables for Research and Implementation." Library & Information Science Research 10 (July 1988): 331-355.

Gers, Ralph and Lillie J. Seward. "Improving Reference Performance: Results of a Statewide Study." Library Journal 110 (November 1, 1985): 32-35.

Janes, Joseph, and Charles R. McClure. "The Web as a Reference Tool: Comparisons with Traditional Sources." Public Libraries 38 (January/February 1999): 30-33, 36-39.

Lancaster, F. Wilfrid. "Question Answering." In his If You Want to Evaluate Your Library..., 151-180. Champaign: University of Illinois, Graduate School of Library and Information Science, 1993. (ON RESERVE IN SLIS LIBRARY)

McClure, Charles R., and others. Statistics, Measures and Quality Standards for Assessing Digital Reference Library Services: Guidelines and Procedures. Draft version, July 30, 2002, available on Web at URL: http://quartz.syr.edu/quality/Quality.pdf

Richardson, John V. "Reference is Better Than We Thought." Library Journal 127 (April 15, 2002): 41-42.

Ross, Catherine Sheldrick, and Kirsti Nilsen. "Has the Internet Changed Anything in Reference? The Library Visit Study, phase 2." Reference and User Services Quarterly 40 (Winter 2000): 147-55.

FOR APRIL 14: EVALUATION OF REFERENCE SERVICES: DATABASE SEARCHING

Baker, Sharon L., and F. Wilfrid Lancaster. "Evaluation of Reference Services: Database Searching." In their Measurement and Evaluation of Library Services, 273-320. 2d ed. Arlington, Va.: Information Resources Press, 1991. (ON RESERVE IN SLIS LIBRARY)

Barnett, Andy. "A Survey of Internet Searches and Their Results." Reference and User Services Quarterly 39 (Winter 1999): 177-81.

Lancaster, F. Wilfrid. "Database Searching." In If You Want to Evaluate Your Library..., 181-219. Champaign: University of Illinois, Graduate School of Library and Information Science, 1993. (ON RESERVE IN SLIS LIBRARY)

Nisonger, Thomas E. "Citation Autobiography: An Investigation of ISI Database Coverage in Determining Author Citedness." College & Research Libraries 65 (March 2004): 152-63.

Nowicki, Stacy. "Student vs. Search Engine: Undergraduates Rank Results for Relevance." Portal 3, no. 3 (2003): 503-515.

Wu, Gang, and Jie Li. " Comparing Web Search Engine Performance in Searching Consumer Health Information: Evaluation and Recommendations." Bulletin of the Medical Library Association 87 (October 1999): 456-61.

FOR APRIL 21: EVALUATION OF TECHNICAL SERVICES

American Library Association. Resources and Technical Services Division. Resource Section. Collection Management and Development Committee and the Acquisitions Committee. Guide to Performance Evaluation of Library Materials Vendors . Chicago: ALA, 1988.

Eldredge, Mary. A United Kingdom Approval Plans and United States Academic Libraries: Are they Necessary and Cost Effective? @ Library Acquisitions: Practice & Theory 18 (Summer 1994): 165-178.

Fowler, David C., and Janet Arcand. "A Serials Acquisitions Cost Study: Presenting a Case for Standard Serials Acquisitions Data Elements." Library Resources & Technical Services 49 (April 2005): 109-22.

Morris, Anne, Julie Woodfield, and J. Eric Davies. "Experimental Evaluation of Selected Electronic Document Delivery Systems." Journal of Librarianship & Information Science 31 (September 1999): 135-44.

Nisonger, Thomas E. "Accessing Information: The Evaluation Research." Collection Management 26, no. 1 (2001): 1-23.

Nisonger, Thomas E. "Cost Analysis of the LIBRIS II Automated Acquisitions System at the University of Texas at Dallas Library." Library Acquisitions: Practice & Theory 11, no. 3 (1987): 229-238.

Nisonger, Thomas E. "Theft and Staffing at a University Library Book Sale." Library Acquisitions: Practice & Theory 13, no. 4, (1989): 343-350.

Orkiszewski, Paul. "A Comparative Study of Amazon.com As a Library Book and Media Vendor." Library Resources & Technical Services 49 (July 2005): 204-09.

Stevens, Peter H. "Who's Number One? Evaluating Acquisitions Departments." Library Collections, Acquisitions & Technical Services 23 (Spring 1999): 79-85.

The Vendor Study Group. "Regional Study of Vendor Performance for In-Print Monographs." Library Acquisitions: Practice & Theory 16, No. 1 (1992): 21-30.

FOR APRIL 28: EVALUATION OF OVERALL LIBRARY PERFORMANCE and TECHNOLOGY AND EVALUATION

Cook, Colleen, Fred Heath, and Bruce Thompson. "'Zones of Tolerance' in Perceptions of Library Service Quality: A LIBQUAL+ Study." Portal 3, no. 1 (2003): 113-23.

Hennen, Thomas J., Jr. A Go Ahead, Name Them: America = s Best Public Libraries. @ American Libraries 30 (January 1999): 72-76.

Hennen, Thomas J., Jr. "Great American Public Libraries: HAPLR Ratings, Round Two." American Libraries 30 (September 1999): 64-68.

Hennen, Thomas J., Jr. "Great American Public Libraries: The 2004 HAPLR Rankings."American Libraries 36 (October 2005): 42-48.

LibQUAL Home Page on the Web: http://www.libqual.org/

Dresang, Eliza T., Melissa Gross, and Leslie Edmonds Holt. "Project CATE: Using Outcome Measures to Assess School-Age Children's Use of Technology in Urban Public Libraries: A Collaborative Research Process." Library & Information Science Research 25 (2003): 19-42.

Everhart, Nancy. "Technology." In her Evaluating the School Library Media Center: Analysis Techniques and Research Practices , 151-77. Englewood, CO: Libraries Unlimited, 1998. (ON RESERVE IN SLIS LIBRARY)

Lancaster, F. W. & Beth Sandore. A Automated Systems in Collection Management. @ In their Technology and Management in Library and Information Services , pp. 55-82. Champaign, Ill.: University of Illinois, Graduate School of Library and Information Science, 1997. (ON RESERVE IN SLIS LIBRARY)

Mathur, Mira. "Advent of Digital Libraries and Measuring Their Performance: A Review." DESIDOC Bulletin of Information Technology 25 (March 2005): 19-25.

Perkins, Gay Helen. "Will Libraries' Web-based Survey Methods Replace Existing Non-Electronic Survey Methods." Information Technology & Libraries 23 (September 2004): 123-26.

NOTICE REGARDING ACADEMIC HONESTY. Students are reminded that double submission of work for academic credit, fabrication, and plagiarism are serious academic offenses that can result in penalties up to and including failure on an assignment or failure in the course. According to the Indiana University Code of Student Rights, Responsibilities, and Conduct , they are defined as follows:

1. Cheating

g. "a student most not submit substantial portions of the same academic work for credit or honors more than once without permission of the instructor or program to whom the work is being submitted."

"2. Fabrication
A student must not falsify or invent any information or data in an academic exercise including, but not limited to, records or reports, laboratory results, and citation to the sources of information.

3. Plagiarism
Plagiarism is defined as presenting someone else's work, including the work of other students, as one's own. Any ideas or materials taken from another source for either written or oral use must be fully acknowledged, unless the information is common knowledge. What is considered "common knowledge" may differ from course to course.

• A student must not adopt or reproduce ideas, opinions, theories, formulas, graphics, or pictures of another person without acknowledgment.
• A student must give credit to the originality of others and acknowledge indebtedness whenever:
• Directly quoting another person's actual words, whether oral or written;
• Using another person's ideas, opinions, or theories;
• Paraphrasing the words, ideas, opinions, or theories of others, whether oral or written;
• Borrowing facts, statistics, or illustrative material; or
• Offering materials assembled or collected by others in the form of projects or collections without acknowledgment. "