Collections Principle 6

Thu, 04/17/2008 - 01:44 — admin

Collections Principle 6: A good collection has mechanisms for collecting data that measure use and usefulness.

Digital collections should be evaluated periodically to monitor usage, assess service effectiveness, demonstrate return on investment, inform collection development, inform strategic planning, and support funding requests. The criteria, methods, and metrics for evaluating collections will vary by the objectives of the collections and the purposes of the evaluation. For example, the collections of the National Science Digital Library are designed to support teaching and learning, so it is appropriate that evaluation measures focus on the educational impact of these collections.

Effective collection management employs a variety of research methods to assess collection usefulness. Observation, surveys, focus groups, interviews, experiments, case studies, and transaction log analyses have been used by digital libraries to assess usage and usability. Each method has its strengths and limitations. To obtain a clear picture of the value of a digital collection is to answer the question: “Who is using what, how, and why?” It is often necessary for collection evaluators to use a combination of methods and measures to answer this question effectively.

The use of the digital collection is closely related to the collection’s content, functionality, usability, and accessibility. Establishing benchmarks for use, collecting usage data over time, and following international standards for measuring use of digital content will enable collection managers to conduct longitudinal collection assessment and compare collection services with those provided by peers. Evaluation is an iterative process. Results of evaluation should inform the design and improvement of a digital collection.

Frameworks and guidance for evaluating digital collections:

Currently the COUNTER standards are dominant for measuring the use of digital collections, but they focus more on vendor-provided data than on collections produced by institutions.

Usage data are somewhat limited when considered alone. When combined with input measures, output measures, or instructional data, they can help shed light on the effectiveness of a digital collection or digital library. Google Analytics (http://www.google.com/analytics/) provides tools to track where users come from and how they use a website.

Resources on collection evaluation methods, standards, and tools:

  • Denise Troll Covey, Usage and Usability Assessment: Library Practices and Concerns (2002) http://www.clir.org/pubs/abstract/pub105abst.html. A good overview of research methods for studying collection use. 
  • International Coalition of Library Consortia (ICOLC), Revised Guidelines for Statistical Measures of Usage of Web-Based Information Resources (2006) http://www.library.yale.edu/consortia/webstats06.htm. Endorses COUNTER and SUSHI and provides guides for recording usage statistics. 
  • Managing Electronic Collections: Strategies from Content to User website http://www.niso.org/news/events/niso/past/Collections-06-wkshp/. Presentations from a NISO workshop held September 28-30, 2006 at Denver, Colorado. Presentations from day one “Understanding users and usage” and day two “Usage statistics wrap-up; practical collection and repository management” are especially useful. 
  • Association of Research Libraries, MINES for Libraries: Measuring the Impact of Networked Electronic Services website http://www.arl.org/stats/initiatives/mines. A web-based survey on user demographics and their reasons for using networked electronic resources. 
  • ANSI/NISO Z39.7-2004, Information Services and Use: Metrics & statistics for libraries and information providers--Data Dictionary (2004) http://www.niso.org/dictionary/. A data dictionary of terms pertaining to use metrics and statistics, includes measures for electronic resources; the main focus is on usage of resources in libraries. 

Some collection assessment studies: