Introduction

Joseph Matthews, as emphasized by Haycock & Romaniuk (2018), describes evaluation as a process for assessing various aspects of a library’s collections, programs, services, and resource utilization, including results, costs, outcomes, impacts, and successes. He further notes that evaluation plays a vital role in guiding decision-making at all levels when libraries implement evidence-based library and information practices. It is vital for planning and delivering library services and serves as the foundation for improving programs (p. 255). 

To gain insight into a particular organization, we focus on the evaluation of a digital library. Khoo & Giersch (2009) explain that effective digital library evaluation involves considering various organizational, methodological, and technical factors. However, challenges such as limited funding, time, and access to users can impact the evaluation process. The authors provide a checklist outlining practical steps for planning digital library evaluations, with a focus on developing a specific logic model. This checklist highlights four key activities: identifying strategic evaluation questions, selecting appropriate evaluation methods, securing stakeholder support, and ensuring sufficient resources. A key emphasis is placed on using strategic evaluation questions to develop a logic model and examining how these activities work together to support the evaluation process. The authors argue that digital library evaluation is a continuous process, where the success of later stages depends on careful preparation in earlier stages. By focusing on evaluation during the planning phase, including defining evaluation questions and setting a budget, future evaluation efforts can be significantly improved (p. 218).

Saracevic (2000) discusses how evaluations can offer valuable insights into enhancing learning, system performance, interface design, electronic publishing, and content quality, particularly in terms of scope and accuracy (p. 355). He explains that evaluation is the process of assessing how well a system, or part of a system, functions in relation to its goals. It examines how effectively the system performs its intended task, its efficiency in terms of cost or effort, or a combination of both. When conducting an evaluation, it is important to specify which aspects are being assessed. Saracevic points out that every system has objectives that can be organized in a hierarchy, with some levels potentially conflicting. These objectives, whether explicitly stated or assumed, must be factored into any evaluation. Evaluations are not uniform; they can vary in levels, objectives, and methods depending on the goals (p. 359-360).

Evaluations are crucial for a number of reasons. Matthews explains that they gather community input to help plan and develop future services, assess how well library services align with the organization’s broader goals, and provide insights for creating new services targeted at specific community groups. They also demonstrate accountability to governing boards, local officials, and the public by justifying budgets and services. Additionally, evaluations inform governing bodies about community needs and assess how well the library meets those needs. They help identify whether service objectives are being met, highlight areas for improvement, and compare the costs and benefits of various services to guide resource allocation. Evaluations provide staff with valuable information on the costs, benefits, impacts, and challenges of library services and help management identify areas for staff training. They also track services over time to generate trend data, allow comparisons with peer libraries, and offer data to advocate for the library. Lastly, evaluations check the effectiveness of existing services, programs, and collections, and prompt the organization to consider the short-term and long-term impacts on library users’ lives (Haycock & Romaniuk, 2018, p. 256-257).

Evidence

My first artifact to prove my mastery in competency N comes from INFO 282. As a reminder, Saracevic (2000) discussed how evaluations can offer valuable insights into enhancing content quality, particularly in terms of scope and accuracy (p. 355).In the course, we explored the concept of scope and scope creep. A project starts with a clear vision or scope, outlining the necessary details and costs to complete the final product. Scope creep refers to the gradual addition of new features that stray from the original scope, often due to unclear definitions or varying interpretations of goals among stakeholders. It can be caused by poor project management, unclear instructions, or missed deadlines. It is acknowledged that scope creep is inevitable when changes occur, and project managers should expect it. To manage this, project managers need to understand their stakeholders, maintain clear communication, and have a strong grasp of the client’s needs. Prioritizing tasks helps ensure the project stays on track and avoids unnecessary delays. Project managers should also anticipate small changes, adjust budgets when needed, and utilize project management tools and tech department support to stay organized. Lastly, project managers must be prepared to reject change requests when necessary and explain how they might impact the overall project.

My second artifact is an instructional lesson from INFO 250. I specifically will discuss the evaluation process of the instruction. For the lesson, I particularly placed formative and summative evaluations that will be implemented throughout the course to assess student progress. A summative evaluation will take place at the end of the course, allowing students to showcase their skills in database research. This final evaluation will include a research paper on a subject of their choice, where students must demonstrate that they have a library card and use it to access the local library’s community database. They will be required to locate at least three references, including at least one peer-reviewed article, to support their stance on their chosen topic. The references must be cited in APA format, including both in-text citations and a reference page. Various formative assessments will be conducted throughout the course to monitor student progress. Surveys will help identify whether students have a library card and previous experience with databases. Blank forms will assess their ability to complete formal documents, while worksheets will help students practice APA formatting and determine if additional review is necessary. I will also observe students as they navigate the database sites to ensure they can find the correct documents for their summative evaluation. If any difficulties arise, I will revisit the material and go over the steps again. Formative evaluations will be conducted using both print and web-based forms. They will need to submit proof of their library card number. Search techniques will be practiced on the library and database websites, while the summative evaluation will be completed in a Google Doc, which will be shared with me. I will provide feedback directly on the Google Doc file. Additionally, I plan to survey the students to gather feedback on whether the lesson needs to be condensed, explained in more depth, and how other students in different classes might benefit from the material. The results of the summative evaluation will provide insight into whether more focus is needed on writing skills, research techniques, or citation practices, allowing me to adjust and improve the course accordingly.

My final artifact comes from INFO 282 where I discuss Accuracy of Status. I would argue these reports act as evaluations. Status reporting is essential for managing the expectations of sponsors and stakeholders in a project. Communication plays a critical role in this process, with team members providing updates to the project manager, who then relays the information to management stakeholders. However, the accuracy of these reports can be problematic. Employees often tend to misreport or put a positive spin on issues to avoid negative perceptions or consequences, which can be influenced by personality traits, work climate, and cultural norms. Higher-level employees may be more likely to misreport due to fear of repercussions, which can result in a lack of honest communication. Despite these challenges, there are significant advantages to ensuring accurate status reporting. Accurate reports allow stakeholders to track progress and identify risks early on, which makes it easier to address problems or ask for assistance. They also help manage the project’s costs and budget more effectively. Accuracy in reporting provides better visibility into the project’s overall status, enabling project managers to make necessary adjustments and maintain control over the project. Furthermore, it encourages learning within the team, helping them gather insights for future projects and fostering motivation as they see the project progressing in the right direction.

Conclusion

In my teaching profession, evaluation procedures are used at every level. Students evaluate their teachers through surveys, though we sometimes question the accuracy of these evaluations, as some students may not take them seriously. Teachers also have the opportunity to complete evaluations to give insight and input for future in-service days and faculty retreats. I need to explore ways to improve and refine the evaluation process within my classroom to make it more effective. Upon finishing my coursework, I need to continue to review and study The Portable MLIS: Insights from The Experts. This proves to be an excellent source of information for understanding particular areas in the information field.

References

Haycock, K., & Romaniuk, M.-J. (2017). The portable MLIS: Insights from the experts, 2nd edition. Libraries Unlimited.

Khoo, M., & Giersch, S. (2009). Planning Digital Library evaluation with Logic Models. Evaluation of Digital Libraries, 217–234. https://doi.org/10.1016/b978-1-84334-484-1.50011-9 

Saracevic, T. (2000). Digital library evaluation: Toward an evolution of concepts. Library Trends. (49) 3, 350-369