Students at SJSU ischool need to demonstrate the ability to evaluate programs and services using measurable criteria I take this competency to include an information professionals ability to assess instruction practices, program assessments, usage surveys, data gathering and analysis practices for the purpose of improving products and services in information organizations. In my background as a teacher and education, I have often had to use measurable criteria to develop student-learning outcomes and evaluate instruction for effectiveness. However, my coursework at SJSU gave me the opportunity to enhance those skills and to apply them to information literacy, evaluating programs, and library services. The evaluation process in information organization can encompass a myriad of elements due to the broad and interdisciplinary nature of Library and information science. In the assessment process, an organization must according to Radcliff (2007), “gather data, analyze & interpret, plan for improvement, share data & plan, and implement the plan”.
Information organizations are often services oriented and goal driven as with many service-based business information organizations planning is required to keep performance at an optimal level. Information organizations need to be certain they are maintaining quality customer-service standards and providing the services that their community needs. Evaluating our services, performance, practices, information literacy instructions, and our collection to ensure it is in line with the needs of our patrons is imperative to being a valued part of the communities in which information organization operate. Information organization, for example, academic libraries often operate under the larger umbrella of institutional goals, missions, and values that they must align themselves with when considering their own goals. Organizational goals are often complicated and change over time as the populations they serve and the resources on which they rely upon change. Underneath main goals, there are often more short-term goals such as increasing the number of attendees at a particular event or reaching a circulation usage benchmark. In order to determine if short-term goals or long-term goals were met, they need to evaluate. Creating clearly defined and measurable goals aid the evaluation process easily determine what success looks like for the program or service were evaluating. The organization must perform assessments to determine how effectively each goal was addressed and if the changes implemented made an impact on the environment or community in which the information organization operates. As Evans & Alire (2013) state the goal of assessment is often to “improve the quality of our operations, services, and programs”. As information organizations, we want to operate optimally within often times constrained budgets while still providing the best possible facilities, programs, and services to our community. Assessment is important to creating and maintaining high standards and improving an organization.
Doing assessments regardless of whether it is on how well our ILI programs implement ARCL frameworks to how well our libraries serve teen populations information professionals must choose the appropriate tools for gathering information and criteria for determining success. In organization planning, the evaluation process should begin with measurable goals and in instruction; student-learning outcomes should be clearly defined and measurable. Instruction should also begin with some benchmark or prior skills assessment to determine what skill levels students began instruction with and an assessment for measuring how those skills changed or did not change due to instruction. The assessment process also can assist in as Radcliff (2007) points out, “identifying the gaps in our instruction and informing us as to what we can do to improve our instruction”(p. 11). Once the gaps in knowledge are identified, a plan is implemented to address those gaps. Some gaps can be easily identified or are observable such as not having enough study room for students and some are more difficult to ascertain such as are we meeting the information literacy needs of our students.
There are a number of different ways to collect the data needed to for the assessment process. User satisfaction, for example, can be measured by a variety of survey and feedback methods. Various services can be measured using tools such as Google Analytics for tracking website usage or database services such as EBSCO can provide usage statistics of databases for your institution that can track how often users and staff engage in certain interactions with systems. Comparing the costs of databases for example from various vendors or comparing the various types of packages offered by a vendor can be vital getting the most resources for your budget. Usage data can be an incredibly useful tool in determining which database our patrons frequently utilize databases and comparing that data to vendor packages can help information professionals maximize the resources they can provide while maintaining their budgets. For libraries, there are several tools and standards that exist that can help streamline the assessment process for the organization such as the LibQUAL survey and ARCL Frameworks for Information Literacy for Higher Education. Choosing the correct tool to gather data is crucial to ensuring the success of any assessment. Knowing what type of information is needed to evaluate programs and having clear objectives about what it is you are trying to assess is essential to make improvements to an organization. Gathering data on student work can happen in a multitude of ways. When assessing student work or student performance in an instructional setting there are several main types of assessments you can implement according to Radcliff (2007):
Formative assessment: in process; progress along the way; development Examples: In-class exercise or reflections on the process of learning.
Summative assessment: the final result; adding pieces of information together to get a sum Examples: final exams, surveys at the end of the course, reflection components of portfolios summarizing the value of a project.
Director authentic assessment: Characterizes assessment that is a result of direct contact with a student. This could be a product, such as a search strategy developed by a student in class, or a portfolio of student work, it also includes tests, annotated bibliographies, and papers.
Indirect assessment: Characterizes assessment information that has been inferred from observation or from surveys. It is subject to interpretation and relies on inferences.
My first piece of evidence is my assignment 8 from my INFO 282 course on Grant Writing and Alternative Funding Resources this assignment was to find a grant or mini-grant to either create a mock grant proposal as though you were a grant writer for a client in this assignment my client was a community college library. This grant proposal would include a letter of introduction, proposal summary, Organization overview, Statement of needs, Project description, Approach/methodology, budget requestion, evaluation process, and an appendix of any additional documents needed for the grant. For this assignment, I chose to create a proposal for a grant put out by the University of Maryland to assist with gaining Open Educational Resources to make textbooks more affordable for college students. In this project, I create an evaluation for determining if we successfully implemented the grant and for measuring and evaluating the success of utilizing the Open Educational Resources. I included this assignment as evidence of my ability to evaluate programs and services using measurable criteria.
My second piece of evidence is my extra credit assignment from my INFO 282 course on Grant Writing and Alternative Funding Resources. This was an extra credit presentation option that allowed students to present on a topic related to grant writing. The topic I chose for this presentation was working with grant clients on evaluation metrics. I noticed that a majority of my classmates struggled with creating evaluation metrics in our discussion posts for that unit. I chose to do a presentation on skills related to creating evaluating metrics. I included this presentation as evidence of my ability to effectively measure and evaluate programs and services using measurable criteria.
My final piece of evidence is my assignment #3 from my INFO 285 course on Research Methods in Library and Information Science Topic: Assessment of Information Literacy Instruction. This assignment asked students to create an assessment design project: Students will develop a small information literacy assessment tool for their user group (identified in the literature review) including writing learning outcomes. I chose to create a rubric to assess student e-portfolios generated from library and information literacy instruction taught in academic library settings. I used performance assessment techniques to determine if information literacy skills improving when ILI utilizes makerspaces.
Evaluating the services and instructional programs are crucial to making improvements to any institution. Using metrics for that evaluation that are measurable is important to ensure that improvements are occurring and that the correct changes are being made to meet the needs of our users. Understanding the evaluation process including how to utilize its tools and how to implement measurable criteria into an evaluation process is invaluable knowledge for information professionals. Information professionals can not improve services if we do not assess how effectively changes is implementing change and determine if that change was effective in addressing the gaps in service or knowledge. Information organization needs to perform optimally in order to stay competitive in this digital age and one way we can do that is by participating in the continuous evaluation of our services, programs, and practices. I enjoyed learning various evaluation tools and improving my ability to implement assessments for a variety of purposes. My time as SJSU has given me a great understanding of the assessment cycle as it applies to information organizations and it has broadened my understanding of assessment. Providing excellent service to patrons is of key importance to me and this knowledge will be applicable in my future as an information profession in a multitude of ways.
Evans, G.E. & Alire, C.A. (2013). Management Basics for Information Professionals. 3rd Edition. Chicago: Neal-Schuman.
Radcliff, C. J. (2007). Practical Guide to Information Literacy Assessment for Academic Librarians. Libraries Unlimited Incorporated.