“The Assessment, Evaluation, and Research competence area (AER) focuses on the ability to use, design, conduct, and critique qualitative and quantitative AER analyses; to manage organizations using AER processes and the results obtained from them; and to shape the political and ethical climate surrounding AER processes and uses on campus.”
(ACPA & NASPA, 2010 p.8)
Assessment is crucial as it holds us accountable for student learning (Schuh, Jones, & Harper, 2009). Within my coursework, I was able to gain knowledge of the best practices in conducting assessment and research, and of the differences between assessment and evaluation. While assessment focuses on whether or not a program's learning outcomes were achieved, evaluation is a means of determining whether the methods of achieving those outcomes were as efficient as they could have been (Creswell, 2014). Having a comprehensive understanding of these differences and of best practices in research methods provides a means of ensuring that the programs and services offered have value and are achieving desired results.
Culp and Dungy (2012) empathize the idea of institutions building a “culture of evidence,” that is, creating an environment that offers student affairs professionals “opportunities to examine their work; make it more effective and efficient; and increase the probability that they will design and implement programs, processes, and services that really matter” (Culp & Dungy, 2012, p. 1). As a Graduate Retention Fellow in Student Involvement and Activities at Salem State University, I was responsible for conducting a retention study within my office. This purpose of this study was to identify the impact employee-sponsored leadership programs (in-services) had on a student's academic success. I reviewed current literature regarding leadership programing and student employment and created a report that identified best practices for institutions to utilize when designing their employee leadership programs (see right). In order to prepare for this study, I had to complete human subjects training through the Collaborative Institutional Training Initiative (CITI). This training addressed such issues as involvement with and awareness of the functions of the Institutional Review Board (IRB), when there is a need to obtain consent from research participants, and conducting research that is moral and ethical.
Within the course Research Methods in Student Affairs, I was introduced to different types of research and the different ways assessment and evaluation can be utilized, whether it’s assessing the quality of a program or information obtained during a lengthy research study. Two assignments, one a qualitative critique and the other a quantitative critique, allowed me to examine in detail the quality of research articles and evaluate their effectiveness. Sample questions from these assignments included:
-
Is the abstract representative of the article and in the correct form?
-
Are key terms identified and explained?
-
Has the author been objective in his/her discussion of the topic?
References
ACPA & NASPA. (2010). Professional competency areas for student affairs practitioners. Washington, DC: Authors.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Thousand Oaks, California: SAGE Publication
Culp, M. M. & Dungy, G. J. (Eds.). (2012). Building a culture of evidence in student affairs: A guide for leaders and practitioners. Washington, DC: NASPA -- Student Affairs Administrators in Higher Education
Schuh, J. H., Jones, S. R., Harper, S. R. (2011). Student services: A handbook for the profession. 5th edition. San Francisco, CA: Wiley and Sons.
Having continuous assessment allows us to identify effective services and programs that are aligned with the mission statement of an institution. Without such methods, offices and departments have no way of being held accountable for their time and resources and could end up putting funds into a program that doesn’t actually have a lot of impact. Everything we do in the field of higher education is in support of students, and it is therefore important that we as professionals build cultures of evidence that demonstrate how our programs and services are being effective with meeting student needs.
The course Assessment and Evaluation in Student Affairs complemented Research Methods as it enabled me to focus more specifically on the assessment application process. Along with a colleague, I was able to design an Assessment Action Plan for Salem State's internship programs (see below). The purpose of the assessment was to identify what professional development was currently taking place at a student's internship site and to determine how/if that development adhered to the standards set for internship programs by the Council for the Advancementof Standards (CAS). Conducting this study provided me with the opportunity to develop goals and learning outcomes based off of Bloom's Taxonomy of Learning Domains. Additionally, I was taught best practices in creating consent and disclosure forms, survey design and implementation, and submitting an IRB proposal.