Developing a Culture of Assessment: Student Affairs Model

The Division of Student Affairs has taken a comprehensive approach to assessment, from annual planning, to setting goals and assessing progress toward those goals, to ultimately using a culture of assessment for continuous improvement.

While staff members within Student Affairs have worked for decades to provide quality educational and social programs and services to students, the efforts have become more strategic in recent years based on the division’s ability to intentionally acquire and assess outcomes-based data and make significant decisions based on the results. The increased recognition garnered by the University’s undergraduate programs in recent years shows that a purposeful culture of assessment can provide significant results.

As the University entered a new era of planning and assessment in the mid-1990s, Student Affairs was one of the first to become actively engaged in assessing the impact of its programs. Reports documented student attendance at educational and social events, the number of tutoring sessions offered, and the number of patients seen at the University Counseling Center or Student Health Service; student feedback on these programs was gathered using targeted surveys. However, because different types of information were collected and the quality was inconsistent, these reports varied considerably across the division. In addition, the data collected and reported focused on inputs and quantitative outcomes such as the number of programs conducted during a term or the number of attendees at an event rather than the student learning and development that resulted from student participation in these programs. The reports were generally isolated to specific programming within each unit, with little regard to programming that was occurring in other sectors within the department, the division, or the University as a whole. The data were not linked to overarching goals of the division or the University for the undergraduate experience, and there was no systematic way of linking assessment to programmatic changes or development.

In 2005, the new vice provost and dean of students led a strategic planning initiative requiring that departmental goals be strategically aligned with specific divisional goals, including the overarching vision of “providing University of Pittsburgh students with the best collegiate experience in the world.” These divisional goals were aligned with the overarching goals of the University. With a clear understanding of critical University goals, such as the retention of students, the dean implemented a strategic planning and assessment initiative that started the process of developing a culture of assessment within the division. This shift led to the understandingthat it was no longer enough to simply create programs and activities for students; programs alone did not necessarily equate to enhancing the student experience. The staff now had to develop and assess programmatic and learning outcomes for everything they did.

During the development of a planning document, senior staff members were required to submit goals that would help to chart the direction of their units toward divisional goals. Directors were required to submit strategies for achieving these goals and, most importantly, intended measurement outcomes. In turn, division goals as well as department goals became intricately linked to performance goals for individual staff members. For example, part of the evaluation of a resident assistant was now based on the retention rates of students on his or her floor. This made resident assistants more deliberate about developing programs that would help their residents to connect to the residence hall community and the larger University community. The Reaching Inside Your Soul for Excellence (RISE) mentoring program, which seeks to retain African American students, is an other example of a program that was developed from outcomes-based data. The program’s effectiveness and viability is closely linked to the intended outcomes: retention, academic performance, and graduation rates.

These were only the first steps in creating a culture of assessment in the division. The following year, goals for student outcomes were introduced as part of the Council of Deans’ initiative on assessment of student learning. Now that the directors within Student Affairs had a taste of the new planning process, including a clear understanding of the importance of appropriate assessment and the methodology required to achieve it, the entire staff within the division could become engaged in the process. This would prove to be critical, as the University was in the process of developing a structured program to educate the whole student—the Outside the Classroom Curriculum. Spearheaded by leadership from within Student Affairs, this University-wide initiative required careful planning to establish 10 key goal areas and associated outcomes that were universally considered important in connecting students to the University, developing the whole student, and helping to position students for success.

For the past few years, directors have worked closely with staff to generatede partmental goals, strategies, and outcomes for the Student Affairs planning document. The Performance Impact Workplace software system has helped to establish goals and competencies for each employee, further enhancing the division’s commitment to assessment.

Other elements have nurtured the culture of assessment. For example, the planning document is visible to all and frequently used. Staff members review goals, strategies, and outcomes at the departmental level, and they share progress reports and celebrations of successes at divisional quarterly meetings. Throughout the year, directors are required to initiate and document assessment strategies in order to report on progress toward achieving goals on quarterly reports. In addition, directors periodically give departmental planning reviews at Student Affairs senior staff meetings and the senior staff retreat. During the formal creation of the new planning document, which generally occurs from November through February, there is intense scrutiny of goals, strategies, and especially outcome measurements and the assessment strategies that bring forth the data. Directors are engaged in the plan and challenge each other to ensure that the right goals, strategies, and outcomes make it to the final document.

The planning document is the clearing house for charting completion of goals or progress. In order for a goal to be considered reached, the stated outcome measurements must have been met or exceeded. In order to state that progress has been made, 75 percent of the measurement must be achieved. If progress has not been made, the goal and outcome measurement are usually carried over to the following year, and strategies for achieving the outcome measurement are evaluated and refined where necessary to ensure that it is met in the future.

The data used to measure whether a goal has been met come from a variety of sources. On a micro level, surveys are distributed directly to students at most student events. In addition, many departments have initiated annual surveys to assess student satisfaction with all of the programsand services offered. These include paper surveys and Web-based surveys that use Survey Monkey or Zoomerang. Data also are extracted from the Quality of Life survey distributed each year in the residence halls to approximately 7,000 students as well as the SERU survey and the UCSUR survey. Various focus groups have been conducted to garner information from students, and strategic benchmarking of other college and university student affairs departments and programs has been part of the process for the past five years.

Assessment is working in Student Affairs. In rating their overall social experience in an internal survey, the number of Pitt students who said they were very satisfied rose more than 10 percentage points between 2007 and 2010. By 2010, almost 99 percent said they were either satisfied or very satisfied with their Pitt social experience. And in 2009, the University celebrated a record retention rate of 92.7 percent for first-year students.

While Student Affairs has clearly developed a culture of assessment, additional assessment data are still needed for certain programs. For example, ongoing assessment of the Outside the Classroom Curriculum program by student focus groups was used to enhance the program in 2011. In addition, the next stage that is planned is to incorporate employer and graduate school admissions evaluations into the assessment mix to ensure intended outcomes of providing students with a competitive advantage in the marketplace or when applying to graduate school. By embracing the challenge of acquiring information of this nature, the University will continue to stay accountable to its mission of helping to educate the whole student and providing an outstanding collegiate experience.