Assessing the Admissions and Recruitment Processes

Throughout the recruitment season, OAFA monitors applications, admissions, yields, and scholarship/financial aid commitments and updates predictive models using these data. Admissions reports are reviewed weekly by the deans and Office of the Provost. At the appropriate times during the recruitment cycle, more comprehensive reviews are conducted, and decisions regarding midcourse adjustments are made in response to these assessments. On the Pittsburgh campus, for example, a comprehensive review is conducted in early December and updated at the end of January and again at the beginning of March, and any major changes to recruitment strategies are contemplated at those times. Similar reviews occur later on the regional campuses, given their recruitment cycles. Copies of the weekly admissions and profile reports are available in the document room.

The outcomes of the recruitment cycle are assessed annually against the established goals in terms of overall number of students recruited, yields, and the characteristics of the class. The primary measures used to assess progress toward recruitment goals include measures of academic aptitude and achievement, including average standardized test scores and high school performance; geographic diversity, including the percent from outside Pennsylvania and the percent from abroad; ethnic/racial and, for some programs, gender diversity; and other measures of student characteristics drawn from responses to the student surveys, such as the CIRP Freshman Survey. Pitt–Johnstown, for example, given its focus on connecting with the real world, benchmarks the average number of hours of community service in high school. Given the demographics of its recruitment area, Pitt–Bradford monitors the percentage of students who are the first in their family to attend college in order to guide the planning and delivery of the academic and support services best suited to the needs of these first-generation and other enrolled students. The Pittsburgh campus considers interest in attending graduate and professional school.

These data are benchmarked against peer and aspirational peer institutions when possible; they are reviewed annually to assess whether the recruitment efforts successfully met goals and to help shape recruitment strategies in coming years. This annual monitoring of progress helps to keep the admissions process focused on longterm goals and provides indications of whena djustments need to be made.

There also is an annual review of the recruitment process that includes a detailed analysis of applications, admit rates, yields, and the admissions processes and strategies. These annual reviews are informed by responses to student and parent surveys (from students who chose to attend the University and those who did not), matriculation patterns of those who did not choose to attend the University of Pittsburgh (from student surveys and using data from the National Student Data Clearinghouse), reviews of specific strategies, and insights from reviews conducted by expert consultants. Based on these annual assessments, goals and strategies are changed or modified for the next recruitment cycle. Freshman profiles, application and yield analyses, responses to surveys, and internal and external reviews of strategies are available in Appendix C22.

Finally, the admissions guidelines developed by the schools and campuses are reviewed periodically. As mentioned earlier, each school and campus develops admissions guidelines in which it articulates those attributes that it believes characterize the students who will be most successful in its programs. The validity of these guidelines as indicators of the fit between the student and the campus is assessed by examining the relationship between the guidelines and student success measured by retention, graduation, and student satisfaction. On the Pittsburgh campus, these reviews are conducted by individual schools and OAFA and also are part of the comprehensive assessment of student success conducted by UCSUR and discussed in the section on assessment of student learning outcomes. Similar analyses also are conducted on the regional campuses. For example, a review at Pitt–Greensburg indicated that high school performance (measured by rank in class) was much more important to the success of students on that campus than SAT scores. This led to a modification of the admissions guidelines to put more focus on high school performance.

Below are several specific examples of how planning and assessment have helped the University to build strong recruitment and awarding programs.

Selected Examples of How Assessment Has Led to Improvements in Recruitment and Admissions Processes

The Pittsburgh campus used student feedback to shape programs and recruitment strategies. Critical to success in effectively attracting students to the University is an understanding of who enrolls and why as well as who chooses not to enroll and some of their reasons. In fall 2000, more than 80 percent of the freshmen on the Pittsburgh campus responding to the CIRP Freshman Survey indicated that they planned to seek advanced degrees. Using this information, OAFA identified an opportunity to build additional partnerships with the graduate and professional schools so that a select population of admitted freshmen now is eligible for guaranteed admission to a graduate or professional school of interest. These guaranteed admissions programs for graduate/professional school have attracted large numbers of qualified applicants with a strong interest in advanced degrees.

These same surveys, along with a review of internal data, indicate that students who attend a campus program are much more likely to choose to attend Pitt. While this is a useful piece of information in assessing on-campus programs, it also has guided the University’s strategies to encourage campus visits. The increased number of students visiting campus and the increasing yields on these students suggest that these strategies have been successful.

OAFA also routinely asks students who do not accept an admissions offer to report where they chose to attend college and the key factorsin their decision to attend that institution. This information also has been useful in helping the University to assess its recruitment strategies. For example, in the 2011 OAFA survey, one of the issues that arose was a lack of access to faculty in the applicants’ intended areas of study. In response, OAFA is creating a new system to better connect students and parents with faculty members in their area of interest and will assign an OAFA staff member to follow up to ensure that the connection is made and that all of the student/parent questions have been answered.

Assessing Marketing Strategies: Pitt–Bradford and Pitt–Johnstown

Ten years ago, Pitt–Bradford was using an untested tagline, “Pennsylvania’s Public Liberal Arts College,” in its marketing and recruitment efforts. An external consultant was hired to survey potential students regarding institutional re call and affinity for the tagline; it was determined to be not only ineffective but, in somecases, a disincentive to a population of students being recruited. That tagline was eliminated, and a different consulting firm was engaged to conduct research and develop a new recruitmentand image campaign with the new Pitt–Bradford tagline “Beyond.” Implementation of that new campaign played a critical role in achieving the 1,500 FTE enrollment target two years ahead of schedule. A reassessment of the current campaign’s effectiveness in mid-2010 resulted in Pitt–Bradford’s decision to continue with its integrated marketing plan.

In response to demographic changes affecting the region, Pitt–Johnstown set a goal of increasing the number of prospects, applicants, and enrolled new students from outside its primary draw area. Internal trend analysis and demographic assessment by an external consultant identified counties that provided the greatest potential for expansion. Recruiting strategies were adjusted; the use of advertising, media, and communications was changed or new initiatives were started; and admissions counselor travel patterns were modified. Over a five-year period, applications and yields increased, and three Pennsylvania counties that had been considered tertiary markets for Pitt–Johnstown advanced to secondary markets.

Quantitative Assessment of Admissions and Awarding Strategies

The University regularly employs the services of a leading enrollment management consulting company, Scannell & Kurz, Inc. (S&K), to assist in the assessment of its admissions and awarding practices, including those regarding use of scholarship awards. S&K has provided comprehensive reviews of the admissions anda warding strategies on all five campuses as well as selected graduate and professional programs on the Pittsburgh campus. These reviews provide useful insights into the strategies, programs, marketing materials, and back-office operations used by the schools and campuses. S&K also reviews the admissions and awarding data for the Pittsburgh campus on a regular basis and provides recommendations that are considered as part of the campus’ annual assessment of its freshman admissions activities.

A comprehensive review of the Pittsburgh campus resulted in new awarding strategies for the freshman class of fall 2004, and the results of the recruitment efforts that year provide evidence of the effectiveness of the new strategy. The average SAT score of the enrolled freshman class improved from 1214 the previous year to 1233, and the tuition discount rate for enrolled freshmen decreased from 22.7 percent to 17.7percent. Annual reviews of the admissions data and appropriate adjustments in recruiting goals and strategies in light of the general recruiting environment (demographics, national economy) have proved it to be a successful strategy. Since 2004, the discount rate remained steady at 18 percent; the profile of the class has continued to improve, with, for example, average SAT score increasing to 1273 in fall 2010; the percent of underrepresented and international students has increased from 15.83 percent in 2004 to 18.45 percent in 2010; and the percent of out-of-state residents increased from 17 percent to 25 percentfor the same period.

Pitt–Greensburg provides another example of the success of this approach of analyzing previous years’ admissions data with respect to academic qualifications, geographic origin, and ethnicity to identify target profiles, estimate the impact of financial awards, and evaluate the success of awarding practices. Over the past five years that this strategy has been inplace, the academic profile of the freshman class at Pitt–Greensburg has improved, with the percent of students in the top fifth of their high school classes increasing from less than 20 percent in fall 2006 to nearly 30 percent in fall 2010. Over the same period, there presentation of underrepresented students has increased from 9.1 percent to 13.3 percent. Efforts to increase the proportion of students coming from outside the primary market area have met with limited success, as the proportion of students from outside Allegheny and Westmoreland counties has increased only slightly more than four percentage points. With this in mind, the campus refocused its efforts and, for fall 2011, anticipates an increase in the percent of students from outside the primary market area to increase to 10 percent.