Tools for Postsecondary Schools: Evidence-Based Strategies for Equitable Student Success

Written by: Lynneah Brown and John Lane, State Higher Education Executive Officers Association

Implementing and sustaining evidence-based comprehensive student support strategies continues to grow in importance as an effective way to help close the success gap for students from low-income families and students of color. Challenges that were facing students before pandemic conditions are now exacerbated and compounded, with a direct impact on student enrollment, persistence, and attainment. The impact of the pandemic also has prompted unprecedented solutions, including historic resources such as HERRF funding (the Higher Education Emergency Relief Fund) in the American Rescue Plan that institutions can use for student support services. The federal Postsecondary Student Success Program and its proposed expansion in the FY23 appropriations bill also demonstrate a commitment to funding support for postsecondary institutions, which has historically been left to the states.

This year, MDRC, in partnership with SHEEO, launched Tools for Postsecondary Schools (TfPS), a technical assistance initiative for 10 state higher education offices and cohorts of the states’ two-year and four-year institutions. The purpose was to help states and institutions implement and scale the essential components of evidence-based comprehensive student support strategies that mitigate attainment disparities. TfPS included: 1) a monthly workshop series for institutions covering topics such as coaching and advising, financial incentives, using data for continuous improvement, and planning for program sustainability; and 2) learning community events for the 10 state higher education offices, designed to provide state policy models that advance scaling and sustaining student support services statewide.

Participating States and Institutions

In the fall and winter of 2021–2022, SHEEO and MDRC engaged 10 state higher education executive officers (“SHEEOs”) for the initiative: California, Hawaii, Indiana, Louisiana, Massachusetts, Minnesota, Montana, New Jersey, Ohio, and Texas. These states represent a mix of governance models (coordinating, governing, policy), regional locations, populations, and policy interests, among other factors. SHEEOs in turn identified three to five institutions to participate in the workshop series, based on criteria and priorities specific to their states, with a total of 32 institutions across three cohorts participating. Institutions convened monthly for the workshop series and completed action plans, and participated in optional coaching, among other activities.

Learning Community Events for State Higher Education Agencies

MDRC and SHEEO hosted two learning community webinars for the higher education agencies in the 10 states to complement institution-level engagement. States have an important role to play in developing statewide policies to support the implementation and scaling of evidence-based student success strategies for several reasons:

• Seventy-five percent of all postsecondary students attend public or state-supported higher education institutions, providing for maximal policy impact.
• Appropriations and other postsecondary funding coordinated statewide can be tied to policy implementation and outcomes.
• States’ higher education agencies regularly coordinate state attainment agendas and can most effectively integrate statewide student support policy development. Additionally, attainment agendas integrate with state workforce development, which relies on constantly improving and sustained completion and placement goals to support state economic development.
• Coordination provides institutional alignment and a consistent statewide policy framework for students who start college at one institution and finish at another.

The webinars featured examples of state funding and implementation for student supports in two participating states: Massachusetts (the Department of Higher Education, DHE) and Montana (the Montana University System, MUS). Their higher education offices described how they successfully secured state resources for student supports and how they developed programs that meet the unique needs of their student populations. These two states provide models for other states across regions, governance models, and levels of support, to scale and sustain strategies.

Massachusetts

Overview of SUCCESS Program

The Massachusetts Department of Higher Education (DHE) is in the final approval stages of a 10-year strategic plan for racial equity to respond to the greatest disparity the department’s analysis of data has identified. Recent state funding to DHE for community colleges has been awarded to provide wraparound support and services to improve outcomes for the most underserved populations including low-income, first-generation, minoritized, students with disabilities, and the LGBTQIA community. These services include programmatic components such as peer mentors, academic skills workshops, field trips to four-year universities, targeted academic, career, transfer, and scholarship advising, among other vital supports.

Funding

In September 2019, the Massachusetts Association of Community Colleges (MACC) described a vision for a coordinated effort to build on the documented success of existing grant-funded programs (e.g., TRIO, Title V) by providing state funding for academic advisors, learning coaches, and peer mentors for all CC students. At the time, the Massachusetts legislature was already considering a funding plan to meet MACC’s call to action. The legislature responded to this urgency by allocating to the DHE $7 million in funding in fiscal year (FY) 2021 and $10.5 million in FY22 to fund MACC’s SUCCESS (Supporting Urgent Community College Equity through Student Services) program. For FY23, MACC and the DHE secured $14 million to support the program. Throughout the process of creating these programs, there has been collaboration among the 15 community colleges, the DHE, and the legislature to help secure the funding. The money was provided for this program due to the influx of funds being provided, but the importance of the message of needing this program was key. Funding helped to put into place needed personnel to provide a myriad of services to students, including coaching, academic advising, and/or case management services. The DHE strategic plan for racial equity will seek additional funds for support.

Institutions and Students

The program serves 15 community colleges within the state of Massachusetts, and over 60% of the students are considered students of color who attend public two-year colleges.

Program Development & Effectiveness

In the process of institutions developing a SUCCESS program on their campuses, there were three questions that campus leaders needed to consider to uphold an effective program: 1) What existing successful practice(s) at your college will you scale up through this initiative? 2) What specific cohorts of students will you be serving? and 3) How will you measure success?

The CCs have drawn from their own experience in customizing supports to best meet the needs of their specific student populations and to fit within their own campus contexts. As a result, each campus has its own mix of students, and the programs developed may look different at one institution versus another. For example, Middlesex College has a Success Scholars Program, and this program focuses on full-time and part-time African American, Asian, Latino, and LGBTQIA students. Northern Essex Community College has a SOAR (Seize Opportunities, Aspire to Rise) Program that builds on the success of their TRIO program, which primarily serves low-income, first-generation Latino students who have at least a 2.0 GPA and have earned at least six credits. Early outcomes for the SUCCESS initiative include over 4,000 students being served in fall 2021, of which 62% were Black and Hispanic students. Overall, 72% of students were retained from fall to spring which puts us on track to meet or exceed the latest fall-to-fall retention rate of 58% for first-time, full-time, degree-seeking students at state-funded Massachusetts community colleges.

DHE Support Services for Student Success

The Department notes that support services for students is a fairly large initiative, prompted not only by raising the inquiries about scale, cohorts, and success measures, but also through assessing the current work in student support services in the state’s nine public universities and 15 community colleges. In addition, DHE compared support services trends in other states, which helped the Department construct the recommendations in use for this program. The processes already in place guided their work and included: 1) assessment of the current state of student success, strategies and procedures (e.g., engaged over 300 stakeholders via interviews and focus groups and completed an analysis of system data); 2) national trends (e.g., conducted peer state review for data comparison and trend assessment); 3) development of key components of the strategic framework; and 4) finalized implementation options, recommendations, and communication (e.g., continued socialization of framework with stakeholders).

From the recommendations, the Department developed four goals for the public higher education system to enhance economic and social mobility for all citizens, mainly students of color:

1. Sixty percent of working-age Massachusetts residents (ages 25–64) will hold an associate degree or higher and an additional 10% of the population will hold a high-quality credential by 2030.
2. Forty-three percent of African American and 32% of Latino Massachusetts residents ages 25–64 will hold an associate degree or higher by 2024.
3. By 2030, the rate at which the Massachusetts public higher education system will graduate African American and Latino students will increase to 51% and 50% respectively, outpacing the current rate of increase by 10%.
4. DHE will also track and report on racial gaps in first-year success metrics incorporated into the performance measurement reporting system, including completion of college-level math and English in the first year, on-time credit accumulation, and persistence to a second year of postsecondary education.

To meet these goals, academic advising is not the sole support. Support starts at college entry by developing early momentum, providing financial support and removing basic need barriers, aligning purpose and program, providing a clear pathway with transfer support, and post-credential planning and engagement.

Collaboration

There has been collaboration across the board, not only with funding, but for the development of the program. SUCCESS represents an innovative, whole-system collaboration among the 15 Massachusetts CCs, MACC, and the DHE. SUCCESS is overseen and governed by a leadership team that consists of college presidents, DHE staff, and other staff from each campus, as well as a cross-agency coordinating committee that includes representatives from MACC, DHE, and each of the 15 community colleges in the state. In addition, a team of success administrators from each campus meets monthly to discuss the day-to-day operations and program issues.

Montana

Overview of Montana 10

The Montana University System’s Montana 10 initiative is a comprehensive student support program that provides financial, academic, and social supports to low-income, rural, and American Indian students at both two-year and four-year colleges across the state. Montana 10 enhances the state’s skilled workforce by dramatically increasing the number of Montanans who have two-year and four-year degrees and does so at a lower cost per degree for Montana taxpayers.

Funding

Montana 10 is currently funded through unearned dollars from the system’s state-general-funded performance-based funding allocation. The state’s performance funding model was developed as a shared policy goal between the Office of the Commissioner of Higher Education and the Montana legislature in 2014 and includes metrics intended to incentivize student success. By directing unearned dollars to the initiative, Montana 10 supports the system’s and state’s intended goals of increasing retention and completion.

Institutions and Students

The MUS is largely an open-access and fully comprehensive system with a total of 16 campuses. The campuses include two-year colleges with as few as 300 full-time equivalency (FTE) and research one (R1) and doctoral offering institutions with as many as 15,000 FTE (please see FTE enrollment dashboard for more information). Montana 10 is a pilot program that was implemented in the fall of 2020 and includes 200 students across three institutions (e.g., one flagship institution and 2 two-year colleges), the University of Montana, Helena College, and Missoula College. Montana 10 is intended to address equity gaps in college attainment, particularly for low-income, rural, and American Indian students. To be eligible, students must be Pell-eligible, first-time, Montana residents.

Program Development & Effectiveness

The purpose for creating Montana 10 was mainly due to the dropout rates for both Pell students and all other students, as well as the dropout rate of students with debt. According to MUS, 60% of Pell students drop out compared to 50% of all other students. Yet, of those, a far greater share of Pell students who drop out do so with debt. Fifty percent of Pell students dropped out with debt (e.g., average debt $8,840) compared to 29% of all other students who dropped out with debt (at an average of $9,273). To address this problem, MUS designed Montana 10, a comprehensive approach that provides support for the challenges that most commonly cause students to drop out, including: 1) the need for financial support, 2) academic under-preparation, and 3) the “hidden” curriculum or the social and cultural knowledge that underserved students frequently have not been provided with but that is necessary to successfully navigate higher education. Montana 10 increases retention and completion rates within the state by providing supports in these three areas. Specifically, Montana 10 provides: 1) financial supports, including scholarships, textbook stipends, and monthly incentives; 2) strategies to help students gain academic momentum, including full-time schedule, corequisite math and writing, and mandatory tutoring; and 3) activities that help students develop a sense of purpose and belonging, including personalized advising, career development, orientation, and freshman seminar.

The initial pilot cohorts of Montana 10 have proven successful. College completion has dramatically increased. Early outcomes show a 16-percentage point increase in retention over a matched sample group. If trends hold, as all signs are pointing toward, the gains in college completion mean that even with the up-front cost of the program, the state will realize a better return on investment because the cost per degree is lower. Below are some outcomes of the program:

• For fall 2020 to fall 2021 retention, the state saw a 16-percentage point increase in retention for their Montana 10 students compared to their matched sample cohort, and a seven-percentage point gain over all other incoming freshmen in the same cohort.
• Montana 10 students saw significant gains for underrepresented groups and for progress toward a degree. During the second year of the program, American Indian students in the Montana 10 program were retained at a rate of 68.8%, compared to 39% of other American Indian Pell-eligible students in the same cohort. Montana 10 students earned nine more credits per year on average than their Pell peers. Gains in credit accumulation will lead to higher on-time completion rates, less debt for students, and a lower cost per degree for the state.
• 73.5% of the Montana 10 scholars were still enrolled in spring 2022 compared to only 50% of Pell-eligible students.
• In spring 2022, the state saw its first cohort graduate and found graduation rates nearly doubled compared to on-time graduation rates for two-year colleges across the entire MUS.

Collaboration

With the backing of promising early results, the Montana Board of Regents has supported expansion of the program. Montana 10 will continue to grow, bringing on a new four-year regional college and a new two-year college (Montana State University Billings and City College) in fall 2022 and two additional four-year colleges in fall 2023 (Montana Technological University and University of Montana Western). By fall 2024, seven MUS institutions will be running Montana 10 and the program will be bringing approximately 500 low-income students into the program each fall, representing nearly half of the Pell students at those campuses. The MUS will continue to work with the Montana legislature on supporting efforts, including Montana 10, to increase the number of Montanans who hold college degrees. The MUS will continue to work with MDRC to conduct a rigorous randomized control trial of Montana 10 starting in fall 2023 to assess its effectiveness as the program scales up. The state continues to push for unimpeachable evidence that that Montana 10 is producing the impacts that are intended at both the state and system levels.

Implications

Implications from the webinars featuring Massachusetts’ SUCCESS Program and Montana 10 are:

1. Essential to scaling equity-based student support strategies statewide is each state defining equity for its circumstances. For Massachusetts, the data revealed disparities by race and ethnicity; therefore, interventions are designed to support students of color. Montana, on the other hand, differs, with a large landmass/small population profile with a significant portion of students in the MUS from tribal communities or who identify as American Indian. Providing higher education opportunities in a largely rural state where historically rural, first-generation, low-income, and American Indian students have a lower likelihood of earning a college degree led to defining equity-based measures that help mitigate disparities for these students.
2. With strategic collaborations between key stakeholders, opportunities exist to create state- and institution-level student support strategies that are adaptive, address historic conditions, and support closing success gaps despite unprecedented challenges. Offering strategic student supports and responding with a comprehensive transition of courses and services online demonstrates that institutions and state higher education systems were able to adapt quickly for students, communities, and their own continuity.
3. State attainment goals are best served by evidence-based strategies responding to available data and the individual needs of students in their local contexts, and not one-size-fits-all approaches. Tools for Postsecondary Schools relies on proven strategies practitioners and researchers teach that are reliable and adaptable for success. These strategies are also scalable and sustainable as they counter inequities by breaking cycles of inadequate student support.

Acknowledgements

SHEEO thanks the MDRC team and state representatives from both Massachusetts and Montana for their contributions to the presentations and the SHEEO Medium post:

• Katie Beal, external affairs associate for development and external affairs, MDRC
• Melissa Wavelet, senior fellow for postsecondary education, MDRC
• Crystine Miller, director of student affairs and student engagement, Montana University System
• Lutful Khan, success senior project director, Massachusetts Association of Community Colleges
• Carlos Santiago, former commissioner of higher education, Massachusetts Department of Higher Education
• Clantha McCurdy, senior deputy commissioner for access and student financial assistance, Massachusetts Department of Higher Education
• Brock Tessman, deputy commissioner for academic, research, and student affairs, Montana University System