Ensuring students are prepared for success in college includes many factors but one critical factor is access to rigorous academic coursework. In collaboration with researchers from the Decision Center for Educational Excellence we partnered with five Arizona School Districts to understand high school course-taking trends in Advanced Placement (AP), Dual Enrollment and International Baccalaureate (IB). From the onset, we wanted to understand who takes rigorous courses across these five districts and to disaggregate the data to examine course taking from an equitable lens. Second, we wanted to understand the impact of these courses on college-going and college persistence. Simply put, did they impact student college-going behaviors.
This initial brief is meant to provide greater context on who takes rigorous courses and the extent to which it impacts college-going. As the reader will see we ran into challenges that forced us to modify some of our analyses. One major challenge that emerged centered on how the districts captured dual enrollment on high school transcripts. Not all districts had processes in place to fully capture who was in dual enrollment courses. As a result, we can only share limited data on who took dual enrollment courses. A second challenge that emerged was the limited numbers of students taking IB courses. This is not surprising since IB is typically a whole school program and only a few schools in these five districts have made that choice. With smaller numbers of IB students we lacked the power to examine impact of IB on college-going or persistence as well. Therefore, our analyses on impact as it relates to college-going and persistence focus on AP course-taking.
The goal of this brief is to inspire a call to action among Arizona’s stakeholders to improve educational opportunities for all students in the state so that we can ensure that every student, regardless of gender, zip code, race or ethnicity, receive a high-quality education that will lead him or her to college if they so choose.
For more than a decade, educational leaders have sought to increase the proportion of students who graduate high school academically prepared for college. Yet, many students continue to graduate high school without having those necessary skills. For example, in ACTs most recently released Condition of College and Career Readiness Report only 26 percent of students were adequately prepared to succeed (ACT, 2019). Similarly, the National Assessment of Educational Progress (NAEP) reported that 37 percent of the nation’s 12th grade students were prepared in Reading, 25 percent in Mathematics, and 22 percent in Science (NAEP, 2015). These numbers are troubling as numerous studies have confirmed that both college attendance and college completion are strongly correlated with academic preparedness.
The economic advantage of having a college degree in both the current and future workforce cannot be overstated. According to the most recently reported data from the U.S. Bureau of Labor and Statistics (2017) the median wages for a person in the United States with a high school diploma and no college is $712 a week. Comparatively, a person with an associate degree makes $836 a week, while an individual with a bachelor’s degree makes on average $1,173 a week. Why is this important? Because a person in the 25th percentile with only a high school diploma would earn about $867,500 in their lifetime, while a person with a bachelor’s degree would earn nearly double at $1,490,600. This means that those individuals with a degree have much more opportunity when it comes to things like home ownership, deciding where they want to live, or how long they might have to work until they can retire.
Over the last two decades one way that educational reformers have tried to prepare more students for the academic rigors of college is by introducing them to what we call “rigorous courses.” This includes AP, Dual Enrollment and IB courses. All of these courses can be taught directly on a high school campus with both AP and IB having an aligned assessment that students take at the end of the course. To receive college credit for either one of these courses a student must score high enough on the assessment. Unlike AP and IB there is no specific standardized assessment for dual enrollment courses. In a dual enrollment course students typically earn credit with a high enough course grade.
A number of prior studies have found positive relationships between rigorous course taking and specific outcomes. But generally, few studies have been able to examine course taking across multiple districts within the same state and across different types of courses (AP, dual enrollment and IB). To better understand how rigorous courses are impacting Arizona students we partnered with Mesa Public Schools, Tempe Union High School District, Tucson Unified School District, Phoenix Union High School District, and Yuma Union High School District to analyze student-course taking patterns and the impacts of such courses. Our goal was to understand three specific questions: Who has access to rigorous courses? Is that access equitable? What impact do rigorous courses have on college-going and college persistence?
This brief has four main sections. In Part I we examine statewide trends in AP course taking since the early 1990s. We focus this effort on AP because it is the most prevalent and has the longest history in the state. In particular, we discuss the growth of AP course taking, the increase in test-taking and the expansion of these courses for low-income and minority students. In Part II, we examine who takes rigorous courses across the five Arizona districts we partnered with for this study. Specifically, we examine how many students took each type of rigorous course in each of the districts. In addition, we disaggregate the data by race/ethnicity, free/reduced lunch status and gender. Part III analyzes the impact of AP course-taking on college-going and persistence. Finally, Part IV highlights a set of recommendations for educational leaders, policy makers and opinion leaders on how to improve rigorous course taking to increase opportunities for students in the state.
PART I: STATEWIDE AP TRENDS
Advanced Placement courses from College Board are the original source of rigorous courses in Arizona. These days, some high schools may have an International Baccalaureate Program or a Cambridge program; even more schools have dual enrollment through their local community colleges or universities. But the longest running and largest offerings of rigorous courses is through AP. Using data available through the Arizona Department of Education (ADE), we matched student-level AP course taking, AP test-taking, test scores and student characteristics to create the first comprehensive profile of Advanced Placement course-taking and test participation in Arizona.
In this section we present data on 4 main questions:
- How many students are taking AP Courses and Examinations in Arizona?
- How do course and test-taking vary when we disaggregate the data?
- What can we learn from descriptive data on AP courses and exam-taking and college going?
- Is it important to take the AP test?
Part I: Section 1: How Many Students Are Taking AP Courses and Examinations in Arizona?
In 2019, 53,746 Arizona students enrolled in 95,961 AP courses. Of those who took AP courses 26% were 12th graders, 42% were 11th graders, 23% were 10th graders, and 9% were 9th graders. The most popular courses were English, US Government, US History, World History and Calculus.
Student Data on AP Courses and Test
In 2018 (the most recent year available from ADE), 34,779 students took 62,281 AP tests. This is significantly less than the AP course takers because AP test-taking is not required to take the course. Eleventh and 12th graders took a majority of the tests. Slightly over three-quarters of the test-takers came from grade 12 (34%) and grade 11 (42%), with the rest from grade 10 (17%) or grade 9 (6%).
Ideally one would take the course taught using the curriculum provided by the College Board and then take the test designed to measure that curriculum. But that is not universal. Looking at student records, between 2014-2018, of the 26,000 who took AP courses & tests each year, around 58% took both the course and the test, 38% took the course but not the test, and 4% took the test but not the course.
Why Wouldn’t Course Takers Take The Test?
Why wouldn’t course takers take the test? Reasons might include the following: inability to pay the $93 fee (although waivers are available), fear that they will not score well, or lack of encouragement by the instructor. And some students just do not have the motivation to take the exam. Students who take the test but not the course so they pursued it on their own, or the student’s AP instruction was accommodated in another course (e.g. an AP German student was enrolled in German 3 but received supplemental instruction).
- Inability to pay the $93 fee (although waivers are available),
- Fear that they will not score well, or lack of encouragement by the instructor.
- Some students just do not have the motivation to take the exam.
- Students who take the test but not the course so they pursued it on their own, or the student’s AP instruction was accommodated in another course (e.g. an AP German student was enrolled in German 3 but received supplemental instruction).
Part I: Section 2: How Does Course and Test Taking Vary When We Disaggregate the Data?
Course and test-taking vary greatly, based on demographic characteristics such as gender, ethnicity and household income. Females and whites were overrepresented in both courses and tests, while lower-income, Hispanic, Native American and rural students were significantly underrepresented. It is notable in the graphs below for both AP course and AP test-taking, the increase for Hispanics is increasing faster than their increase in the student body and Native American students are showing a notable decline in course-taking.
AP Course Enrollment by Race and Ethnicity
White & Asian Students Enroll in AP Courses at Disproportionately Higher Rates; Hispanic & Native American Students Enroll at Disproportionately Lower Rates; Native American Participation in AP Courses is Noticeably Declining.
AP Test Taking by Race and Ethnicity
White & Asian Students Take AP Tests at Disproportionately Higher Rates; Hispanic & Native American & African American Students Take Tests at Disproportionately Lower Rates.
Rural Students Enroll in AP Courses & Take AP Tests at Lower Rates
Looking at participation by rural students and by gender, we see low and unchanging participation rates for rural students and low and slightly declining rates by males.
AP Course Enrollment and Test Taking by Gender
Looking at participation by gender, we see low and slightly declining rates by males. The higher AP rates for females are consistent with higher female college-going rates.
AP Enrollment and Test Taking by Poverty Indicators
For the poverty indicator, we included students that were flagged as having been on free/reduced meals at any time in grades 7-12, which covers about two-thirds (64%) of students. Another option was to flag students who were on free/ reduced meals any time during high school (about 57% of all Arizona students), but this would then be an undercount since high schoolers often choose not to participate in the lunch program in high school. A third option was to flag those who had participated in free/reduced meals at any time in K-12 (about 70% of Arizona students), but we decided that this would be an overcount of current poverty.
Part I: Section 3: What Can We Learn From Descriptive Data on AP Course and Exam Taking and College-Going?
Overall, 1 in 4 students (26%) take an AP course and 1 in 6 (16%) take an AP test. Those who take both the AP course and AP test enroll in post-secondary education at high rates (80%), with over half of those students going to a 4-year college/university (60%), with the rest going to a community college. Two out of three of those who take the course but not the test go on to post-secondary education, with most of these enrolling at a community college (65%) instead of a 4-year college/university (35%). In contrast, only 3 in 10 of those who do not take a course or test go on to further education, with most (78%) going to a community college. We do not include those who only took the test because there are so few of these students.
Part I: Section 4: Is It Important to Take The AP Test?
Taking an AP course and test is important, but how one does can also matter. If one does not master the material, receives a low grade in the course or a low score of 1 or 2 on the test, it may signal to a prospective college and to the student that they are not ready for college-level work.
AP tests are scored on a 1 to 5 scale, with 3 to 5 considered ‘passing,’ although whether a student receives college credit and whether that credit is for a required class or for elective credit is up to each university. A score of 4 or 5 is considered solidly passing, and some preliminary analysis suggests that the Arizona students scoring at these levels tend to do better in college - in terms of overall GPA, credits attempted and credits earned - than students scoring a 3. This is consistent with the literature that the benefits of AP are largely reserved for those who take the test and score a 3-5 on the test.
Looking at average scores, most are in the 2.0 to 3.0 range. This means that for many courses, a large number of students are not scoring at the ‘passing’ level. Of the 33 courses examined, only 9 courses had an average score above 3.0 that were statistically significant, compared to not taking the course – Physics2, Calculus BC, Macroeconomics, Microeconomics, Psychology, Chinese, Art History, Art Studio, and Seminar – and in 3 of them (Calculus BC, Chinese, Art History), the higher mean score was for the group that chose not to take the course at all. For more detailed analysis of test-taking in Arizona, see Section 3 in the Appendix.
AP course-taking and test-taking have increased greatly over the last 20 years. When you add in dual enrollment, IB and other similar programs, the increase is even larger. Access to more rigorous instruction is a good news story for Arizona education.
However, access is not equal across the state and student subgroups. Higher-income, white and female subgroups are overrepresented in both course taking and test-taking, while lower-income, Hispanic, Native American and rural students are underrepresented. Is this because AP courses are less likely to be offered, or because the number of seats is limited? Or perhaps students are not motivated to take the courses? Are high school counselors & advisors not placing certain types of students in these courses? This needs further study.
PART II: WHO TAKES RIGOROUS COURSES IN 5 ARIZONA SCHOOL DISTRICTS?
While AP courses are by far the most dominant source of rigorous courses in the state, Arizona school districts have a number of other options to help prepare students for college. Figure 15 (below) shows estimates of participation in rigorous courses based on analysis of course titles in the Student-Teacher-Course database. In 2019, AP topped the list with 28%; Dual or concurrent enrollment at 10%; Cambridge 4%; and IB at 3%.
In this section, we move beyond statewide trends in AP to a deeper analysis of rigorous course taking in 5 Arizona School Districts: Mesa Public Schools, Tempe Union High School District, Tucson Unified School District, Phoenix Union High School District, and Yuma Union High School District. In particular, we present data on the following 3 questions:
- What are the numbers and proportion of 12th-grade students taking rigorous courses including AP, IB and known dual enrollment?
- What types of rigorous courses were students taking (e.g., AP English, IB, etc.)?
- What did AP course taking look like across the five districts when we disaggregate for race/ethnicity, free and reduced lunch status and gender?
What are the numbers and proportion of 12th-grade students taking rigorous courses including AP, IB and known dual enrollment?
Overall, a large proportion of students in each district took rigorous courses. Indeed, between 39 to 62 percent of seniors in the five districts took at least one rigorous course. Figure 16 shows the number and proportion of seniors who: (1) did not take a rigorous course; (2) took at least one rigorous course in any subject; (3) took an English/Reading rigorous course; (4) took a mathematics rigorous course; (5) took a rigorous course that was not an English/Reading or mathematics course.
Our first major goal was to understand who was taking rigorous courses. To understand this question, we collected transcript data for all students (Grades 9-12) to see who was taking rigorous courses and when. To paint a clearer picture of who is taking rigorous courses we present the data for seniors since their transcripts give us a complete picture of student course-taking over their high school period.
What types of rigorous courses were students taking (e.g., AP English, IB, etc.)?
Across the five districts we also wanted to understand which type of course was more prevalent and available to students. Figure 17 (below) shows the number of students who took each of the three types of rigorous courses. By far, AP was the most prevalent type of course taken. A total of 6,515 seniors took at least one AP course; 2,973 students took a Dual Enrollment course; and 1,154 students at least one IB Course. Here we remind the reader of the challenges in collecting dual enrollment data described in the introduction.
What did AP course taking look like across the five districts when we disaggregate for race/ethnicity, free and reduced lunch status and gender?
Given that we knew that we did not have the complete counts of students taking dual enrollment courses and that IB was school concentrated we shift our last descriptive analyses to a focus on
Table 1: Rigorous course taking by race/ethnicity
Table 2: Rigorous course taking by free/reduced lunch status
Figure 18: The breakdown of rigorous course taking by gender. Each table shows the representation of a demographic group in all students and the representation of that same group among AP students. If taking AP courses was equitable, these two metrics would be very similar, but often they are not. Thus, these tables show us that there is room to increase equity for many of the sub-populations. For example, Asian, White, females, and students not enrolled in free/reduced lunch are typically overrepresented in rigorous course enrollments.
A major goal for this study was to understand the impacts of rigorous course taking on college-going and persistence while also compensating for the non-random nature of AP test taking (see Section 2). The driving hypothesis around this goal is that, if students take and are successful in rigorous courses, they will be more likely to enroll in and persist in college. Unfortunately, as described above we only obtained partial data on dual enrollment and few students actually had an opportunity to participate in IB. As a result, we modified our impact analysis to examine the impact of taking AP courses instead of taking any rigorous course.
To measure the difference between those who take advanced courses and those who do not, but are similar, we employ propensity score weighting with regression adjustment (see Appendix section 1). This method first estimates the likelihood of a student taking advanced courses based on the covariates available, which includes academic ability, and estimates a probability of taking advanced courses. Second, the results of the likelihood model are then incorporated into the impact analysis. The goal of this procedure is to create AP taking and non-AP taking groups that are very similar on a variety of characteristics (ability, gender, Race/Ethnicity, number of courses taken). In some cases, creating comparable groups was not possible for a specific district, and we then do not report their result. A more detailed description of the methodology can be found in Appendix A.
Overall, our analysis shows large and statistically significant impacts of students taking rigorous courses on attending college. This is true for any type of college (two or four year), but we see larger effects for four-year colleges and universities. Our analysis also shows large statistically significant effects based upon free/reduced lunch status and race/ ethnicity. Unfortunately, we did not find statistically significant results for college persistence.
In this section we provide the reader with a with a set of figures for each set of analyses. In each of the figures we identify the difference between the chance of matriculation and persistence for those taking AP courses a statistically created comparison chance of matriculation or persistence. We then graphically present these effects and their margin of error by district with an overall summary effect. We provide the reader with the following analyses:
Impact of Taking One or More AP Courses on College-Going
Impact of Taking One or More AP Courses on College-Going to a Four-Year College
Impact of Taking at Least One AP Math and One English/Reading Course on College-Going
Impact of Taking One or More AP Courses for Latino Student Attending College (Two or Four-Year)
Impact of Taking One or More AP Courses on College Persistence
PART IV: CONCLUSIONS
Advanced Placement test-taking and course-taking has steadily increased over the last twenty years. Arizona students’ access to these rigorous courses is more widespread than ever. An increase in rigorous course taking is a positive trend for Arizona’s education system. Today one in four Arizona seniors (26%) have taken an AP exam.
One in four taking an AP exam is a significant number. But about half of seniors enroll in college right after high school and only around half of those complete a degree. The better-prepared students are in high school for post-secondary content, through things like challenging course work, the more likely they are to be successful in college.
Also, not every student subgroup has a quarter taking an AP exam — lower-income, male, Hispanic, and Native American students are underrepresented in these more rigorous courses and exams, as well as in postsecondary enrollment. There is still significant room for improvement in AP participation overall and for these underrepresented groups.
Why aren’t more students taking AP? One reason may be a lack of available rigorous courses. A recent analysis showed 235 Arizona high schools offer no AP or dual enrollment courses. Unless they offer an alternative program, such as IB or actual college courses, students in these schools do not have access to rigorous courses. Many of these are small schools and charters where they may not have enough students to fill a whole class. Expanding access to online AP or college-level classes and promoting participation may meet the needs of advanced students in these schools. The sudden shift to universal online instruction by schools due to Covid-19 may make schools more open to offering these online options to students prepared for them.
Where one went to school was important to student access and outcomes. We saw some unique patterns in our partner districts.
Mesa had the highest AP participation rate (62%). Mesa also has thousands in dual enrollment courses and an IB program at one school, so this does not reflect a single focus on AP. Despite more rigorous course taking, Mesa has a lower college-going rate than the other districts. Strikingly, the college-going rate for those who did not take AP was much lower than other districts. In Mesa, not taking AP was a strong predictor of not going to college. Whether this was due to students’ lack of preparedness, the fact that so many do take rigorous courses so a smaller number are in this group, or guidance practices that exclude some students is not known but deserves further examination. It should be noted that Mesa has recently started a number of new initiatives that may increase college-going that are not reflected in this study.
In Phoenix Union High School District we saw a slightly lower AP rate, but it should be noted that they have strong ties to Phoenix College and other community colleges so many may be taking dual enrollment. What was striking about Phoenix Union is the equity in AP access. Hispanics are slightly overrepresented in AP classes (83% vs. 81% in the student body) and students from low-income families are almost equally represented (92% vs. 93.5%).
Tempe Union High School District is notable for the high percent of students in AP, and the percent of both AP students and students like those students who did not take AP, enrolling in college. This was true overall and for Hispanics and other groups. This indicates that Tempe has strategies that are effective with these types of students. Further study on what they do to be effective and sharing that with the other districts could have a positive impact.
Tucson had the second-highest AP taking (61%) and had over 1,500 taking dual enrollment. TUSD had the highest AP English taking, which probably reflects an emphasis on students getting freshman requirements (e.g., EN 101) out of the way. Whites were only slightly overrepresented in AP in Tucson
The patterns in Yuma are similar to the other districts. Taking AP is associated with increased college-going compared to comparable students who did not take AP. Although university enrollment is lower in Yuma, the high schools have some of the highest college-going rates in the state, especially if you account for factors such as household income and parental education.
Recognizing the importance of access to rigorous courses, we offer a set of recommendations that have implications for districts and schools, state educational leaders, and communities.
Recommendations For Districts and Schools
The following are recommendations for districts and schools:
- Establish a curriculum and support structures that allow all students to enroll in rigorous courses.
- Create equitable advising policies and practices that promote and allow for all students to take rigorous courses during high school
Think equity and opportunity here. If all students are not provided a curriculum that allows them access to rigorous courses, we have to ask ourselves who are we leaving out. To ensure that all students can reap the benefits of rigorous courses districts and schools have to make sure that all students are enrolled in a curriculum that leads to the option of rigorous courses. This is especially important in the middle school grades and 9th and 10th grade in high school where we most often see divergent curricular paths. This will take collaboration between partner districts where there is not unification as well as within districts that are unified.
Create equitable advising policies and practices that promote and allow for all students to take rigorous courses during high school
Not all high school advisers think that all high school students should go to college. But who should make that decision? We believe that decision should be left up to each individual student. This means that high school guidance counselors should not be guiding students out of rigorous courses or even failing to provide it as an option. In some of our own work we have identified a number of students who wish they would have learned more about rigorous courses earlier in high school and that their counselor would have talked to them more about the opportunities they present (e.g., college credit). To ensure that counselors are not steering students out of these courses we argue that districts and schools need to reinforce training that ensures that all students are provided with rigorous course options.
Recommendations For State and Educational Leaders
- Increase opportunities for students to take rigorous courses across the state including online opportunities.
- Support students by providing funding so that there is no cost for taking dual enrollment courses or AP examinations. At a minimum provide state-wide fee waivers for low-income students.
Increase opportunities for students to take rigorous courses across the state including online opportunities.
Plain and simple – students need more opportunities to take rigorous courses in Arizona. Students do not have equal access for a multitude of reasons. Be it budget shortfalls, remoteness of a school, or even the difficulty of finding qualified teachers. State educational leaders must work hard to identify practices that work against these inequities. One example of how this could be done is through online course-taking like Helios’ partnerships with the Arizona Student Opportunity Collaborative. This program allows students from multiple schools and districts to take online courses with a common teacher. Arizona leadership is strong and innovative. Let’s figure out how to create more opportunities state-wide.
Support students by providing funding so that there is no cost for taking dual enrollment courses or AP examinations. At a minimum provide state-wide fee waivers for low-income students.
No student should be denied access to rigorous courses because neither they nor their family have the resources to pay for them. Nationwide, more states are providing for the costs of these courses and at a minimum providing waivers for low-income students. If, as a state, we believe in equitable educational opportunities then we should find the resources to give everyone access to rigorous courses. In the past Federal funding has helped to support waivers for low-income students. For 2019-20 Arizona used state dollars to sustain waivers. To sustain them over the long term we will need new state policies to continue them in the future.
Recommendations for Communities and Nonprofits
Influence and support local schools with initiatives/programs that include increased opportunities for low-income and underserved students.
Influence and support local schools with initiatives/programs that include increased opportunities for low-income and underserved students.
Communities and nonprofits have a wealth of power in influencing districts and schools. Both should be asking the question: What can I do to improve education in my community? Some options might include the further development of mentor programs, additional tutoring supports, or even funding for dual enrollment courses or AP testing. Real power comes through collaboration and community. It’s up to you make a difference.
Section 1: Methodology of Impact Estimates
The gold standard of any statistical estimation of “impact” of an intervention on an outcome is a randomized trial. Through randomization, the exposure of a treatment (e.g., taking an AP course) is not correlated with any factor (e.g., academic ability) that also influences an outcome (e.g., matriculation). This allows researchers to estimate an independent effect of a treatment on outcomes.
In many cases, randomization is simply not possible, either administratively or ethically. This is the case with AP courses. In these cases, a quasi-experimental design (QED) is often desired to provide plausible model-based estimates of impacts. This is necessary for this study because AP course takers and non-takers are not equivalent (see the results of Section 3). The QED approach we then implement is propensity score weighting with regression adjustment (IPWRA).
The IPWRA approach is essentially a two-step process. The first step is to create a model that relates information about all sample members, X, such as academic ability, gender, Race/Ethnicity, and number of courses taken, to the treatment event T, which in this case is taking at least one AP course. This model produces a probability function Λ(□Pr Pr (T))=f(X). From this estimated function, each case is assigned a “propensity” of taking an AP course, noted as e, which is the predicted probability of taking an AP course based on the variables in X. Note that many cases which do take an AP course have propensity values near 1, but some AP takers may have lower propensities. The latter cases are individuals who take AP courses but statistically resemble those who typically do not take AP courses. Conversely, many cases which do not take AP courses have propensities near 0, but some who do not take courses will have higher propensities. This latter group represents cases that did not take AP courses, but statistically resemble those who do take AP courses. The cases with propensities that differ substantially from their actual behavior become a key element in the impact estimation process.
The second step is to weight the data. The weight for each case is based on actual behavior, T, coded as 1 for taking AP courses and 0 otherwise, and the predicted propensity, e, which ranges from near 1 for a high-likelihood of taking an AP course to near 0 for a low-likelihood of taking an AP course. The weight for each case is then
Which equates to giving higher weights to AP takers who have low propensities for taking AP courses or those who don’t take AP classes with high propensities for taking AP courses. This is exemplified in the following hypothetical four cases: A, B, C, and D.
Cases A and C have propensities that reflect actual behavior. Case A did take an AP course, and have a predicted propensity of .8 (near 1). Case C did not take an AP course and has a propensity of .2 (near 0). The weight for each case is 1.25. The other cases, B and D, have propensities that differ from actual behavior. Case B did take an AP course, but has a low propensity for doing so, and case D did not, but has a high propensity for doing so. These cases have much larger weights (5), and thus the analysis is based far more on the differences in the outcomes between B and D than on the differences between A and C. In research parlance, we are basing the analysis more on the counterfactual conditions than on the factual conditions. The analysis essentially is a weighted difference in means
We evaluated the success of our weights produced for the analysis by examining the differences between the AP course takers and non-takers on the variables in X using the weights. For example, in many cases, the differences between AP course takers and non-takers was more than 1 standard deviation in academic ability and 9th grade GPA, indicating the obvious fact that AP course takers tend to score better on standardized tests and enter high school with better grades. After weighing the data based on predicted propensities, these standardized differences on academic ability (and other factors) were less than a quarter standard deviation. A quarter standard deviation difference, when combined with statistical regression controls, is considered adequate for creating matched groups according to the What Works Clearinghouse. When the weighted difference on any covariate was .25 or more on a standard scale, the analysis for that district is not reported.
Our analysis adds further rigor by also adding the covariates X into the analysis of the weighted mean differences through regression. We produced our estimates using Stata 16’s IPWRA procedure.
Academic achievement is one of the many student variables we used as a control covariate and in creating our propensity weights. However, students varied widely on the exact constellation of test scores available due to missed tests or district data limitations. Generally, student scores on AZ Merit tests for high school math and reading subjects were available as well as ACT test scores. To create an ability score for all students, we used a full information maximum likelihood model that assessed the correlation among all available test scores and covariates, then predicted an ability “factor” score for each student based on available information regardless of missing data on any one item.
Section 2: Course & Test Taking & College-Going By Household Income
More than one in five students (21%) from middle income and richer families take both an AP course and test, almost double the rate for lower income students (11%). Both groups of course and test takers go on to further education at high levels (85% higher income, 75% lower income), but many more lower income students attend community college (50%) than the higher income students (35%). This is despite the fact that there is a great deal of financial aid for high-achieving low income students. For example, an Obama Scholar at ASU would have to pay far fewer out-of-pocket expenses than if that same student had chosen to attend a community college.
More higher income students took just the course (13%) than lower income students (9%). Almost three in four of the 13% went on to college, while just over half of the lower income students (55%) did so. For those not taking any AP courses, the pattern of higher income students going on to post-secondary at higher rates (40% vs. 25%) and more likely to go to a 4-year college/university (28% vs. 18%) also held true.
Student AP Course and Test Taking and College Outcomes by Household Income: Students with Poverty Flag = 1
Student AP Course and Test Taking and College Outcomes by Household Income: Students Without Poverty Flag
Rural students are much less likely to take an AP course and test (9%) or even just a course (5%). Rural students have the same abilities as other students, so it is likely due to lack of course offerings and, possibly, less interest in the courses because of future education and work plans.
Three out of four of those rural students who take both the course and test go on to post-secondary education, with half going to a 4-year college/university. Of those taking just the course and not the test, only 60% pursue further education, and most of those go to a community college. About a third of the rural students who take neither AP courses nor tests go on to a college (35%), with most of those going to a community college (80%).
Student AP Course and Test Taking and College Outcomes for Rural Students
Section 3: AP Test Scores
Mean AP Scores By Subject: Statewide
The table below shows the percent of students scoring a 3 and the percent scoring a 4 or 5 by AP course and by subgroup. In the most popular courses, one-fifth to one third are scoring a 4 or a 5, and an additional 20% to 30% are scoring a 3. In other words, only 48-54% are getting a ‘passing’ score, and that does not include those who chose not to test!
Scoring Patterns By Subject: Statewide, By Household Income, By Rural/Urban
For lower-income and rural students, the passing rates are even lower. For example, the median score of lower-income students is typically around 2, and sometimes even less, while higher-income students have median scores of 3 or higher.
Median Scores By Subject: Statewide & By Household Income
This raises questions about prior preparation for the more rigorous course and the AP course rigor and fidelity to the College Board curriculum. As you can see below from the distribution of scores by poverty flag, the benefits go disproportionately to those who had higher family incomes. It may not even be advisable for those from lower income families to attempt these expensive exams if they are not well prepared, since more likely than not, they will score so low that they will see minimal or no benefits in future college classes. (Warne et al, 2015)
Distribution of AP Test Scores By Household Income
Who Does & Does Not Benefit From Taking AP Tests?