Rigorous Coursework in Arizona

Part I: Section 1: How Many Students Are Taking AP Courses and Examinations in Arizona?

In 2019, 53,746 Arizona students enrolled in 95,961 AP courses. Of those who took AP courses 26% were 12th graders, 42% were 11th graders, 23% were 10th graders, and 9% were 9th graders. The most popular courses were English, US Government, US History, World History and Calculus.

Student Data on AP Courses and Test

In 2018 (the most recent year available from ADE), 34,779 students took 62,281 AP tests. This is significantly less than the AP course takers because AP test-taking is not required to take the course. Eleventh and 12th graders took a majority of the tests. Slightly over three-quarters of the test-takers came from grade 12 (34%) and grade 11 (42%), with the rest from grade 10 (17%) or grade 9 (6%).

Ideally one would take the course taught using the curriculum provided by the College Board and then take the test designed to measure that curriculum. But that is not universal. Looking at student records, between 2014-2018, of the 26,000 who took AP courses & tests each year, around 58% took both the course and the test, 38% took the course but not the test, and 4% took the test but not the course.

Why Wouldn’t Course Takers Take The Test?

Why wouldn’t course takers take the test? Reasons might include the following: inability to pay the $93 fee (although waivers are available), fear that they will not score well, or lack of encouragement by the instructor. And some students just do not have the motivation to take the exam. Students who take the test but not the course so they pursued it on their own, or the student’s AP instruction was accommodated in another course (e.g. an AP German student was enrolled in German 3 but received supplemental instruction).

• Inability to pay the $93 fee (although waivers are available),
• Fear that they will not score well, or lack of encouragement by the instructor.
• Some students just do not have the motivation to take the exam.
• Students who take the test but not the course so they pursued it on their own, or the student’s AP instruction was accommodated in another course (e.g. an AP German student was enrolled in German 3 but received supplemental instruction).

Part I: Section 2: How Does Course and Test Taking Vary When We Disaggregate the Data?

Course and test-taking vary greatly, based on demographic characteristics such as gender, ethnicity and household income. Females and whites were overrepresented in both courses and tests, while lower-income, Hispanic, Native American and rural students were significantly underrepresented. It is notable in the graphs below for both AP course and AP test-taking, the increase for Hispanics is increasing faster than their increase in the student body and Native American students are showing a notable decline in course-taking.

AP Course Enrollment by Race and Ethnicity 

White & Asian Students Enroll in AP Courses at Disproportionately Higher Rates; Hispanic & Native American Students Enroll at Disproportionately Lower Rates; Native American Participation in AP Courses is Noticeably Declining. 

AP Test Taking by Race and Ethnicity 

White & Asian Students Take AP Tests at Disproportionately Higher Rates; Hispanic & Native American & African American Students Take Tests at Disproportionately Lower Rates. 

 

Rural Students Enroll in AP Courses & Take AP Tests at Lower Rates

Looking at participation by rural students and by gender, we see low and unchanging participation rates for rural students and low and slightly declining rates by males.

             

 

AP Course Enrollment and Test Taking by Gender

Looking at participation by gender, we see low and slightly declining rates by males. The higher AP rates for females are consistent with higher female college-going rates.

             

 

AP Enrollment and Test Taking by Poverty Indicators

For the poverty indicator, we included students that were flagged as having been on free/reduced meals at any time in grades 7-12, which covers about two-thirds (64%) of students. Another option was to flag students who were on free/ reduced meals any time during high school (about 57% of all Arizona students), but this would then be an undercount since high schoolers often choose not to participate in the lunch program in high school. A third option was to flag those who had participated in free/reduced meals at any time in K-12 (about 70% of Arizona students), but we decided that this would be an overcount of current poverty.

             

 

Part I: Section 4: Is It Important to Take The AP Test?

Taking an AP course and test is important, but how one does can also matter. If one does not master the material, receives a low grade in the course or a low score of 1 or 2 on the test, it may signal to a prospective college and to the student that they are not ready for college-level work.

AP tests are scored on a 1 to 5 scale, with 3 to 5 considered ‘passing,’ although whether a student receives college credit and whether that credit is for a required class or for elective credit is up to each university. A score of 4 or 5 is considered solidly passing, and some preliminary analysis suggests that the Arizona students scoring at these levels tend to do better in college - in terms of overall GPA, credits attempted and credits earned - than students scoring a 3. This is consistent with the literature that the benefits of AP are largely reserved for those who take the test and score a 3-5 on the test.

Looking at average scores, most are in the 2.0 to 3.0 range. This means that for many courses, a large number of students are not scoring at the ‘passing’ level. Of the 33 courses examined, only 9 courses had an average score above 3.0 that were statistically significant, compared to not taking the course – Physics2, Calculus BC, Macroeconomics, Microeconomics, Psychology, Chinese, Art History, Art Studio, and Seminar – and in 3 of them (Calculus BC, Chinese, Art History), the higher mean score was for the group that chose not to take the course at all. For more detailed analysis of test-taking in Arizona, see Section 3 in the Appendix.

AP course-taking and test-taking have increased greatly over the last 20 years. When you add in dual enrollment, IB and other similar programs, the increase is even larger. Access to more rigorous instruction is a good news story for Arizona education.

However, access is not equal across the state and student subgroups. Higher-income, white and female subgroups are overrepresented in both course taking and test-taking, while lower-income, Hispanic, Native American and rural students are underrepresented. Is this because AP courses are less likely to be offered, or because the number of seats is limited? Or perhaps students are not motivated to take the courses? Are high school counselors & advisors not placing certain types of students in these courses? This needs further study.

What are the numbers and proportion of 12th-grade students taking rigorous courses including AP, IB and known dual enrollment?

Overall, a large proportion of students in each district took rigorous courses. Indeed, between 39 to 62 percent of seniors in the five districts took at least one rigorous course. Figure 16 shows the number and proportion of seniors who: (1) did not take a rigorous course; (2) took at least one rigorous course in any subject; (3) took an English/Reading rigorous course; (4) took a mathematics rigorous course; (5) took a rigorous course that was not an English/Reading or mathematics course.

Our first major goal was to understand who was taking rigorous courses. To understand this question, we collected transcript data for all students (Grades 9-12) to see who was taking rigorous courses and when. To paint a clearer picture of who is taking rigorous courses we present the data for seniors since their transcripts give us a complete picture of student course-taking over their high school period.

What types of rigorous courses were students taking (e.g., AP English, IB, etc.)?

Across the five districts we also wanted to understand which type of course was more prevalent and available to students. Figure 17 (below) shows the number of students who took each of the three types of rigorous courses. By far, AP was the most prevalent type of course taken. A total of 6,515 seniors took at least one AP course; 2,973 students took a Dual Enrollment course; and 1,154 students at least one IB Course. Here we remind the reader of the challenges in collecting dual enrollment data described in the introduction.

Recommendations For Districts and Schools

The following are recommendations for districts and schools: 

1. Establish a curriculum and support structures that allow all students to enroll in rigorous courses. 
2. Create equitable advising policies and practices that promote and allow for all students to take rigorous courses during high school

Think equity and opportunity here. If all students are not provided a curriculum that allows them access to rigorous courses, we have to ask ourselves who are we leaving out. To ensure that all students can reap the benefits of rigorous courses districts and schools have to make sure that all students are enrolled in a curriculum that leads to the option of rigorous courses. This is especially important in the middle school grades and 9th and 10th grade in high school where we most often see divergent curricular paths. This will take collaboration between partner districts where there is not unification as well as within districts that are unified.

Create equitable advising policies and practices that promote and allow for all students to take rigorous courses during high school

Not all high school advisers think that all high school students should go to college. But who should make that decision? We believe that decision should be left up to each individual student. This means that high school guidance counselors should not be guiding students out of rigorous courses or even failing to provide it as an option. In some of our own work we have identified a number of students who wish they would have learned more about rigorous courses earlier in high school and that their counselor would have talked to them more about the opportunities they present (e.g., college credit). To ensure that counselors are not steering students out of these courses we argue that districts and schools need to reinforce training that ensures that all students are provided with rigorous course options.

Recommendations For State and Educational Leaders

1. Increase opportunities for students to take rigorous courses across the state including online opportunities.
2. Support students by providing funding so that there is no cost for taking dual enrollment courses or AP examinations. At a minimum provide state-wide fee waivers for low-income students.

Increase opportunities for students to take rigorous courses across the state including online opportunities.

Plain and simple – students need more opportunities to take rigorous courses in Arizona. Students do not have equal access for a multitude of reasons. Be it budget shortfalls, remoteness of a school, or even the difficulty of finding qualified teachers. State educational leaders must work hard to identify practices that work against these inequities. One example of how this could be done is through online course-taking like Helios’ partnerships with the Arizona Student Opportunity Collaborative. This program allows students from multiple schools and districts to take online courses with a common teacher. Arizona leadership is strong and innovative. Let’s figure out how to create more opportunities state-wide.

Support students by providing funding so that there is no cost for taking dual enrollment courses or AP examinations. At a minimum provide state-wide fee waivers for low-income students.

No student should be denied access to rigorous courses because neither they nor their family have the resources to pay for them. Nationwide, more states are providing for the costs of these courses and at a minimum providing waivers for low-income students. If, as a state, we believe in equitable educational opportunities then we should find the resources to give everyone access to rigorous courses. In the past Federal funding has helped to support waivers for low-income students. For 2019-20 Arizona used state dollars to sustain waivers. To sustain them over the long term we will need new state policies to continue them in the future.

Recommendations for Communities and Nonprofits

Influence and support local schools with initiatives/programs that include increased opportunities for low-income and underserved students.

Influence and support local schools with initiatives/programs that include increased opportunities for low-income and underserved students.

Communities and nonprofits have a wealth of power in influencing districts and schools. Both should be asking the question: What can I do to improve education in my community? Some options might include the further development of mentor programs, additional tutoring supports, or even funding for dual enrollment courses or AP testing. Real power comes through collaboration and community. It’s up to you make a difference.

Section 1: Methodology of Impact Estimates

The gold standard of any statistical estimation of “impact” of an intervention on an outcome is a randomized trial. Through randomization, the exposure of a treatment (e.g., taking an AP course) is not correlated with any factor (e.g., academic ability) that also influences an outcome (e.g., matriculation). This allows researchers to estimate an independent effect of a treatment on outcomes.

In many cases, randomization is simply not possible, either administratively or ethically. This is the case with AP courses. In these cases, a quasi-experimental design (QED) is often desired to provide plausible model-based estimates of impacts. This is necessary for this study because AP course takers and non-takers are not equivalent (see the results of Section 3). The QED approach we then implement is propensity score weighting with regression adjustment (IPWRA).

The IPWRA approach is essentially a two-step process. The first step is to create a model that relates information about all sample members, X, such as academic ability, gender, Race/Ethnicity, and number of courses taken, to the treatment event T, which in this case is taking at least one AP course. This model produces a probability function Λ(□Pr Pr (T))=f(X). From this estimated function, each case is assigned a “propensity” of taking an AP course, noted as e, which is the predicted probability of taking an AP course based on the variables in X. Note that many cases which do take an AP course have propensity values near 1, but some AP takers may have lower propensities. The latter cases are individuals who take AP courses but statistically resemble those who typically do not take AP courses. Conversely, many cases which do not take AP courses have propensities near 0, but some who do not take courses will have higher propensities. This latter group represents cases that did not take AP courses, but statistically resemble those who do take AP courses. The cases with propensities that differ substantially from their actual behavior become a key element in the impact estimation process.

The second step is to weight the data. The weight for each case is based on actual behavior, T, coded as 1 for taking AP courses and 0 otherwise, and the predicted propensity, e, which ranges from near 1 for a high-likelihood of taking an AP course to near 0 for a low-likelihood of taking an AP course. The weight for each case is then

Which equates to giving higher weights to AP takers who have low propensities for taking AP courses or those who don’t take AP classes with high propensities for taking AP courses. This is exemplified in the following hypothetical four cases: A, B, C, and D.

Cases A and C have propensities that reflect actual behavior. Case A did take an AP course, and have a predicted propensity of .8 (near 1). Case C did not take an AP course and has a propensity of .2 (near 0). The weight for each case is 1.25. The other cases, B and D, have propensities that differ from actual behavior. Case B did take an AP course, but has a low propensity for doing so, and case D did not, but has a high propensity for doing so. These cases have much larger weights (5), and thus the analysis is based far more on the differences in the outcomes between B and D than on the differences between A and C. In research parlance, we are basing the analysis more on the counterfactual conditions than on the factual conditions. The analysis essentially is a weighted difference in means 

We evaluated the success of our weights produced for the analysis by examining the differences between the AP course takers and non-takers on the variables in X using the weights. For example, in many cases, the differences between AP course takers and non-takers was more than 1 standard deviation in academic ability and 9th grade GPA, indicating the obvious fact that AP course takers tend to score better on standardized tests and enter high school with better grades. After weighing the data based on predicted propensities, these standardized differences on academic ability (and other factors) were less than a quarter standard deviation. A quarter standard deviation difference, when combined with statistical regression controls, is considered adequate for creating matched groups according to the What Works Clearinghouse. When the weighted difference on any covariate was .25 or more on a standard scale, the analysis for that district is not reported.

Our analysis adds further rigor by also adding the covariates X into the analysis of the weighted mean differences through regression. We produced our estimates using Stata 16’s IPWRA procedure.

Academic achievement is one of the many student variables we used as a control covariate and in creating our propensity weights. However, students varied widely on the exact constellation of test scores available due to missed tests or district data limitations. Generally, student scores on AZ Merit tests for high school math and reading subjects were available as well as ACT test scores. To create an ability score for all students, we used a full information maximum likelihood model that assessed the correlation among all available test scores and covariates, then predicted an ability “factor” score for each student based on available information regardless of missing data on any one item.

Section 2: Course & Test Taking & College-Going By Household Income

More than one in five students (21%) from middle income and richer families take both an AP course and test, almost double the rate for lower income students (11%). Both groups of course and test takers go on to further education at high levels (85% higher income, 75% lower income), but many more lower income students attend community college (50%) than the higher income students (35%). This is despite the fact that there is a great deal of financial aid for high-achieving low income students. For example, an Obama Scholar at ASU would have to pay far fewer out-of-pocket expenses than if that same student had chosen to attend a community college.

More higher income students took just the course (13%) than lower income students (9%). Almost three in four of the 13% went on to college, while just over half of the lower income students (55%) did so. For those not taking any AP courses, the pattern of higher income students going on to post-secondary at higher rates (40% vs. 25%) and more likely to go to a 4-year college/university (28% vs. 18%) also held true.

Student AP Course and Test Taking and College Outcomes by Household Income: Students with Poverty Flag = 1

Student AP Course and Test Taking and College Outcomes by Household Income: Students Without Poverty Flag

Rural Students

Rural students are much less likely to take an AP course and test (9%) or even just a course (5%). Rural students have the same abilities as other students, so it is likely due to lack of course offerings and, possibly, less interest in the courses because of future education and work plans.

Three out of four of those rural students who take both the course and test go on to post-secondary education, with half going to a 4-year college/university. Of those taking just the course and not the test, only 60% pursue further education, and most of those go to a community college. About a third of the rural students who take neither AP courses nor tests go on to a college (35%), with most of those going to a community college (80%).

Student AP Course and Test Taking and College Outcomes for Rural Students

Section 3: AP Test Scores

Mean AP Scores By Subject: Statewide

The table below shows the percent of students scoring a 3 and the percent scoring a 4 or 5 by AP course and by subgroup. In the most popular courses, one-fifth to one third are scoring a 4 or a 5, and an additional 20% to 30% are scoring a 3. In other words, only 48-54% are getting a ‘passing’ score, and that does not include those who chose not to test!

Scoring Patterns By Subject: Statewide, By Household Income, By Rural/Urban

For lower-income and rural students, the passing rates are even lower. For example, the median score of lower-income students is typically around 2, and sometimes even less, while higher-income students have median scores of 3 or higher.

Median Scores By Subject: Statewide & By Household Income

This raises questions about prior preparation for the more rigorous course and the AP course rigor and fidelity to the College Board curriculum. As you can see below from the distribution of scores by poverty flag, the benefits go disproportionately to those who had higher family incomes. It may not even be advisable for those from lower income families to attempt these expensive exams if they are not well prepared, since more likely than not, they will score so low that they will see minimal or no benefits in future college classes. (Warne et al, 2015)

Distribution of AP Test Scores By Household Income

Who Does & Does Not Benefit From Taking AP Tests?