Dashboard Metrics
Advanced Course Mastery
Background
District Dashboard Metrics
Because of the importance of rigorous coursework to postsecondary success and to support district efforts to increase advanced course completion, several districts across the country, including Charlotte, Chicago, Dallas and Oakland, have incorporated into their internal performance management dashboards metrics to monitor advanced course enrollment and performance. In focus group discussions conducted with over 2,600 educators throughout Texas, 95% of those surveyed found advanced course enrollment and performance information useful when reported at the campus level. When reported at the student level, the percent of respondents finding the information useful was notably lower (less than 90%). The implication is that this metric, though still valuable to teachers in preparing their individual students for advanced coursework, may be even more useful to district leaders in developing strategic, district-wide approaches to increasing enrollment and success in advanced courses.
Primary Metric
- Advanced Course Mastery: percent of students who scored at or above the specified criterion on required AP and IB exams in the four core subject areas .
User Interface
Dashboard Example
Figure 1 shows the Advanced Course Mastery metric as seen on the district dashboard.
Figure 1 TSDS District Advanced Course Mastery
Status Definition
“Advanced courses” include AP courses and IB courses. Dual enrollment courses are not included as there are no uniform set of exams used to determine mastery.
The Status column is the percentage of students that scored at or above the state criterion on all of the AP and IB exams taken in ELA, mathematics, science, and social studies in the most recent year. The Status indicator is determined by district. The TSDS default threshold is 45%. The actual percentage is displayed with the indicator.
Table 2 defines the Status indicators.
Metric Status Indicator | Comparisons | ||||
Metric Name | Sub-metric Name | Metric Status Indicator | Trend | Campus Attainment | |
Advanced Course Mastery | ELA | Red (Percentage) = if percentage of students mastering all AP exams (score must be greater than or equal to 3) and IB exams (score must be greater than or equal to 4) in the ELA subject area in the most recent year is less than the district goal Green (Percentage) = if percentage of students mastering all APexams and IB exams in the ELA subject area in the most recent year is greater than or equal to the district goal | N/A | The number of campuses meeting their campus goal over the number of campuses with data for the metric | |
Mathematics | Red (Percentage) = if percentage of students mastering all AP exams (score must be greater than or equal to 3) and IB exams (score must be greater than or equal to 4) in Mathematics for the most recent year is less than the district goal Green (Percentage) = if percentage of students mastering all AP course exams and/or IB exams in Mathematics for the most recent year is greater than or equal to the district goal | N/A | The number of campuses meeting their campus goal over the number of campuses with data for the metric | ||
Science | Red (Percentage) = if percentage of students mastering at least one advanced course (score must be greater than or equal to 3) or IB (score must be greater than or equal to 4) in the most recent year is less than the district goal Green (Percentage) = if percentage of students mastering at least one advanced course or IB in the most recent year is greater than or equal to the district goal | N/A | The number of campuses meeting their campus goal over the number of campuses with data for the metric | ||
Social Studies | Red (Percentage) = if percentage of students mastering at least one advanced course (score must be greater than or equal to 3) or IB (score must be greater than or equal to 4) in the most recent year is less than the district goal Green (Percentage) = if percentage of students mastering at least one advanced course or IB in the most recent year is greater than or equal to the district goal | N/A | The number of campuses meeting their campus goal over the number of campuses with data for the metric | ||
Trends
The dashboards do not currently support trends for this metric. The UI supports the trend, but the ETL package has not been developed.
Trend Definition
Trend Indicators: Objective is to indicate Advanced Course Mastery | |||
| Up green | If the value for the current time frame is more than 5 percentage points higher than the prior time frame, then display a gray arrow trending upward. | |
| Unchanged | If the value for the current time frame is less than or equal to 5 percentage points higher or less than or equal to 5 percentage points lower the prior time frame, then display two gray arrows pointing outward indicating no direction. | |
| Down red | If the value for the current time frame is more than 5 percentage points lower than the prior time frame, then display a gray arrow trending downward. | |
The trend does not show for the first occurrence of a time frame since data is not available.
Delta Definition
The delta appears only in the campus list and displays the individual campus goals rather than the district goal.
Table 4 Delta Indicator for TSDS District Advance Course Mastery Metric | |||
Metric Name | Sub-Metric Name | Campus Goal | Delta |
Advanced Course Mastery | ELA | Set campus goal | Red (Delta Percentage) = if the calculated percentage of students mastering all AP course and IB exams is less than the campus goal Black (Delta Percentage) = if the calculated percentage of students completing all AP and IB exams is greater than or equal to the campus goal |
Mathematics | Set campus goal | Red (Delta Percentage) = if the calculated percentage of students mastering at least one advanced course or IB is less than the campus goal Black (Delta Percentage) = if the calculated percentage of students completing at least one advanced course or IB is greater than or equal to the campus goal | |
Science | Set campus goal | Red (Delta Percentage) = if the calculated percentage of students mastering at least one advanced course or IB is less than the campus goal Black (Delta Percentage) = if the calculated percentage of students completing at least one advanced course or IB is greater than or equal to the campus goal | |
Social Studies | Set campus goal | Red (Delta Percentage) = if the calculated percentage of students mastering at least one advanced course or IB is less than the campus goal Black (Delta Percentage) = if the calculated percentage of students completing at least one advanced course or IB is greater than or equal to the campus goal | |
Periodicity
Recommended Load Characteristics | |
Calendar | Beginning and end of school year |
Frequency of data load | Twice a year |
Latency | 7-8 months |
Interchange schema | Interchange-StudentGrades.xsd |
Tooltips
The standard tooltips for the metric definition, column headers, and help functions display for this metric.
- Green metric status: OK
- Red metric status: LOW
- 'More' indicates drill down views for this metric
is ‘No change from the prior period'
is ‘Getting worse from the prior period’
is 'Getting better from the prior period'
Business Rules
The Advanced Course Mastery metric is the percentage of current students mastering (scoring at or above national/state-defined criterion) on all of their AP and IB exams taken in the most recent school year for the following subject areas:
- ELA
- Mathematics
- Science
- Social Studies
Mastery of an AP subject is defined as earning a score of 3 or greater on the AP exam. For IB courses, the mastery of a subject is defined as earning a score of 4 or more on the IB exam.
Data Assumptions
“Advanced Courses” are those in which a nationally or internationally accredited exam is taken:
- AP courses
- IB courses
Performance in Pre-AP, Pre-IB, and dual credit courses for which a student does not also participate in an AP or IB exam will not be considered in this metric.
Computed Values
The following calculation must be performed for each core subject area (ELA, Mathematics, Science and Social Studies) to determine the percentage of students mastering the course.
Table 6 Calculated Values for TSDS Campus Advanced Course Mastery Metric | |
Metric | Calculation |
Advanced Course Mastery | (Total of all current students who mastered all AP and IB exams taken in the most recent school year / (Total of all current students that took one or more AP or IB examinations in the most recent school year) |
Data Anomalies
Data Anomalies
- A student may have missing transcript data in which an AP or IB examination may have been taken in a prior year. This campus metric only pertains to the most recent year.
- Some files from the examination providers do not include critical data that is required for precise matching of students (e.g., social security number, date of birth). In these instances, there may be duplicate names and manual intervention is required to assign students to their data.
- Students who sat for the exam but canceled their scores will not be reflected in this metric.
Date of Refresh
Best practice is for the date of last data refresh to appear next the metric in the following format:
- (Through May 2010)
Implementation Considerations
Transfer Students (Late Enrollments)
Transfer students may not have complete transcripts from which to identify prior year advanced courses or mastery exams.
Advanced Course Definition
This metric includes AP and IB courses, as well as those defined as advanced by both the state and locally defined by the district. This mapping looks at the course characteristic type as coded locally. In the case of courses that are both AP and another type such as GT, the course should have the characteristic type of AP. If the other characteristic is recorded, the course will not appear as 'advanced.'
Drill Downs
Drill Down Views
The dashboards include the option to see more detail that is associated with a metric. Table 7 lists the drill down views that are recommended for this metric.
| Campus List | List of campuses and whether or not they meet their goal for this metric for the specified period. |