Once the loading box has disappeared, results will be displayed in the Results tabs.


Many youth within the Tennessee foster care and juvenile justice systems are prescribed psychotrophic medications for a variety of reasons. Despite the potency of these medications and the potential for misuse, there is little oversight of the prescription of psychotrophics to youth in state care. As a result, many are concerned these medications are often prescribed in frequencies, quantities, or combinations that are inappropriate.

Our analysis has two aims: 1) to identify prescribers with outlying rates of potentially problematic prescriptions of psychopharmacologic medications after accounting for varying case-mixes; and 2) provide relevant information for interventions with outlying prescribers. To do this, we emulate a methodology originally developed by the Centers for Medicare and Medicaid Services for predicting Risk-Standardized Mortality Rates (RSMR) for hospitals. Within our context, we estimate Risk-Standardized Red Flag Rates (RSRFR) for each prescriber. Prescribers are then identified as outliers on the basis of a high rate of “red flag” prescriptions.

The funnel plots at the top of the 'Results' panel show the model output, illustrating where each prescriber is relative to his or her peers in terms of RSRFR. Points on the right-hand funnel plot can be selected, generating supplemental tables that detail characteristics of the youth treated, as well as the prescriptions issued by the selected prescriber.


The dataset used in this analysis was compiled from three sources:

  1. Prescription claims data from Magellan Health Services
  2. Youth administrative data from the Tennessee Department of Children's Services (DCS)
  3. Census block poverty data from the US Census Bureau's 2011-2015 5-year American Community Survey

Magellan Health Services data included all prescriptions for qualifying psychotropic medications to youth in DCS care from 6/22/16 to 12/26/16. Qualifying psychotropic medications included antidepressants, antipsychotics, stimulants, and mood stabilizers. Prescribers with fewer than 10 prescriptions and automatic refill prescriptions were excluded from the analysis. Our final dataset consisted of 21,124 prescriptions across 3,485 youth and 289 prescribers.

Classification of Red Flags

Prescriptions were labeled as “red flag prescriptions” if they met any of the following criteria: they were a) issued to youth five years of age or younger, b) issued to a youth at the same time as three or more additional prescriptions, c) one of two or more prescriptions issued at the same time in a given class (e.g. two or more antidepressants, antipsychotics, stimulants, or stabilizers), or d) fell outside the maximum recommended dosage range, as defined by Magellan.


We fit a Generalized Linear Mixed Model (GLMM) to these data at the prescription level with a logit link and normally distributed random effects to account for within-prescriber correlation. Our fixed effects include the following youth-level covariates:

  • Age
  • Gender
  • Race
  • Adjudication status
  • Commitment region
  • Number of days in custody until prescription date
  • Level of care
  • CANS recommendation level
  • Percent poverty associated with the youth's removal address

Age, the number of days in custody until prescription, and percent poverty are included as continuous covariates and modeled with restricted cubic splines, while the remaining covariates are categorical. Youths' gender has two levels: “Male” or “Female,” while adjudication status has the levels “Delinquent” and “Dependent/Neglect/Unruly.” Commitment region has 12 levels, while level of care has 3 levels: “1,” “2,” “3/4.” Finally, CANS recommendation level is categorized as either “1,” “2,” or “3/4.”

Prescribers were classified as falling within the “Alert” and “Alarm” zones if their RSRFR exceeded control limits based off of 95% and 99.8% “exact” confidence intervals around the mean RSRFR, respectively. Prescribers with estimated RSRFPR values in the Alarm zone, therefore have adjusted red flag rates that are statistically different at the 0.002 level than would be expected for their number of prescriptions if they had the average RSRFR value.

Calculation and Interpretation of Risk-Standardized Red Flag Rate

The Risk-Standardized Red Flag Rate (RSRFR) was calculated using the following formula:

$$\text{RSRFR}_i = \frac{\text{Predicted number of red flags for prescriber i}} {\text{Expected number of red flags for prescriber i}} \times \frac{\text{Total number of red flags for all prescribers}} {\text{Total number of prescriptions for all prescribers}}$$

Where the predicted number of red flags for Prescriber i is the model’s prediction for Prescriber i, estimated from Prescriber i's covariates and random effect, while the expected number of red flags for Prescriber i is the model’s prediction using only covariate values.

The RSRFPR gives us a measurement of each prescriber’s predicted to expected ratio, standardized to the population of prescribers included within the model. More explicitly, the expected value for a prescriber is the number of red flags that we would estimate that prescriber to issue after adjusting for the case complexity of their patients, as measured by the covariates included within the model. The predicted value gives the estimated number of red flags for that prescriber after accounting for their case complexity and the variation between prescribers that was unaccounted for by covariate values. This variation is due to unmeasured differences between prescribers that could not be incorporated into the model.

The predicted to expected ratio, therefore tells us how many times larger the predicted number of red flags is for a prescriber than would be expected for the average prescriber who prescribed to kids with the same measured characteristics. This rate is then multiplied by the overall rate of red flags among all prescribers in order to anchor the relative predicted to expected measurement to the observed population of prescribers.