ALTHOUGH COMMITMENT TO the use of evidence-based practices in probation is widespread, debate continues about the design, use, and benefits of assessing risks and needs using an actuarial tool (Baird, 2009; Bogue, Campbell, & Clawson, 2004). The use of an assessment tool is the undeniable basis of evidence-based practice and the ability of actuarial risk assessment tools to improve prediction over clinical judgment of probation officers has been documented in a thorough review of relevant literature (Gottfredson & Moriarty, 2006). Even with these strong foundations, probation departments remain far from the goal of universal adoption of risk/ needs assessments. This may in part involve a lack of trust, by probation officers and justice partners, in the ability of the tools to accurately predict the probability of recidivism.
The key to a more widespread use of risk/ needs assessment tools is improved trust that actuarial tools are reliably able to place offenders that pose different risks to the community in categories that accurately reflect those risks. In other words, the risk assessment tool must be statistically valid. The use of meta analysis is the gold standard in considering a practice to be evidence based. In the assessment tool validation literature, for example, the adult assessment tool, the Level of Service Inventory (LSI), has been subjected to at least 47 studies of its predictive validity (Vose, Cullen, & Smith, 2008). The current article seeks to add to the published documentation supporting the validity of the San Diego Risk and Resiliency Checkup.
The goal of evidence-based practice is to reduce recidivism among probationers. The first step to this goal is understanding the offender’s risks and needs. The results of the assessment must provide accurate information as to the risk that the probationer poses to the community and the criminogenic needs that must be addressed in order to reduce recidivism. Adherence to the risk principle means that offenders must be supervised according to the risk that they pose to the community (Lowenkamp & Latessa, 2004). Research shows that resources are used most effectively when they are focused on high-risk offenders. In addition, the risk/need responsivity model asserts that the provision of community-based services must be based on the criminogenic needs of the individual offender in order to reduce rates of recidivism (Bonta & Andrews, 2007).
As the assessment of risk and need is fundamental to later decisions about the youth’s level of supervision as well as what services he or she will receive while under supervision, it is especially important that risk assessment tools be grounded in empirical research. A tool that combines a validated risk assessment with an intuitive needs component allows community corrections officers to effectively balance accountability and rehabilitation. The question remains: how well does the San Diego Regional Resiliency Check-up (SDRRC) function?
The SDRRC is a 60-item measure of risk and protective factors related to a juvenile’s functioning. Risk factors are designed to measure the level of dysfunction in a youth’s life, while protective factors measure the extent to which a minor has “protection” or positive factors in his/her life. The risk and protective items measured on the SDRRC are divided into six domains: individual factors, delinquency factors, family factors, educational factors, substance use factors, and peer factors. The SDRRC is designed to be administered at the time of the initial investigation and then again every six months during the period of community supervision.
When the SDRRC was created by Brad Bogue of the Justice System Assessment and Training in Boulder, Colorado, he and his team established the content validity by conducting a thorough review of the literature to ascertain which protective and risk factors were most highly predicative of recidivism (Bogue, personal communication 4/28/2007). Later research established the predictive validity of the SDRRC (Little, n.d.; Susan Turner & Fain, 2006; S. Turner, Fain, & Sehgal, 2005) . The next two paragraphs describe the predictive validity research.
In 2002, Little, in an unpublished study, examined data on approximately 2700 adolescents. Little examined the Total Resiliency, Total Protective, and Total Risk scores obtained from the SDRRC and their relationship to criminal activity. She found that the Total Resiliency, Total Protective, and Total Risk scales correlated with both prior and recidivistic activity, and the Total Resiliency score correlated at the highest level (past criminal behavior r=-.25, future criminal behavior r=-.15). The Total Resiliency score (overall SDRRC score) provided predictive validity over and above other known risk factors, such as past behavior and demographic variables. (Little, n.d.).
In 2006, Turner and Fain examined the predictive validity of the SDRRC with the Total Resiliency score and recidivism, categorized into “low,” “medium,” and “high” categories (N=1200), with a follow-up period of 12 months. Results indicated that the Total Resiliency score was significantly related to juvenile recidivism. They also correlated other SDRRC subscale scores with recidivism. Excluding the Total Resiliency score, the strongest correlation with recidivism was the Total Protective subscale score (r = -.25). The risk and protective subscale correlations ranged from r = -.12 to r = -.21, p<.05.
These two studies suggest that the SDRRC merits consideration as an evidence-based assessment tool. The current research aims to strengthen the evidence showing the SDRRC has predictive validity by replicating Turner and Fain’s published research. Our replication project measures the relationship between SDRRC risk scores and criminal recidivism in order to determine which score is the best predictor of recidivism. Additionally, based on the empirical evidence, we suggest revised cutoff scores for the tool that creates categories of youth under probation supervision that allow for reasonable groupings of youth who can be expected to recidivate at different levels.
Predictive Validity Study
Sample. Information for all 6766 juveniles referred to the San Diego County Probation Department in 2004 who had an SDRRC completed was collected. For youth with multiple SDRRC administrations in 2004, the first SDRRC administered in 2004 was used for the current study. Youth under 11 and over 15, those with missing or inaccurate data, youth who had their records sealed by the court, those who were not wards of the court under welfare and institutions code 601 or 602, and those who lived outside San Diego County at the time they were assessed were removed from the sample. The final sample consisted of 2076 juveniles.
Procedure. Independent and dependent variables were collected through probation department electronic databases and through San Diego County District Attorney automated files. In order to ensure that youth in the sample remained in San Diego County during the follow-up period, electronic contact records were reviewed to look for mention of moving out of state or otherwise not being available to recidivate. Table 1 shows all of the variables used in the study.
Dependent Variable. The dependent variable for this project was recidivism. Recidivism is defined as the filing of a juvenile petition or the adult parallel, the filing of an adult complaint, after the first administration of the SDRRC in 2004. The instant offense was defined as the petition filed immediately prior to the 2004 SDRRC administration. Any petitions filed prior to the instant offense were considered prior criminal activity.
San Diego Regional Resiliency Check-up. The SDRRC produces three scores: the Total Resiliency Score (range -60 to 60), the Total Protective Score (range 0 to 60), and the Total Risk Score (range -60 to 0). The six domains measured in the SDRRC are delinquency, education, family, peer, substance use, and individual. Each domain includes 10 items; five protective items are scored in a positive direction and five risk items are scored in a negative direction. All of the domain scores range from -10 to 10. Each scale is scored in such a manner that the higher a score is, the more resilient the youth is. This means that a high resiliency score should be correlated with a low recidivism rate.
Description of Sample. The average age of the 2076 youth in the sample at the time of the SDRRC assessment in 2004 was 14.3 years and ranged from 11 to 15 years. The sample was 72 percent male and 28 percent female. Nearly half of the sample was Hispanic (49 percent), 20 percent were African-American, an additional 20 percent were Caucasian, with the remaining youth categorized as other (11 percent).
Ethnic groups scored differently on the SDRRC Total Resiliency Score. Hispanics had the lowest total resiliency score at 9.5 (lower scores indicate higher risk of recidivism). African-Americans had the second lowest score, 11.0, and Caucasians scored at the highest average score, 14.2. Statistically significant differences were observed between the Caucasian score and the Hispanic group.
SDRRC scores did not differ significantly across gender. Gender did, however, play a role in differences in recidivism. Males were significantly more likely to recidivate than females. Rates of recidivism also differed across ethnic groups. Post-hoc testing revealed that while African Americans and Mexican/Hispanics do not recidivate at significantly different levels, Caucasians differ from both other groups at a statistically significant level (p < .01). Please see tables 2 and 3 for information on recidivism rates across gender and ethnicity.
Characteristics of the SDRRC
Past research on the SDRRC suggests that the SDRRC subscales and scales are highly inter-correlated (Little, n.d.; Turner & Fain, 2006). In order to determine whether or not this was true within the current sample, the correlations between all total scale scores and domain scale scores were computed and can be seen in Table 4. The current sample has high correlations between each of the domain scales and the total scales; therefore we examined whether using any individual domain score would improve prediction of recidivism over the power of any of the summary scores.
Probation officers introduce bias into their scoring of the SDRRC. The amount of variance accounted for by probation officer was calculated using intraclass correlation coefficients (ICCs) (see Table 5). ICCs provide information about the degree of lack of independence in scores. An ICC greater than .05 indicates that nesting should be used in analyses. Given that all total scores and domain scale scores on the SDRRC have ICCs greater than .05, all analyses in this project were conducted controlling for probation officer bias through multilevel modeling.
Predictive validity. In order to determine which unique total scale or domain subscale serves as the best predictor for later delinquent behavior, correlations were run examining the relationship between recidivism and the various SDRRC scores (see Table 6). The Delinquency domain subscale score was one of the two best predictors of recidivism ( r = -38, p < .01). The Total Risk score was the second. The Delinquency domain score provides the most consistent predictive ability across ethnic and gender groups.
Once the bivariate correlations were complete, analyses were conducted to determine whether, in combination with the Delinquency domain score, the juvenile delinquent’s ethnicity, gender, or age were associated with later delinquent behavior. Table 7 provides the results of the logistic regression analysis. The results show that age at first offense was the most significant variable related to later delinquent behavior. This finding is in line with almost all research examining predictors of recidivism. The results also show that the Delinquency domain score is significantly related to recidivism.
The multi-variate findings lend support to the use of the Delinquency domain scale as a predictor of risk of recidivism. A youth’s ethnicity (African-American and Hispanic) was also significantly related to recidivism. This is consistent with the current study’s bivariate analyses, which revealed that African-American and Hispanic youth had significantly higher rates of recidivism. Being male was also found to be significantly related to recidivism, which again is consistent with previous findings that males have significantly higher base rates of recidivism, as compared to females. Interactions between the delinquency domain score and ethnicity were non-significant, indicating that ethnicity does not change the relationship between delinquency and recidivism.
Based on the combined strength of the previous and the current research, we conclude that the SDRRC is a validated risk/need assessment tool grounded in empirical research. The Delinquency domain scale score shows predictive validity over and above other common factors related to recidivistic behavior, such as age at first offense. While all of the total scales and domain scales are predictive of recidivistic behavior, the Delinquency domain scale had the strongest and most consistent relationship.
Use of the Delinquency subscale rather than the Total Resiliency score eliminates some of the previously noted problems of highly correlated domain scores and provides a static risk score that is useful for probation departments that want to place youth into supervision levels based on the risk that they pose to the community. The next step that is required in order for probation officers to make appropriate decisions about placement of youth in supervision is the creation of empirically based cutoff scores.
Useful cutoff scores are associated with recidivism rates. To establish cutoff scores, the recidivism rates associated with each delinquency score (i.e. -10, -9, -8, etc.) were examined. Cutoffs were created by using the overall sample Delinquency domain score mean and one standard deviation above and below the mean. These new groupings can be seen, along with their recidivism rates, in Table 8.
Use of the Delinquency domain score and the new delinquency cutoffs allows for better utilization of the SDRRC. This revision allows the SDRRC to separate the risk and needs assessment in order to more precisely address both.
Limitations and Future Directions of Research. Whenever a measurement of criminal behavior is chosen, there are inherent limitations in that choice. In this case, using criminal petitions means that those offenses that result in official intervention are used. Using officially documented criminal behavior may underestimate the amount of actual criminal behavior exhibited. A useful addition to this body of research would be a replication study using self-reported criminal behavior.
The truncation of the sample to younger adolescents was a useful starting point, but research could be more valuable if the sample were expanded to older adolescents.
While the severity groupings created for this project were done using recidivism data, it is important that these be replicated in future work. This replication would allow for a more confident use of the groupings as tools to determine the most appropriate level of supervision using an evidence-based approach.
Finally, the use of only one of the domain scale scores leaves open the question of what to do with the remainder of the SDRRC. The San Diego County Probation Department research unit is currently designing a research project that will use discriminate analysis to determine how the 50 remaining items on the SDRRC can best be configured to provide officers
and youth information that can be used to create customized and useful case plans.