Christopher T. Lowenkamp
Richard Lemke
Edward Latessa
University of Cincinnati
Pretrial Assessment
History of Pretrial Assessments
The Use of Pretrial Risk Assessment Tools
The Current Project
Risk and Need Measures
Outcome Measures
Data Collection Procedures
Data Analysis
Sample
Item Selection
Validation
Limitations
Conclusions
THE LAST TWO decades have witnessed a sharp increase in the use of offender assessment instruments across every stage of thecriminal justice system. The reasons for this increase in use are generally centered on the problems of overcrowding and shrinking monetary resources (Jones, 1996). These conditions are ubiquitous across every level of the criminal justice system, forcing correctional institutions at all levels to accommodate themselves to these expanding populations with diminishing resources. Correctional institutions (Beck, Karberg, and Harrison, 2002; Sabol and Couture, 2008), community corrections agencies (Glaze and Bonczar, 2007), and jails (Sabol and Minton, 2008) have had significant increases in their respective populations over the last decade. While population growth has slowed for jail populations in the last eight years, it has not declined. Despite a continual growth in jail capacity over the last few years, estimates calculate that local jails operated at 96 percent capacity, which is significantly higher than years before (Sabol and Minton, 2008).
In the search for assistance, many agencies have employed classification techniques that develop typologies of an individual’s level of risk. Researchers have summarized the numerous ways these typologies benefit the agencies that utilize them, including: minimizing the subjective personal bias that exists in decision making, improving the placement of individuals for treatment and public safety, aiding in protecting against legal scrutiny, and improving the allocation of resources (Andrews and Bonta, 2007; Andrews, 1995; Bonta, 1996; Jones, 1996; Latessa and Allen, 2003; Van Voorhis, Braswell and Lester, 2004). While a majority of these efforts are based on probation or prison populations, the underlying classification principles are not limited to these agencies and are also applicable to pretrial goals and decisions.
Pretrial Assessment
Pretrial agencies are routinely required to perform numerous services at the initial phases of the criminal justice process. These include the gathering of information about the defendant in order to make an informed decision about pretrial release, assessing the likelihood that a defendant will fail to appear to a mandatory court appointment or will be re-arrested, providing recommendations regarding conditional release including supervision, and facilitating voluntary participation in treatment programs. Some agencies have even implemented drug screening (Henry and Clark, 1999) as part of their routine duties.
The pretrial agencies’ most fundamental decision lies in the recommendation made to the courts regarding whether an individual is detained or released and, if released, with (or without) some conditional requirements. The bail decision generally considers two factors: 1) the likelihood that a defendant will appear for subsequent court dates and 2) the likelihood that public safety will not be jeopardized as a result of the defendant’s return to their community. This is an important decision, since it must strike a balance between the individual’s personal freedom and the public safety. Unlike agents in subsequent stages of the criminal justice process, pretrial agencies have the unique responsibility of working with arrestees prior to disposition. Individuals they see may or may not eventually be convicted of the offenses for which they were arrested. The defendant is presumed innocent and the law requires the treatment of the defendant to reflect that presumption. Therefore, it is even more vital that the decision be made in a consistent and fair manner to ensure the protection of defendants’ rights and the integrity of the criminal justice process.
Since the arrestee is presumed innocent, the American Bar Association (American Bar Association, 2002), the National Association of Pretrial Services Agencies (National Association of Pretrial Services Agencies, 2004), and the National Institute of Justice (Mahoney et al., 2001) have issued strong recommendations about the adoption and use of objective standardized guidelines or criteria (i.e., risk assessments) in assisting court and pretrial agents in making bail decisions. These organizations believe that improved bail decisions provide substantial benefits to the defendants and the criminal justice system, including increased public safety, protection of civil liberties with minimal disruption in the lives of those presumed innocent, efficiently managed jail space and staff, and a reduction in disparity for bail decisions/release decisions (VanNostrand, 2003, 2007).
A structured objective classification process aids pretrial services in the replacement of subjective judgment with objective criteria in the determination of a defendant’s risk to the community and the likelihood of nonattendance during court proceedings. Previous research indicates that bail decisions have often been made arbitrarily and that little information, with the exception of previous criminal history and current charges, was provided to judicial officers to assist in the determination of appropriate bond amounts or the conditions of bail (Maxwell, 1999). Since that time, there has been a professional movement to incorporate explicit, objective criteria into pretrial decision making (Clark and Henry, 2003). This has been done with the rationale that consistently applied explicit criteria can assist in minimizing the disparate outcomes resulting from arbitrary and subjective decisions.
Classification instruments at the pretrial stage are designed to identify arrestees who are more likely to be a danger to the public and less likely to appear in court once released. In addition, these tools aid the decision maker in choosing which arrestees should receive available services and, perhaps just as important, which individuals do not need those services. Such information has substantial implications for referral processes, placement, and supervision decisions, and can impact significantly the allocation of financial resources. Integral to the identification and classification of risk is the principle that each category should have differential official responses. In other words, arrestees who are assessed at different risks elicit different responses from that agency. To accurately identify high- and low-risk arrestees, an objective and validated assessment instrument should be used.
Evidence from correctional research also indicates that providing intensive services and supervision to low-risk offenders does little to change their likelihood of recidivism and, worse, occasionally increases it. Conversely, high-risk offenders often need services and supervision more intensive in nature and of longer duration (Lowenkamp and Latessa 2002; Lowenkamp, Pealer, Smith, and Latessa 2006). The adoption of this principle has been widespread, since the supporting evidence transcends community corrections (Brown 1996; Lowenkamp and Latessa 2002), institutional corrections (Andrews and Bonta 2007; Armstrong and Bourgon 2001), or existing pretrial assessments (VanNostrand, 2003, 2007).
Accurate classification of pretrial defendants has substantial implications for the bail process, affecting both the decision to release and the conditions imposed on the defendant once released. By efficiently assessing and releasing defendants who pose little danger to the public or are likely to comply with court dates, the process upholds and affirms the arrestee’s constitutional protections and minimizes the infringement of their day-today lives. For jails, releasing these low-risk arrestees frees previously occupied bed space in a jail system that is nearing capacity. By classifying some individuals as low risk, agencies may then reallocate resources (community referrals, supervision decisions, etc.) to higher-risk clients.
History of Pretrial Assessments
While the growing empirical support of these principles is recent, the principles and underlying premises (equality, resource management, public safety) are not, and the recognition of these goals was the impetus behind the development and implementation of pretrial services agencies nationwide. In 1961, the Vera Institute created the first pretrial screening program, the Manhattan bail project, which sought to aid pretrial agents in making release decisions. The factors incorporated included the defendants’ ties to the area, employment status, education, and prior criminal record. As the first such program of its kind, the Manhattan bail project produced two significant findings. The first was that financial limitations of defendants, and the frequent use of finances as bail, forced many defendants to remain in pretrial custody. Second, individuals with strong ties to the community were likely to appear at required court proceedings, even if they were not assigned financial bail. The experiences and findings from the Manhattan bail project were integral to the bail reform movement, which emphasized releasing defendants pending disposition of their trial without financial surety. However, in order to do this, information about the defendant was needed prior to release. Because of the risk of releasing a charged individual into the community, the accuracy and verification of the information was considered paramount.
Recently, several jurisdictions have attempted to develop and implement objective risk assessments. In 2003, the Urban Institute Justice Policy Center developed and validated a risk prediction instrument on a large sample of Washington, DC pretrial defendants (Winterfield, Coggeshall, Harrell 2003). The resulting instrument comprised 22 items in two separate subscales; the safety risk scale and the appearance risk scale. Nearly all the incorporated items that predicted either of the outcomes were components of criminal history, demographics (i.e., age, citizenship), current criminal charge, and drug involvement/testing.
Individuals assessed on the Urban Risk Prediction Instrument were categorized in one of five risk categories that represented the five alternative formal responses utilized in the jurisdiction. The tool categories had moderate correlations, with .21 for predicting failure to appear and .16 for recidivism. The authors suggest that the moderate values may have been a result of the formulated cut-off scores, since nearly half the sample was classified at moderate risk.
The work of the Vera and Urban Institutes notwithstanding, there have been few multi-site, racially diverse, empirically validated pretrial assessments for use in the United States. One recent attempt is the Virginia pretrial risk assessment, which comprises nine risk factors (VanNostrand, 2003). Six of the nine risk factors are derived from an individual’s criminal history, and three incorporate additional factors such as residential stability, employment, and drug use. The risk assessment is administered by interviewing the defendant; however, individuals conducting the assessment are required to verify all possible information provided by the defendant. The author reported that the assessment successfully differentiated defendants into the five risk categories, which had significant differences in failure rates. These failure rates held up even when gender, race, and income were isolated and examined.
The Use of Pretrial Risk Assessment Tools
The development and implementation of a standardized tool is often costly, and is a task that few criminal justice agencies have the finances or experience to undertake (Jones, 1996). In the most recent survey of pretrial services programs by the U.S. Department of Justice (Clark and Henry, 2003), the authors found that fewer than 1 in 4 pretrial programs relied exclusively on objective criteria when making bail recommendations. Of those agencies using a risk assessment process, 39 percent reported adopting a classification scheme from another jurisdiction, while only 25 percent reported developing it by using data from their own jurisdiction.
It appears that jurisdictions often rely on implementing pre-existing tools derived for similar purposes but on different samples. Given that it is unlikely for a single instrument to have universal applicability, research has suggested that adopted assessments should be piloted and validated on the jurisdiction implementing the tool, since the instrument or its classification scales may not be valid for the agency’s specific population (Gottfredson & Moriarty, 2006; Jones, 1996; Wright, Clear, & Dickson, 1984). Specifically, it should be shown that the instrument can successfully predict the outcomes of interest for the population being served (Flores, Travis & Latessa, 2003; Lowenkamp & Latessa, 2002).
When choosing an assessment tool for implementation in a new jurisdiction, research has identified characteristics of effective assessment practices. These characteristics include choosing an assessment that 1) contains items that are theoretically derived, 2) contains multiple measures of the constructs being assessed, and 3) measures multiple domains that are empirically related to the behavior being predicted (Monahan and Steadman 1994; Monahan 1996; Bonta 1996). These characteristics appear to be relevant regardless of the correctional outcomes being predicted.
The Current Project
The current study details the construction and validation of a pretrial risk assessment. This assessment is the first component of a larger project that includes development of distinct tools for each stage of the criminal justice system, including probation, prison intake, and community re-entry. The overall purpose of the project is to create a risk assessment process that promotes more effective classification and efficient resource allocation and facilitates risk communication across criminal justice agencies.
Risk and Need Measures
The data collection tool integrated structured interview questions and a self-report questionnaire. The data collection tool consisted of 63 items covering 8 theoretical risk and need domains. These domains are: criminal behavior history, pretrial supervision history, drug/alcohol abuse, employment and attitudes about employment, residential stability, mental and medical health, criminal attitudes and orientation, and criminal associations. Table 1 reports the various domains and the number of individual items within each category.
Outcome Measures
Individual cases were tracked using multiple methods, including county court records, reports from pretrial supervision officers, and a computerized database of criminal records that contained detailed case file information on individuals’ criminal histories. The outcomes tracked for each defendant were whether they failed to attend a mandatory court appearance (FTA) and whether any new offense occurred while they were released pending sentencing. Both of these outcomes were coded dichotomously, with a value of 1 indicating the occurrence of this outcome and a 0 indicating that the outcome had not occurred for that case.
Data Collection Procedures
Defendants who were referred to five pretrial services agencies in two states during the data collection phase were asked by pretrial caseworkers to participate in the research. Those who volunteered were provided with a secluded interview space where the data collection tool was administered. The first half of the data collection tool was administered by a trained research staff member, who used the structured data collection tool to gather information on criminal history, probation history, employment, mental and medical health, and residency. Once the interview was completed, defendants then filled out a selfreport questionnaire. Information obtained was verified, when possible, through a review of the individual’s case file information. To ensure valid and candid responses from participants, a series of measures were employed, including the acquisition of a certificate of confidentiality and verification with official records. Once the interview data were collected, defendants were tracked until case sentencing and outcome data were coded and entered into the database.
Data Analysis
Once data collection was complete, a split-half methodology was applied to the complete pretrial sample (N=342). This procedure randomly divides our single research sample into two separate but equal subsamples: construction and validation. The construction subsample was used to identify individual items from the data collection tool that had a significant relationship with the outcome measures. Once identified, these items were organized into a draft assessment instrument and both the individual item correlations and the total assessment scores were re-examined, using only those cases in the validation sample. While the limitation of this procedure is the reduction of the sample size, the benefit is that it minimizes error from overmatching the assessment to a specific sample.
Sample
The pretrial project utilized multiple pretrial agencies across two states to generate a total sample of 342 adult defendants placed on pretrial release (with or without supervision) between June of 2006 and July of 2007. Table 2 presents the description of the sample and results from the outcome measures. The sample ranged in age between 18 and 59 years (Mean=33.8) and was primarily male (74.3 percent), non-white (58.8 percent), had been arrested on a felony (59.6 percent), had a high-school education or greater (57 percent) and was single (66 percent). With respect to outcome measures, 18.4 percent of the defendants failed to appear at their court appointment, and 15.8 percent were rearrested during pretrial supervision.
Item Selection
From a total of 64 possible predictors, items were selected based on their relationship to the two outcomes of interest: failure to appear and new arrest under pretrial supervision. To determine which items were selected, chisquare statistics were utilized. These statistics identified six predictive items. Table 3 shows these six factors. It is important to note that the number reported in the table represents only the number of individuals who received a FTA or were arrested while under supervision. As indicated below, six factors emerged as significant predictors (p<.05) of a subsequent FTA. These factors can be subsumed under three domains; criminal history (age of defendant at first arrest, the number of previous FTAs, three or more prior jail incarcerations), drug use (any drug use history, severity of reported problems resulting from use), and employment (employment status at time of arrest). In terms of criminal history, individuals who were more likely to have a warrant for failure to appear were arrested at a younger age (x2=3.93, p <.05), had a previous failure to appear (x2=4.52, p <.05), and had at least one jail sentence imposed upon conviction (x2=7.18, p <.05). In addition, these defendants were more likely to be unemployed (x2=7.31, p <.05), report recent drug use (x2=5.17, p <.05), and report greater personal and legal problems when it came to drug use (x2=6.06, p <.05).
Six of the items that were identified as significant using the construction sample were combined into a tentative pretrial risk assessment instrument. Two additional items, residential stability and the number of occurrences of FTA within the last two years, were incorporated into the assessment despite their statistical non-significance. The decision to include these items was based on two factors. First, both of these items have traditionally been held as predictors of failure to appear and success on pretrial supervision. Second, a review of the instrument by field staff and the judiciary indicated that these items, regardless of their statistical relationship with outcomes, were required for the instrument to have face validity. Therefore, these items were added to the model and had little effect on predictive power. The relationship these items have with FTA and arrest will be reviewed again following a fairly large field test of the instrument. Table 4 elaborates on the individual items and scoring weight assigned to each of those items.
Table 5 presents the range of possible risk values (0–10) for the pretrial assessment, the quantity and proportion of the sample that was assigned to each value, and the failure rates of the two outcome measures. When the table is examined in terms of the distribution of assessment scores, the largest group of the sample (n=38) scored a five on the assessment, with the second largest group scoring either a four (n=35) or a three (n=27). The overall distribution of scores is bell-shaped, with most individuals falling into the center of the distribution, with fewer cases at either extreme end. It is important to note that this distribution holds regardless of whether the sample is examined in its entirety or the examiner focuses exclusively on the validation sample.
A higher score corresponds to a greater likelihood of failing to appear or supervision failure. For example, 14.8 percent of the individuals who received a score of 3 received a FTA, and 7.4 percent were arrested for a new crime under supervision. Comparatively, 37.5 of the individuals who received a score of 8 received a FTA, and 25 percent were arrested for a new crime under supervision.
A series of cross-tabulation analyses were used to create cut-off values in the distribution of risk scores of the construction sample. From these cross-tabulations, an optimal cut-off score was devised with three categories. Individuals who scored a 0-3 on the risk assessment were classified as low risk, or individuals unlikely either to receive a FTA or to commit a new offense while under supervision. Individuals who scored a 4-7 on the risk assessment were classified as medium risk, and were substantively more likely than low-risk individuals to reoffend in either outcome measure. High-risk individuals scored an 8, 9, or 10 on the assessment and were the most likely classification to fail at either outcome. A majority of the defendants in the construction sample were classified as medium risk (n=108), followed by low risk (n=52), with fewer defendants classified as high risk (n=20). Table 6 reports the number of defendants assigned to each risk level and failure rates attributed to those risk levels.
Validation
With the assessment constructed and risk level cutoffs created, the next analyses have the purpose of testing the assessment on the validation sample. To test the linear relationship between the pretrial assessment score and outcomes, a series of correlations were conducted. Correlations were conducted for the whole sample, the construction and validation samples, and by gender, and are reported in Table 7. Overall, the total score was significantly correlated with both outcome measures. Without exception, the assessment score had consistent correlations above .23 with failure to appear for each sample. The total score is also significantly correlated with a new arrest while under supervision. Significant correlations range from .211 to .235. When the sample was disaggregated by gender, the relationship between risk assessment total score and new arrest was not significant for female defendants. However, the assessment score was predictive for female defendants when failure to appear was the outcome of interest. In fact, the correlation for female defendants was noticeably larger in magnitude than it was for male defendants.
The practical utility of a risk assessment lies in its ability to accurately distinguish between risk groups of defendants (low, medium, and high) for the purposes of case planning, resource allocation, and supervision. Therefore, the second series of analyses examines the ability of the pretrial assessment to distinguish among risk groups by evaluating their respective failure rates. As demonstrated by the analysis, there is considerable difference in the failure rates between risk categories for both failure to appear (x2= 11.65, p<.01) and new arrest while under supervision (x2= 9.17, p<.01). Specific to predicting failure to appear, 2.1 percent of those classified as low risk re-offended, compared to 19.6 percent of medium-risk defendants, and 33.3 percent of high-risk defendants. A similar significant distribution was found for new arrests under pretrial supervision. Only 4.3 percent of those classified as low-risk defendants were arrested, compared to 19.6 percent of medium-risk defendants, and 33.3 percent for high-risk groups. These failure rates are illustrated in Graph 1. 1
Limitations
The current study has some methodological limitations that should be addressed. The first is the relatively small size of the sample. Due to the size of the sample and the voluntary nature of the research, arguments could be made that this sample is likely to represent a subgroup of pretrial defendants who are lower risk or who are more apt to comply with supervision requirements. If this is the case, there is a chance that the sample characteristics could have limited the robustness of the predictive ability of items and the assessment in its entirety. This problem is inherent in the initial study of any assessment and only additional studies and samples will confirm either argument. In addition, the applicability of the pretrial assessment for special offending populations (sex offenders, white collar criminals, etc.) should be tentative until additional studies can determine the robustness of the predictive validity for those samples.
Conclusions
The current study attempted to add to the risk assessment literature by constructing and validating a pretrial risk instrument. Using data from 342 adult offenders, eight items were selected to comprise the pretrial instrument, including age at first arrest, history of FTA, recent occurrence of FTA, prior jail incarcerations, employment status, drug use, drug-related problems, and residential stability. With the validation sample, the pretrial assessment score was found to be significantly correlated with both failure to appear (r=.263) and new arrest (r=.235). When the cut-off scores were examined, the pretrial instrument successfully differentiated between low-risk defendants, medium-risk defendants and high-risk defendants.
In addition to the construction and validation of the pretrial risk assessment, the current project had the advantage of incorporating and examining additional factors in the data collection process, such as criminal thinking and criminal peers (Andrews and Bonta 2007), and obstacles to adhering to court requirements (medical and mental health, lack of transportation, etc.), that could impact the ability or desire of an individual to adhere to court requirements. For the current project, none of the items in these domains was significant in predicting FTA or new arrest while under pretrial supervision. This lack of significance could be due to the operationalization of these variables (i.e., use of self report for criminal thinking and criminal peers) or under-reporting of criminal values or criminal acquaintances/friends for fear of self-incrimination, or these variables could be less important when predicting FTA. For example, having criminal peers may not as greatly influence an individual’s probability of appearing in court as engaging in crime in general.
Notwithstanding the limitation reported above, the pretrial assessment was found to have good predictive validity for a population of adult defendants. The ability to identify higher-risk defendants for the purposes of safety and the allocation of agency resources is an important aspect of a valid risk assessment. Just as important, risk assessments should also accurately identify those low-risk defendants who do not need such supervision or detention. As Lowenkamp and Latessa (2004) suggest, assigning intense supervision or preventative detention to low-risk defendants either removes the individual from pro-social aspects of their life or exposes them to risk factors that were previously nonexistent in the defendant’s life. Either way, these actions put the defendant at greater risk of recidivism or negative supervision outcomes.
In addition, a multi-site pilot of the pretrial assessment is currently underway that promises to address several of the noted limitations. First, the implementation of pilot sites will address identified deficits in the characteristics of the sample. Pilot sites are comprised of both rural and urban jurisdictions, which will allow for a more geographically diverse sample. Furthermore, the inclusion of a greater sample of female defendants will assist in addressing whether the sample size is the driving force behind the non-significant correlation for new arrest, or if the eight variables included in the device are gender sensitive.
Second, to address inconsistencies between the reported findings and past research, some variables were incorporated into the pilot assessment instrument, permitting agencies to pilot case-managing specific questions or allow for the data-driven clarification of existing items. For example, if a defendant lives in the same residence for 3 years (a score of 0 on item #8), but that residence is in another jurisdiction, would the defendant be at greater risk of failing to appear? Data from the pilot regarding these follow-up questions will better assist our understanding of pretrial predictors and the current assessment instrument.