Volume 72 Number 2
Federal Probation
 
     
     
 
Risk/Needs Assessment:
Is This the Best We Can Do?
 

Scott VanBenschoten
Office of Probation and Pretrial Services
Administrative Office of the U.S. Courts

The perception and use of risk and needs assessment tools has radically evolved over the past 30 years. The field of probation is beginning to accept what the academic literature has demonstrated for decades: objective actuarial risk/needs instruments more accurately predict risk and identify criminogenic needs than the clinical judgment of officers. This acceptance is demonstrated by the number of probation systems that continue to adopt the use of risk/needs assessment tools as part of their offender assessment process (Champion, 1993) (Hubbard et al., 2001). However, although probation administrators are adopting the use of risk/needs tools, are officers actually using these tools during supervision? Despite increased agency adoption of actuarial tools, research indicates that officers are not connecting either risk or criminogenic needs to the supervision of offenders (Bonta, Rugge, Sedo & Coles, 2004) (Lowenkamp, Latessa & Holsinger, 2006).

An opportunity now exists to develop a tool that officers will incorporate into their daily decision making. The federal probation system plans to develop a new risk/needs assessment tool that provides the officer with the necessary information to create a supervision plan that will increase the likelihood of recidivism reduction. If developed correctly, this tool will remove existing barriers and reduce officer resistance to actuarial risk and needs assessment. The following is an examination of why risk/needs assessment is needed in the supervision process and what technical characteristics are necessary as the federal probation system develops the next generation of risk/needs assessment tools.

The Purpose of a Risk/Needs Tool: Why is it Necessary?

The cornerstone of effective supervision is the use of a risk/needs tool. One of the foundational principles of community-based supervision is the Risk Principle (Andrews, Bonta & Hodge, 1990). This principle states that the intensity of an offender’s supervision and treatment must be proportional to his or her level of risk. Offenders with a high risk of recidivism must be intensely supervised and receive comprehensive treatment services. Conversely, offenders with a low risk of recidivism should receive minimal services. Recent research indicates that the failure to follow the risk principle leads to higher recidivism rates (Lowenkamp & Latessa, 2004).

Although determining an offender’s general risk level is critical, identifying the specific dynamic risks in an offender’s life that drive future crime is equally important. The dynamic risk factors, also known as criminogenic needs, lay a blueprint for the development of a case plan that will reduce recidivism.

In short, the primary reason to use a risk/needs tool is to help officers both identify which offenders need intensive intervention and what type of intervention is required. If officers are guessing about which offenders on their caseloads need intensive supervision and what type of interventions are required, then the field of probation will fail to produce optimal resultas. Probation agencies are responsible for reducing or at least stabilizing the likelihood of future criminal activity. The failure of officers to effectively use a risk/needs tool may actually be increasing the likelihood that offenders will further victimize society.

There are secondary reasons that encourage probation agencies to utilize a risk/needs tool. Risk/needs tools offer a mechanism for organization leaders to create an equitable workflow, effectively budget resources and develop a statistical understanding about what type of population is being served. These are all important byproducts of risk/needs assessment tools, but should never be the core reason for systems to implement risk and needs assessment. Unfortunately, more often than not, probation agencies use risk tools in the administrative capacity more than they do for case planning (Hubbard et al. 2001). Why are administrators more comfortable with these assessment tools than officers? Could it be that officers do not find these tools effective in their daily jobs?

The History of Risk and Needs Assessment

The evolution of risk assessment is understood through the development of successive “generations” of tools. Each generation utilized the most advanced methods of the time to predict the risk of recidivism and then applied the results of the assessment to supervision strategies. As the academic field of criminal justice developed, so did the understanding of the etiology of criminal behavior. This tradition continues today, with researchers continually refining their understanding of criminal behavior and the associated enhancements to risk/needs prediction tools.

The first generation of risk prediction is defined as an officer’s use of his or her clinical judgment. As officers gain knowledge and experience they begin to develop a “sense” of who is going to fail under community-based supervision. This clinical judgment extends into the case plan as officers filter their experience and knowledge to determine what interventions will protect the public and reduce long-tern recidivism. It is both intuitive and demonstrated in research that clinical judgment is an unreliable form of assessment. As articulated by Harris (2006), actuarial assessment consistently outperforms clinical (officer) judgment. Clinical judgment is wrought with bias and subjectivity. Additionally, officers often overlook important information while trivial information can be overemphasized (Latessa, 2003).

Clinical judgment was replaced in the late 1920’s with the Burgess Model of assessment (Latessa, 2003). This second generation tool used an objective scale to measure static offender characteristics. Each characteristic was given an equal weight and the total number of characteristics that were present in an offender were added together to produce a final score. Although rudimentary, the Burgess Scale ushered in a new era in risk assessment; objective scales that focused on inter-rater reliability (Connolly, 2003). Second generation tools were continually developed with further refinement. The most prominently utilized second generation scale in the United States was the Salient Factors Score, which was developed in the early 1970’s (Hoffman & Beck, 1974).

In the late 1970’s, the State of Wisconsin ushered in the third generation of risk assessment with the development of the Wisconsin Client Management Classification System. This tool utilized many of the past methods of static assessment, but introduced the key ingredient of a third generation tool; dynamic risk factors (Connolly, 2003). Unlike second generation tools, this new generation mixed factors that an offender could modify in the future (e.g. substance use) with traditional static factors to predict risk and needs. This change made risk/needs assessment more helpful with case planning. Although basic in the early version of these tools, the third generation of risk and needs assessment introduced the concept of measuring both negative and positive offender change over time (Bonta & Andrews, 2006). Later third generation tools, such as the Level of Service Inventory-Revised, refined the use of dynamic factors and included these factors in a needs assessment component to be used in tandem with the risk prediction score (Latessa, 2002).

The leap to fourth generation risk/needs assessment is defined by the integration of the assessment process and case planning. This integration creates a systematic intervention and monitoring system based on the results of a much larger series of assessed characteristics than is present in most third generation tools (Bonta & Andrews, 2007) The Level of Service Case Management Inventory is the most notable fourth generation risk/needs tool.

Risk Assessment in Federal Probation

The federal probation system has a long history of using risk assessment tools. This history begins in the 1970s with the use of the U.S.D.C. 75 Scale, which was later modified into the Risk Prediction Scale 80. This tool was used with probationers, while the Salient Factors Score, developed by the U.S. Sentencing Commission, was used on parolees. In 1991, the Criminal Law Committee, fearing that these scales were outdated, asked that a new and more predictive risk tool be developed. Over the course of 6 years the Federal Judicial Center developed the Risk Prediction Index (RPI), which was deployed as the mandatory risk tool in September of 1997 (Lombard & Hooper, 1998). Twelve years later, the RPI remains the primary risk prediction tool in the federal probation system.

The RPI is a classic second generation risk tool. The RPI contains 8 questions, 7 of which are static. Once calculated, the RPI produces a risk score that ranges from 0 to 9. The instructional guide for the RPI states that the RPI should never be revised or recalculated unless there was an error in the original calculation. This same guide further states that the results are not to be used to create a supervision level.

The purpose of the RPI is to aid officers in developing a case supervision plan, but it is very unclear how this can be accomplished. Without the benefit of case stratification system (supervision levels), the ability to detect change over time, identification of future criminal drivers and a direct connection between a risk assessment and a case plan, federal probation officers are not taking advantage of the advances in modern risk and need assessment.

In response to advancements in the risk/needs assessment field, several federal probation districts sought out and implemented off-the-shelf risk/needs assessment tools. These tools exceed what the RPI can provide, but still have limitations.

Risk/needs Assessment: A Call for Change

Now that risk/needs tools are being integrated into probation offices across the country, it is the responsibility of probation system leaders, academia and private industry to provide risk/needs tools that are useful for officers. Those who develop and research risk/needs tools have convinced probation system leaders that actuarial assessment is necessary and critical, yet officers are not as quick to accept and adopt the use of the currently available tools. This lack of acceptance can be seen by officers feeling that conducting a risk/needs assessment is not worth their time or even if the assessment is completed the outputs are ignored (Latessa, Cullen, & Gendreau, 2002).

The common assumption in the literature is that resistance to the use of a risk/needs tool comes from officers who believe their judgment is being threatened (Harris, 2006). Although feeling threatened can elicit a strong response, would officers defy the help of a tool if the benefits were self-evident? If you need to explain and continually market the use of risk/needs tools to officers, can the tool really be all that helpful? The disconnect between agencies adopting risk/needs tools and officers failing to apply the tools to their work may be explained by a simple statement: the currently available risk/needs tools are fraught with limitations that make them hard for an officer to use in his or her daily work.

No matter how well a particular tool predicts recidivism, it can only be judged on the connection between the outputs and supervision activity. The best tool ever developed is useless unless officers apply the outputs to shape how a case in managed. If officers don’t see how the risk/needs tool can help them better manage a case, then it relegates the tool to a data gathering instrument for administrators and researchers. Our field can do better and in fact we must do better, so the momentum developing in the field of risk assessment is not squandered.

How can the extensive data about risk and need prediction be operationalized into a risk/needs tool that officers perceive to be helpful rather than a hindrance? What are the risk/needs tool characteristics that are required to bridge this gap? It is time to consider the possibilities of a new generation of risk/needs tools. A generation of tools that translates complex and abstract academic research into simple and realistic case plans. A generation of tools that keeps pace with the sudden changes that often occur in an offender’s life. A generation of tools that finds the proper balance between accuracy and size.

The Limitations of Currently Available Risk/needs Tools

Each generation of risk/needs assessment tools has advantages and disadvantages; however, the common denominator among all of the tools is that they all produce better results than clinical judgment. Despite the discussion of the limitation of currently available risk/needs assessments that follows, the use of any validated actuarial risk/needs assessment tool is better than allowing officers to make judgments based on experience.

Statistical Accuracy: The accuracy of both the risk and need prediction is the most critical component to the future of risk assessment. Although companies that market off-the-shelf instruments make strong claims about the predictive statistical quality of their instrument, the question is deceptively complex. Because an instrument predicts well in the aggregate does not mean it predicts risk with every subpopulation. For example, an instrument may predict recidivism for the entire federal population with acceptable statistical validity; however, the type of questions asked in the tool may over-categorize female offenders as high risk because of work patterns associated with staying home to care for children. Additionally, the total risk score may be accurate, but many of the tool’s dynamic risk (criminogenic needs) subscales may not be valid. For example, a tool may accurately predict an offender’s risk, but the antisocial subscale may not be a valid measure on which to based referrals for service. Although actuarial risk prediction is stronger than clinical judgment, it is possible to see major improvement (Connolly, 2003).

Applicability of Off-The-Shelf Tools to Every Office: In addition to the internal statistical challenges mentioned above, every risk/needs assessment tool was developed using a specific sample. Often these sample sets were small and homogonous. This creates two very difficult problems for probation offices with diverse populations. First, if a risk/needs construction sample size were small, not enough people would fail in order to make accurate predictive models for certain events. This phenomenon, known as the “base rate problem”, can produce a lack of validity when the tool is applied in populations different from the original sample (Soderstrom & Leitner, 1997). For example, if a risk/needs tool were constructed using a sample that contained a small number of sex offenders who failed, it would be difficult to predict sex offender failure in any population in which the tool was applied. In addition to the issues associated with the base rate, a risk/needs tool is limited by the representation of the construction sample. A tool should only be used to predict risk on a population that is representative of the original construction sample (Connolly, 2003). For example, assessing risk on a population that is largely Native American and rural with a tool that was developed on a population of urban African Americans will not produce optimal results.

Dynamic Risk Assessment: Dynamic risk/needs assessment uses offender characteristics that can change to determine the overall risk score. If a tool is dynamic, the risk score will change as the offender’s condition improves or deteriorates. A significant limitation of the currently available risk/needs tools is that they are dynamic as they appear. There are several tools that identify a series of dynamic risk factors, each of which contribute to the overall risk score, but these subscales are derived using largely static questions. Currently available risk/needs tools allow the officer to develop an initial case plan based on statistically valid factors, but they do not offer the ability to reassess in real time as an offender’s circumstances change. For example, if an offender who has been free of substance use for the past 5 months suddenly tests positive for cocaine and verbalizes increasing antisocial thoughts, an officer knows that the offender’s risk of recidivism has increased. Many of the currently available tools would not be able to accurately adjust the risk score because they are unable to measure the presence and significance of the change.

Length of the Assessment: The length of assessment can vary greatly among the various off-the-shelf risk/needs tools. In general, second generation tools are the shortest and the fourth generation tools are the longest. The LS/CMI is one of the longer commercially available tools, with 124 questions; it requires an interview and a file review in order to complete the entire process. Some of the more comprehensive third generation and fourth generation tools can take 2 hours to complete. As officers struggle to supervise ever increasing caseloads, the thought of adding hours of work for each offender generates officer resistance. This concept can be understood by a simple cost/benefit equation. Is the amount of time required by an officer to complete the risk/needs assessment worth the payoff in improved case planning and supervision?

Face Validity and High Profile Offenders:Officers often perceive sex offenders, offenders with mental health issues and those with a propensity for violence as needing special attention. Often this need for special attention is confused with risk, when in fact the factors used to predict future criminal behavior for these types of offenders are the same for all offenders (Andrews and Bonta, 2006). What separates these types of offenders from the general offender population is that if they re-offend the level of damage to the community is great and the public relations consequences could be catastrophic.

When an officer completes a risk/needs assessment for one of these high-profile offenders, the risk/needs tool may place the offender into a low-risk category. Although most risk/needs tools offer an override option, the low-risk placement can damage the tool’s face validity and increase officer resistance. Some existing risk/needs tools attempt to assess for the risk of violence, the presence of a mental health condition, and past history of being a perpetrator, but no tool seamlessly integrates these items into a valid assessment that informs a case plan for offenders.

Case Planning: Risk/needs assessment must never be seen independent from its goal: informing the case plan. When a case plan is completed, how does an officer make sense of the outputs and develop a meaningful case plan? Current risk and needs assessment identifies the presence and severity of the risk and needs but fails to interpret what these outputs mean. As outputs of risk/needs tools become more sophisticated, officers should be better able to match offenders to appropriate services. Without a direct link between the assessment and the type, duration and intensity of treatment, all that has been accomplished through the use of a risk/needs assessment tool is lost by forcing the officer to make educated guesses.

The Next Generation of Risk/needs Prediction

It is easy to identify problems with the currently available technologies, but what enhancements need to be made to move the field of risk assessment forward? The following are a series of desired goals that the Federal Probation System is seeking to achieve in the development of the next generation of risk/needs assessment tools.

Improved Statistical Accuracy: With a clean slate, the development of the next generation of risk/needs assessment must be able to accurately predict the risk of recidivism and provide the identification of criminogenic needs. This includes developing scales for each criminogenic need that are statistically valid as stand alone assessments. For example, if a risk/needs assessment demonstrates that an offender has antisocial personality disorder, an officer can feel comfortable that this is a valid need that requires a specific intervention.

The current process of risk/needs assessment construction looks at what characteristics an offender has at intake and compares those characteristics to recidivism rates. Although this is a good starting point, there are countless variables that impact an individual over the course of supervision that can influence recidivism rates. Also, the interplay between variables can be complex and difficult to disentangle. To address these and other complex statistical concerns, it is critical to consider alternative statistical methods for predicting risk and needs.

Improving the Generalizability of Risk/needs Tools: The federal probation system has a unique opportunity to create a risk/needs tool that can be generalized to any probation system in the United States. With a caseload of over 100,000 offenders in every state and major city that includes every culture, race, gender and socioeconomic status in the United States, the federal probation system can build a large construction sample to mitigate under-representation and base rate issues that plague the currently available tools. If constructed correctly, questions such as “Is the tool valid for female offenders?” and “Does the tool accurately predict for Native American populations?” will be answerable.

Length of the Assessment: How can a risk/needs tool be large enough to be reliable and valid, yet take as little of an officer time as possible? The primary answer to this question is to integrate the case management system and the risk assessment tool. A risk/needs tool should be able to pull as much information from the case management system as possible in order to reduce redundant data entry. Static data should not require reentry for reassessments. These simple improvements can save an officer’s time.

The Development of an Event-Driven Risk/needs Tool: Officers see offenders in real time. They see offenders as ever-changing people who are often in a period of rapid change. These changes, good or bad, have the potential for quick and profound impacts on the risk of recidivism. Officers cannot wait for 3 or 6 month intervals to reassess changes when the world is moving in real time. It is time that dynamic risk prediction become just that; dynamic. Research indicates that dynamic factors better predict the risk of recidivism than static factors (Connolly, 2003)..

In order to achieve a truly dynamic tool, it must be integrated into the case management system. The risk/needs tool must be responsive to changes in an offender’s condition (e.g. positive drug tests, loss of employment, and change of address). The only way to provide real-time changes in dynamic factors to the risk/needs tool is by integrating the case management system and the risk/needs tool.

Face Validity and High-Profile Cases: In order to improve an officer’s perception that a risk/needs tool is effective for all types of offenders it must address their concerns about low-risk but potentially violent high-profile offenders. Officers can use additional tools that predict violence and combine the results to create a case plan. For example, the Psychopathy Checklist Screening Version (PC-SV) can predict the likelihood of violent recidivism ( Wilson, 2003). Although it is tempting to use a tool like the PS-SV, and then combine it with the outputs from the risk/needs tools, research indicates that this approach produces practical and validity concerns (Campbell, French & Gendreau, 2007). This prediction can become more complicated with additional, potentially violent subpopulations, such as sex offenders. Any development of a new generation of risk assessment should include trailer assessments that provide officers information about not just the risk of recidivism, but also the potential for community harm if recidivism occurs. These results should be integrated into the tool outputs and subsequent case plan.

For example, if a first-offense pedophile scores as a low-risk on a risk/needs tool, should an officer funnel him or her onto an administrative caseload with other low-risk offenders? Instead, the officer should be armed with the information about offender propensity to commit another sex offense and the level of violence he or she poses to the community.

The next generation of risk/needs assessments should provide screening questions that can expand into comprehensive assessments for high profile potentially violent populations. At a minimum, any new tool should include screening and assessment sections for sex offenders, mentally disordered offenders with threat/control-override symptomatology, and offenders with a propensity for violence. The next generation of tools should also connect the outputs of the trailer assessment to the case plan.

Case Planning: Several existing risk/needs tools have begun the process of connecting the outputs from the assessment to the case plan. This includes connecting possible supervision strategies to assessment outputs. Through business rules built into these systems, officers are forced to at least address the criminogenic needs that are identified.

Although this is a good starting point, the next generation of risk/needs assessment must take the next step. Tools must begin identifying what services, in what duration with what level of intensity will produce the best outcomes based on the assessed needs. This advancement in assessment will require a tremendous amount of research and advanced statistical methodology, but the field of probation must demand statistically valid connections between risk/needs assessment, case planning and outcomes.

Conclusion

The field of risk/needs assessment has seen advancement over the past decade, but more must be demanded. The federal probation system has the opportunity to create a tool that officers determine they need instead of a tool that they are forced to use. With advancements in both the statistical integrity and practical applicability, the opportunity exists for a new generation in the field of risk assessment. Some of these recommendations require the development of statistical technologies that are yet to be developed but advancements can and must be made to improve risk/needs assessment and case planning in order to make officers more effective in the recidivism reduction.