Timothy P. Cadigan
Chief, Data Analysis Branch
Office of Probation and Pretrial Services
Administrative Office of the U.S. Courts
THE APPLICATION of evidence-based practices (EBPs) could potentially revolutionize the field of pretrial services. Pretrial services programs across the country are looking to apply these practices in hopes of seeing tangible results in the form of increased release rates, while maintaining or improving appearance and safety rates. Yet the revolution seems stalled as pretrial services agencies ponder questions about the applicability of post-conviction EBPs to achieving their outcomes: ensuring a defendant’s appearance in court and protecting the community from crime. There are significant issues to consider: Do post-conviction evidence-based practices that were developed to reduce long-term recidivism rates impact these unique pretrial outcomes? And does the application of post-conviction supervision EBPs infringe on the constitutional rights of individuals not convicted of a crime?
The numerous potential problems of applying post conviction EBPs in a pretrial setting are easy to envision:
- an officer places a defendant in a post-conviction EBP program and subjects the defendant to situations where he or she must admit to criminal behaviors or risk program failure.
- an officer employs motivational interviewing in an effort to resolve the defendant’s ambivalence about his or her drug use, resulting in the defendant’s admission of heroin use, which was previously undocumented.
Given possible scenarios such as these, the carte blanche application of post-conviction supervision EBP strategies in pretrial services may be dangerous.1
How does federal pretrial services address this shift toward relying on evidence in a proactive manner? The answer lies in going back to the basics to develop evidence-based practices that are specific to pretrial services outcomes. Pretrial services must develop specific EBP practices that fit into the context of being least restrictive and “reasonably necessary” to assure a defendant’s appearance in court and the safety of the community pending trial. Drifting from this core mission toward a focus on changing defendant behavior over the long term is simply outside of the scope of the Bail Reform Act of 1984. The system must refocus on the core mission and both develop and disseminate proof that specific practices being employed by officers optimize pretrial outcomes.
Cullen, Myer, and Latessa capture the essence of the problem quite nicely in “Eight Lessons From Moneyball.” Pretrial services and baseball “share the common plight of being social domains in which irrational practices, based on unsystematic observation and common sense, have been enshrined with legitimacy.”2 If it doesn’t “feel right” or “pass the smell test,” it’s dismissed no matter the level and degree of data to support it. The opposite caution is warranted, however, since the importance of data quality cannot be overstated. If we’re going to make decisions using our data, that data needs to be accurate to within acceptable industry standards: between 98 and 99 percent accuracy, depending on the importance of the variable. Failure to get the data accurate dooms any initiative that purports to employ data as the basis for program decisions.
What are the first steps? The federal pretrial services system must develop methods to make bail recommendations to judges that are nationally consistent and backed by evidence. Who conducts the interview or in what district a defendant finds himself or herself charged with a crime in should not affect the bail recommendation as much as the risk the defendant poses to the community. The initial focus of any pretrial services EBP effort should be the development of a risk prediction tool that is empirically based. The federal system has begun the process of developing a risk prediction device and should be able to test that device in 2009.
Three pretrial services offices ( Hawaii, Michigan Eastern, and Oregon) in the federal system are currently working towards implementing evidence-based practices in their districts. Each office is focusing on different pieces. For example, Hawaii has been trying to integrate motivational interviewing into the pretrial services program and Michigan Eastern has been working on an ongoing methodology to obtain judicial feedback on pretrial services reports. Those initiatives are part of a larger effort in the federal system known as Research 2 Results (R2R). The R2R program provides 18 federal probation and pretrial services offices extra funding in an effort to “pilot” these processes for the system. One of the early lessons learned is the need for an organizational assessment: Is the office in a position to take on this endeavor and if so, what areas do they most need to work on to prepare them for the initiative?
For the most part, federal pretrial services offices have achieved the organizational assessment by employing an outside contractor, experienced in implementing evidence-based practices in a pretrial services setting, to perform the assessment and provide the office with feedback on the readiness of the office to proceed. Generally, those reports include recommendations for the office to follow to prepare for implementation.
Below is an excerpt from an organizational assessment report that identifies key components and activities to achieve readiness3:
The purpose of the assessment was to examine the efficiency and effectiveness of the district with an emphasis on the organization culture and consistency with Legal and Evidence Based Practices (LEBP). LEBP are interventions and practices that are consistent with the legal and constitutional rights afforded to accused persons awaiting trial and methods research have proven to be effective in decreasing failures to appear in court and danger to the community during the pretrial stage. The organization assessment included the following components of a district’s readiness:
- assess the organization culture through the administration of an organization culture survey, conduct on-site interviews, and observe the climate/culture of the agency;
- identify other potential areas of improvement relating to organization development, including but not limited to, leadership development, organizational performance, succession planning, diversity management, and process improvement;
- conduct Pretrial Services Legal and Evidence Based Practices (LEBP) training;
- examine the current operations of the agency in relation to LEBP for pretrial services; and
- lead a session to guide the LEBP committee through a preliminary planning process intended to address any identified issues related to organization development and to begin the implementation of LEBP.
Performing 90 more of these visits to cover all districts could be time consuming and costly. The question, then, is whether other methodologies could assist the federal pretrial services system in determining the readiness of its district offices for evidence-based practices. One solution that comes to mind is the program review process. The federal system currently conducts staggered program reviews of its probation and pretrial services district offices. The Office of Probation and Pretrial Services attempts to complete 20 reviews annually. A written report is prepared for each review and submitted to the chief judge with a copy to the chief pretrial services/probation officer. Could those program review reports be mined to identify potential issues of concern for pretrial services in implementing evidence-based-practices?
The federal system is embarking on a unique opportunity and needs to approach that opportunity without preconceived ideas or limitations. To enable the federal system to better navigate these uncharted waters we need to employ a compass rather than a map. Soon after we embark on our journey the wind will blow and sands will shift and our carefully planned map to EBP nirvana could be worthless. However, our compass will still work in the changed environment. Therefore, we should not limit ourselves to the map of evidence-based practices, but employ the compass of R2R. This embodies the essence of what we are trying to accomplish, making programmatic decisions using research as the only yardstick. A given practice either works to achieve our goals and mission or it does not; if the latter is the case, it should be discontinued.
“The Guide to Developing Legal and Evidence based Practices for Pretrial Services Investigations and Supervision”, above, is an attempt to provide a pictorial representation of some of the same concepts, goals, and steps provided in this text.
Mission, Goals, and Outcomes
What is the primary mission of pretrial services? While it may seem clear to those of us who have been around for 20 years, we need to do a better job of inculcating in new staff the mission of our program. The Pretrial Services Agency in Washington DC offers an excellent example of such inculcation. This agency provides the pretrial services function to all cases in the District of Columbia; therefore, although they are a federal agency, their caseload range is quite unique. One of the things they have done outstandingly well is educate their staff in the mission of their agency. Staff know what they are about. Their mission is conceived in a sense as a “calling” and something anyone would be proud to participate in.
A number of other functions for pretrial services officers have bubbled up, and we need to consider whether these are functions that pretrial services should be performing or whether they distract from the core mission:
- Is pretrial services an opportunity to get a jump start on the presentence investigation?
- obtain verification of employment, education, etc.
- obtain written prior criminal record information
- obtain prior presentence reports.
- Is pretrial services an opportunity to prepare the defendant for his period of incarceration?
- Is pretrial services an opportunity to set, process and pay for mental health evaluations of detained defendants?
This list is by no means exhaustive, and many other mission-creep functions are being performed by various pretrial services offices. The point is that all of these non-core tasks must be considered, evaluated, and prioritized in light of their impact on the primary mission and goals of the pretrial services program. An excellent place to start is with the Pretrial Services Act of 1982 (ACT).
The Pretrial Services Act of 1982 began a process of getting pretrial services into the fabric of the federal criminal justice system. The subject matter of the vast majority of text in committee and subcommittee reports and testimony concerns the reduction of unnecessary detention. In fact, several senators and congressmen influential in ultimately passing the ACT, including Biden, Kennedy, and Hughes, cited the reduction of unnecessary detention as their primary motivation in voting for the Act and subsequently passed legislation that included the Bail Reform Act of 1984.
The federal pretrial services system—and pretrial services generally for that matter—has published, monitored, and based its operations on crucial outcome measures since its inception in the 1960s and 1970s. While we have not yet performed the requisite formal studies of our practices to fully qualify as evidence-based, we are by no means subordinate in the area of evidence-based practices to our post-conviction colleagues. However, we place ourselves in that subordinate position with the current mentality of adopting post-conviction evidence-based practices, rather than embarking on the testing and documentation of our own successful pretrial services practices.
The fourth and final report on the Implementation of Title II of the Speedy Trial Act of 1974 was published on June 29, 1979. That report, based on the initial research that demonstrated the documented evidence of the effectiveness of pretrial services practices, concluded that pretrial services was a good thing and should be expanded in the federal system. Unfortunately it was not followed up in any meaningful way with continued research. In fact, a brilliant time series research design prepared by Roger LeBouef for that report was never performed. If it had been performed, it would have provided hard data on the effectiveness of pretrial services in the federal system.
To successfully implement R2R, the federal system needs to refocus itself on its core mission of increasing release rates while maximizing appearance and safety rates. If we agree that these should be our mission and goals, we need outcomes to measure our progress. While we were preparing data for a study of the federal system to be published in 2009, it became readily apparent that our outcome data is faulty. Lack of documentation and clear definitions, ambiguity and a host of other factors have contributed to the situation and need to be rectified.Doing so may seem straightforward, but there are a myriad of obstacles to be overcome. Let’s use the example of failure-to-appear, a common pretrial services outcome. This is complicated right off the bat as the federal system is apparently one of the only systems that doesn’t routinely set a next court date for every defendant who appears. Please code the following scenarios given the above fact:
The defendant absconds from supervision, with no court date set;
The defendant does not appear for sentencing on Friday at 10 but comes to the courthouse on Monday at 3 pm.; and
The defendant cuts off his location monitoring equipment in his San Diego home and heads for the border, only to be arrested within hours by the Border Patrol.
Of the examples, which are FTAs and which are not? Should the lapsing of time have an impact on determining if a particular behavior is in fact an FTA? As soon as you settle on those rules, a scenario can be envisioned that we would feel was clearly a FTA but is outside the identified parameters. Simply put, there is no perfect solution to operationalizing or defining such nebulous concepts. Great care must be taken in developing those definitions, which should continue to be open to revision, but at some point the program needs to adopt the best definition it can and live with it and its imperfections.
Other complications surround the issue of technical violations. Are they successes? Are they failures? Or perhaps they are something in between. There are arguments to be made on each side. Identifying drug use through testing, which leads to drug treatment and ultimately the defendant achieving and maintaining a sober lifestyle could be considered a positive, if the outcome line is drawn at the end; it could be considered negative if the outcome line is drawn at the time of substance use. The response of the judicial officer also comes into play, as behavior that is tolerated in one court is not tolerated in another court. In analyzing our treatments and programs in the future, we will need to develop a methodology that successfully categorizes technical violations, since they account for approximately 20 percent of our outcomes.Conclusion
A research-to-results approach has much to offer pretrial services offices and the federal system generally. Fully achieving that potential requires significant work to be done in preparing for implementation. The most important part of that work is achieving buy-in of staff to the mission of increasing release rates while maintaining appearance and safety rates. There are significant factors that have caused the release rate to decline so dramatically. In all likelihood a substantial percentage of the pretrial services population cannot and probably should not be released. However, there is a group in the middle of our population in terms of risk of appearance and safety that could and should be released. We as a system need to begin to identify those individuals and develop the necessary tools and programs so that judicial officers are comfortable in releasing those defendants. Using a research focus to improve our treatments and programs will help us develop a pretrial services system that provides judicial officers with strong recommendations for release, backed by a proven supervision program to “reasonably assure” appearance and the safety of the community, and detention cost savings for our system and our defendants.