By Robert Sullivan, Director, Johnson County’s Department of Corrections
Leveraging a University Partnership
In May 2016, Johnson County partnered with the University of Chicago’s Data Science for Social Good (DSSG) Fellowship to develop a predictive analytics model that would promote intergovernmental opportunities for cross-system cooperation and coordination while increasing our ability to proactively connect individuals to effective care. We provided data from 2010 to 2015 about individual interactions with three county systems, including criminal justice, emergency medical services (EMS), and the county mental health center. The criminal justice data included inmate characteristics, jail bookings, bond amounts, charges, pretrial services, court cases, sentencing, and probation data. The EMS data included patient characteristics, medical impression, transportation, triage, call time, and location. The mental health data included patient characteristics, diagnosis, treatment modalities, service dates, programs enrolled, and discharges.
In order to comply with the requirements of the Health Insurance Portability and Accountability Act of 1996 (HIPAA), Johnson County de-identified every record prior to submission. We allowed only four staff members, responsible for refining and preparing records for statistical analysis, to have access to the shared database, all of whom were HIPAA trained and Criminal Justice Information Services (CJIS) vetted. The names, addresses, and Social Security numbers of every individual were replaced with identifiers that were unique to each person so that individual-level data was linked across all three systems. By linking individuals across datasets, we were able to see how people moved between systems.
Conducting an Initial Review
An initial review of the data found 127,000 unique individuals had touched at least 1 of the 3 systems in the last 6 years. There were 12,000 people who overlapped with 2 or all 3 departments, and over 1,000 that interacted with all 3. For the development of the Early Intervention System, we focused on individuals who had previous interactions with mental health and the jail as our population of interest, representing 4,400 individuals.
We applied machine-learning methods to predict which individuals in our target population were most at risk of entering jail in the next year. Since all of the records were linked, the project team was able to compile timelines for individual interactions and generate over 250 individual-level variables to predict future jail bookings. Sample variables included age at first jail booking, number of bookings in the previous year, total days of therapy received, number of therapists seen, transported to hospital by EMS, mean time between jail bookings, and mean time between EMS calls.
Identifying Frequent Utilizers and Patterns
The DSSG project team reserved 2015 data for testing and then trained different suites of models, including decision trees, random forests, logistic regressions, and gradient boosting, for each year from 2010-2014. Using those results, they selected the best model and then used that model to generate risk scores for every individual as of January 1, 2015. Johnson County has enough resources to support 200 individuals at risk of being booked into jail, so the DSSG team evaluated the models based on the precision of the 200 highest predictions.
The best performer was a random forest model that had a precision rate of 52 percent. In other words, 104 of the individuals identified in our dataset as at risk of being booked in the next 12 months were booked in 2015. In contrast, in our overall population of interest, only 1 person in 10 was booked in 2015, meaning that the Early Intervention System provided a 5-fold improvement to our predictive capabilities. These 104 individuals went to jail for an average of 2 months each, for a total of 6,987 days. This average length of stay was twice as long as in the general population. On average, at the time of booking, it had been 2 years since these individuals’ last contact with the Mental Health Center.
Moving from Research to System Improvements
Due to our model’s early success, we have been able to recruit new partners. In addition to the three systems we started with, we now have agreements with our jurisdiction’s two largest police departments to share arrest, calls for service, and case disposition data, and we have similar agreements with Johnson County Health and Environment and Community Corrections. Adding more diverse datasets to our prototyping will increase our Early Intervention System’s precision rate, help us generate additional predictive features, and shorten our predictive window from 12 months to 30 days. We will use the same prototyping to build additional models to predict other risky, complex patterns of social service interactions, for example to identify individuals with mental illness at risk of using emergency medical services for non-emergent issues or to detect those at risk of dropping out of mental health services prematurely. Johnson County is on track to operationalize the Early Intervention System in 2018.
Background
Johnson County is located in northeast Kansas and is adjacent to both Kansas City, Missouri and Kansas City, Kansas. It is home to approximately 580,000 citizens, contributing to about a quarter of the total population in the Kansas City metropolitan area. Much like every other place in the country, Johnson County has worked hard to safely store and secure its data. However, rather than letting data sit in dozens of system silos, either to be forgotten or useful only to individual departments, our county chose to integrate system data so it could be useful to all of our criminal justice and human service partners.
In the early 1990s, we created the Justice Information Management System (JIMS). JIMS is a fully integrated record management system that supports our public safety partners. This means a single database tracks a person from booking into the jail via the prosecutor’s office, throughout the entire court process, and often onto pretrial supervision or probation. In 2017, we began an inter-governmental collaborative effort to put the necessary infrastructure in place for JIMS to host every one of our municipal police departments in one record management system.
In 2008, Johnson County developed My Resource Connection (MyRC) after an internal audit found gaps in services provided by the county, as well as duplications and inefficiencies in service delivery, such as one Johnson County resident receiving services from 27 different agencies and 17 case managers. MyRC is a web-based application that enables probation and human service programs to contribute client data to a shared database. A nightly process then identifies individuals who were booked into jail that day or receiving services from multiple programs as mutual clients. Staff in program areas utilize MyRC to obtain information about other services a particular client is receiving, contact information of other professionals serving that client, and information about other individuals residing in the client’s home. Staff access to client data varies depending on whether they work for a HIPAA-covered entity or are a HIPAA business associate.
Due to the success of JIMS and MyRC, and the dedicated staff who support these systems, Johnson County was able to position itself for the next evolution in its county’s data development and make way for the creation of the Early Intervention System using machine learning techniques, leveraged data, and predictive analytics.
About the Author
Until recently, Robert Sullivan was the Criminal Justice Coordinator for Johnson County, Kansas. He provided support for the County Manager’s Office on all criminal justice-related matters and staff support for the Criminal Justice Advisory Council. Currently, Robert is serving as director for Johnson County’s Department of Corrections.