|Behavioral Health Faces Challenges Collecting Performance Data|
Behavioral healthcare providers are eager to create industry-wide core
performance measures. Individual facilities collect voluminous data within their
own organizations. However, a pilot test of key benchmarking indicators found
that substantial challenges remain in gathering, reporting, and comparing data
across systems. The National Association of Psychiatric Health Systems (NAPHS)
White Paper: Lessons Learned from Pilot Testing of the NAPHS Benchmarking
Indicators identified measures that hold great promise for the field. More
importantly, the participants in this 2000 pilot test demonstrated their
willingness to share this critical data. The project was coordinated by the
NAPHS Benchmarking Committee, chaired by NAPHS Immediate Past President Peter
Panzarino, M.D., chairman of the Department of Psychiatry at Cedars Sinai
Medical Center in Los Angeles.
The pilot test, which focused on nine indicators chosen from an earlier consensus-driven process of the NAPHS Benchmarking Committee, analyzed data from 48 organizations offering a total of 637,241 days of inpatient care; 601,595 days of residential care; and 161,993 days of partial hospital care. The facilities represented all levels of care (including inpatient, residential, partial hospitalization, and outpatient services) and all populations (including child, adolescent, and adult). The indicators reviewed in the pilot test were:
-- adverse drug reactions
-- completed suicide
-- attempted suicide
-- symptom/function measure
-- patient satisfaction
-- peer review
While no specific data is reported in WHITE PAPER: Lessons Learned from Pilot Testing of the NAPHS Benchmarking Indicators, the document outlines challenges identified in the testing phase. Commentary is provided on each of the indicators in the pilot test.
CHALLENGES FOR THE FIELD
According to the White Paper, any data-collection effort -- and any national core performance measures in particular -- must:
-- provide value in the data generated that is in proportion to the intensity of the data collection effort. Allocation of limited resources needs to be directed to the collection of the most important data.
-- focus on indicators that provide the most useful clinical and operational data possible within the scope of the data currently available within organizations.
-- recognize that definitional decisions have significant policy implications. Widely diverse state, local, and national standards often lead organizations to adopt similar -- but not identical -- definitions. This makes meaningful data-collection extremely difficult -- and potentially costly. If organizations use operational definitions that are different from the question asked, they are not able (short of going to primary sources such as the patient record or reviewing their entire data base and applying the study definitions) to report the data.
An example of the challenges of definitional variations became clear in looking at restraint/seclusion indicators. In this pilot test, there was great variation in the definitions of restraint and seclusion, particularly as they relate to children and adolescents. Reporting mechanisms appear to be in place in all organizations for collecting data about restraint and seclusion. If the field (in conjunction with regulatory and accrediting organizations) can agree on consistent definitions, the possibility for developing meaningful benchmarks seems to be very strong.
Similarly, the definition of attempted suicide used in this pilot test shows how difficult it is to standardize definitions. Because of the wide variation in incidences reported, we suspect that some organizations reported incidences that were not of the severity of the operational definition used in the pilot (actually or potentially life-threatening or resulted in the need for urgent intervention).
According to the report, indicators with the most promise for benchmarking across systems are those that have generally similar definitions, are collected by a majority of sites, are retrievable, and do not require extensive re-collection of data.
"As this pilot test demonstrates, there is high interest and commitment by behavioral health providers to work towards core performance measures that will help organizations improve the quality of care and be responsive to the needs of those who seek mental health care," said NAPHS Executive Director Mark Covall. "We have learned a great deal about the importance of focusing on data that is relevant to clinical operations and collected in ways that conserve limited resources. We look forward to sharing the knowledge we have gained about what works and the challenges ahead. Our experience can help to inform other national efforts to identify core performance measures and will help us to improve our own association's data collection efforts," he said.
NAPHS Immediate Past President Peter Panzarino, M.D., of Cedars Sinai Medical Center, CA was the chair of the Benchmarking Committee which oversaw the pilot test. Other committee members included James M. Cole, Devereux Foundation, PA; Allen S. Daniels, Ed.D., University Managed Care/University Psychiatric Services, OH; Naakesh A. Dewan, M.D., University Managed Care/University Psychiatric Services, OH; Susan Eisen, Ph.D., McLean Hospital, MA; Frank Ghinassi, Ph.D., Western Psychiatric Institute & Clinic/University of Pittsburgh Medical Center, PA; Leonard S. Goldstein, M.D., Integrated Behavioral Care, VA; Gay C. Hartigan, Liberty Management Group, NJ; John Lehnhoff, Ph.D., Richard Young Center, NE; Robert Mansfield, Carrier Foundation, NJ; William Nolan, Ph.D., consultant, TN; Richard T. Palmisano, Brattleboro Retreat, VT; Howard Waxman, Ph.D., Belmont Center for Comprehensive Treatment, PA; Martin Schappell, Universal Health Services, PA; Kathleen McCann, R.N., D.N.Sc., and Carole Szpak, NAPHS Staff Liaisons.
The WHITE PAPER: Lessons Learned from Pilot Testing of the NAPHS Benchmarking Indicators, is $40"prepaid"from the National Association of Psychiatric Health Systems, 325 Seventh Street, NW, Suite 625, Washington, DC, 20004. Call 202/393-6700, Ext. 15, for ordering information (Mastercard and Visa accepted
---National Association of Psychiatric Health Systems
Articles in The Science of Mental Health are written by the originating institution. This article was originally posted to Newswise. Newswise maintains a comprehensive database of news releases from top institutions engaged in scientific, medical, liberal arts and business research. The friendly interface allows you to search, browse or download any article or abstract.