city and county of san francisco - sf, dph · city and county of san francisco department of public...
TRANSCRIPT
City and County of San Francisco DEPARTMENT OF PUBLIC HEALTH
Edwin M. Lee
Mayor
BEHAVIORAL HEALTH SERVICES
Quality Improvement Work Plan Evaluation Report
FY 2016-2017
2
Table of Contents
Introduction……………………………………………………………………………………………………………………. 3
Service Delivery Capacity ………………………………………………………………………………………………… 3
Access to Care ………………………………………………………………………………………………………………. 9
Beneficiary Satisfaction …………………………………………………………………………………………………. 14
Service Delivery and Clinical Issues ………………………………………………………………………………… 20
Performance and Areas for Improvement...………………………………………………….................... 27
Continuity and Coordination of Care. ……………………………………………………………………………… 34
Provider Appeals ………………………………………………………………………………………………............ 37
3
INTRODUCTION This report describes the results of the San Francisco County Behavioral Health Services (BHS) Quality
Improvement Work Plan for Fiscal Year 2016-2017. Each section provides the objectives, activities, data sources and results for our endeavors in each of the main content areas.
This report is divided into the following content areas:
I. Service Delivery Capacity II. Access to Care
III. Beneficiary Satisfaction IV. Service Delivery and Clinical Issues
V. Performance and Areas for Improvement
VI. Continuity and Coordination of Care VII. Provider Appeals
WORK PLAN REPORT
I. SERVICE DELIVERY CAPACITY
GOAL I. Ensure that the number, type, geographic distribution and cultural and linguistic
competency of behavioral health services is appropriate for the client population. Based on an analysis of service locations, set goals for the number, type, and geographic distribution
of services.
San Francisco City and County is dedicated to ensuring that services are accessible on multiple levels. In
addition to ensuring that services are distributed geographically to meet the needs of San Franciscans, we are committed to providing culturally and linguistically competent behavioral health services to a
diverse population. Chinese, Russian, Spanish, Tagalog, and Vietnamese constitute our five threshold languages, although services are available in other languages dependent on clinicians’ linguistic capacity,
or through interpreter services.
Objective 1: Behavioral Health Services programs will be located primarily in the neighborhoods in
which the majority of our clients reside.
Action(s)
1. Describe the number, type, and geographic distribution of county-funded behavioral health service programs. Review geographic location of services and assess appropriateness given client density by
June 30, 2017.
Objective 1 Results:
Density maps were produced to illustrate the geographic distribution of clients served and treatment
programs during calendar year 2016. The darker the blue shading, the greater the density of clients
residing in that area; programs are represented by a red dot.
Overall the locations of clinics are well positioned in the areas of the city where our clients live. In relative terms, mental health clients living in the Hunters Point and Outer Mission neighborhoods have fewer
programs in their neighborhoods, but the distance to programs is very short (within one mile). There are
few substance use clinics in the Outer Mission neighborhood as well, but many more in the Hunters Point neighborhood as compared to mental health clinics.
4
5
6
The table below shows the number of mental health programs by modality of service and neighborhood.
In previous years the numbers reported have been consolidated by location, meaning that if more than one program existed at a given address with the same modality of care it was only counted once. The
decision was made to count all programs, to better reflect not only location but our capacity to provide
services within the various levels of care. Since this change was made, it is difficult to compare these numbers to past years. One notable program that was added this year is the Instituto De La Raza Full
Service Partnership (FSP) program, our first FSP program for children 0 to 5 years old, which started to bill in August 2016.
Number of Mental Health Treatment Programs by Neighborhood
Neighborhood Crisis
Day
Treatment ICM Inpatient Outpatient Residential Total
Bayview 3 0 2 0 14 0 19
Bernal Heights 0 0 0 0 5 3 8
Castro/Upper Market 0 0 0 0 1 0 1
Chinatown 0 0 1 0 2 0 3
Downtown/Civic Center 0 0 5 0 17 0 22
Excelsior 0 0 0 0 1 0 1
Haight Ashbury 0 0 0 2 8 6 16
Inner Sunset 0 0 0 0 0 1 1
Mission 2 1 7 1 28 3 42
Nob Hill 0 0 0 2 0 0 2
Ocean View 0 0 0 0 3 0 3
Outer Richmond 0 0 0 0 11 0 11
Outer Sunset 0 0 0 0 4 1 5
Pacific Heights 0 0 0 2 0 1 3
Parkside 1 0 0 0 7 0 8
Potrero Hill 0 1 0 0 2 0 3
Presidio Heights 0 0 0 0 0 1 1
Russian Hill 0 1 0 0 1 0 2
South of Market 2 1 7 1 20 3 34
Twin Peaks 0 0 0 0 4 0 4
Visitacion Valley 0 0 0 0 2 0 2
West of Twin Peaks 0 0 0 0 2 0 2
Western Addition 0 0 0 0 12 1 13
Not in SF 0 0 0 9 16 2 27
Totals 8 4 22 17 160 22 233
Objective 2: Clients will report satisfaction with the convenience and cultural appropriateness of
behavioral health service programs, as indicated by an average score of 4 or higher on these items in the consumer perception survey, by June 30, 2017.
Action(s)
1. Conduct system wide consumer perception survey on the schedule determined by DHCS.
2. Assess client satisfaction results for location and cultural and linguistic competence items.
7
Objective 2 Results:
Mental Health Consumer Perception Survey Results for Culture and Location Consumer Perception Surveys were distributed to all mental health clients who received a face-to-face
service during a one-week period in May 2017, including youth, families, adults, and older adults. The
surveys were completed in clinic waiting rooms and dropped into secure survey collection boxes. We received over 3,500 surveys from mental health treatment clients.
Several questions on our Consumer Perception Survey address client perception of sensitivity to cultural
background, as well as convenience of location of services. The table below shows the question, the average response (based on a Likert scale where 1= Strongly Disagree and 5= Strongly Agree) and the
number of clients who answered that question. Overall, results show that clients agreed to strongly
agreed that service providers were sensitive to their cultural and linguistic needs, and that services were provided in a convenient location.
Comparing the current year’s results to the prior year, satisfaction within Adult Mental Health programs
has increased slightly for clients’ perception of sensitivity to cultural background and also improved
slightly for convenience of location of services.
Results from Adult Consumer Perception Survey (MHSIP)
Question Average Rating
FY 15-16
N=2602
Average Rating
FY 16-17
N=2566
1. Staff were sensitive to my cultural background
(race, religion, language, etc.). 4.27 4.33
2. The location of services was convenient for me. 4.26 4.32
Comparing the current year's results to the prior year for the Youth Services Survey, satisfaction has increased both for clients’ perception of sensitivity to cultural/ethnic background and for convenience of
location of services.
Results from Youth Consumer Perception Survey (YSS)
Question Average Rating FY 15-16
N= 542
Average Rating FY 16-17
N= 575
1. Staff were sensitive to my cultural/ethnic background.
4.27 4.38
2. The location of services was convenient for me/us. 4.20 4.30
Comparing the current year's results to the prior year for the Youth Services Satisfaction Survey for
Family/Caregivers, satisfaction has slightly improved for clients’ perception of sensitivity to cultural/ethnic background and slightly decreased for convenience of location of services.
Results from Family/Caregiver Perception Survey (YSS-F)
Question Average Rating
FY 15-16 N= 630
Average Rating
FY 16-17 N=710
1. Staff were sensitive to my cultural/ethnic background.
4.56 4.57
2. The location of services was convenient for me/us. 4.48 4.45
8
Substance Use Treatment Satisfaction Results for Culture and Location Treatment Satisfaction Surveys were distributed to all substance use treatment clients who received a
face-to-face service during a one-week period in March 2017. The surveys were completed in clinic waiting rooms and dropped into secure survey collection boxes. We received nearly 2,500 surveys from
Substance Use Treatment clients.
Several questions on our Treatment Satisfaction Survey address client perception of sensitivity to cultural
background, as well as convenience of location of services. The table below shows the question, the average response (based on a Likert scale where 1= Strongly Disagree and 5= Strongly Agree) and the
number of clients who answered that question. Overall, results show that clients agreed to strongly agreed that service providers were sensitive to their cultural and linguistic needs, and that services were
provided in a convenient location.
Comparing the current year’s results to the prior year, satisfaction has remained virtually the same
regarding clients’ perception of sensitivity to cultural background and convenience of location of substance use services.
Results from Substance Abuse Treatment Satisfaction Survey
Question Average Rating
FY 15-16 N=2431
Average Rating
FY 16-17 N=2492
1. Staff were sensitive to my cultural background (race, religion, language, etc.).
4.48 4.47
2. The location of services was convenient. 4.45 4.41
Objective 3: By June 30, 2017, enhance existing substance use services in compliance with Drug
Medi-Cal (DMC) Organized Delivery System (ODS) waiver for all existing and approximately 3,000 new clients.
Action(s)
1. Hire DMC-ODS Project Manager to manage waiver.
2. Hire DMC-ODS Quality Management and System of Care positions.
3. Implement ASAM-based AVATAR residential authorizations.
Objective 3 Results:
Erik Dubon, MBA, was hired as the new SUD Program Manager to oversee the day-to-day operations of
the DMC/ODS Waiver Program. Erik has over twenty years of experience working with non-profit
organizations and the SF Department of Public Health. In his various roles, he has worked with all communities, providing direct services for HIV/Prevention Services, Mental Services, and Substance Use populations. He has also worked in the administration part of the contracting process with the SFDPH.
Quality Management hired a SUD data analyst, Sherry Lam, to conduct analyses and develop reports for
monitoring SUD performance metrics. Several additional SUD Program Manager positions, as well as a dedicated SUD Quality Improvement Coordinator position, are in process after a lengthy HR delay.
The ASAM Level of Care form was completed in June 2017 and piloted by one residential substance use
program for central authorization in August 2017. Some lessons learned from the pilot were the need for
a client pre-admission prior to authorization so program could access ASAM Level of Care form in AVATAR and thus the need for client pre-admit status to be changed to admit once client arrived at program site
for residential stay. There is a planned roll-out for all programs starting November, 1, 2017. One remaining outstanding issue to be resolved is a lack of agreed upon point person or department
responsible for this pre-admission enrollment.
9
Objective 4: Expand enrollment of Full-Service Partnership (FSP) families with children aged birth to 5 years old with high needs from 5 to 22, by June 30, 2017.
Action(s)
1. Provide outreach through partnering with local Community Based Organizations, developing a
collaboration with in-home day care providers, participating in community events, and going door-to-door to inform eligible families of the new service.
Objective 4 Results:
Instituto De La Raza's Full Service Partnership (FSP) program, our first FSP program for children 0 to 5
years old, began in August 2016. The program, called Strong Parents and Resilient Kids (S.P.A.R.K.), served fourteen families in FY 16-17. These families all have children aged birth to five years old and
either reside in the Sunnydale Housing project or have an open CPS case. Of the fourteen, eight had never received services prior to engagement with the S.P.A.R.K. program and all but one child had been
exposed to domestic and/or community violence. The program continues to have capacity for more families, but due to distrust in the community, caregiver engagement has been challenging, with four
referred cases extending at least ninety days before the family was ready to start formal treatment, and
three additional cases are still in the engagement phase.
In order to expand enrollment in this past fiscal year, S.P.A.R.K. developed a strong referral relationship with Mercy Housing Resident Services Coordinators. The S.P.A.R.K. team also provided outreach at over
fifteen community events to increase awareness about the program while building relationships with
residents in a non-threatening and organic way. In addition to community events, S.P.A.R.K. program staff have met with fifteen community agencies serving their target population, which led to three
ongoing intensive collaborative projects such as weekly Reading/Tutoring support for at-risk youth, month-long trauma workshops for Latina Mother Groups, and six-week Preventing Long-term Anger and
Aggression in Youth (P.L.A.A.Y.) support groups for African American male teens. In the next fiscal year, S.P.A.R.K. will continue to expand their services to families in need with plans to outreach to specific
Early Learning Centers and Family Childcare Homes while also providing activities such as Family
Drumming and Infant Massage.
II. ACCESS TO CARE
GOAL II.a. Ensure timeliness of routine and urgent mental health appointments.
Objective 1: At least 90% of individuals requesting behavioral health outpatient services will be offered an appointment within 10 business days of the request by June 30, 2017.
Actions
1. Monitor time from request for services to first offered appointment quarterly using the Timely Access
Log in Avatar, and determine areas for improvement.
2. To monitor compliance with completion of Timely Access Log, provide programs with a biannual analysis of logged requests relative to number of new episodes per program.
Objectives 1 Results:
BHS Quality Management extracted data from the Timely Access Log in Avatar to report on the timeliness of routine mental health appointments offered during FY16-17. As seen in the chart below, the 10 day
standard was met 87% of the time, with similar rates for AOA and CYF services. The average number of
days to the first offered appointment was approximately 3 days.
Based on feedback at our December 2016 EQRO Review, we changed our methodology for calculating this indicator for FY 16-17. Previously we excluded entries with an “attestation,” a clinical assessment by
a qualified professional that the client could wait longer than 10 days without any untoward
10
consequences. The revised method includes all initial requests for services, with or without an attestation.1
Time to First Offered Appointment – Routine Initial Mental Health Appointments
All Services
Adult Services
Children’s Services
Days to 1st Offered Appointment
3.12 days
(mean)
0 days (median)
7.01 Std. Dev.
2.94 days
(mean)
0 days
(median)
6.81 Std. Dev.
3.83 days
(mean)
2 days (median)
7.93 Std. Dev.
MHP standard or goal 10 business
days
10 business
days
10 business
days
Percent of appointments that meet
this standard 87.4% 87.6% 86.7%
Time to First Offered Appointment – Annual Trends
FY
2012-2013
FY
2013-2014
FY
2014-2015
FY
2015-2016
FY
2016-2017
Average Days to 1st Offered Appointment 4.7 4.6 3.7 2.4 3.12
Percent Offered within 10 days 89% 91% 95% 98% 87%
To track compliance with recording initial requests for services on the Timely Access Log, quality
management created a Tableau dashboard that shows the number of entries on the log compared to the number of new treatment episodes opened by month, by program. This dashboard was shared and
discussed in SOC-QIC meetings. In addition, each program is monitored on a performance objective
comparing the number of entries on their Timely Access Log to the number of new episodes opened, with the expectation that the number of requests for service should always be the same or higher than the
number of episodes opened. Approximately two-thirds of our civil service and contract providers met this compliance objective in FY 16-17. Programs that did not meet the objective are required to write a plan
of correction for how they will improve their compliance with use of the Timely Access Log. The chart
below shows the performance objective results for completion of the timely access log.
1 In order to ascertain the extent to which including entries with an attestation impacted the data we also calculated
this metric with those entries excluded. The result with the attestation entries excluded was a mean of 2 days, and almost 98% compliance with the 10 day standard.
11
Objective 2: 100% of individuals assessed as having urgent conditions will be served within 24 hours
of initial contact.
Action(s)
1. On a quarterly basis, monitor number of individuals entered on outpatient Timely Access Log as needing an "urgent" appointment, and whether their episode of care was opened in an urgent care
clinic within 24 hours.
Objective 2 Results:
Data was extracted from our Timely Access Log for entries designated as “Crisis.” There were a total of 70 Crisis entries, and of those there were 55 entries with sufficient identifying information recorded to
match to Avatar. Of the 55 entries we matched to Avatar billings, 53 had a subsequent billed service. Just over 85% of the appointments (45) were offered within one day, meeting our goal. It is possible
that some of the 8 clients with longer than one day of wait time received services not billed in Avatar, such as behavioral health services in primary care.
Timely Access for Urgent Appointments
Number of
Requests for
Urgent Appt
Number of “Crisis” entry clients from Timely Access Log
found (matched) in Avatar 70
Number of those clients with a billed service subsequent to the Timely Access Log date
53
Number of clients with a billed service within 1 day 45
Objective 3: At least 70% of individuals discharged from inpatient psychiatric services will be seen by a prescriber (MD/NP) within 14 business days by June 30, 2017.
Action(s)
1. On a quarterly basis, monitor time from inpatient hospital discharge to next contact with psychiatrist or nurse practitioner.
2. BHS to participate in monthly hospital discharge meeting at Zuckerberg San Francisco General
(ZSFG) Hospital to improve transitions in care through more timely coordination.
Objective 3 Results:
Time to MD/NP Service after Hospital Discharge
We track the time from hospital discharge to prescriber appointment on a run chart to look at trends over time. As seen in the chart below, our mean rate of compliance with our 14 day standard was 74.9%
during FY16-17; this is an increase from last year’s mean of 69%.
12
Improving Transitions between ZSFG and BHS Programs
In the fall of 2015 a committee was formed that included staff from psychiatric inpatient services at
Zuckerberg San Francisco General Hospital (ZSFG), ZSFG Psychiatric Emergency Services (PES), and Specialty Behavioral Health Services (BHS). The overarching aim of this workgroup was to improve the
collaboration between these entities, and to facilitate more efficient transitions of care for consumers,
and thereby reduce psychiatric re-admissions. This workgroup has established written protocols and procedures for communication between ZSFG Psychiatric Inpatient and PES, and the programs BHS that
specify timeframes for communication from the hospital, return calls from BHS, standard scripts for
communication, and the essential content of these communications.
The protocol was communicated to all outpatient programs and hospital staff by program directors and
hospital administration. The implementation plan was tested in September 2016 and a feedback mechanism was put in place to report and correct any problems in the implementation of these two
protocols, as part of their adoption. In the six months since implementation, BHS has received only two notifications of instances when the protocol was not followed, and staff involved were informed of the
correct procedures to follow in the future.
Objective 4: Reduce psychiatric hospital 30-day readmissions to below the statewide average of 19%
by June 30, 2016.
Action(s)
1. Monitor psychiatric rehospitalization rates on quarterly basis.
2. Continue to monitor program performance objective requiring “no more than 15% of psychiatric inpatient hospital discharges occurring during FY15-16 will be followed by a readmission within 30
days.”
13
Objective 4 Results:
BHS Quality Management tracks the rates of 30-day recidivism to the hospital on a monthly basis, based on all hospital discharges recorded in Avatar (see the chart below). The mean for FY16-17 was 17.3%, a
negligible increase from the mean recidivism rate of 16.4% during FY15-16. While there were minor
fluctuations, the rate overall has been steadily declining for several years, as illustrated by the trend line in the chart below.
Programs have a performance objective aiming to reduce preventable rehospitalizations, which applies to all clients who were enrolled in the program prior to the initial psychiatric hospitalization, and remained
active in the program for at least the next 30 days following hospital discharge. The goal is that less than 15% of clients will be rehospitalized within 30 days.
Performance objectives are rated on a scale of 0 – 5, with a 5 being a perfect score (achieving the target). Results of this performance objective for FY16-17 indicated that 50 out of 63 programs (79%)
achieved a perfect score of ‘5’on the objective; seven programs achieved a score of ‘4’; three programs scored ‘3’ points, and the remaining three programs scored less than ‘3’ points.
GOAL II.b. All calls to the BHS 24/7 toll-free access line will be answered by live service
providers in the language of the caller, and will gather all required information to ensure the
caller receives the appropriate information or referral needed.
Objective 1: By June 30, 2017, 100% of calls will be triaged to staff who speaks the language of the caller. If a caller speaks a language not spoken by staff, the Language Line will be used.
14
Action(s)
1. Monitor the quality and responsiveness of calls to the BHS 24/7 toll-free access line and provide immediate feedback.
Objective 2: By June 30, 2017, 100% of calls will be screened for crisis situations and will be referred
appropriately.
Action(s)
1. Monitor the screening and referral process of crisis calls to the BHS 24/7 toll-free access line.
Objective 3: By June 30, 2017, regular test call results for both the business and after-hours 24/7 Access Line will have a 100% success rate.
Actions
1. Continue four independent test calls per month, two during business hours and two after hours,
conducted by consumer volunteers, clinical interns, and BHS QM staff and provided feedback to
Access Coordinator.
2. Continue to meet monthly with Access Coordinator to discuss and document improvements made in
response to test call results.
Objectives 1-3 Results:
The Access Coordinator continued to monitor the quality and responsiveness of all calls, including crisis
calls, to the BHS 24/7 Access Line through daily log reviews, weekly Access staff meetings, weekly
meetings with the after-hours contract agency, San Francisco Suicide Prevention (SFSP), and monthly meetings with the Quality Improvement Coordinator to identify any areas for improvement.
Throughout the fiscal year, both the Behavioral Health Access Center's (BHAC) team covering business
hours and SFSP's after-hours team continued to score close to 100% on bi-weekly test calls. Following the Department of Health Care Services' Triennial Medi-Cal Review findings in April 2017, BHS began to
specifically assign one test call a month to include a request to file a grievance. From May-July 2017,
grievance test calls have consistently scored 100% on both calls and log portions of the analysis.
III. BENEFICIARY SATISFACTION
GOAL III.a. Monitor beneficiary/family satisfaction at least annually.
Objective 1: By June 30, 2017, at least 80% of clients will report being satisfied with their care, as indicated by an average score of 4.0 or higher on the Consumer Perception Survey.
Actions
1. Collect and analyze consumer perception results from all mental health and substance abuse
treatment programs to determine areas for improvement.
2. Provide individualized feedback to programs regarding client satisfaction.
Objective 1 Results:
Mental Health Satisfaction Survey Results Satisfaction surveys were distributed to all mental health clients who received a face-to-face service
during a one-week period in May 2017, including youth, families, adults, and older adults. Substance Abuse Treatment Programs surveys were collected in March 2017. The surveys were completed in clinic
waiting rooms and dropped into secure survey collection boxes. Each program received a report of their satisfaction survey results, showing item level results. Results at the program and system level were
posted to the DPH website.
15
For the Adult and Older Adult mental health programs, 89.5% of clients reported overall satisfaction, with a mean score of 4.29 out of 5.00 (which was virtually the same as Spring 2016 at 89.1% and 4.26 on
average). Of the 2566 adult respondents, the three items with the highest satisfaction rating were (1) “I like the services that I received here,” (2) “Services were available at times that were good for me,” and
(3) “I would recommend this agency to a friend or family member.” The three items with the lowest
satisfaction ratings were (1) “Staff told me what side effects to watch out for,” (2) “I, not staff, decided my treatment goals” and (3) “I was able to see a psychiatrist when I wanted to.” It should be noted that
even the lowest rated items still fell into the “satisfied” range.
For the Children, Youth, and Family mental health programs, 92.8% of our youth and families reported overall satisfaction, with the mean score for youth at 4.28 and the mean score for families at 4.48.
These scores have remained same since our last survey. Among the 575 youth respondents, the three
items with the highest satisfaction rating were (1) “Staff spoke with me in a way that I understood,” (2) “Staff treated me with respect,” and (3) “Staff respected my religious/spiritual beliefs.” The three items
with the lowest satisfaction ratings for youth were (1) “I helped to choose my services,” (2) “I got as much help as I needed,” and (3) “I helped to choose my treatment goals.” It should be noted that the
means were in the “satisfied” range even for the items with the lowest ratings.
Among the 710 parents or caregivers responding about their child’s services, the three items with the
highest satisfaction ratings were (1) “Staff spoke with me in a way that I understood,” (2) “Staff treated me with respect,” and (3) “Staff were sensitive to my family’s religious/spiritual beliefs.” Among parents
or caregivers, the lowest rated items were (1) “My family got the help we wanted for my child,” (2) “I helped to choose my child’s services,” and (3) “I helped to choose my child’s treatment goals.” Again,
the mean scores even in the “lower” rated categories were still within the “satisfied” range.
SUD Satisfaction Survey Results
For Substance Abuse programs, 92.3% of clients reported overall satisfaction in this fiscal year, with a mean score of 4.48 out of 5 (results are slightly lower than Spring 2016, which was at 93.6% of clients
reported overall satisfaction and mean score of 4.49 out of 5.00). The three items with the highest
satisfaction ratings were (1) “I felt welcomed here,” (2) “Staff spoke with me in a way I understood,” and (3) “Staff treated me with respect.” The three lowest rated items were (1) “I was able to get all the
help/services that I needed,” (2) “The location was convenient (public transportation, distance, parking, etc)," and (3) " Services were available when I needed them.”
Open-ended Comments Report This year, Quality Management has been able to provide programs with comments that clients wrote into
the open-ended section of their surveys. Comments were transcribed, with personal information redacted. Comments were mostly praise and thanks, but also contained some specific feedback for
programs that resulted in programmatic change. In addition to displaying the words of the comments as a word cloud, text analysis was used to find the sentiment and emotion content of the comments for
each program. Sentiment refers to the positive or negative meaning of words. The emotion content of
comments was assessed for the emotions of joy, trust, anticipation, surprise, anger, disgust, fear, and sadness.
Objective 2: By June 30, 2017, decrease by 30% the number of mental health providers needing to
complete Plan of Action related to client satisfaction survey return rate from 19 to 13 mental health programs.
Actions
1. Provide Peer Volunteer to clinics to support completion of client satisfaction survey.
2. Expand training and technical assistance to individual programs.
16
Objective 2 Results:
This year, only one mental health program required a Plan of Action for having an insufficient survey return rate, which vastly exceeded our targeted reduction. Since the target was reached, this objective
will be replaced with a new satisfaction objective specific to improving access and engagement items for
FY 17-18.
Quality Management engaged in a number of activities that be believe contributed to this improvement. First, survey administration instructions were modified to include more detail covering the methods and
best practices for conducting the twice yearly Consumer Perception Surveys and Substance Abuse Treatment Satisfaction Survey. This included a new “Staff Fact Sheet” and a “Client Fact Sheet” in all
threshold languages, which explained the purpose of the survey, confidentiality of the information, and
how the information would be used to improve services. The instructions also gave more detail on how to print, reproduce, and staple surveys so that they are scannable. Quality Management staff conducted
presentations at Adult and Child, Youth and Family Provider meetings to review the survey instructions and reinforce the importance of conducting the surveys. Our survey coordinator also provided
individualized training and technical assistance to administrative assistants and managers who called,
emailed, and/or visited with questions pertaining to the client satisfaction surveys.
Quality Management also piloted sending a volunteer to help to support the completion of client satisfaction surveys at a clinic in Nov 2015, May 2016, and Nov 2016 to determine if expanding this
service to multiple clinics was viable. The coordination of volunteers proved to be resource and time intensive. Quality Management determined that it did not have the capacity to expand the service to all
clinics; therefore, the volunteer coordination has been discontinued. Even without the peer support,
clinics were able to demonstrate improvements in survey completion rates.
GOAL III.b. Evaluate beneficiary grievances, appeals, and fair hearings at least annually.
Objective 1: Continue to review grievances, appeals, and fair hearings and identify system improvement issues.
Actions
1. Collect and analyze grievances, appeals, fair hearings, and requests to change persons providing
services in order to examine patterns that may inform the need for changes in policy or programming.
2. The Risk Management Committee will analyze trend reports in order to identify any areas of
needed improvement. Areas for improvement will be presented to SOC-QIC and/or other management, provider, and consumer forums.
Objective 1 Results:
Information pertaining to grievances, appeals and State Fair Hearings is regularly collected and analyzed
by the Risk Management section of Quality Management (QM). Grievances and appeals are filed in
person or by phone directly to the Behavioral Health Access Center’s Officer of the Day, who then provides them to the QM Grievance Officer. Grievances and appeals filed by mail are sent directly to our
QM Grievance Officer. The protocols governing the processing of grievances and appeals, the informing materials, filing forms, and posters are current and providers are aware of these protocols.
An annual risk management report for FY 2016-17 was presented in several quality improvement committee forums, including the Adult/Older Adult QIC, System of Care QIC, the Children, Youth & Family
QIC, and the Risk Management Committee. Findings were also presented at our Adult/Older Adult and the Child, Youth & Family Provider meetings. In these meetings, a summary of all grievances/appeals
received during FY 2016-17, broken out by type, were presented and discussed (see table below, Grievances/Appeals by Category).
17
Quality Management oversees the problem resolution processes for clients who are seeking a formal
avenue to work out any issue or to review any decision pertaining to Behavioral Health Services. These processes involve either a grievance or an appeal process, and upon its completion, clients may request a
State Fair Hearing. Quality Management collects and analyzes aggregate information from these processes as well in order to examine patterns or trends that may inform the need for changes in policy
or programming. There were no system issues identified in FY 2016-17.
During FY 2016-17, there were no appeals or expedited appeals filed. There was one request for a State
Fair Hearing involving a Drug Medi-Cal provider, which was eventually withdrawn by the claimant. Compared to the prior fiscal year, FY 2016-17 had a 25% overall increase in total number of grievances.
All grievances were pertaining to services within the adult system of care. There were no grievances filed
within the children’s system. At the conclusion of the investigation of grievances, 18% were determined to have merit. Seventy percent of the grievances pertained to mental health services, and 30% to
substance use services. Fifty-six percent of the mental health grievances pertained to outpatient services, whereas 82% of the substance use grievances pertained to residential treatment programs.
Overall, there were slight decreases noted in grievances pertaining to Finances, Patient’s Rights, and
Quality of Care issues. Compared to the prior FY, there was a significant percentage decrease in
grievances relating to Medication (-50%). There were slight increases noted in grievances relating to Access to Care, Lost Property, Operational, and Physical Environment. Compared to the prior FY, there
was a significant percentage increase of grievances relating to Change of Provider (+200%), Staff Behaviors (+41%), Treatment Aspects (+75%), and Peer Behaviors (+100%). These four categories of
grievances together comprised 63% of all grievances.
In regard to grievances relating to Change of Provider in FY 16-17, there were 6 grievances (6.4/%): four
were granted and two were denied, both for clinical reasons and need for continuity of care. While a grievance relating to Change of Provider is usually, although not necessarily, about a client’s request to
change a provider, all six grievances were requesting to change their providers due to dissatisfaction about their current providers. All six grievances were filed by female clients and all occurred at the
outpatient mental health level of care. One of the denied requests pertained to a culture-specific issue
whereby the client did not want an Asian-focused program nor her monies managed by the program; however, the program had no such focus and is not involved in money management. One of the
granted requests also pertained to a culture-specific issue and, while the client did not state this as a reason for requesting a change of provider, she was successfully transferred to an Asian counselor.
BHS Grievances/Appeals by Category
July 2016 – June 2017 Total Number = 94
(Appeals = 0 Grievances = 94)
Grievance/Appeal Category Number
Access –Service not Accessible 2
Access – Timeliness of Service 2
Access – Other Access Issues 7
QOC – Staff Behaviors 31
QOC – Treatment Aspects 14
QOC - Medication 5
QOC – Other QOC Issues 4
Change of Provider 6
Other - Finances 2
Other – Lost Property 2
Other - Operational 5
Other – Patient’s Rights 5
Other – Peer Behaviors 8
Other – Physical Environment 1
18
The Risk Management Committee meets on a monthly basis and is comprised of Quality Management staff, the Deputy Medical Director, clinical pharmacists, and service providers. Information about
grievances, appeals, and incident reports are entered into a Risk Management database, and then reviewed for patterns that may inform the need for changes in policy or programming. These trend
reports are routinely analyzed by the Risk Management Committee and the System of Care Age Directors.
The table below reflects incident reports by event category submitted by providers within our Adult &
Older Adult System of Care.
* Deleted event category effective 7/1/15 ** Added event category effective 7/1/15
The table below reflects incident reports by event category submitted by providers within our Child,
Youth & Family system of care.
* Deleted event category effective 7/1/15 ** Added event category effective 7/1/15
Quality of Care Report Summary for Adult/Older Adult SOC by Event Category
FY 2013-
14
FY
2014-15
FY
2015-16
FY
2016-17
% Diff
Prior FY
Violent Behavior (includes DTO) 130 105 171 157 -8%
Sexual Assault/Misconduct 7 9 14 11 -21%
Medication Related 24 30 100 99 -1%
Suicide Attempt 14 30 35 20 -43%
Unethical Conduct 7 2 5 5 0
Client Death 152 109 170 150 -12%
Mandatory Reporting 33 62 125 95 -24%
Service Disruption** 10 17 +70%
Involuntary Hold* (DTS/GD in OTHER) 144 128
Injury, Accident, Acute Medical Problem 255 280 369 388 +5%
Inapprop Tx, Delay, D/C, Documentation* 6 5
PHI Breach** 5 5 0 AWOL/Unauthorized Absence 24 Hr Setting 157 220 289 281 -3%
Other (includes DTS/GD) 43 44 125 197 +58%
Total 972 1024 1418 1425 +.5%
Quality of Care Report Summary for
CYF SOC by Event Category
FY
2013-14
FY
2014-15
FY
2015-16
FY
2016-17
% Diff
Prior
FY
Violent Behavior 42 55 99 54 -46%
Sexual Assault/Misconduct 6 8 2 2 0
Suicide Attempt 1 3 8 6 -25%
Medication Related 4 1 24 3 -88%
Unethical Conduct 1 2 7 3 -57%
Client Death 1 2 1 0 -100%
Mandatory Reporting 32 38 90 108 +20%
Facility Damage* (in SER DIS or VIO) 6 3
Service Disruption** 3 6 +100%
On-Site Accidents* (in INJURY) 0 1
Injury, Accident, Acute Medical Problem** 21 15 -29%
AWOL/Unauthorized Absence 24 Hr Setting 228 98 15 26 +73%
Absenteeism* 1 0
Client Records* (in PHI BREACH) 0 1
PHI Breach** 4 9 +125%
Needlestick* (in INJURY) 0 0
Other 99 43 30 49 +63%
Total 421 255 304 281 -8%
19
Based upon these trend reports, subsequent recommendations for quality improvement activities were made in a number of forums such as the Medication Use and Improvement Committee, the Adult/Older
Adult QIC, the Children, Youth & Family QIC, and the System of Care QIC.
FY 16-17 system issues and/or recommendations identified by the Risk Management Committee include
the following:
Medication Related
Recommend a review of the current prescribing guidelines for ADHD
Recommend a system plan for dealing with benzodiazepine dependence
The need for provider guidelines to help determine when clients are appropriate (or not
appropriate) for medication-only services
Difficulty trouble shooting problems with third party payers of medication services
Not all outpatient mental health programs have the ability to administer long-acting injectable antipsychotics
Require all BHS InfoScriber users prescribing clozapine to use the Clozapine REMS as utilization
of the REMS Program would have prevented 50% of the clozapine-related incidents occurring in
2015 to August 2016
Program Capacity and Client Acuity
Recognition that service disruptions due to clinic site closures for remodeling can have
unforeseen deleterious effects on some patients
Consider that selected programs be equipped with automated external defibrillators (AED)
To enhance the effectiveness of payee services by including payees as part of a patient’s treatment team with plans for money management that are clinically driven and developed in
close collaboration with other members of a patient’s treatment team
Difficulty getting clients admitted to inpatient hospitalization for much needed stabilization
despite the direct requests of clients’ community treatment providers
Practice/Service Delivery
The dearth of trauma specialized services and structured, non-residential programs (e.g., day
treatment, intensive outpatient programs)
The possibility that entering needed residential services will cause clients to lose pre-existing
housing and become homeless
Restriction of two episodes of residential (non-crisis) treatment per year
Programs should routinely request ROI when split treatment is provided by multiple behavioral health services and permission is required in order to collaborate and coordinate treatment
efforts
Development of community care plans (versus program specific treatment plans) which may help integrate care and mitigate the confidentiality barriers which can prevent substance abuse and
mental health programs from communicating
Not all BHS funded programs are reflected on the MHS 140 report or have access to and/or utilize AVATAR/CCMS database
Staff Related
Staff capacity issues
Patient contact information should be regularly updated in the medical record
20
IV. IDENTIFY AND ADDRESS SERVICE DELIVERY AND CLINICAL ISSUES
GOAL IV.a Ensure staff are engaging in appropriate prescribing practices.
Objective 1: By June 30, 2017, improve diagnosing and prescribing for adult ADHD clients.
Actions
1. Form subcommittee to create workflow process, identify screening tool, and non-pharmacologic
tools for adult ADHD diagnosis and treatment.
2. Monitor adult ADHD diagnosing and prescribing rates quarterly.
Objective 1 Results:
A multi-disciplinary subcommittee was created in March 2016 with the goal of improving and
standardizing SFDPH’s systems approach to the diagnosis and treatment of Adult ADHD. This group met monthly to create guidelines for a workflow process. Please see this link for more information about the
guideline workflow identified: https://www.sfdph.org/dph/comupg/oservices/mentalHlth/CBHS/
The screening tool that was identified is the ASRS-v1.1
(https://www.hcp.med.harvard.edu/ncs/ftpdir/adhd/18Q_ASRS_English.pdf) This tool was chosen because it has been used by the World Health organization to measure attention and focus. Also, it is in
the public domain, and available in many threshold languages, making it accessible for most clients in the diverse community of San Francisco’s safety net. A functional impairment scale, the Self-report Adult
Symptoms and Role Impairment Inventory, was also provided as a supplemental screening tool.
Work on finding and using non-pharmacologic tools for adult ADHD diagnosis and treatment continues. A
review of these findings show that mindfulness, time management, and organization techniques prove to be effective for people diagnosed with Adult ADHD.
Additionally, we had a system-wide training by a national expert in the field by Dr. Craig Surman, an
Assistant Professor of Psychiatry at Harvard Medical School and Scientific Coordinator of the Adult ADHD
Research Program at Massachusetts General Hospital. The training was focused on accurately diagnosing and appropriately treating Adult ADHD, and was held on October 26, 2016 with 200+ providers and
prescribers in attendance.
Although the system still views this as an important activity, we abandoned it as a PIP after our 2016
EQRO feedback due to concerns about measurement and not meeting the standards associated with PIPs. Because we are not pursuing it as a PIP, we have not measured these activities quarterly.
The following bar graphs show the number of adults diagnosed with ADHD in FY 15-16 and 16-17:
Number of ADHD diagnoses of adult clients
255
356
0
50
100
150
200
250
300
350
400
FY15-16 FY16-17
21
The chart shows that the diagnosis of ADHD has increased from 255 in FY15-16 to 356 in FY16-17. Although Whites constitute the majority of clients diagnosed, increases were evident in virtually all
gender and ethnicity groups.
The following Chart shows the proportion within each ethnic group receiving treatment services who have
the diagnosis. Among adults, Whites are substantially more likely to receive the diagnosis than Asian Americans, Latinos, African Americans, Native Americans, or Native Hawaiian/Pacific Islanders.
Percent Diagnosed with ADHD
Episodes open during FY16-17
2.2%
1.6%2.0%
6.3%
1.9% 1.8%
3.1%
4.2%
0.0%
1.0%
2.0%
3.0%
4.0%
5.0%
6.0%
7.0%
African-American
/Black
(54 / 2353)
Asian
(37 / 2376)
Latino/a
(35 / 1734)
Multi-ethnic
(14 / 223)
Native American
(3 / 157)
Native Hawaiian
or Other Pacific
Islander
(2 / 110)
Unknown /
No Entry /
Other
(33/1052)
White
(183 / 4324)
The following Chart documents the number of clients using an ADHD medication. Our criteria for use of
the medication is that they have been written prescriptions that extend for 60 days within each 90-day quarter.
Number of adult clients on ADHD medications
110103
97
108
0
20
40
60
80
100
120
Jul-16 Aug-16 Sep-16 Oct-16 Nov-16 Dec-16 Jan-17 Feb-17 Mar-17 Apr-17
As can be seen, the number of clients using the medications is relatively constant over time.
Objective 2: By June 30, 2017, reduce the % of clients screening positive for benzodiazepine in methadone programs.
Actions
1. Programs to report baseline positive screen rates and quarterly thereafter.
Across all methadone programs in FY 16-17, the range of client positive screens for benzodiazepine was between 10-15%. This relatively low rate has remained stable and is not expected to change, so
prescribing improvement efforts will focus on a different priority for the next fiscal year.
22
GOAL IV.b. Increase the use of Group Therapy as a modality of care for mental health treatment.
Objective 1: By June 30, 2017, implement two cohorts (~16 clients) of evidence-based Supporting
Father Involvement (SFI) program.
Actions
1. Graduate at least 50% of enrolled clients from SFI program.
Objective 1 Results:
In FY 2016-17, the Wellness Center delivered SFI to six fathers, two of whom attended at least half of the sessions, and four of whom attended the graduation. Due to challenges with recruitment and
retention, the decision was made to delay starting a second cohort and focus on better understanding the needs of fathers in the Sunnydale community. A focus group with 24 father was held on September 27,
2017. We are using the findings from the focus group to guide programming for the Fatherhood Initiative programming at the Wellness Center and ensuring it aligns with the priorities and preferences of
the community.
Some initial changes are:
Providing low-threshold participation events, such as social events, at the Wellness Center to develop
trust and buy-in from the community
Adding content, either prior to beginning SFI or incorporated into the curriculum, that addresses the effects of trauma and racial discrimination on fathers
Reworking the incentive plan to maximize participant motivation
Holding a multi-day cross-training between the SFI developer and Wellness Center staff that includes
the SFI training components as well as training and conversation about the local context in which SFI
is delivered at the Wellness Center.
The second SFI series is planned for January 19, 22, and 23, 2018 and the dates are confirmed with the SFI developers.
GOAL IV.c. Expand the Trauma-informed Systems (TIS) Initiative.
Objective 1: By June 30, 2017, implement a workforce training on the principles of a trauma-informed system.
Actions
1. Train a team of at least 2 trainers capable of presenting the training to remaining staff from both
ZSFGH and Population Health.
2. Develop workforce healing circles in collaboration with the Black and African American Health Initiative.
3. Pilot a Leadership Learning Community with a multi-department, cross discipline cohort.
4. Disseminate a preliminary Trauma Informed Leadership toolkit of resources.
Objective 1 Results:
The Trauma Informed Systems (TIS) Initiative has continued to expand through training and piloting
programs designed to sustain, support and embed a culture of knowledgeable responses to trauma. To date, one TIS trainer is dedicated to reaching out and training ZSFGH and Population Health staff. The
TIS coordinator is working diligently with ZSFGH leadership to identify TIS trainers and a coordinator.
The TIS team worked in collaboration with the Black and African American Health Initiative to develop two Healing Circles for DPH staff. The Healing Circles where piloted in the fall of 2016. Both were well
attended.
The Trauma Informed Systems Leadership Learning Community began meeting in January of 2016 and
consists of leadership from Children, Youth and Families as well as 4 other child-serving departments in the city. The leadership learning community is partnered with staff Champions from various roles and
23
levels within each participating organization to support a collaborative, multi-level model. Participants meet regularly to develop strategies, discuss progress and share resources to promote TIS
implementation within the workplace. Leadership from across agencies meets quarterly to discuss progress and share strategies on creating and sustaining TIS policies and practices. Champions meet
monthly to participate in and design strategies aimed at organizational change, relevant to Trauma
Informed Principles and Competencies. Leadership and Champions collaborated on TIS strategies informed by individualized agency results from an organizational assessment called the Tool for a Trauma
Informed Worklife (TTIW) Survey that was administered in May 2016. Leadership and Champions continue to review workplace policies and procedures in consideration of TIS principles and practices, and
are preparing to re-administer the TTIW and collect relevant data to evaluate progress.
A preliminary Leadership Toolkit was disseminated as part of a pilot, consisting of a literature review of
existing TIS literature, quality improvement/management tools, and the Trauma Informed Worklife survey and the Trauma Informed Agency Assessment. This toolkit is being further developed in
consideration of results and lessons from the pilot.
GOAL IV.d. Expand implementation of Wellness and Recovery (W&R) Practices in behavioral
health programs.
Objective 1: By June 30, 2017, BHS will test a new process for transitioning clients from ICM/FSP to Outpatient services (“step down”) with at least 10 ICM/FSP clients.
Actions
1. Convene a series of collaborative working meetings with representatives from the ICM/FSP programs
and Outpatient clinics, sponsored by BHS and facilitated by a trained consultant.
2. Identify programmatic and systemic barriers to transitioning clients to OP services, as well as conditions/processes that currently work well.
3. Decide on a set of recovery criteria or conditions that identify clients’ readiness for referral to lower
intensity services (i.e., graduation or “step down”).
4. Solicit solutions or pathways to transitioning clients to OP care (brainstorm).
5. Create small tests of change (PDSAs) that explore solutions for transitioning clients and expand testing
to multiple sites and implement best learning across BHS.
Objective 1 Results:
Actions:
1. Convene a series of collaborative working meetings with representatives from the ICM/FSP programs and Outpatient clinics, sponsored by BHS and facilitated by a trained consultant.
From April 7, 2017 through June 16, 2017, six meetings were convened with Intensive Case Management
(ICM) and Outpatient (OP) providers, consumer advocates and peer employees, facilitators from Learning For Action (LFA), and BHS administrative staff. All six meetings were consistently well attended.
The goals of the convenings were to: 1) Build relationships between providers of ICM and Outpatient programs
2) Clarify the problem to address (clients getting lost between ICM and outpatient services)
3) Identify barriers and potential solutions to supporting clients in the referral and linkage to OP 4) Identify a topic for an MHSA Innovation proposal
Convening
#
Date Attendance
(sign-ins)
1 April 7 24
2 April 21 37
3 May 5 42
4 May 19 20
5 June 2 26
6 June 16 23
24
An ICM-OP Flow workgroup consisting of BHS administrators from the System of Care, Quality Management (QM) and MHSA, collaborated with an evaluation consulting firm, LFA, to coordinate the six
convenings, define meeting objectives, discussion topics, and arrange logistics. LFA facilitated the discussions and produced detailed notes for the workgroup.
2. Identify programmatic and systemic barriers to transitioning clients to OP services, as well as conditions/processes that currently work.
Transitioning clients from ICM to standard outpatient services has been a challenge for many years. Past
efforts to identify and understand the causes and conditions that hamper client transitions include 1) the
Advancing Recovery Practices Learning Collaborative with Citywide Forensics that focused on client graduations and transitions to South of Market Mental health Services (2013-14); 2) the San Francisco
Health Network QI Academy teams that focused on transitions to primary care; 3) the ICM/FSP Recovery initiative meetings that gave rise to the need to strengthen communication and support for clients
transitioning to OP clinics, and 4) the Psychiatric Inpatient transition effort that focused on “Gold Card” clients or fast-tracking linkage to outpatient care.
Causes and conditions that impact ICM client transitions to OP care are captured in the fishbone diagram below. Starred items are those providers identified as within our locus of control to improve.
Low success
in
Transitioning
Clients to
OP care
MATERIALS / IT SUPPORT
PROCESSES / WORKFLOW HUMAN FACTORS
Transitions in Care: Root Cause Analysis
ICM Case Mgrs worry
about clients relapsing
Benefit program rules (SSI) can
discourage full time employment
Lack of safe and stable
housing for clients
ENVIRONMENT
Drugs are prevalent and
constantly available to clients
ICM and OP clinics are
usually separate geographically
Clients may feel attached
to their ICM Case Mgr
Clients may fear less
easy access to support
OP Case Mgrs may perceive ICM
client as “high need”, more
demanding
ICM Case Mgr may resist
Replacing “recovered” client
w/a new, “higher need” client
OP clinicians have very high caseloads
Poor communication / lack of communication
protocols between ICM and OP providers
No clear protocol defined by BHS
No agreed upon set of criteria or conditions
agreed upon to assess client readiness
ICM and OP clinicians have a lot
of documentation requirements
No standard designated position at OPs
to handle intakes (varies by clinic)
No single Checklist
Form in use for BHS
Avatar does not have a
specific section with transitions
data (e.g. Checklist or tracking
referrals)
Avatar does not have data
or reports that identify clients
ready for transition
In addition to these potential root causes, there were also several system-level factors that were
indentified by the group as contributors to the problem, including a large gap in service and support between ICM and Outpatient levels of care; and the sense that peer employees are undervalued and not
yet fully integrated into BHS service delivery.
25
3. Decide on a set of recovery criteria or conditions that identify clients’ readiness for referral to lower intensity of services (i.e. “graduation” or “step down”).
A set of criteria has not yet been developed. The six convenings concluded by identifying potential
solutions to facilitate referrals and linkage. One set of solutions will focus on improving communications
and protocols for referring clients from an ICM to OP services.
4. Solicit solutions or pathways to transitioning clients to OP care (brainstorm).
The convenings generated a set of potential solutions for transitioning clients. They fall into two major focus areas:
a. Improving communications and protocols between ICM and OP providers to facilitate transitions to OP, and
b. Proposing the creation of a clinically supervised, outpatient based peer transition support team.
The details will be worked and tested for learning, adaptation and implementation, in volunteer clinics
over the next year. The next steps will be to convene the Communications and protocols workgroup (in July) and schedule regular planning and working meetings over the next six months to develop and move
forward ideas to improve communication. In particular, the workgroup plans to identify the steps involved in an ICM contacting an OP clinics to discuss a transfer, the OP response process when receiving
requests for transfers, protocols for conducting warm hand offs, etc.
The Workgroup is drafting an MHSA Innovations proposal, to be completed in September 2017, aimed at
funding a peer transition support team to help facilitate transitions from ICM to OP services. This level of care transition support was identified as a system “gap” that contributes to clients falling through the
cracks after discharge from the ICM level of care. Pending approval of BHS leadership, the San Francisco Board of Supervisors and the Mental Health Services Oversight and Accountability Commission
(MHSOAC), we anticipate funding will be available in 2018.
5. Create small tests of change (PDSAs) that explore solutions for transitioning clients and expand
testing to multiple sites and implement best learning.
No tests of change were completed on this project, as the planning phase went through the end of the
fiscal year (see above).
GOAL IV.e. Improve clinical supervision.
Objective 1: By June 30, 2017, train 60 Clinical Supervisors across CYF and A/OA SOC in the Clinical
Supervision Model.
Actions
1. Finalize Best Practice Manual and adoption of standards.
2. Develop curriculum and identify trainers.
3. Develop Best Practice Clinical Supervision Toolkit.
4. Launch 10-month Clinical Supervision Training Institute, including 101, specialty trainings, and
learning communities.
5. Complete pre-post test to evaluate program.
Objective 1 Results:
The Best Practice Manual was completed in June 2016 and will be disseminated to the participants in
September when they arrive for their 2-day foundational training. The manual describes the SOC values that frame the material for the training, the model the will be implemented, and the toolkit to support
staff in supervising their clinical staff.
26
The Academy trainers were identified through an RFQ process. We have a lead trainer for each component of the model: Yale School on Supervision will present the foundational training (BHS 101) in a
2-day format; UCSF Infant Parent Program staff will facilitate the bi-monthly learning communities and present the initial Specialty training Reflective Practice in Clinical Supervision; and Nicholas Ladany, Ph.D.
will present the two final Specialty trainings; Supervising to Effective Clinical Care and Creating a Trusting Supervisory Relationship. These three components comprise our 10 month training Academy.
The best practice tool kit was developed using resources from our trainers and other experts in the field of clinical supervision. The purpose of the toolkit is to support our staff with concrete resources that can
be utilized to structure the supervision process, augment the skills that the supervisor possesses, and as a way to respond to the feedback that was shared during the workgroup process. This toolkit is a living
document that will be updated as new resources are identified.
We were unable to launch the 10 month Academy in January 2017 because of contracting barriers with
our primary trainer for the BHS 101 component. Once the barriers were resolved it was not feasible to launch the model in FY 16/17, so it was postponed until its current launch date in September 2017.
Drs. Richie Rubio and Farahnaz Farahmand led the design and development of an evaluation plan, which includes collecting data from clinical supervisors and clinicians (the clinical supervisors’ supervisees)
across all of BHS Civil Service clinical staff. This will allow data to be linked by supervisory pairs and includes a comparison group of those who are not attending the Clinical Supervision Training & Learning
Collaborative. The evaluation survey includes the following domains: 1) demographics of clinical supervisors and clinicians, 2) supervision practice, 3) supervisory relationships, 4) supervisor / clinician
competency, 5) job satisfaction, and 6) burn out. In addition, we incorporated the system lens/priorities
of trauma-informed systems and racial humility.
The survey was distributed to all BHS staff for completion of a “pre-test” in January 2017. Given delays to the launch of the training and learning academy, the survey was re-administered to all clinical supervisors
and clinicians in August 2017 to better capture the baseline of these measures before the launch of the
trainings in September, and to capture the current snapshot of supervisory pairs. Survey results are currently being analyzed. Another survey will be distributed in January 2018 (mid-way through the
academy) and again in July 2018 at the completion of the academy.
GOAL IV.f. Increase use of evidence-based practices.
Objective 1: By June 30, 2017, provide Dialectical Behavioral Treatment (DBT) program to 25 clients
in 2 CYF behavioral health programs.
Actions
1. Provide on-going training and clinical consultation to identified clinics.
2. Monitor use of hospital and crisis utilization of CYF clients, as well as client suicide risk and DBT providers’ burnout.
Objective 1 Results:
Training & Consultation Since the start of services in July 1, 2016, our UCSF DBT Consultant has been providing monthly training
and clinical case consultation at the DBT clinic site. Intensive training to the two new clinicians was provided by our DBT Clinical Supervisor and our UCSF Consultant. The UCSF consultant will continue
monthly training and case consultations services for the team this next fiscal year.
Monitoring/Research
Data collection of client level evaluation data and clinician burnout data started simultaneously with the beginning of the treatment program in July 2016. System level data is still in the process of being
gathered and integrated so UCSF staff, when resourced, can analyze the data to examine the impact the DBT clinic is having on crisis utilization and hospitalization, in addition to therapist burn out and skill
acquisition.
27
DBT clinicians completed a satisfaction measure following the initial DBT trainings and again after 1 year of monthly consultation. The ratings provided are averages of 7 clinicians post-training satisfaction, and 3
clinicians following 1 year of consultation using a five point scale (1=not enjoyable/helpful, 2=somewhat enjoyable/helpful, 3= enjoyable/helpful, 4= very enjoyable/helpful, 5=extremely enjoyable/helpful).
Post-training/pre-
consultation Average
Post-1 year of
consultation
How much did you enjoy the DBT-A consultation and training? 3.9
4.3
How helpful was the consultation to aid you in treating suicidal patients? 4.0
5
Please provide an overall rating of the program with respect to how helpful it was to you in learning how to implement DBT-A. 4.1
4.3
V. ASSESS PERFORMANCE AND IDENTIFY AREAS FOR IMPROVEMENT
GOAL V.a. Use quantitative measures to assess performance and to identify and prioritize
area(s) for improvement.
Objective 1: By June 30, 2017, clients will improve on at least 30% of their actionable items on the
Adults Needs and Strengths Assessment (ANSA).
Actions
1. Develop and disseminate quarterly reports tracking program and client-level outcomes.
2. Facilitate "data reflection" meetings with clinics to explore the meaning of ANSA results relative to clinical practice, and identify areas for improvement.
Objective 1 Results:
We have continued to produce quarterly reports that display client improvement on the ANSA, both at the program- and the item-levels. The program-level report displays the percent of clients who showed
improvement on 30% of their actionable items (those rated 2 or 3 on the time 1 ANSA). The item level report displays the number of clients who had actionable needs at time 1 and the number of those clients
who improved on each of those items at the time 2 (most recent in FY 16-17) ANSA.
These reports are posted on the DPH public website. The link to the most up-to-date (for the first 3
quarters of FY16-17) summary report is: https://www.sfdph.org/dph/files/CBHSdocs/ANSADocs/FY16-
17ObjectiveA.2ANS_Outcomes_Summary_Report_7.1.16-3.31.17.pdf
The BHS program performance objectives state that 40% of clients with actionable needs at the earlier
ANSA should improve on 30% of those actionable items. Using the 3rd quarter results, only 2 programs have not met this benchmark.
The summary report is supplemented by an item level report, which shows specifically which needs and
strengths are most prevalent in each program, and the extent to which clients improved on each item.
The link to the item-level report is: https://www.sfdph.org/dph/files/CBHSdocs/ANSADocs/FY16-17ObjectiveA.2ANSA_Outcomes_Item-
Level_Report_7.1.16-3.31.17_wCJCBHS.pdf
The first page of the report contains a system-wide summary. It shows that Depression and Anxiety are
the most prevalent client needs in the system, affecting 63% and 57% of the cohort of clients,
28
respectively. The report also shows that for this fiscal year, about 39% of clients improved on Depression and 37% improved on Anxiety.
The final ANSA reports for FY16-17 will be completed and posted by September 15, 2017.
We have taken findings from qualitative interviews conducted with clinical staff and developed new ANSA
reports highlighting what clinicians say is most important to their work: a focus on clinical care. One of these, called the Traffic Light Report, allows for quick comparison of ANSA ratings over time and debuted
in October 2016. The findings are color-coded: green for improvement, yellow if they stayed the same, and red for deterioration. The report resides within Avatar and can be generated at the click of a mouse
for any client with at least two ANSAs.
A team of quality management and system of care staff has been visiting BHS clinics to introduce and
discuss the Traffic Light Report. Within the fiscal year, 17 meetings took place with staff at a variety of agencies, including those specializing in transitional age youth, older adults, outpatient mental health,
hospital settings, residential settings, and intensive case management. Proposed improvement-focused uses of the report include clinical case consultation, supervision, and service utilization review at the clinic
level. A subset of clinicians additionally welcomed the use of the ANSA as a communication tool between
client and clinician. Clinics have largely welcomed the data reflection initiative, indicating that they are receptive to training and quality improvement activities in service of their clients.
Objective 2: By June 30, 2017, clients will improve on at least 50% of their actionable items on the
Child and Adolescent Needs and Strengths Assessment (CANS).
Actions
1. Develop and disseminate quarterly reports tracking program and client-level outcomes.
3. Facilitate "data reflection" meetings with clinics to explore the meaning of CANS results relative to
clinical practice, and identify areas for improvement.
Objective 2 Results:
Two types of quarterly reports were developed to reflect performance on the CANS objective: program-
level summary report and item-level report. These reports are both posted online quarterly, and are
discussed in Executive QI and CYF management meetings, as well as CYF provider meetings. These
reports are also the basis for data reflection activities conducted by programs with their clinicians and
staff.
The program-level summary report is a one-page graph that contains a bar for each CYF-SOC program
that represents the percentage of clients in each program who are meeting the objective. This report
displays the percent of clients who showed improvement on 50% of their actionable items (those rated
as either a '2' or '3', indicating either a moderate or a severe need at the previous or time 1 CANS). The
BHS performance objectives state a goal that 40% of clients with actionable needs at the previous CANS
should improve on at least 50% of their actionable items. Across all CYF programs, 47% of clients met
the benchmark of improving on at least 50% of their actionable items on the CANS. The link to the
program-level report is: https://www.sfdph.org/dph/files/CBHSdocs/CANS-
CalOMS/PerformObjA2a_Q4_FY1617.pdf
The item-level report is a more detailed report that illustrates each item on the CANS, the number of
clients who had an actionable need at the previous or time 1 CANS, and the number of clients who
improved on that item at the current or time 2 CANS. The item-level report allows programs to identify
areas where they are doing well, and determine their intervention and training needs. The first four
pages of that report contain a system-wide summary. It shows that Anxiety, Depression, and Anger
Control (for the 5 to 18 age group); and Adjustment to Trauma, Regulatory problems, and Anxiety (for
the 0 to 4 age group) are the most prevalent client needs in the system. In terms of improvement among
these prevalent needs, Anger Control (48%) showed the most improvement for the 5 to 18 age group,
while Regulatory Problems (62%) showed the most improvement for the 0 to 4 age group. The report
29
also includes item-level reports for each program in the CYF system of care. The link to the item-level
report is: https://www.sfdph.org/dph/files/CBHSdocs/CANS-
CalOMS/PerformObjA2_CANSItemLevel_Q4_FY1617.pdf
The following data reflection activities were facilitated for the FY 2016 to 2017:
(1) Data Reflection Assist Workshop (DRAW). Four 2-hour DRAW workshops were facilitated for
clinic/program staff (i.e., clinicians, case workers, clinic managers/directors, QM personnel, data
analysts) on September 26, 28, 29, and 30, 2017. The DRAW sessions aimed to facilitate data reflection activities and coach staff on how to effectively utilize CANS reports to engage in this
reflection process. The workshops provided guidance on how to reflect on client outcomes with the three questions on the Data Reflection Form: (1) How can we understand this data? Please discuss
findings; (2) What other information do you use to understand your clients’ needs, strengths, and
progress; and (3) What is your plan moving forward after reflecting on your program’s CANS outcomes? Staff who participated were then encouraged to (a) facilitate a data reflection process
with their program’s management/leadership team and staff, and (b) submit a data reflection form that summarizes the outcome of their data reflection activities. The following flow chart shows the
timeline of this data reflection process:
(2) Identifying clinical practice improvement opportunities from data reflection outcomes. Twenty-three CYF-SOC programs conducted data reflection with their staff and submitted a data
reflection form. These forms were reviewed and scored by QM staff and CYF Management. Qualitative data analysis was also conducted to generate the most prevalent themes from the
completed data reflection forms. The themes that emerged were used to facilitate conversations
around CYF priorities for clinical practice improvement opportunities. The results of the qualitative analysis were also presented in different venues for further data reflection aimed at seeking
recommendations for clinical practice improvement. These venues included the SF-DPH Executive QI meeting, and the CYF-SOC Providers’ Meetings. The following clinical practice improvement
opportunities were identified: Family-Centered system of care, substance abuse integration, reflective
practice in data and clinical supervision, school-based interventions, strength-based assessment and intervention, and primary care collaboration.
(3) CYF-SOC System-Wide Data Reflection. CANS Item-Level reports are generated by QM on a
quarterly basis. These reports include system-wide as well as program-specific data. The system wide
data is presented for data reflection at the CYF-SOC providers’ meeting, which happens on the third Tuesday of each month. The program-specific reports are provided to each clinic/program on a
quarterly basis and they are encouraged to conduct data reflection activities with their staff.
(4) CYF-SOC Program-Specific Data Reflection. A few programs requested QM and CYF management to facilitate a data reflection meeting with their staff/clinicians: Alternative Family
Services (AFS) Outpatient Clinic; Alternative Family Services (AFS) Visitation program; Chinatown
Child Development Center (CCDC); Edgewood Center for Children and Families (separate meetings for some of their programs: Outpatient Program, Crisis Stabilization Unit, Residential); Foster Care
Mental Health (FCMH); Instituto Familiar de la Raza (IFR); and Mission Family Center (MFC).
30
GOAL V.b. Implement Quality Improvement Training Academy.
Objective 1: By December 31 2016, 6 BHS clinics and the Executive Team will complete a year-long Quality Improvement Training Academy.
Actions
1. Six clinic teams and the BHS Exec Team will participate in the year-long QI Training Academy
(including monthly trainings and twice monthly team meetings) and receive individualized coaching on
QI implementation.
2. Teams will apply QI tools to address system priorities.
3. Teams will use data to monitor progress on their QI projects.
Objective 1 Results:
In November 2016, six multi-disciplinary outpatient clinic teams ((Chinatown North Beach Mental Health
(CTNB), Foster Care Mental Health (FCMH), Mission Family Center (MFC), Mission Mental Health (MMH), South of Market Mental Health (SOMMH), and Sunset Mental Health (SMH)) along with a BHS Executive
team completed a year-long quality improvement project primarily focused on access-related system priorities within the QI Training Academy structure which consisted of 9 half-day in-person training
sessions, two remote webinars, individualized coaching and data support. In addition to the training
sessions, clinic teams met twice a month with their assigned coach and data support to discuss potential countermeasures to test, review data to monitor progress, and plan for next steps in the QI process.
Teams also met once a month with their assigned Executive Sponsor to address any system barriers that may be hindering the QI progress. Coaches participated in a monthly 2-hour coaching session with two
consultant Master Coaches to learn and practice various QI concepts and tools.
As a final product of the project, all six teams created a final report which utilized an “A3” Problem
Solving and Project Documentation tool from the LEAN method. The report included information on clinic background, current conditions, problem statement, target/goal, analysis (e.g., process map, Pareto
chart, root cause, etc.), countermeasures, results (e.g., run chart with target/goal data), lessons learned, and next steps. Project posters were displayed at the last training session in November for other clinic
teams to observe, ask questions, and provide recommendations/guidance for further QI work if the team
chose to continue after the end of the QI Training Academy.
Objective 2: By June 30, 2017, three new BHS teams will participate in the 2017 Quality Improvement Training Academy.
Actions
1. Beginning in January 2017, 3 new teams will participate in monthly trainings and twice-monthly team meetings to apply QI tools to solve priority clinic or system problems. Each team will be provided a
coach and data support person.
Objective 2 Results:
Two multi-disciplinary clinic teams, one mental health (Special Programs for Youth- SPY) and one
substance use (Treatment Access Program-TAP), participated in the QI Training Academy to work on clinic priorities and received individualized coaching and data support.
CLINIC PROJECT TOPIC QI TARGET/GOAL COACH
SPY Improving Client
Discharge Process
Increase the percentage of completed
client discharge summaries monthly from 50% to 75%.
Michelle Meier
TAP Implementing
ASAM/Level of Care tool
Reduce the number of people who
wait for entry into residential substance use disorder treatment
from 18.2% to less than 10%.
Tom Bleecker
31
In this reporting period, there were 5 half-day in-person training sessions, 10 in-clinic QI team meetings,
and 3 2-hour coaching sessions. As of June 30, 2016, teams submitted a mid-year progress report
which utilized an “A3” Problem Solving and Project Documentation tool from the LEAN method. The report included information on clinic background, current conditions, problem statement, target/goal,
analysis tool (e.g., process map, Pareto chart, root cause, etc.) and run chart with baseline and one
additional data point. Teams will continue to participate in the QI Training Academy program through November of the next fiscal year.
GOAL V.c. Improve Clinical Documentation.
Objective 1: By June 30, 2017, all mental health clinical staff will have participated in an in-depth clinical documentation training.
Actions
1. BHS Clinical Documentation Specialist will organize large system-wide documentation trainings in Fall
2016.
2. Brown-bag "drop-in" documentation clinics will be provided for each level of care on a monthly basis.
3. Clinic-level consultation and coaching will be provided to improve clinic specific documentation
challenges.
4. Documentation "Super Heroes" will be identified in each clinic, and will have responsibility for being
the on-site contact person for documentation resources and questions.
Objective 1 Results:
In December 2016, Quality Management hired a Clinical Documentation Specialist with an overall goal of
developing a Clinical Documentation Improvement Program (CDIP) that includes comprehensive training,
individualized clinic coaching, and responding to documentation consultation needs. The CDIP position works closely with the DPH Compliance Officer to provide training and coaching to programs after they
have received compliance audits and compliance checks.
In FY16-17:
Documentation Trainings & Workshops: a comprehensive (8 hour) documentation training curriculum was
created and implemented for CYF and A/OA staff. Medical staff received a specific 3.5 hour training. The table below shows the date of training, length and target audience.
Dates Time Location Targeted Section
10/28/2016 9am-5pm Moran Hall, Laguna Honda Hospital CYF
10/31/2016 9am-3:30pm Koret Auditorium, 100 Larkin St A/OA
11/3/2016 9am-5pm Moran Hall, Laguna Honda Hospital Medical (9a-12:30) and
Open Training (1p-5p)
11/4/2016 9am-5pm Moran Hall, Laguna Honda Hospital CYF
11/7/2016 9am-5pm Koret Auditorium, 100 Larkin St A/OA
11/14/2016 9am-5pm Moran Hall, Laguna Honda Hospital Medical (9a-12:30) and Open Training (1p-5p)
11/18/2016 9am-5pm Moran Hall, Laguna Honda Hospital CYF
11/21/2016 9am-5pm Koret Auditorium, 100 Larkin St A/OA
12/5/2016 9am-5pm Koret Auditorium, 100 Larkin St CYF
12/12/2016 9am-5pm Moran Hall, Laguna Honda Hospital A/OA
12/15/2016 9am-5pm Moran Hall, Laguna Honda Hospital Medical (9a-12:30) and
Open Training (1p-5p)
12/16/2016 9am-5pm Moran Hall, Laguna Honda Hospital CYF
12/19/2016 9am-5pm Koret Auditorium, 100 Larkin St A/OA
In the process of planning the trainings, CDIP, QM and SOC decided to target the trainings to the staff
responsible for management, oversight and documentation training at the clinics (i.e., improving
organizational infrastructure to provide training) rather than attempting to train every BHS staff member
32
within a single year. Over 300 (unduplicated) managers, supervisors, program directors received the
comprehensive documentation curriculum in FY16-17.
In addition, targeted workshops (90 minutes) were conducted that focused on a rotating topic
(assessment, client plan, progress notes). In total, 10 workshops held over 6 months (4 workshops on Assessment; 3 Workshops on Treatment Planning; 3 Workshops on Progress Notes) and 90 staff
(unduplicated) participated.
Coaching & Consultation: staff- and agency-level coaching and consultation was provided to groups
whose needs were identified through BHS management (e.g., request from an AOA Program Manager to meet with a clinic), and through DPH Office of Compliance (e.g., after completing a formal audit), as well
as through self-referrals (e.g., an agency emails with a request for coaching). These coaching and
consultation sessions included:
Coaching:
o SF-FIRST (initial coaching; two trainings in staff meetings) o Mission Mental Health (initial coaching)
o Mission ACT (initial coaching; one training in staff meeting) o Westside (one documentation training to staff)
o Mobile Crisis (one training to staff)
o South of Market Mental Health (initial coaching) o Laguna Honda Hospital (initial coaching; 3 workshop series on documentation)
o Chinatown Child Development Center, CCDC (documentation training) o Baker Places (documentation training)
o Group Home providers (overview of Medi-Cal system in preparation for CCR)
Consultation: o Provider Organization-level consultation (e.g., AIIM Higher Program-consultation on
implementing group therapy; CCDC, review clinical documentation and give feedback/guidance to program directors);
o Staff-level consultation (e.g., answer questions via email, phone and in-person at Provider Meetings)
o System-level consultation (e.g., supporting A/OA and CYF Leadership on clinical
documentation questions and consultation).
In the process of establishing the “Superheroes” program (i.e., the clinic-level staff member responsible for documentation resources and questions), CDIP, SOC and DPH Compliance observed that programs
did not necessarily have staff with sufficient knowledge and expertise to discharge the duty (e.g., Civil
Service programs generally do not have any QM staff and in some instances, are mainly staffed by case managers with limited scope of practice). The decision was made to have DPH Compliance lead the task
of having programs identify an agency-level Compliance Officer, given the expectations and requirements through the Health and Human Services’ (US Department of HHS) Office of the Inspector General (OIG).
Objective 2: By June 30, 2017, develop a clinic-level structured quality assurance process to proactively identify documentation problems.
Actions
1. Train clinics on an internal chart review tool; implement a new clinical documentation manual and a
"desk reference" tool.
2. Work with Compliance to develop analytic reports to identify clinics with specific documentation problems.
3. Sample a percentage of charts, selected based on analytic reports, and conduct external clinical documentation improvement review. Provide feedback and coaching to clinics based on results.
33
Objective 2 Results:
Documentation Tools-Chart Review Tool, Documentation Manual, Desk Reference: Designing, developing and implementing clinical documentation tools was a major improvement area. The mental health chart
review tool was updated based on DHCS’ updated Annual Review Protocol—with that update, trainings
will be conducted in December 2017 and February 2018 by Compliance.
The new Specialty Mental Health Services (SMHS) documentation manual for Outpatient services continued in the FY16-17 year. The process was delayed due to the impending release of the
Information Notice for chart documentation. However, the new documentation manual is expected to be published on 10/31/2017. In FY16-17, the decision was made to additionally create a documentation
manual for other core constituencies (Psychiatric Emergency Services/PES; Inpatient Hospital/SFGH) —
those manuals are expected to be finalized in December 2017.
To round out the “suite” of documentation support tools for providers, the SMHS Outpatient Desk Reference was published (September 2016) and revised and republished (July 2017). The Clinical
Documentation Specialist utilized this tool in ongoing trainings through FY 16-17 with both Civil Service
and CBO mental health providers.
During the FY16-17 year, the decision was made to also create mental health Desk Reference tools for specific constituencies (Private Provider Network/PPN, PES, ZSFG)—those Desk Reference Tools are also
expected to be finalized around December 2017.
For SUD, a Documentation Manual and Chart Audit Tool were finalized in December 2016 by DPH
Compliance.
File Review Processes with Analytic Reports: The Clinical Documentation Specialist developed reports to summarize Compliance findings. In May 2017, a Compliance Data Analyst was hired to expand the
reports beyond a simple summary (i.e., “drill down” to identify staff-level problems).
From January 2016 through April 2016, CYF-SOC and CDIP collaborated on a pilot project to create a
risk-based framework to prioritize programs and the client charts to review (about seven clinics). Cases were sampled based on length of stay (newly opened; six to nine months; more than one year).
Following the review, a client-level improvement report was created and provided to the program with
required disallowances (if applicable) and recommendations for improvement. As a final step, Program Directors met with CYF-SOC leadership to create a plan to implement improvement recommendations
from the report. During the same period, A/OA SOC conducted a project to support Civil Service and CBO programs to improve the use of existing Avatar reports to identify charts at risk for late assessments and
treatment plans. AOA Program Managers printed and compiled reports for programs (e.g., performance objective reports on percent of assessments completed), educated providers on the use of the reports
and then delegated the chart review and corrections to the programs directly.
As part of the Triennial Plan of Correction, BHS will take these pilot activities to the next step in FY17-18
through enhancements to the electronic health record and reports coupled with mandatory structured file review processes at the clinic- and system-levels.
Objective 3: By June 30, 2017, ensure that Drug Medi-Cal programs have the appropriate documentation training and are prepared to begin billing Drug-Medi-Cal.
Actions
1. Compliance Team will conduct at least 7 pre-billing readiness reviews.
Objective 3 Results:
Using the newly completed SUD documentation tools, DPH Compliance completed four pre-billing readiness reviews (in June 2017) and expect to complete twelve more in FY 17-18.
34
VI. CONTINUITY AND COORDINATION OF CARE
GOAL VI.a. Ensure that beneficiaries have access to integrated primary and behavioral
health care.
Objective 1: By June 30, 2017, process and outcome data dashboards will be developed for the remaining Behavioral Health Homes (Sunset Mental Health, Mission Mental Health, and Chinatown North
Beach Mental Health) to monitor access to and outcomes for Health Home clients.
Actions
1. Hire a QI analyst who will engage in a collaborative process with Health Home staff to build the
process and outcome dashboards, and participate in monthly dashboard review meetings.
2. All four Behavioral Health Homes will identify at least one quality improvement goal and engage in a
PDSA cycle to improve a clinic process or patient health outcome by the end of June 2017.
Objective 1 Results:
Action 1: A Health Program Coordinator (HPC) was hired in April 2016 to serve as the QI analyst for the Behavioral
Health Homes (BHH). The QI analyst works with the BHHs on quality improvement projects, performance metrics, and evaluation. During the fiscal year 2016-17, the analyst engaged and
collaborated with all four Behavioral Health Homes and their leadership and staff to build processes for
monitoring access to and outcomes for BHH clients and began the groundwork necessary to develop data dashboards for the BHHs. The analyst participated in monthly meetings with each BHH as well as a
monthly joint meeting with BHH leadership and administration.
Action 2:
Last fiscal year (2015-16), Quality Management, in collaboration with the BHHs, created a list of metrics that measure access and quality of client care: (1) Process measures include access metrics such as
referrals and engagement rate, no-show rate, clinician productivity and time to third next available appointment; and (2) Outcomes measures include rates of health screening (e.g., smoking, hypertension,
immunization, cancer) and health improvement measures (e.g., lipids, blood pressure, and HbA1c in good control).
During this fiscal year (2016-17), each BHH reviewed the metrics to be included in a process and outcome data dashboard and confirmed that the chosen metrics are useful and actionable. The BHH at
South of Market Mental Health (SOMMH) continues to review a process data dashboard on a monthly basis. However, we have not been able to create an outcomes data dashboard for SOMMH since the
beginning of this fiscal year due to issues with the data system (i2iTracks) that collects, consolidates, and
retrieves population health data from our EHR. Consequently, efforts to develop an outcomes data dashboard for the other BHHs had to be postponed due to these issues. The QI analyst, however,
continues to collaborate and confer with the Department’s primary care QI team to troubleshoot the problem and identify possible workarounds for obtaining primary care data for the BHHs. The analyst also
continues to collaborate with the primary care QI team to gain access for the BHHs to certain measures
and reports such as the third next available appointment and no-show rate.
The limitations of the i2iTracks data system to accurately retrieve primary care data has been a significant barrier this year to meeting our objective. During this time, we have focused on collaborating
with the three BHHs (Mission Mental Health, Sunset Mental Health, and Chinatown Mental Health) to develop a process data dashboard. To do so, the analyst conducted site visits with each BHH and worked
closely with the staff to determine how the BHH collected process-related data such as number of
referrals and new patients, and helped streamline and improve the way the BHHs track data. The analyst has developed a process data dashboard for the Chinatown Mental Health BHH, and is currently working
with Sunset Mental Health and Mission Mental Health to organize and streamline their process for collecting and tracking data. The analyst also worked with leadership to develop and administer a staff
and client survey to learn about their experience with the BHH and obtain feedback for improving their
experience.
35
Finally, SOMMH identified a quality improvement goal to decrease the no-show rate for the BHH. This PDSA project is still in process while the analyst works with the primary care QI team to access data
necessary for the PDSA. Specifically, the BHH is awaiting data on specific clients with poor show rates. The BHH plans to target these clients with intensive outreach in an attempt to get them to attend their
medical appointments, with the overall goal of decreasing the BHH’s no-show rate.
GOAL VI.b. Improve adequacy and effectiveness of services to youth in Foster Care.
Objective 1: By June 30, 2016, 75% of foster care clients will complete three visits with provider
organization within 45 calendar days of referral for Specialty Mental Health Services.
Actions
1. Monitor percentage of foster care clients referred to Specialty Mental Health provider meeting
engagement criteria.
Objective 1 Results:
Outcome: Objective was not attained (see Figure below). Overall, only 34% (n = 38) of FCMH clients
who were referred for child and/or family therapy (N = 113) completed at least three visits within 45 days of their first visit or session. On a monthly basis, this rate ranged from as low as 0% in June 2017 to
as high as 75% in January2017.
Processes leading to Outcomes: FCMH is still developing procedures and tools for monitoring of service
linkage and satisfaction. However, recent improvements in service engagement may be a response to cultural, procedural and staffing changes in the agency. This includes the following processes:
1. Increased, explicit emphasis on care coordination and monitoring responsibilities as part of FCMH’s work.
2. Participation in CFT meetings has augmented clients' buy-in and attachment. It allows increased family voice and choice, and allows MH staff to address specific concerns, describe treatment
approaches, and provide the family with someone to contact if they have problems with a referral. 3. Pods support collaboration between clinicians and case managers so they can work together to
decide among referral options, and to share information about client engagement, agency waiting
lists, etc. 4. Improved and increased communication with Human Services Agency Psychiatric Social Workers.
5. Existence of three Health and Wellness coordinators who support the monitoring efforts to track client referrals, follow up with families, address barriers, etc.
Challenges Our clients face several barriers to successful service engagement. Additional discussion with our
colleagues at other agencies would help us understand which if any of these factors may be limiting child and family participation in treatment. These barriers include the following:
Children and youth in foster care are often moved from placement to placement, or between foster
care placement and home; geographic changes and communication lapses mean that attendance at services is likely to suffer.
Our child and youth clients can be very hard to reach, whether they are in foster care or at home
with a parent whose stressors include child welfare involvement and a number of service participation requirements.
Service engagement is made more difficult by the very nature of human responses to trauma – social
withdrawal, mistrust, and avoidance of painful subjects.
36
Notes: The height of each bar (with the number on top of it) represents the number of clients who were referred to child and/or family therapy providers. The teal bar (with the number in it) represents the number of clients who engaged in child and/or family therapy. Engagement is operationalized as a client showing up in at least 3 sessions within 45 days of initial appointment. This number is also represented as a percent under each bar.
Successes
There were several notable successes in our FCMH program this year, as follows:
By refining FCMH in-house communication procedures, the speed of service linkage for children identified as meeting medical necessity has improved. The new pod structure made it possible for
CANS clinicians and case managers to discuss screening results and determine linkage options immediately after the CANS is completed.
Our Health and Wellness Coordinators (HWCs) work with the Human Services Agency (HSA) to conduct quarterly reviews of service needs for all youth identified as members or potential members
of the Katie A. subclass. We also developed a mental health screening tool for use by HSA protective
service workers and supervisors.
The HWCs began (in summer 2017) the second annual round of clinical review of all open child
welfare cases, to ensure that each child: o Is assigned to an FCMH pod
o Received an initial mental health screening
o Was linked to services if needed, and has had regular re-assessments o Receives regular follow-up screenings if the child/parent refused services or the child did not
meet medical necessity initially.
37
VII. MONITOR PROVIDER APPEALS
GOAL VII. Appeals from Private Provider Network clinicians will be tracked and evaluated at
least annually.
Objective 1: By June 2017, a report of the number and type of Private Provider Network provider appeals will be evaluated for trends.
Actions
1. Gather all appeals from PPN clinicians and create trend report, sorted by provider and reason for
appeal. Present results to SOC-QIC for action if necessary.
Objective 1 Results:
During the period from July 1, 2016 to June 30, 2017, the San Francisco Mental Health Plan Claims Unit
received appeals from nine Private Provider Network (SFPPN) Providers that were forwarded to Private
Provider Network Director for review and appeal decision. These nine SFPPN Providers submitted appeals that covered 170 separate services/dates to SFPPN Clients. One SFPPN Provider and his medical
billing company were responsible for 153 claims due to an extenuating circumstance beyond their control. We have had no additional late claims submitted by this SFPPN Provider since the workflow issue was
identified.
All of the SFPPN Providers were sent a letter by the either the SFPPN Acting Director or the SFPPN
Director that approved the appealed claims for payment on a one-time courtesy exception to the timely submission requirement, which also noted that all future claims must be received in a timely manner. No
SFPPN Provider requested a repeat courtesy exception and, if a second instance of late submission occurs, those claims will be denied.