2. Evaluation Findings

Evaluation Methodology

The purpose of the evaluation of the Portals Medical Transition Project was to determine the success of the program in implementing the three components of the project to date and to identify recommendations for improvement of the processes. Three primary questions guided the evaluation process including:

• How successful was the project in supporting youth and their families in the health care transition process?
• How effective was the project in supporting family practitioners in assuming the care of youth with special health care needs into their practices?
• How helpful were the videoconferences in supporting services coordinators' understanding of transition issues for youth?

Data Collection Process

Four primary evaluation strategies were used to assess the effectiveness of the project. These strategies included:

• Documentation of the number of participants in each of the three components of the Portals Medical Transition Project.
• Completion of satisfaction surveys by youth, family members and services coordinators on the effectiveness of the clinic in supporting health care transition.
• Completion of satisfaction surveys by physicians on the effectiveness of the orientation, resource materials, and clinic observations on their ability to serve youth with special health care needs.
• Completion of satisfaction surveys by services coordinators on the helpfulness of the educational videoconferences in understanding issues related to transition.

Number of Program Participants

Participants in the program included:

• Nineteen (19) youth and their families who participated in the clinics since the clinics began. These youth were seen between October 2005 and September 2007. Additionally, a staffing was held for one youth with the Transition Team, the Services Coordinator, and HHS staff. (The family had to cancel attendance at the clinic at the last minute.). Resource information was mailed to another six youth and their families, who were unable to participate in clinic. Of these youth, five were in college (four on MHCP and one on A & D Waiver.)
  
• The youth seen in clinic represented a wide range of medical diagnoses, including autism, seizure disorder, cerebral palsy, mental illness, cognitive disabilities, traumatic brain injury, scoliosis, vision problems, paraplegia due to accident, brain tumor, and spina bifida. They ranged in age from 15 to 21 years. The families were geographically distributed across the state, with nine living in the metropolitan area and the remainder living throughout the northeast, southeast central and western parts of Nebraska. Two of the families were Spanish speaking and required an interpreter and translation of reports and Transition Resource materials.
• Although the majority of the children who are on A & D Waiver do not have significant cognitive impairments, the majority (80%) of the children that were referred from A & D Waiver and were seen in the clinic did have cognitive impairments of which most were severe. For youth on the MHCP, only 27% of the youth who came to clinic had severe cognitive impairments. More of these youth had goals of competitive employment and/or post secondary education.
• 15 videoconferences were broadcast for services coordinators throughout the state.
• Eight physicians shadowed the clinic team and participated in training to date.

Dr. Bradley Schaefer met individually with the 10 faculty of the University of Nebraska Medical Center, Family Practice and Internal Medicine. These physicians agreed to observe one of the Transition Clinics. Eight have observed clinic and participated in the training orientation to enhance their understanding of the health care needs of youth with special health care needs and to better prepare them to serve this population as they become adults. The remaining two physicians were unable to participate, due to scheduling conflicts.

“There was enough time to get all of my questions answered. The group was caring, was able to get to the point. Overall, it was wonderful.” ........a parent

Result: Effectiveness of Clinic on Youth’s Transition

Clinic Model

As part of the grant’s continuous improvement process, consumers were asked to provide feedback regarding the clinic. Twenty-three (23) consumers (family members and the youth) completed a satisfaction survey, either through a telephone interview or completion of paper/pencil survey. It was prioritized to obtain youth feedback whenever possible.

“I liked the separate evaluations and I shared one of the resources with my teachers.” .......a youth

Overall, the consumers rated the clinic positively (3.43) (based on a 4-point Likert scale with 1=strongly disagree and 4=strongly agree). They agreed the clinic addressed their concerns (3.83). The consumers reported their ideas were valued (3.56) and that they were part of the decision making process (3.32), suggesting implementation of family-centered principles.

One parent recommended that the team needs to interpret the medical information in a more understandable way. Another recommended the report be sent to their physician sooner, so recommendations could be implemented quicker. They noted that the handouts and materials were a valuable resource.

Results Based the Youth’s and Family’s Response

	Strongly Disagree	Disagree	Agree	Strongly Agree	Not Rated
1. My concerns were addressed by the clinic team.	1	2	3 43% (10)	4 57% (13)
2. The information provided was clear and understandable.	1	2 9% (2)	3 48% (11)	4 43% (10)
3. My ideas were valued by the team members.	1	2	3 48% (11)	4 52% (12)
4. I liked the way the assessments were structured.	1	2	3 65% (15)	4 35% (8)
5. The resource materials provided were helpful.	1	2	3 43% (10)	4 57% (13)
6. I felt part of the decision-making process.	1	2 9% (2)	3 56% (13)	4 35% (8)
7. The clinic team provided me with information about community resources that could help meet my needs.	1	2 9% (2)	3 52% (12)	4 39% (9)
8. The clinic helped me and/or my family get other needed services.	1	2 22% (5)	3 39% (9)	4 39% (9)
9. Overall I found this clinic to be very helpful.	1	2	3 62% (14)	4 40% (9)
10. The information from this clinic will help me as I (my youth) transition to adult services.	1	2	3 65% (15)	4 35% (8)

The services coordinators (8) who were involved in the clinic process rated the clinic positively (3.63). They reported that they were impressed with the holistic interdisciplinary approach, the thoroughness of the report and the later follow-up. They noted that handouts and materials were a valuable resource.

Results Based on Services Coordinators’ Responses

	Strongly Disagree	Disagree	Agree	Strongly Agree	Not Rated
1. The clinic team addressed the family’s concerns.	1	2	3 17% (1)	4 83% (6)
2. The information provided was clear and understandable.	1	2	3 17% (1)	4 83% (6)
3. My ideas were valued by the team members.	1	2	3 57% (4)	4 43% (3)
4. I liked the way the assessments were structured.	1	2	3 83% (6)	4 17% (1)
5. The resource materials provided were helpful.	1	2	3 25% (2)	4 75% (6)
6. I felt part of the decision-making process.	1	2	3 57% (4)	4 43% (3)
7. The clinic team provided the family with information about community resources that could help meet their needs.	1	2	3 50% (4)	4 50% (4)
8. The clinic helped the family get other needed services.	1	2	3 63% (5)	4 37% (3)
9. Overall I found this clinic to be very helpful for the family.	1	2	3 37% (3)	4 63% (5)
10. The information from this clinic will help the youth transition to adult services.	1	2	3 43% (3)	4 57% (4)

“I liked the Youth Transition Plan. it will be very helpful to me as a provider when I take over the youth’s care.” ........a physician

Initial feedback from seven family practice physicians who participated in the clinic as part of a training effort found the experience to be very positive (3.48) [based on a 4-point Likert scale with 1=strongly disagree and 4=strongly agree]. They had limited experience in their current practice with young adults with disabilities. They felt that as a result of this experience that they were more likely to increase the number of young adults with disabilities that they would follow in their practice (3.29). It was noted the comprehensive list of resources they were given would be very valuable in their practice

Results for the Physicians

	Strongly Disagree	Disagree	Agree	Strongly Agree	Not Rated
1. I found the following experiences to be very helpful in learning about addressing the needs of young adults with childhood on-set disabilities or health problems:
• Observing the clinic.	1	2	3 29% (2)	4 71% (5)
• Talking with the clinic staff.	1	2 29% (2)	3 14% (1)	4 57% (4)
• Receiving the notebook of curriculum and resource materials.	1	2	3 57% (4)	4 43% (3)
• Discussion about curriculum and resources.	1	2	3	4
2. I felt prepared to care for a young adult with childhood on-set disabilities prior to participating in this transition clinic.	1	2	3	4
3. I feel prepared to care for a young adult with childhood on-set disabilities after my participation in this transition clinic.	1	2	3	4
4. I am willing to increase the number of young adults with disabilities that I follow in my practice or to begin serving this population.	1	2	3	4
5. I am already serving this population of young adults with disabilities in my practice.	1 25% (1)	2 50% (2)	3 25% (1)	4
6. I plan to use some of the materials and resources in the notebook as part of my training of residents or medical students.	1 20% (1)	2	3 40% (2)	4 40% (2)

Focus Group Findings

A focus group was conducted with the majority of the clinic team members in February of 2007. A summary of their input is provided in the following:

The clinic procedures were reviewed and modified to run more effectively. One of the primary purposes of the pilot was to test out a proposed model for an interdisciplinary transition clinic and modify procedures to increase its effectiveness. The team reflected that over the previous year and a half the team has refined the clinic process to increase its effectiveness of addressing the transition needs of the youth. For example, a summary plan was developed to augment the written report for the family and the nurse specialist now meets with the family to summarize the recommendations of the team on the day of the visit. Overall, they felt they have developed into a cohesive team that worked well together and the clinic format provided an opportunity to learn from each other. Although the clinic initially was conceived to primarily address the health issues related to transition, the team has addressed a wide spectrum of issues. The ability to use a holistic approach has helped to address the needs of the family. All team members felt the clinic via tele-health was very successful and recommend continuing to make this approach available for families who live a great distance from the Omaha area.

A parent suggested that families be given a list of the professionals they will see at clinic. This suggestion was implemented. Another parent suggested that we not use as much technical medical language in our report. We re-wrote the report of that youth to make it more family friendly and we have kept this in mind in the reports completed since then.

One of the ongoing issues, especially during the first year, was the limited referrals to the clinic. During that time period, most referrals were from the Medically Handicapped Children’s Program. Referrals from the Child A & D Waiver increased during the second year. No referrals have been received from the Adult A & D Waiver, although they provide the Waiver Services for youth beginning at age 18. More marketing is needed for the clinic to meet its capacity.

Clinic staff have found some barriers in accessing needed resources for families. For example, extensive work was done with one youth who is diabetic who needed funding for necessary supplies after she turned age 21. Accessing adult specialists, e.g., orthopedist, who have the expertise to serve persons with special health care needs, continues to be a challenge.

The clinic benefited the youth and families. The team identified a number of ways the youth and families benefited from participation in the clinic. The team facilitated a dialog for the family and youth to express their perspectives about transition, the youth’s goals for the future, and begin to discuss what steps needed to be taken. The medical evaluations provided useful information about the youth’s current medical condition, identifying issues that needed to be addressed (e.g., inappropriate medications, misdiagnoses, behavioral issues). Medications were reviewed and recommendations were made as needed. Nutrition and exercise were an issue for many youth and for several these areas were being addressed for the first time. In order to address the broad spectrum of issues presented, the team identified appropriate community resources and supported the family in making these referrals. For example, clinic staff helped youth and their family’s access community resources such as respite, free dental care and connecting the youth with opportunities for social interactions.

The project staff have continued to increase the number of resource materials that are made available for youth and their families, as well as information about services that may be of interest to them in their area. As needed, project staff research services and added resources based on the goals and needs of the youth and their family.