Second phase of the MUSCO Initiative impact study
Results and outlook
In the summer of 2024, almost 50 Initiative collaborators took part in the second phase of the impact study. The main objective was to assess the impact and effects of the experiments carried out, in comparison with the first phase of the impact study done in 2021. As a reminder, the first phase aimed to measure the Initiative’s relevance to improving care and services rendered. At the time, it documented the changes observed on the part of the families who had benefited from the various projects, but also mainly on the part of the professionals involved. With a view to continuous improvement, the results of this study provided food for thought and helped to readjust MUSCO’s orientations thereafter.
In this second, more recent phase of the study, the aim was to analyze the data in order to draw lessons, document the evolution of partnership work and validate potential strategies for sustainability.
The NISKA cooperative: a key role in evaluation
The NISKA work cooperative (website in French only), which had been entrusted with the mandate for the first impact study, also took charge of this second phase, promoting a participatory approach to the analysis of the data collected. Their approach is based on the methodological principles of collective impact, continuous cooperation and systemic change.
“Beyond traditional evaluation, measuring social impact helps to encourage continuous improvement, mobilize stakeholders, reinforce the impact of our actions and communicate their added value more effectively. In the case of MUSCO, this evaluation helps to give value to the progress made in terms of collaboration. The process highlights the evolution of the level of trust between individuals and the importance of the structures put in place to increase the space for cooperation”, Isabelle Mercieca, Collective Development Consultant, NISKA.
Creation of a new evaluation committee
To ensure the rigor of the evaluation, and as had been the case during the first phase, an evaluation committee was set up to accompany the process. This committee, made up of members from the four centers and families involved in MUSCO’s governance, validated the consultation objectives, co-constructed the evaluation strategy, supervised data collection, ensured the application of the methodology and participated in the co-analysis of the results. It also contributed to the drafting of the recommendations.
The committee members are :
- Aurélie Vigné, Manager, MUSCO
- Caroline Marie Fidalgo, Patient-Expert, member of MUSCO’s Advisory Pole
- Camille Brosseau, Inter-establishment Navigator, MUSCO
- Claude Nadeau, Clinical-Administrative Coordinator, Technical and Technological Aids Cluster, Marie Enfant Rehabilitation Centre
- Gwenaëlle Trottet, Clinical-Administrative Coordinator, Trauma and Burn Trajectory, CHU Sainte-Justine
- Jacques Boissonneault, Executive Director, Shriners Hospitals for Children – Canada
- Lise Gagnon, Head of Care and Services, Psychoeducation, Specialized Education, CHU Sainte-Justine
- Marie Beauséjour, Researcher, Université de Sherbrooke
- Stéphanie Renaud Beaudin, Patient-Expert, member of the MUSCO Advisory Pole
Lessons learned and results of the impact study
Data collection took place from May to July 2024 in various forms: focus groups, interviews and online forms to complete interviews. A period of co-analysis of the results took place between September and October, enabling MUSCO collaborators to share their feedback on the results of the data collection.
This collective work aimed to highlight the positive effects of MUSCO, while emphasizing the strengths but also the persistent challenges, and this, in several areas identified during the first phase:
- Patient experience of care and services;
- Collaboration between families and caregivers;
- Collaborations between caregivers and facilities;
- Collaborations between the four centers and external partners.
An initial meeting enabled the evaluation committee to analyze the data collected, using a metaphor: that of a ship that moves forward thanks to the wind in its sails, stays afloat thanks to its solid hull, but sometimes slows down because of an anchor that rubs against the bottom of the water. This image facilitated the appropriation of the results during a second co-analysis session, organized this time with all collaborators who had taken part in the focus groups and interviews. Held face-to-face on October 28, this second analysis meeting compared the 13 issues identified in 2021 with those identified in 2024, highlighting the changes that had taken place between the two impact assessment phases.
“My participation in the impact study was a rich opportunity to gain access to a multitude of perspectives on MUSCO’s many projects,” explains Marie Beauséjour, evaluation committee member, “and to observe unexpected results on the scope of MUSCO’s actions, which the qualitative approaches enabled us to explore,” she adds.
These various exchanges enabled the NISKA cooperative to produce a final evaluation report (in French only) that includes the recommendations and reflections shared by the collaborators who contributed to both phases of the impact study. To the professionals and families who shared their time and thoughts, and who believe in the benefits of collaboration: thank you for helping to make the most of all the advances made possible by MUSCO! We invite you to read and re-read the final reports that summarize all your discussions.
In conclusion, a key point to take away from this study is that culture change in an environment as complex as the healthcare system is a long process. Collaboration and mobilization of stakeholders require sustained effort over the long term. Now it’s up to each and every one of us to do our part to ensure that the changes we make are sustainable, and that our collective efforts last.