Evaluation

Evaluation of Family Programs

Evaluation of programs is central to Overcoming Barriers’ mission.  Starting with the first pilot Family Camp program in 2008 we collected data through pre and post surveys of families and in 2013 we began a collaboration with Dr. Michael Saini to evaluate programs more thoroughly moving forward. (Survey)

Collaboration with Dr. Saini, University of Toronto

Dr. Michael Saini, Associate Professor of the Factor-Inwentash Faculty of Social Work at the University of Toronto (visit website) is collaborating with OCB to develop, implement and analyze the evaluation of Overcoming Barriers’ programs.

Presentation at AFCC National Conference

2015: Dr. Saini, Sevil Deljavan, Ph.D. (student), and Dr. Deutsch, presented “An Exploration of Children’s Experiences in the Overcoming Barriers Family Camp” at the 2015  AFCC National Conference in New Orleans.  View or download a PDF of the 2015 OBFC Poster here.

2014: Dr. Saini and Shely Polak Ph.D. (candidate) and Dr. Deutsch presented “An Evaluation Study of the Overcoming Barriers Family Camp at the 2014 AFCC National Conference in Toronto.  View or download a PDF of the 2014 OBFC Poster here.

Overview of Evaluation

The development and implementation of the evaluation meets the urgent need to ensure that services offered to families involved in family courts are based on evidence-informed principles and guided by quality assurance for providing both efficient and effective services to help families resolve their disputes.

Reasons why this evaluation project is especially valuable in the field:

  1. The project is managed by an experienced and highly respected third party (Dr. Saini at U. of Toronto)
  2. The evaluation looks at participants pre- and post-intervention attitudes and beliefs
  3. The project includes the voice of the child

In the first pilot of the evaluation protocol, parents attending the Overcoming Barriers Family Camp in Vermont (July 2013) were asked to complete pre and post online surveys based on their experiences of attending the camp. In addition, the evaluation team completed individual interviews with both parents and children. The pilot evaluation also includes a nine-month follow-up with each parent. It is anticipated that the pilot evaluation project will produce process and outcome data to inform interested parties about the efficiency and effectiveness of the programs in changing knowledge, attitudes and behaviors of participants.

The pilot evaluation protocol was designed to be applicable to a range of intensive programs who’s components are consistent with the Overcoming Barriers Approach including: the Overcoming Barriers Family Camp, Family Intensive, and other similar programs that will be operated by outside groups in the future. The results of the pilot evaluation will be published in 2015.

The evaluation strategy has been developed based on the core mission and values of Overcoming Barriers in assisting families where a child is resisting contact with one parent.  Key evaluation questions have been developed to assist in monitoring and evaluating the impact of the program and expansion of services, including, but not limited to:

  • Relevance: Are Overcoming Barriers programs relevant to the stakeholders involved in family disputes (judges, lawyers, mental health professionals, children, parents, etc.)?
  • Program Implementation: How have these programs been implemented with reference to the original objectives?
  • Program Administration and Operation: Are the services offered administered and operated satisfactorily from the viewpoint of clients and stakeholders?
  • Impact: What impact have the programs had on the families and stakeholders engaged in family court matters?
  • Sustainability and Expansion: Are Overcoming Barriers programs adequately designed to support the development of new and expanding services as required?
  • Costs and Productivity: What are the costs of delivering the services and is there any evidence that the services are cost effective for families involved in family courts.

Visit Dr. Saini’s homepage at the University of Toronto (click here).

Family Camp Pilot Program 2008-2010

Overcoming Barriers conducted exit interviews with participants and collected evaluations at the 2008-2010 Family Camp programs.  A detailed description of the components of Family Camp and the results of the first three years of surveys were written up by Drs. Deutsch, Sullivan and Ward and published in the Family Court Review in 2010 (click here to view an excerpt of this article).

In 2012 as Overcoming Barriers launched as an independent 501(c)(3) the organization committed to further developing evaluation methods and better understanding outcomes for families.