From Outputs to Outcomes – Making the Shift

For museums to have meaningful impacts on the cultural problem of climate change, they will need to shift from a focus on outputs to outcomes. (Douglas Worts commenting on Joy Davis’ recent post Agentic Professionals: How individual museum workers can champion climate justice)

So what might a shift to outcomes-based program evaluation look like?

In 2013 I* had the privilege of working with the Ontario Museum Association in association with the Ontario Ministry of Tourism, Culture and Sport to develop an Outcomes-Based Program Planning & Evaluation Webinar and Workbook. These materials are online and available free of charge. Much of the following post is drawn from these resources.

What is Outcomes-Based Planning & Evaluation (OBPE)?

Outcomes-Based Program Evaluation is a systematic way to plan user-centered programs and to measure whether they have achieved their goals. OBPE goes beyond documenting what you did and measures what difference you made in the life of your audience. How has your audience changed?

Why OBPE?

The mission of the Coalition of Museums for Climate Justice is to mobilize and support Canadian museum workers and their organizations in building public awareness, mitigation and resilience in response to climate change. Ultimately, we hope the work of the Coalition will have an impact not only on museum workers and their organizations but also on their publics, the citizens of Canada and the world.

Planning for and evaluating the outcomes of museum programs and exhibits helps us to focus on and document the changes we want to see – public awareness, mitigation and resilience in response to climate change.

Being able to express and demonstrate what difference museums’ climate justice programs and exhibits might and do make is invaluable. Why?

  1. Funds for museums are decreasing.

  2. Community needs are increasing.

Previous evaluation measures focused on money, numbers of people served and on client satisfaction . These are outputs. These measures don’t really assess the impact of programs and exhibits on museum & cultural heritage audiences. We need to know if our efforts are making a difference in the lives of our citizens – and, if they are, to be able to share those results with others in a meaningful way. We need to know the outcomes and impacts.

See National Museums Liverpool – Making a Difference

So what are the components of OBPE?

Model

Inputs – The materials and resources that the program uses in its activities. These are often easy to identify and are common to many organizations and programs. For example: equipment, staff, facilities, standards (Standards for Community Museums in Ontario), information, etc.

Activities – These are the tasks, or processes, that the program undertakes with/to the audience in order to meet the audience’s needs; for example, recruiting participants, promoting the program, networking, training. Note that when identifying the activities in a program, the focus is still pretty much on the organization or program itself, not on changes in the audience.

Outputs – These are the most immediate direct products or results of your program. Program outputs are typically measured in numbers. For example, the number of  … people who participated, classes who completed a program, materials developed, workshops given, supplies consumed, web site hits, etc. Outputs create the potential for outcomes to occur

OutcomesThese are changes that occur to people, organizations and communities as a result of your program. These changes, or outcomes, are usually expressed in terms of:

  • Knowledge and skills
  • Behaviours
  • Values, conditions and status

Impact – This describes your vision of a preferred future and underlines why the project is important. It refers to the longer-term change that you hope your project will help create or to which your project may contribute.

Photo by Caroline Grondin, Upsplash
Photo by Caroline Grondin, Upsplash

Splash & Ripple

Imagine your program is like a rock thrown into a pond. Your program makes a splash. The different outcomes measured are the ripples.

Inputs are like the person & the rock.

Activities are like dropping the rock.

Outputs are like creating the splash.

The ripples spreading from the splash are like your Outcomes,

that later become your Impacts.

The edge of the pond represents the Situation.

(Plan:Net Limited & Stratecona Research Group 2008)

In my next post, I’ll focus on measuring outcomes.

In the meantime, check out these Free Resources …

Betts, Tracy. “The Nonprofit Social Media Decision Guide.” idealware. idealware. October 2011.

Diamond, Judy, Jessica J. Luke, and David H. Uttal. Practical Evaluation Guide: Tool for Museums and Other Informal Educational Settings. Second. Lanham: AltaMira Press, 2009.

Indiana University IUPUI & Institute for Museum & Library Services. What is Outcomes Based Planning & Evaluation? IMLS. 2006-2010.

McNamara, Carter. Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources.

Museums Association of Saskatchewan. Evaluation Workbook: A Basic Understanding of the Program Outcome Evaluation Model. Regina: Museums Association of Saskatchewan, 1999.

Patton, Michael Quinn. Utilization-Focused Evaluation: The New Century Text. 3rd. Beverly Hills: Sage Publications, 1997.

Plan:Net Limited & Stratecona Research Group. “Splash & Ripple: Using Outcomes to Design & Manage Community Activities.” Plan:Net. PLAN:NET LIMITED. 2008.

Plan:Net Limited. “Splash & Ripple.” Health Canada. n.d.

Thomson, Gareth, and Jenn Hoffman. “Measuring the Success of Environmental Education Programs.Canadian Parks & Wilderness Society.


*I am M. Christine Castle, PhD, a Canadian museum educator and consultant and a member of the Coalition of Museums for Climate Justice Advisory Group. More info on me here.

One Person Can Make A Difference – Building the Coastal Connections Temporary Exhibit at the Britannia Shipyards National Historic Site [Case Story]

This week’s Guest Post is by Brooke Lees, Curator, Britannia Shipyards National Historic Site, Richmond, B.C. Brooke shares the story of how an historic site can mount a meaningful exhibit about the impact of climate change on our oceans with little money but many committed community partners.

The Britannia Shipyards National Historic Site sits on the picturesque Steveston waterfront in Richmond, BC. The site is owned and operated by the City of Richmond, and a small team of dedicated staff are working with partners to complete a multi-phased restoration of the site’s cultural landscape and 14 heritage buildings. Each year, new permanent exhibits open to the public that focus on maritime heritage and the social and living conditions of the multicultural people who worked on the waterfront. Exhibit themes focus on telling the stories of the past, interpreting them as relevant to the present, and commenting on implications for the future.

Exhibit panels - credit Joel Baziuk
Exhibit panels – credit Joel Baziuk

The new temporary exhibit, “Our Coastal Connection,” at the Britannia Shipyards National Historic Site explores our historic ties to our local rivers and west coast environments, and how this has sparked the imagination of many local groups working to celebrate and protect our waters for the future. Over 20 groups are profiled within the exhibit, all of which have dedicated remarkable efforts in areas of water conservation, education, awareness, recycling and entrepreneurship.

The exhibit started with the hope of spreading awareness about the current state of our oceans.

As we go about our daily activities, we forget that our collective actions on land are affecting ecosystems far off shore and out of our direct line of sight,” said Brooke Lees, curator of the exhibit. “A staggering amount of plastic is finding its way into our global oceans every year – 8 million metric tons, which is equivalent in weight to 20,000 fully loaded 747 airplanes. As you can imagine, this is wreaking havoc on marine life and ocean environments. This problem is of course also negatively impacting fish stocks – something vital to our fishing community and the very foundation of the historic fishing village of Steveston.”

“The exhibit outlines the issues, and showcases water stewardship and educational initiatives that began with the efforts of local individuals. Many local people are banding together to implement creative solutions to one of the greatest problems of our time – and this is truly inspiring. These people deserve to be highlighted, and I’m hopeful that their work will inspire others to remember that one person can make a difference, and that every effort at home and in the community can have far reaching impacts.”

Exhibit displays - credit Joel Baziuk
Exhibit displays – credit Joel Baziuk

Although permanent exhibits at Britannia are funded through capital projects, there is no annual budget for temporary exhibits, maintenance or updates.

“In order to continue providing visitors with the latest research and new engaging relevant topics, we have to get creative,” says Brooke. “We work closely with partner groups that provide us with in-kind donations and we do our own research, writing, design and installation.”

Our Coastal Connection was researched, designed and installed with a budget of only $2,000.

“We had the opportunity to work with many enthusiastic people who felt compelled to share their stories and photographs, donate objects for display and provide their research for use in the exhibit.” Brooke explained. “It was a wonderful community effort, and the resulting exhibit is something we are all extremely proud of.”

An open house held on Saturday, June 3 launched the exhibit, with over 250 people in attendance. Visitors explored the exhibit, and met representatives from the Emerald Sea Protection Society, Fraser Riverkeeper and the T. Buck Suzuki Environmental Foundation. These exhibit partner groups captured visitors’ unique water stories, challenged them to be water champions, showed them what’s lurking in the depths under the waves, and gave them a sneak preview of cool new technologies soon to be released.

Exhibit demonstrations - credit Joel Baziuk
Exhibit demonstrations – credit Joel Baziuk

The exhibit open house was engaging, interactive and extremely well received. Visitors were fascinated with the photographs and information demonstrating the amount of plastic affecting our oceans  – many learning about this environmental crisis for the first time. Visitors left the exhibit with a greater understanding of current marine conservation issues, and were inspired to make changes at home to incorporate ‘greener’ habits in their daily routines.

“These ‘aha’ moments are what we are considering a success in terms of exhibit impact,” says Brooke. “And many of our visitors are recording their feelings in our exhibit guest book, complete with pledges of what they promise to do to help.”

Exhibit activity - credit Joel Baziuk
Exhibit activity – credit Joel Baziuk

Our Coastal Connection will be in place for one year, and will reach an audience of over 120,000 visitors to the Britannia Shipyards National Historic Site. Volunteers and staff are excited to welcome the public to the new exhibit, and to inspire people to remember why we love our waters and what we can do to keep our rivers and oceans clean and safe for everyone to enjoy.

For more information about the exhibit, visit www.richmond.ca/culture/sites/britannia/events or contact britannia@richmond.ca or 604-238-8050. Britannia Shipyards National Historic Site is located at 5180 Westwater Drive in Richmond BC.