What questions might an evaluation of a research-practice partnership answer?
There are three kinds of questions that partnerships often seek to answer about their work: 1) how well is the partnership working, 2) to what extent is the partnership’s research impacting the district and students, and 3) to what extent is the partnership impacting the researchers and their work. There are many sub-questions under these three broad ones, and some questions may be easier to answer than others.
The first set of questions is turned on the partnership itself. Are we building trust? Are we responsive to district needs and concerns? Are we building the district’s capacity to take up research? Are we building researchers’ capacity to work collaboratively with districts?
The second set of questions cover areas of potential impact, including changes in policy, district leader practices, teacher practices, and student outcomes.
The third set of questions focus on how engaging in partnerships affect the researchers, including changes to the research questions they pursue, their understanding of core problems of practice, and their abilities to communicate with practitioners.
What kinds of data can be collected to assess RPPs and their work?
At a basic level, RPPs can assess their productivity by documenting the number of reports, policy briefs, and derivative products created. RPPs can also collect data on media coverage of their research and the number of presentations to various stakeholders.
A deeper level of assessment goes beyond an accounting of activities and products. An RPP might conduct a content review of their projects to understand the partnership’s contributions to important policy and practice issues for the district.
RPPs can also assess the quality of their partnership activities and processes. Some RPPs have collected annual data on:
- The utility of the partnership: Does the partnership contribute to policy or program improvements or influence the day-to-day work of partners?
- The quality of the research: Does the partnership produce timely research? Does the partnership produce research that allows district representatives to confidently take action based on findings?
- Communication within the partnership: Are structures in place to ensure that all partners have meaningful opportunities to inform the work, exchange information, and communicate with each other?
- Access within the partnership: Are the partners connected to key decision makers?
How have RPPs used evaluations to improve their work?
RPPs can use evaluation of their partnership work in a variety of ways. For example:
- For internal purposes to improve their work. Information about both process and impact can guide the continuous improvement of partnerships.
- To communicate their impact to external stakeholders and funders, particularly the value of partnership research to district decision making and outcomes. Stanford University/San Francisco Unified School District (SFUSD) Partnership Director Laura Wentworth explains, “We have found that while our documentation helps us internally tell the story of impact, it also prepares us for external opportunities to tell a compelling story.”
- To inform tool development. For example, REL Northwest used their evaluation findings to develop a toolkit to assess partners’ data capacity needs, track the progress of joint research projects, and better engage district partners and stakeholders. The Stanford/SFUSD Partnership is developing standards for assessing whether research projects are worthy of support from district and university leaders. Both tools should help partnerships develop research that is more relevant to district needs.
What makes for an effective research-practice partnership?
Begin the assessment or evaluation by clarifying the definition of “effective.” Effective at what? A new partnership, for example, might focus on the process by which partners work together. More mature RPPs are better poised for evaluations of impact. For example, these partnerships might be ready to assess the district’s capacity to turn research findings into improvements. Much of this capacity depends on how individual leaders are engaged in research findings and can apply them.
Researcher Cynthia Coburn suggests, “We know that good working relationships are essential. Having structures and processes to facilitate open and active communication is important, and having regular meetings and processes in place to ensure mutuality are signs that partnerships might be effective.” She adds that cooperation in understanding research findings is an additional sign of effectiveness. “A lot of reports go into piles because people don’t have time to make meaning of them.”
New work by Coburn and Joshua Glazer is assessing partnerships in terms of their “absorptive capacity,” or the capacity of an organization to incorporate knowledge from external sources. This concept is promising because it may help partnerships better support districts’ capacity to use research.