As members of the American Evaluation Association (AEA), our core values reflect those of the Association. One of AEA’s Guiding Principles for evaluation focuses on common good and equity: Evaluators strive to contribute to the common good and advancement of an equitable and just society. Our recent client, the Network of the National Library of Medicine (NNLM) at the University of Washington, contracted our team with this goal in mind. In partnership with their team of experts, Informed designed a suite of resources for conducting evaluations with specific populations – K-12 Education, LGBTQIA+, Race & Ethnicity, and Rural Health – that encourages sustainability in M&E processes and procedures. These resources were compiled and shared with NNLM members via virtual training courses and the NNLM website.
Though many of our contracts are in international contexts, we were excited to partner with this domestic organization because of their strong stance in promoting social justice and equity through evaluation. As evaluators, we always seek to amplify the voices of those often overlooked, those with a different story to tell than those in the majority. By centering these voices, we acknowledge and amplify groups who have been historically marginalized by society. For more information on how to conduct evaluations through this lens, the Center for Culturally Responsive Evaluation and Assessment (CREA) has numerous resources from various subject-matter experts.
With any evaluation, we at Informed are grateful for the opportunity to learn alongside our client. The focus on social justice and equity for marginalized communities meant we needed to learn how best to center these voices and support those who engage in programming with these groups. Through this process, we learned three key lessons:
Be adaptive
Consult the experts
Design for accessibility
1) Be adaptive
At the beginning of the consultancy, we knew we would be developing content for use in a web format. As the project advanced, it was clear that the best way for us to successfully complete the project was to also take on the role of adding the content to the website. While this is not our team’s area of expertise, our team’s flexibility was key to producing an output you can access here.
Initially, our hope was that the material could be built into a highly interaction user interface. Working in pre-determined style sheets, however, meant that our desired interactivity was somewhat limited. Through consultation with the NNLM web and evaluation teams, we settled on an approach that could accommodate the tabbed layout of the broader site while providing the user with numerous internal and external links that assisted in facilitating engagement with numerous materials. This allows users to navigate through an entire pathway or to jump between pathways at different steps in the process.
2) Consult the experts
This project required not only expertise in evaluation but also subject-matter expertise on several populations. Our team did extensive research on evaluation approaches that centered these groups, however, we knew that we could only really create resources of value by consulting members of these groups. NNLM assisted us by locating these experts on staff and within their membership, helping us not only improve the quality of the output but also set the stage for its uptake by other NNLM members. Expert consultation should always be a key step in the completion of any evaluation activity.
3) Design for accessibility
While inclusive design is important for us as evaluators to consider when writing reports or designing visualizations, we learned through this project just how critical that need is when creating our final product for use on a website. This process included a lesson in screen readers, learning that we needed to consider all of the following and more:
Consistent title/header levels in our code
Colors that are compliant with the Americans with Disabilities Act of 1990 and consistent throughout the site
Consistent layout on each website page that presents an optimal user experience
Spare and intentional use of images
Since we worked with an internal web team at NNLM, many accessibility controls were already set for us, but if you are working on your own web-based solution for evaluation, I recommend this article on designing for accessibility by Pablo Stanley. Pablo covers all of the essential considerations for accessibility and provides resources to help you ensure your site is accessible for all users.