Topic 9: Quality assurance, credibility and usability of CAPs

Submitted by Robin Hocquet | published 30th Aug 2021 | last updated 13th Sep 2021

Background: The KE4CAP Synthesis Report

This topic page forms a chapter of the KE4CAP Synthesis Report, which brings together learning from across KE4CAP's activities, including five virtual knowledge exchange events, three bilateral knowledge exchange event series, and the KE4CAP Survey (see Further Resources for links to relevant pages). The KE4CAP project is providing a global forum for developers and operators of climate adaptation knowledge platforms (CAPs) to come together to share knowledge and best practices, and to work together to address common and emerging challenges.

This early summary (September 2021) aims to provide food for thought for the KE4CAP Synthesis Workshop and will be followed by a more detailed and further illustrated synthesis report (and further developed topic pages) in December 2021.

Introduction: Quality assurance, credibility and usability

As the need for robust climate action increases, interest by CAP users, providers, and funders in quality assurance processes and demonstrating such is increasing. This interest includes developing standardised procedures for quality assurance that ensure and demonstrate the relevance, usability, legitimacy and credibility of the content and user friendliness in accessing that content. Procedures of particular interest are those that allow for evaluating the quality of source data and knowledge (including standards and stewardship), those that ensure credibility, transparency and pedigree, and those that are based on an understanding of user needs and capacities.

Lessons Learnt

Adopted practices  

  • Boards, advisory panels and user groups: These bodies are seen as legitimate components of the platform governance structure, dedicated to quality assurance and helping to ensure the credibility and usability of the data and products hosted on the platform.
  • Internal routines and expertise: For decisions on a daily basis some providers rely on internally established routines, either formally agreed or organically evolved, and the expertise of their team for quality assurance.
  • Dedicated quality assurance processes: A few platforms have put in place a dedicated quality assurance process with associated criteria and indicators.

Selected innovations 

CAPA (Alps). A standard operational procedure has been established for reviewing and editing new content proposed for publication on the platform. This includes direct communication with the external authors/editors, comprehensive and clear criteria for the selection of relevant resources, and the provision of online, step-by-step guidance for external editors.

CCCS (Canada). A process based on requiring users to register with their email address when downloading data provides an opportunity to contact them if future changes or updates are required.   

Shared challenges 

  • Building capacity and resources to establish and sustain quality assurance processes and deliver ongoing improvements based on the results of those processes. A phased approach and integrating such within the governance structure has been suggested but these can be challenging in terms of scope and timing. There are also difficulties associated with agreed benchmarks and understanding of appropriate metrics beyond just web statistics.
  • Establishing and maintaining an effective level of user engagement, critical to defining and developing robust standards and QA/QC approaches. Included in this is the need to review the scope of engagement to include appropriate representation of both existing and new users as the platform evolves.
  • A further challenge relates to third party data and information on the platform. For such information there are often limits to the application of QA/AC procedures and the ability to update or address errors, with reliance on the good will and capabilities of third parties. This can be particularly challenging if the information was created by a project that is no longer active.
  • Identifying the need to act when digital tags within content are no longer valid, have been changed or a new area of interest has been added. Dealing with these changes requires continuous monitoring to retain the confidence of the intended users.