Volume 12, Issue 6 (Nov & Dec 2022)                   J Research Health 2022, 12(6): 363-376 | Back to browse issues page

XML Print

Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:

Mutatina B, Kanyarutokye Basaza R, Kawulukusi Sewankambo N, Lavis J N. Identifying Key Steps in Developing a One-stop Shop for Health Policy and System Information in a Limited-resource Setting: A Case Study. J Research Health 2022; 12 (6) :363-376
URL: http://jrh.gmu.ac.ir/article-1-2095-en.html
1- Clinical Epidemiology Unit, School of Medicine, College of Health Sciences, Makerere University, Kampala, Uganda. , bonifacemutatina@yahoo.com
2- Uganda Christian University, Mukono, Uganda.
3- School of Medicine, College of Health Sciences, Makerere University, Kampala, Uganda.
4- Department of Health Research Methods, Evidence and Impact, McMaster University, Ontario, Canada.
Full-Text [PDF 1259 kb]   (141 Downloads)     |   Abstract (HTML)  (448 Views)
Full-Text:   (29 Views)
1. Introduction
Evidence-based health decision-making has increasingly attracted global attention in the recent past [123]. Several initiatives are underway in the form of global and regional networks as well as national efforts to promote evidence-based decision-making. The different initiatives have led to tremendous growth in the production of knowledge translation (KT) products, such as rapid response summaries, evidence briefs for policy, dialogue reports, systematic reviews, and impact evaluation reports [4]. To disseminate these KT products, some initiatives, especially those based in high-income countries, have implemented online one-stop shops for both global research evidence and local policy-relevant documents to provide answers about health interventions and health systems. Examples of one-stop shops for global evidence include the health systems evidence (HSE), a repository of reviews and economic evaluations on how to strengthen health systems and get the right programs, services, and products to those who need them [5]. Others are the Cochrane and Campbell libraries which are collections of high-quality, independent evidence for informing policy and practice [6, 7]. On the other hand, one-stop shops for local policy-relevant documents include the health technology assessment (HTA) database search interface, which is specific for the dissemination of Canadian health technology assessment (HTA) reports [8].
In low and middle income countries (LMICs), such online KT resources, especially those that focus on local policy-relevant documents for addressing questions about health policy and systems are lacking. Limited understanding exists on the development process of such resources in LMICs, with less developed infrastructure (equipment, electricity supply, internet access, and connectivity) and inadequate human resource capacity. Given the limitations in LMICs and complexities involved in creating the virtual KT resources, such undertaking requires strong theoretical research underpinnings which are lacking. The use of the available frameworks, such as the knowledge-to-action (KTA) framework [9] and the ADDIE (analysis, design, development, implementation, evaluation) model [10] have been limited to the development of summaries of evidence, continuing health care education modules and decision aids [11]. Where studies have considered online KT resources, the focus has been on translating evidence into clinical practice in the developed world and not on local policy documents in LMICs [12]. 
This study aimed to critically analyse and understand the development process of a virtual KT resource for local policy and systems-relevant information in LMCs. This paper provides a step-by-step inventory for setting up a virtual KT resource in limited-resource settings.
2. Methods 
We conducted a case study of the development process of the Uganda clearinghouse for health policy and system (UCHPS) developed by supporting the use of research evidence (SURE) in the African health systems project in 2011 [13]. The SURE project was a mechanism to strengthen evidence-based policy-making in Africa. The UCHPS is an online repository for Uganda-specific health policy and systems-relevant information, including health policies, strategies, plans, rapid response summaries, evidence briefs for policy, and policy dialogue reports. The resource seeks to increase access to decision-relevant information for decision-making about the health system and interventions in Uganda. The case study design was chosen because it was optimal to answer our objective which sought to answer “how” the platform was developed within a limited resource setting context. Further, we did not have the option to manipulate the behaviour of those involved or variables that are likely to be influential [141516].
Defining the case
The case (i.e., unit of analysis) in this study was defined as “the development process of the Uganda clearinghouse for health policy and system”. We wanted to fully understand how the knowledge translation platform was developed right from the inception stage in a resource-limited setting. We used the term “development process” to mean any or all stages, from inception to platform revision and update [16]. Each stage was studied holistically. We gained insights from the Levac et al model to define the unit of analysis. This model builds on the KTA framework and the ADDIE model [9, 10]. It recommends four steps for developing online KT resources designed to translate evidence into practice. The steps include developing evidence-based, user-centered content, tailoring content to an online format (including user-testing), evaluating impact, sharing results, and disseminating knowledge [12]. The model was particularly selected because it integrates principles of instructional design to inform the development of KT resources. Instructional design principles have been shown to provide clear guidance for developing replicable and high-quality instructions [10]. 
Data sources, sampling, and recruitment
We collected data from key informant interviews, participant observations, and archival records to develop a comprehensive account of the case under investigation. Key informants were purposively sampled from a wide range of stakeholders, including members of the regional East Africa community health policy initiative (REACH-PI), SURE staff, and members of the advisory group for setting up the clearinghouse. Purposive sampling, which aimed to achieve maximum variation, was utilized to ensure that a range of key informants was sampled based on their role in the development process. We used both emailed invitation letters and follow-up phone calls to recruit key informants. We conducted telephone and face-to-face interviews using a specially developed guide (Appendix 1).

Appendix 1: Key Informant Interview Guide

Position ---------------------------------------------
Members of the REACH-PI regional and international sounding boards/SURE staff/members of the advisory group (specify) --------------------------------------------
Could you briefly tell me what you know about the Uganda clearinghouse for health policy and systems Research? (Check more using the following bullet questions)
What is it for?
How did it arise? 
At what stage did you get involved in the development process of the Uganda clearinghouse for health policy and systems research?
Did you involve in the inception stage (i.e. when the idea of creating the clearinghouse started), interface design stage, procurement of the IT firm (including drafting terms of reference [TOR]), or part of the advisory panel?
Briefly describe for me exactly what this stage involves 
How did you contribute to the development process of Uganda clearinghouse for health policy and systems research?
Briefly tell me about at least 2 things you have contributed to the development of the clearinghouse
Were these contributions done in a group or alone?
Do you think your contribution was necessary?
For how long have you been a part or in contact with the Uganda Clearinghouse for health policy and systems research?
If you participated at the inception stage, how did the idea of starting the clearinghouse come about?
Who initiated the idea of starting up the Uganda clearinghouse for health policy and systems research?
When and where did this idea start?
What exactly inspired the idea? What factors /conditions led to this idea?
How has this idea evolved? Has it changed from the original idea or has remained the same?
As an individual, what motivated your participation in the development process of the clearinghouse? 
What impact did you want to have from the development of the Clearinghouse? 
By participating in development process of Uganda clearinghouse, would you kindly tell me about your experience from when you 1st got involved to date?
What lessons have you learnt from the process that you think will help other people planning to develop a clearinghouse?
What challenges have you encountered during your participation and how have you addressed them?
Does the development process of Uganda clearinghouse for health policy and systems research meet your expectations? 
What do you think would have been done differently during this process?
By contributin to the development process of Uganda Clearinghouse for health policy and systems research, how do you see its future?
Do you have any other information that you consider useful to this study?
Thank you for your time. Please feel free to contact us

The telephone interviews were used to collect data from respondents who were not readily available for face-to-face interviews, thus helping to reduce the non-response level. We also used participant observation to complement the key informant interviews and to increase the rigor of the study. The principal investigator (PI) kept notes on descriptions and depictions of events, participants, and activities during the development of the clearinghouse in a field journal. 
The participant’s observation findings provided a better understanding of the context and phenomenon [17]. This method sought to provide an opportunity for the PI to check for non-verbal expression of feelings, to observe events that informants were unable or unwilling to share, and to observe situations that informants described in interviews. It also sought to make the PI aware of distortions or inaccuracies in the description provided by the informants [18]. Lastly, we reviewed the documents. We looked at documents that contained information about the Uganda clearinghouse for health policy and system. This involved reviewing different documents that guided the setting up of the clearinghouse, including the REACH-PI prospectus, SURE clearinghouse proposal and minutes of advisory group meetings, usability evaluation, and scoping review study reports [192021].
Data analysis
Data collected in the form of interview transcripts, observation notes, and synthesis of the documents was entered into ATLAS.ti for analysis. The analysis involving thematic coding was done step by step, and started with reading and re-reading the data to get familiar with what the data entailed to form comprehensive codes. Data were collapsed into codes for more efficient analysis. To enhance the accuracy of the analysis, the PI (MB) regularly consulted with another investigator (RB) to generate consensus on the codes. Codes were then combined into over-arching themes that accurately depict the data to list candidate themes for further analysis. We looked at how the themes supported the data and the over-arching perspectives to have coherent recognition of how the themes are patterned to tell an accurate story about the data. The insights from the analysis of the interviews, participant observation notes and archival records enabled us to develop comprehensive accounts that highlighted how the clearinghouse developed.
3. Results
Generating the idea to create the clearinghouse

The idea to set up the UCHPS was suggested in the 1st meeting to prepare a proposal to European :union: for SURE in the African health systems project in 2006. The idea was majorly influenced by the increased production of KT products at that time. This was in addition to the evidence from the literature reviewed during the meeting, which pointed to the difficulty of finding local health policy information packages in one place as a barrier to the use of research evidence. Thus, the team came up with the idea of developing one-stop shops in African countries that would provide handy information to decision-makers when faced with urgent questions about health policy and systems. A member of the technical working group that developed the SURE proposal recounts:
“The 1st time we started speaking about the clearinghouse was when we were planning; we were sitting in Oslo to write a proposal to submit to EU.…………… We came up with the idea of developing one-stop shops that would provide timely information because there had been some studies pointing at the scarcity or the challenges in finding documents, policy documents, strategic plans or demographic data, or every kind of relevant policy document across Africa.” 
Determining the focus of the uganda clearinghouse
The focus of the clearinghouse was influenced by the anecdotal data and literature review during the initial stages, suggesting that health policymakers in limited-resource settings faced difficulties in accessing health policy information for decision-making; they were often unable to rapidly identify well-indexed decision-making information when pressing issues emerged. In particular, Uganda lacked a one-stop shop where such information was easily accessible. Thus, the resource was specifically designed as a repository of Uganda-specific health policy and system documents, including policies, strategies, plans, guidelines, rapid response summaries, evidence briefs for policy, and dialogue reports. A REACH-PI member explained: 
“So policymakers always struggle to find information quickly when they need it; it was the concern when developing the idea. Considering that so-called international platforms that people can use, or access, our policymakers in Uganda will find it difficult to access information on specific Uganda health policies from such platforms. Secondly, these platforms may not contain information on document policy that is highly relevant to Uganda. As such, we found it necessary to think about creating a platform that has information specifically about Uganda.”
Mobilizing resources to setup the clearinghouse
Although the idea to setup the clearinghouse started when writing the proposal to the European :union: for the SURE project, this particular grant did not support its development. A more focused grant was solicited from World Health Organization in June 2011 by the Uganda SURE team to support the development of the clearinghouse [20]. In the proposal, the team sought to set up a clearinghouse to provide convenient, quick, and easy access to a wide range of reliable and contextual research evidence for health policy and systems that are needed for informed policy decisions. The specific objectives of the proposal included developing a structure of key components of the clearinghouse, actively engaging policymakers, researchers, and other stakeholders in the development of a user-friendly design and format for the clearinghouse, and carrying out stakeholder user testing and a pilot survey of the clearinghouse. A member of the SURE team narrates:
“…We were the 1st people to write a short proposal to WHO, we received US$40,000 to help set up the structure.”
Developing a structure of key components for a clearinghouse
The SURE project team, consisting of the project coordinator, research scientists, rapid response officer, and information technology (IT) officer, developed the initial design of the clearinghouse architectural as a procedure document to guide the structural development of the key components for the clearinghouse (Figure 1).

The PI was part of the SURE project team as a research scientist. He participated in developing the initial and final architectural design of the clearinghouse (Figures 1 and 2).

This was an iterative process, which involved the review of websites targeting health policymakers and health managers as well information systems for other groups such as clinicians. The process aimed at a structure that is easily navigable and searchable. A respondent who was part of the SURE Project team reported: 
“I was part of the whole process from the beginning. I participated in developing the proposal and then we implemented it. We worked closely with the IT personnel, back and forth, to develop the architectural structure and prototypes……... We aimed at a website that would feature policy briefs, policy dialogue reports, report summaries, the SURE tool, and also other evidenced policy networks. As such, we reviewed other similar libraries and websites”. 
According to the terms of reference document, the scope of work for the IT team includes designing an interface for a web-based resource, reviewing websites targeting health policymakers and health managers to inform the design, and uploading health research information relevant to the Ugandan health system. Other tasks included providing links to other relevant websites, databases, repositories, hosting, maintaining, and updating the web-based resource, and building the capacity of SURE staff to upload the documents [20]. The expected key deliverable included a structure for a web-based clearinghouse that is easily navigable and searchable according to types of policy documents, a database of contextualized research products, links to other websites and databases unlimited space and search engine optimization with advanced search functionality [20]. 
To accomplish the assignment, the IT team regularly held consultative meetings with the advisory group and SURE team to present their progress on the assignment. In the 1st meeting, the IT firm presented its understanding of the scope of work and the SURE team provided further clarification. In the subsequent meetings, the team presented multiple prototypes and revisions made based on the feedback from the previous meetings. The discussions are guided by the architectural design as a key procedure document (Figure 1) and terms of reference.
Engagement with policy makers, researchers, and other stakeholders 
The advisory group played a key role in the development process of the clearinghouse. This was a gender-balanced group of five resource persons comprising two health policymakers, a health manager, a researcher, and a representative from civil society. This group worked with the SURE, and IT teams to review the development of the clearinghouse structure as well as identify information needs for the database. Three consultative meetings were held with the advisory group. The advisory group made several recommendations, including clarifying the title of the one-stop center. Most members felt that the word ‘clearinghouse’ in the title was somewhat vague and confusing to the target users. Further elaboration was suggested in only one sentence below the main title to explain the function of the resource center. Thus, the title reads as:
‘The Uganda clearinghouse for health policy and systems’ 
‘A public resource for health policy and systems evidence’
The advisory group also suggested a vision statement to further qualify the title as follows:
‘One-stop center of excellence for providing health policy and systems evidence for decision-making in Uganda’
The advisory group recommended that the clearinghouse should be designed to target technical staff supporting policymakers’ needs for research information, policymakers, health managers, donors, and international organizations. Other potential users include researchers and civil society. Regarding the interface design, the group advised creating a node specifically for knowledge translation products of the SURE project, such as the evidence briefs for policy, rapid response summaries, and other resources. Members also suggested a tab on the SURE team and other participants as well as the clearinghouse setup process. Lastly, the research team worked closely with the advisory group to develop a list of potential document sources to guide the searching process for the clearinghouse content.
Identifying, mobilizing, and indexing the content of the clearinghouse
In the proposal for developing the UCHPS, the SURE team highlighted the procedure of identifying and mobilizing policy-relevant documents [19]. During the document identification, a team of researchers at the SURE project periodically visited websites and offices of relevant institutions (both governmental and non-governmental) to identify documents related to the new national policy to be included in the clearinghouse. The team physically located documents that were not yet published on the website but were available either as hard copies or soft copies on desktops at these institutions. The hard copies are scanned and uploaded to the website with the permission of the authors. A previously generated list of potential document sources guided this process. The search engine Google was used to locate such websites, which were then navigated by the tabs and menus available on the homepage (such as policy documents and guidelines, e-library, resources, publications, and legislation). Furthermore, at the time of developing the clearinghouse, there was a lack of documented evidence on the available health policy and systems-relevant documents in Uganda that could inform whether the most crucial documents were included. Subsequently, a scoping review of health policy and systems documents was conducted to demonstrate the availability of Uganda-specific policy documents [21]. This scoping review helped describe the nature (i.e. type, coverage of national priority areas, frequency of health-system topics) and volume of the documents and trends over time. Importantly, it provided a step-by-step inventory for identifying and characterizing such documents, which was vital for building the clearinghouse content.
Two strategies, which included email newsletters, and presentations at meetings and workshops, were majorly used to disseminate the platform to potential users. A total of 1000 potential users including policymakers, health policy advisors, health managers, and researchers were sent an e-mail introducing the Uganda clearinghouse and providing a link to the site. Feedback was provided by some recipients who indicated that the resource would be appreciated and helpful. The research team followed up with potential users closely and recorded no non-responders. A policymaker of the ministry of health replied in response to the e-newsletter:
“Thank you very much. This is very useful. I appreciate. It will go a long way in facilitating our work and contributing to timely evidence-based decisions. I assume the website is updated from time to time.”
Monitoring and evaluation
User experiences with the Uganda clearinghouse for health policy and systems were evaluated between January and May 2016. [22]. The evaluation aimed to explore the user perceptions and experiences of resources in terms of effectiveness, efficiency, user satisfaction, and learnability. In-depth interviews were conducted with potential users with knowledge of health policy, who use the internet and have never used the clearinghouse before. These included policymakers at the national level, health policy advisors, health managers, and researchers. The results of this evaluation showed that the resource was effective in using for successful information seeking and was perceived as efficient in the sense that participants felt that it was worth investing their physical and mental effort to use this site because they can find information in one place. In terms of user satisfaction, the results showed respondents were impressed with the clearinghouse of its findability on the web, visual appearance, and content. The users also paid high attention to the credibility of the clearinghouse because it was hosted by a reputable research and academic institution. However, inadequate background information about the site and a lack of current information is reported. Findings from user testing informed the revision and updating of the content of the clearinghouse in the subsequent step. 
Revision and updating the content of the clearinghouse
Following the scoping review study that identified and provided a refined categorization of the policy-relevant documents necessary to build and index the content of the clearinghouse [21], the original architectural design was modified (Figure 2). The design was further refined with findings of user testing [22]. In the new architectural design, policy-relevant documents were indexed as policies, strategies, plans, guidelines, rapid response summaries, evidence briefs for policy, dialogue reports, and “other reports” for the major categories. The documents were further sub-categorized according to the national health priority issues identified in the scoping review (Figure 2) [21]. From time to time, new documents are identified and uploaded into the clearinghouse.
Our findings show that the development process of the UCHPS follows seven steps as summarized in Figure 3.

However, while the process seemed to be stepwise, it was an iterative rather than linear process. Notably, the iteration occurred within and between the process steps. Particularly, the “within step iteration” occurred in the development of the structure of key components of the clearinghouse (step 2), where the IT team made back-and-forth consultations with the research team and key stakeholders on multiple prototypes. The examples of the “between steps iterations” included modifying the initial structure of the clearinghouse and indexing the content following the scoping review of policy-relevant documents in Uganda and user-testing in the subsequent steps (Figure 3).
4. Discussion
Principal findings 

We found out that the development of the Uganda clearinghouse for health policy and the system was a seven-step process, characterized by iterations within and between the steps. The essential components of the process included developing the concept, prototyping of the key structure, engaging with policymakers, researchers, and other stakeholders, identifying and indexing the content, including conducting a scoping review of policy-relevant documents, dissemination of the resource, user-testing, and updating of the resource. 
Findings concerning other studies
Different research efforts have previously recommended guiding the development of knowledge translation (KT) resources. The KTA framework is one such effort that guides the process of developing KT initiatives [9]. More efforts include Levac et al’s framework from which we gained insights to define the unit of analysis in this study [12]. Levac et al’s recommendations were developed by integrating the KTA framework and the ADDIE model along with practical experiences in creating and evaluating online KT resources for physical therapists [12]. Lavis et al also focused on developing and refining ‘one-stop shop’ methods for research evidence about health systems [5]. Our study is consistent with the previous research work that the development of KT resources follows a step-wise process (iterative), although we identified a different number of key steps compared to some studies [5, 9, 10, 12, 2324252627]. For instance, Levac et al’s model propose four broader recommendations with several specific steps while our study identified a seven-step process. However, it should be noted that our study focused on a KT resource to increase access to local policy and system-relevant information while most of the studies reviewed focused on resources for clinical practice, development of summaries of evidence, and decision aids. Also, given the challenges in a limited resource setting, we were interested in the whole development process, including the dynamics at the inception stage. Such processes may be missed in many frameworks starting from the content development stage. Previous work highlights the significance of an iterative development process in supporting a participatory approach and encouraging user feedback to improve the final products [2829]. 
Among the essential components of the process, we identified “concept development” involving need realization. The knowledge-to-action gap necessitates the development of evidence-based tools to provide knowledge in clear and user-friendly formats with concise recommendations [9]. Realization of such a gap leads to idea generation [30, 31, 32]. 
Our study found out that the idea to create the clearinghouse resulted from the increase in the local production of KT (such as rapid response summaries, brief evidence for policy, dialogue reports, systematic reviews, and impact evaluation reports) to promote evidence-based decision-making in Uganda along with the need to mobilize them in one place that is easily accessible. The idea was also influenced by personal experiences and research evidence. Different studies have emphasized the importance of using evidence to identify a problem as critical to exploring how the issue is addressed and how it can be framed [9, 30, 33]. In addition, some studies have also identified other factors such as debates and struggles that can also lead to a need realization process [33, 34, 35]. 
We also identified iterative prototyping and engaging with key stakeholders as critical steps to refine the structure and resource focus. Evidence-based decision-making is facilitated by easy access to packaged and reliable research results [6]. As such, iterative prototyping and stakeholder engagement are key components to produce objectively stronger final products with good designs that are user-friendly [33, 34, 3536373839]. Stakeholder engagement, including needs assessment and continuous involvement of end users, has been widely highlighted as key in formulating and designing KT resources [9, 10, 12, 34]. The ADDIE model, an instructional design framework previously used in the development of KT resources highlights the design phase as one of its 5 key elements [10].
Another critical step in the development process of the online KT resources is the identification and indexing of the content. This is underscored by Mutatina et al in a scoping review study that suggested a step-by-step process for identifying and indexing policy-relevant documents [21]. Several other studies have emphasized the need for a scoping review of a body of literature when the topic has not yet been extensively reviewed, mapped, and categorized [40, 4142, 4344]. It has been noted that proper indexing of the online KT resource content according to what is considered most relevant may facilitate its usage [5, 21]. The study identified the last three steps of the process, dissemination, which involves creating public awareness about the resource through newsletters, e-mails, and presentations among others, user testing of the resource as well as revising and updating the resource based on the feedback from stakeholder engagement and user experiences. 
Strength and limitation
To our knowledge, this is the only study that has provided a comprehensive account of the entire development process of a one-stop shop for local evidence-based health policy and system information, especially in a limited-resource setting. Further, the study utilized a case-study design involving multiple methods of data collection, which sought to increase the quality of the findings and provide a better understanding of the phenomenon. However, in trying to keep to the interpretive approach in qualitative research, we recognize that our subjective feelings as researchers could have influenced the findings. In addition, due to limited resources, we were unable to go further and identify underlying factors that could have influenced the clearinghouse development process. 
5. Conclusion
Implications for research

Our study identifies a seven-step process inventory for developing a one-stop shop for evidence-based health policy and system information in a limited-resource setting. Researchers, especially those in similar settings, can apply this inventory to develop related systems. Our work complements previous efforts that have focused on one-stop shops for global research evidence and local policy documents in developed countries. In particular, our study focuses on the previously neglected limited resource setting context and attempts to go a step further to study the entire development process of a one-stop shop. The available evidence indicates that previous studies have focused on some parts as opposed to the whole development process [5, 6, 8, 21, 454647]. Further, future research that goes beyond usability evaluation and includes the evaluation of the clearinghouse’s impact can also contribute to future inventory refinement. 
Implications for policy and practice
We hope that when researchers follow this inventory to develop one-stop shops for local evidence, policymakers and stakeholders can quickly identify decision-relevant information in one place when faced with questions about health systems. This can contribute to increase the prospects of using evidence in policy-making, especially in limited-resource settings [48].

Ethical Considerations
Compliance with ethical guidelines

This study was approved by the Makerere University College of Health Sciences, (Code: 2012-182). A major ethical concern was the PI’s significant contribution to developing the initial and final architectural design of the Uganda clearinghouse for health policy and system research, which is the focus of this study. However, this also provided an opportunity to utilize participant observation as one of the qualitative methods for data collection, which contributed to provide a better understanding of the context and phenomenon. By directly participating in the process, the PI was able to examine the non-verbal expressions of feelings, observe events that informants were unable or unwilling to share, and observe situations that informants described in interviews.

The International Development Research Centre (IDRC) funded this study through the International Research Chair initiatives (IRCI) in evidence-informed health policies and systems. IDRC/IRCI was a collaboration between Makerere University and McMaster University with project No.: 104519-008.

Authors' contributions
Conceptualization and design, data collection and data analysis, interpretation of findings, writing, and editing: Boniface Mutatina, Nelson Kawulukusi, Sewankambo, and John Norman Lavis; Guiding the conceptualization and contributing to the writing and editing of the manuscript: Robert Kanyarutokye Basaza.

Conflict of interest
All the authors declare no conflict of interest.

We thank the research assistants, the data entry clerks, and all those who contributed to conduct this study. We also acknowledge the support from SURE project staff, the evidence-informed policy network (EVIPNet) Africa REACH-PI, and the office of the principal, Makerere University College of Health Sciences.

  1. World Health Organization (WHO). World report on knowledge for better health: Strengthening health systems. Geneva: World Health Organization; 2004. [Link]
  2. World Health Organization (WHO). Ministerial summit on health research. Geneva: World Health Organization; 2005. [Link]
  3. The Lancet. The bamako call to action: Research for health. Lancet. 2008; 372(9653):1855-926. [DOI:10.1016/S0140-6736(08)61789-4] [PMID]
  4. Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D: Evidence summaries: The evolution of a rapid review approach. Syst Rev. 2012; 1:10 [DOI:10.1186/2046-4053-1-10] [PMID] [PMCID]
  5. Lavis JN, Wilson MG, Moat KA, Hammill AC, Boyko JA, Grimshaw JM, et al. Developing and refining the methods for a ‘one-stop shop’ for research evidence about health systems. Health Res Policy Syst. 2015; 13:10. [DOI:10.1186/1478-4505-13-10] [PMID] [PMCID]
  6. Rosenbaum SE, Glenton C, Cracknell J. User experiences of evidence-based online resources for health professionals: User testing of the cochrane library. BMC Med Inform Decis Mak. 2008; 8:34. [DOI:10.1186/1472-6947-8-34] [PMID] [PMCID]
  7. Davies P, Boruch R. The campbell collaboration. Does for public policy what cochrane does for health. BMJ. 2001; 323(7308):294-5. [DOI:10.1136/bmj.323.7308.294] [PMID] [PMCID]
  8. Faith A, Bekhuis T. HTA database Canadian repository. J Med Libr Assoc. 2015; 103(4):239-40. [DOI:10.3163/1536-5050.103.4.021] [PMCID]
  9. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: Time for a map? J Contin Educ Health Prof. 2006; 26(1):13-24. [DOI:10.1002/chp.47] [PMID] [PMCID]
  10. Peterson C. Bringing ADDIE to life: Instructional design at its best. J Educ Multimed Hypermedia. 2003; 12(3):227-41. [Link]
  11. Field B, Booth A, Ilott I, Gerrish K. Using the knowledge to action framework in practice: A citation analysis and systematic review. Implement Sci. 2014; 9:172. [DOI:10.1186/s13012-014-0172-2] [PMID] [PMCID]
  12. Levac D, Glegg SM, Camden C, Rivard LM, Missiuna C. Best practice recommendations for the development, implementation, and evaluation of online knowledge translation resources in rehabilitation. Phys Ther. 2015; 95(4):648-62. [DOI:10.2522/ptj.20130500] [PMID]
  13. World Health Organization (WHO). Evidence-informed policy-making: Supporting the use of research evidence (SURE) for policy in african health systems. Geneva: World Health Organization; 2021. [Link]
  14. Yin RK. Case study research: Design and methods. London: Sage; 2009. [Link]
  15. Baxter P, Jack S. Qualitative case study methodology: Study design and implementation for novice researchers. Qual Report. 2008; 13(4):544-59. [Link]
  16. Creswell J. Qualitative inquiry & research design: Choosing among five approaches. London: Sage; 2007. [Link]
  17. Dewalt, KM, Dewalt BR. Participant observation: A guide for fieldworkers. Walnut Creek, Lanham: AltaMira Press; 2002. [Link]
  18. Marshall C, Rossman, GB. Designing qualitative research. Newbury Park, London: Sage; 1995. [Link]
  19. Nabudere H, Sewankambo NK. Grant Proposal for setting up the Uganda Clearinghouse for Health Policy and systems Research (unpublished); Makerere University College of Health Sciences, Kampala, Uganda; 2011.
  20. Nabudere H, Mutatina B. Terms of reference for developing the IT core structure of the Uganda Clearinghouse for Health Policy and Systems Research (un published).  Makerere University College of Health Sciences, Kampala, Uganda; 2011.
  21. Mutatina B, Basaza R, Obuku E, Lavis JN, Sewankambo N. Identifying and characterizing health policy and system-relevant documents in uganda: A scoping review to develop a framework for the development of a one-stop shop. Health Res Policy Syst. 2017; 15(1):7. [DOI:10.1186/s12961-017-0170-3] [PMID] [PMCID]
  22. Mutatina B, Basaza R, Sewankambo NK, Lavis JN. Evaluating user experiences of a clearing house for health policy and systems. Health Info Libr J. 2019; 36(2):168-78. [DOI:10.1111/hir.12257] [PMID]
  23. Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: Recommendations resulting from an international workshop. J Med Internet Res. 2017; 19(6):e232. [DOI:10.2196/jmir.7126] [PMID] [PMCID]
  24. Lenz R, Elstner T, Siegele H, Kuhn KA. A practical approach to process support in health information systems. J Am Med Inform Assoc. 2002; 9(6):571-85. [DOI:10.1197/jamia.M1016] [PMID] [PMCID]
  25. Dow S, Heddleston K, Klemmer S. The efficacy of prototyping under time constraints. Paper presented at: Proceedings of the Seventh ACM Conference on Creativity and Cognition. 27-30 October 2009; Berkeley, California. [DOI:10.1145/1640233.1640260]
  26. Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000; 321(7262):694-6. [DOI:10.1136/bmj.321.7262.694] [PMID] [PMCID]
  27. Liddy C, Maranger J, Afkham A, Keely E. Ten steps to establishing an e-consultation service to improve access to specialist care. Telemed J E Health. 2013; 19(12):982-90. [DOI:10.1089/tmj.2013.0056] [PMID] [PMCID]
  28. Timpka T, Sjoberg C, Hallberg N, Eriksson H, Lindblom P, Hedblom P, et al. Participatory design of computer supported organizational learning in health care: Methods and experiences. Proc Annu Symp Comput Appl Med Care. 1995; 800-4. [PMID] [PMCID]
  29. Brender J. Methodology for constructive assessment of IT based systems in an organisational. Int J Med Inf. 1999; 56(1-3):67-86. [DOI:10.1016/S1386-5056(99)00042-8] [PMID]
  30. Lavis JN, Wilson MG, Oxman AD, Lewin S, Fretheim A. Support tools for evidence-informed health policymaking (STP) 4: Using research evidence to clarify a problem. Health Res Policy Syst. 2009; 7 Suppl 1(Suppl 1):S4. [DOI:10.1186/1478-4505-7-S1-S4] [PMID] [PMCID]
  31. Grewal A, Kataria H, Dhawan I. Literature search for research planning and identification of research problem. Indian J Anaesth. 2016; 60(9):635-39. [DOI:10.4103/0019-5049.190618] [PMID] [PMCID]
  32. De Jong JPJ, Den Hartog DN. Measuring innovative work behaviour. Creat Innovation Manage. 2010; 19(1):23-36. [DOI:10.1111/j.1467-8691.2010.00547.x]
  33. Lewin S, Oxman AD, Lavis JN, Fretheim A, García Martí S, Munabi-Babigumira S. Support tools for evidence-informed policymaking in health 11: Finding and using evidence about local conditions. Health Res Policy Syst. 2009; 7 Suppl 1(Suppl 1):S11. [DOI:10.1186/1478-4505-7-S1-S11] [PMID] [PMCID]
  34. Rochefort DA, Cobb RW. Problem definition, agenda access, and policy choice. Policy Stud J. 1993; 21(1):56-71. [DOI:10.1111/j.1541-0072.1993.tb01453.x]
  35. Stone D. Policy paradox: The art of political decision making. New York: WW Norton and Company; 1997. [Link]
  36. Lorenzi NM, Kouroubali A, Detmer DE, Bloomrosen M. How to successfully select and implement electronic health records (EHR) in small ambulatory practice settings. BMC Med Inform Decis Mak. 2009; 9:15. [DOI:10.1186/1472-6947-9-15] [PMID] [PMCID]
  37. Hartzler A, McCarty CA, Rasmussen LV, Williams MS, Brilliant M, Bowton EA, et al. Stakeholder engagement: A key component of integrating genomic information into electronic health records. Genet Med. 2013; 15(10):792-801. [DOI:10.1038/gim.2013.127] [PMID] [PMCID]
  38. Arevian AC, O’Hora J, Jones F, Mango J, Jones L, Williams PG, et al. Participatory technology development to enhance community resilience. Ethn Dis. 2018; 28(Suppl 2):493-502. [DOI:10.18865/ed.28.S2.493] [PMID] [PMCID]
  39. Documet PI, McDonough BL, Van Nostrand E. Engaging stakeholders at every opportunity: The experience of the emergency law inventory. Am J Public Health. 2018; 108(S5):S394-5. [DOI:10.2105/AJPH.2018.304615] [PMID] [PMCID]
  40. Berg M. Implementing information systems in health care organizations: Myths and challenges. Int J Med Inform. 2001; 64(2-3):143-56. [DOI:10.1016/S1386-5056(01)00200-3] [PMID]
  41. Mays N, Roberts E, Popay J. Synthesizing research evidence. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organisation and delivery of health services: Research methods. London: Routledge; 2001. [Link]
  42. Hanneke R, Asada Y, Lieberman L, Neubauer LC, Fagen M. The scoping review method: Mapping the literature in structural change public health interventions. London: Sage; 2017. [DOI:10.4135/9781473999008]
  43. Brien SE, Lorenzetti DL, Lewis S, Kennedy J, Ghali WA. Overview of a formal scoping review on health system report cards. Implement Sci. 2010; 5:2. [DOI:10.1186/1748-5908-5-2] [PMID] [PMCID]
  44. Weeks LC, Strudsholm T. A scoping review of research on complementary and alternative medicine (CAM) and the mass media: Looking back, moving forward. BMC Complement Altern Med. 2008; 8:43. [DOI:10.1186/1472-6882-8-43] [PMID] [PMCID]
  45. Tilahun B, Kauppinen T, Keßler C, Fritz F. Design and development of a linked open data-based health information representation and visualization system: Potentials and preliminary evaluation. JMIR Med Inform. 2014; 2(2):e31 [DOI:10.2196/medinform.3531] [PMID] [PMCID]
  46. Judy J. Usability Assessment of academic digital libraries: Effectiveness, efficiency, satisfaction, and learnability. Libri. 2005; 55(2-3):96-121 [DOI:10.1515/LIBR.2005.96]
  47. Kowalewski K, Lavis JN, Wilson M, Carter N. Supporting evidence-informed health policy making: The development and contents of an online repository of policy-relevant documents addressing healthcare renewal in Canada. Healthc Policy. 2014; 10(2):27-37. [DOI:10.12927/hcpol.2015.24034]
  48. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014; 14:2. [DOI:10.1186/1472-6963-14-2] [PMID] [PMCID]

Type of Study: Orginal Article |
Received: 2022/07/2 | Accepted: 2022/09/24 | Published: 2022/10/30

Add your comments about this article : Your username or Email:

Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

© 2023 CC BY-NC 4.0 | Journal of Research and Health

Designed & Developed by : Yektaweb