training effectiveness review of literature

joann starks: good afternoon, everyone. iam joann starks of sedl in austin, texas and i will be moderating today’s webinar entitledintroduction to reviewing and synthesizing qualitative evidence. it is the first in aseries of four webinars that make up an online workshop on qualitative research synthesis.i also want to thank my colleague, ann williams, for her logistical and technical support fortoday’s session. the webinar is offered through the center on knowledge translationfor disability and rehabilitation research, ktdrr, which is funded by the national instituteon disability and rehabilitation research. the ktdrr is sponsoring a community of practiceon evidence for disability and rehabilitation or d&r research. evidence in the field ofdisability and rehabilitation often includes

studies that follow a variety of qualitativeresearch paradigms. such evidence is difficult to summarize using traditional systematicresearch review procedures. the goal of this series of web-based workshops is to introduced&r researchers to the methodology of qualitative evidence reviews. participants will be provideda state-of-the-art overview on current approaches and will learn to apply those to the literaturebase. ongoing innovative initiatives at review-producing institutions will be highlighted. today, ourspeaker is karin hannes, assistant professor at the methodology of educational sciencesresearch group at the catholic university or ku leuven in belgium. she has a backgroundin adult education as well as medical and social sciences. karin currently teaches qualitativeresearch methodology to undergraduates and

masters students. she has been teaching evidence-basedpractice and systematic review courses for over a decade, both in public health and educationalsciences. karin is the founder of the belgian campbell group, co-convenor of the cochranequalitative research group, and co-author of the cochrane handbook for systematic reviewsof effectiveness. she has published several books and articles on qualitative evidencesynthesis, particularly on the critical appraisal of qualitative research. she also specializesin visual research methodology. thank you, karin, for agreeing to conduct this introductorysession today on reviewing and synthesizing qualitative evidence. if you’re ready togo, please take it away. karin hannes: yes. thank you, joann, for such a nice introduction.i have indeed been asked to introduce you

to the fantastic and exciting world of systematicreviews and then, more specifically, the qualitative evidence part of it. i’ll do my very bestto give it a bit of sex appeal and so by the end of this talk, i would be hoping that youwould all be motivated to start your own review projects. i just want to outline what i’mgoing to talk within this particular presentation. what i want to speak to you about is how iactually got triggered by qualitative evidence synthesis, hoping that this would also leadyou into seeing that searching for evidence, looking at evidence, actually, has nothingto do with research or science so much, but is or should be some sort of a common attitudethat people should adopt. i also want to clarify what qualitative research is and in my opinionwhat sort of evidence it may generate. on

top of that, i’d like to show how it’sgoing to contribute to effectiveness reviews, so how it differs from them, and i’ll givea quick but very brief insight in potential approaches that can be used when you considerqualitative evidence synthesis, how you can build your own review protocol, and i willillustrate these things with some work examples. how did i get triggered by qualitative evidencesynthesis? let’s start with the very beginning. it’s always a good point to start. meetemma, and emma is the youngest in our family and i’m going to use her as a case to explainwhy effectiveness reviews have failed to me and more specifically what has been my worstevidence-based case scenario so far. emma is actually born on the sixth of october,2010 and is a little sister of door and polle,

you might see on this slide. apart from alot of joy, she also brought a lot frustration in me, and i don’t know how many mothersi have in the room but believe me after having been pregnant for the third time, it becomesreally, really hard to control your body especially your weight, and many moms will be able toconfirm that. after my third pregnancy, i not only gained 6 pounds that did not automaticallydisappear again, but i further gained weight to the extent that i did not fit half of mycloset anymore. i was interested in actually knowing what can i do to actually controlthe weight gain and actually get rid of the extra pounds? if you don’t know the answerto your question, think a minute about where you would go look for it. so i did that andi went looking in the cochrane and campbell

library to see whether i could find reviewsthat could provide me with an answer to that query. i found this cochrane review on “dietor exercise, or both, for weight reduction in women after childbirth.” the answer tomy question from that review was that women who exercise did not lose significantly moreweight than women who were in the usual care group. that sort of comforted me so it meantthat i didn’t have to go out running or cycling for the upcoming five months. i alsolearned that women who took part in a diet or diet plus exercise lost more weight thanwomen in the usual care. there was no difference in the magnitude of weight loss between dietand diet plus exercise groups and the intervention seemed not to affect breastfeeding performanceat first, and then i thought that was a very

important trigger for me. i found this studyin the journal of the american college of nutrition stating that those who ate cerealswere lowering rates compared to those who ate meat and eggs, bread, or even skippedbreakfast. so my simple logical reasoning actually was that if a diet helps to loseweight after pregnancy and if cereals are proven to work well as a diet, then actuallythe consumption of those cereals should lead to weight loss after my pregnancy. right?wrong, because it didn’t. after having consumed bowls of cereals in several mornings for severalmonths, i didn’t achieve any effect and yes, that’s the moment where you actuallystart panicking and thinking about, “gosh, i’m not normal. i’m not like this averageperson where it works. what is happening to

me and what am i doing wrong? am i not followingthe protocol? did i maybe buy the wrong type of cereals?” i was thinking and thinking,and then realized that there must have been something that i had overlooked. maybe therewas some sort of alternative explanation for not achieving the effectiveness the reviewactually had promised to me. instead of mourning about my weight, i started to go and dig alittle bit deeper into a different sort of literature and i came across a few qualitativearticles discussing, for example, the role of social support in weight loss, diet issues,and so on; and also some of the barriers that have been perceived by mothers who engagedin weight loss programs i learned from these studies, i learned a lot of things about whyi had such a hard time. i learned from the

first study that female relatives, husbands,and the right sort of people around you are the primary source of emotional, instrumental,and informational support. having just moved from australia to belgium at that point intime, my social network, for example, was really thin. my family was living far fromme and while i tried to reserve some time to exercise, there are lots but actually verylimited. only i didn’t see that at that point in time. so i learned a lot about facilitatingfactors for engaging in programs with weight loss. from the second study, it highlighteda lot of barriers and facilitators that women had experienced. the two of them may be applyingto me. the first thing was the unhealthy eating habits and because of that statement in thestudy, i started logging what i actually ate

during daytime. while it wasn’t a lot, ithink the things i did eat contained a lot of fat, not in the least the cheese crackersi was consuming on a daily basis. secondly, i also suffered from some sort of light depression.i wasn’t feeling good about myself. i was no longer able to hold my breath long enoughfor diving. i couldn’t get my foot off the ground in dance class, and a lot of thesethings actually came together in that situation. so i looked carefully at the conclusions ofthe studies and they pleaded actually for community-based family-oriented programs toincrease the chance of successful weight reduction, which was not something that i had found inthe previous effectiveness review. the conclusion of study 2 - weight loss intervention shouldaddress the psychological effects of childbearing,

affordability, and perceptions of body image- was not something that was particularly taken into account in the programs describedin that particular review. so it reminds me a bit of this advertisement that displaysa bald middle-aged man in his early 50s with a message this and then they show him somesort of liquid. it’s the only approved in clinical tests to grow hair. then if you turnto the next image, then you see the same bald middle-aged man with hair growing all overhis body, his nose, his ears, his hands, except on his head with the message that individualresults may actually vary. after seeing that, i thought this is a perfect example of a wrongeffect but i now no longer panic because i’ve learned i may not be that average person andthere’s nothing abnormal about that. it

happens to a lot of other people as well.what i learned was that there are different sources of evidence that may need to be consideredand that qualitative evidence had been proven to be very valuable to me to explain a certainsituation. this is one of the most famous quotes in the history of systematic reviews.it’s archie cochrane, and the cochrane collaboration disseminated systematic reviews in healthcare,actually named the organization after this person. he stated that, “it’s surely agreat criticism of our profession,” meaning the health profession, “that we have notorganized a critical summary adapted periodically of all relevant randomized controlled trials.”while i’m thinking that’s very true, i think it’s also a great criticism of ourprofession that we have been foolish enough

to think that critical summaries of relevantrandomized controlled trials would provide us with the right answer for each type ofquery because we already learned that individual results may vary and that rcts can’t explainevery sort of outcome. what we now are about to learn is that rcts are further very limitedin the amount of questions they are able to answer as well. we use to see evidence interms of effectiveness research. it’s often mentioned in the context of trying to establishsome kind of causal relationship. i don’t know whether anyone of you ever looks at thetelevision series sherlock holmes, but sherlock always goes like, “watson, i know what causedthat.” then watson who’s down to earth says then, “but we have only administereda few interviews and gone on to sight visits.

should you not collect evidence that is morerobust?” indeed, if you talk in terms of causal effects, qualitative techniques maybe the worst choice to make but if you talk about evidence in a different fashion andconsider, for example, evidence of feasibility, the extent to which an intervention is practical,cultural, financially possible within a given context. then the picture actually changesalso when you want to assess the appropriateness of interventions, which is the extent to whichan intervention fits with a situation how it relates to the context in which it is given.the rcts are not able to provide you a lot of relevant information for that. the samewith evidence of meaningfulness or the extent to which an intervention is positively ornegatively experienced by your target group

or how it relates to people’s personal experience,opinions, failures, beliefs, or interpretations. so we have long neglected the whole bunchof questions because we couldn’t quite fit them into the straightjacket of an rct. apartfrom these types of evidence that firmly link into intervention research, there are otherquestions we might be asking like what’s the evidence of the cost benefit of a particularintervention? what are the lived experiences of people with a certain condition or livingin some sort of deprived situation that we do not know a lot about? what actually dopeople value or not in an intervention or maybe just in daily life? so i always wonderedwhat if archie cochrane had thought about organizing a critical summary adapted periodicallyof all relevant qualitative research studies.

now, that would’ve made the difference becausethen we might have had about 6,000 mixed-method reviews that provide us with a much more in-depthunderstanding of a condition, an intervention. not all sorts of questions require a mixed-methodapproach. for example, questions related to understanding the meaning of a particularphenomenon such as how people make sense of a particular chronic disease, or why theybehave or feel the way they do. these questions may be explored in a stand-alone qualitativeevidence synthesis. they would provide another information on their own, but that would bethe easy way out because mixing evidence is really hard. it’s methodologically challengingand we’re still working on the development of methods to actually smoothen the integrationof combining quantitative and qualitative

evidence. why is that so? some people wouldargue that it’s ridiculous to think that that is going to work, that we can just infact mix apples with oranges, but i actually support gene glass in saying that it is, ofcourse, about mixing apples and oranges, mixed-method reviews. yes, in the study of fruit, nothingelse would be sensible, so comparing apples to oranges to him would be the only endeavorworthy of true scientists, thus yes. when you look at it, comparing apples to applesis trivial, in a sense. in order to be able to mix different strands of evidence, so inorder to let them inform each other, you need to be able to understand what exactly a quantitativeand a qualitative study is and what you can do with it at the meta level. to my understanding,most of you have already had some sort of

introduction in quantitative studies. my jobis actually to reveal a bit more about qualitative basic studies and how they can be used ona meta level. first of all, i think it’s an inquiry of meaning. it addresses a differentsort of questions that go into the what of a phenomenon, the why things are what theyare, and the how people cope or deal with them. i very often tell my kids the storyof the three little pigs and in case you’ve never heard about it, it’s a story aboutmama pig who goes to the market and tells her three little piggies to find some sortof shelter because there’s a big bad wolf running around. so each of the pigs buildsa house. the first piggy builds a house of straw which in effect the least solids butthe piggy then has plenty time to play afterwards.

the second one builds a wooden house thatrequires a bit more work, but still there’s plenty of time to play. the third one buildsa brick house and is laughed at by his brothers because he’s all sweating while they areactually playing and having all the fun. then the wolf comes. he blows down the straw andthe wooden house, but he can’t blow down the brick house. that house then gave shelterto the three pigs in the end. when i tell this story, what occurs to me is that my kidsstart asking many questions about it such as, “why weren’t the first two pigs smartenough to build a brick house? were they too lazy? why did mommy not take them with herto the market?” these are all questions about the meaning of the story. they seldomask, “well, did these pigs really exist

or could they really talk with each other?”that would be easy because i could answer that with a yes or a no. the other questionsthough, they require some sort of deeper understanding. it’s what i would call a rich, deep, thickinsightful or illuminated understanding. so many of our kids, many adults actually seethe world in terms of meaning and that’s what qualitative studies try to understand.looking at evidence, we can look at both strengths and how they can inform each other. for example,i came across this effectiveness review on rehabilitation in a housing community integrationafter an acute traumatic brain injury. it’s a systematic review presented to me by chetmeyers who is one of the facilitators of the series. this effectiveness review actuallyfound that a lot of community integration

programs show positive results and shouldbe studied more rigorously. the others actually recommend that to further establish that post-acutetraumatic brain injury rehabilitation intervention, in order for them to improve community integration,they should think about intervention strategies that are based on injury severity, for example.they should take better care about their control groups. they should engage in longer-termfollow-up. if you look at it, these are all very instrumental suggestions to actuallyimprove the designs more than anything else. if you now look at qualitative evidence, lookingat the same topic of injuries and how people deal with it, then you see that these papersactually look beyond numbers. numbers are very bad in capturing experiences, nor dothey allow us to really conduct an in-depth

exploration of a phenomenon. so i came acrossthis study from gauvin-lepage that really spoke to me in the sense that it succeededin capturing the lived experience of these people and how difficult it was to actuallyreintegrate. in fact, the difficulties mentioned were not that much related to severity ofthe condition, which was suggested in the quantitative review, but rather to the responsefrom the environment. this should actually certainly be taken into account when we startpromoting interventions for community integration because it tells us that once you focus onknowing and understanding what life was before for these people that once you focus on managingthe psychological imbalance that this condition actually evokes, and on reframing the expectationsof the environment rather than focusing on

the individual with the limitation itselfbecause these are all things that would help people recover their social roles from beforethe injury. we know that famous cartoon where an interviewer goes up to a man in the streetand tells him to describe how he feels. the man then actually says, “the way i feelis actually hard to quantify.” “well,” said the interviewer, “how hard on a scalefrom 1 to 10 would that be?” so it really shows that if we’ve trained in a particulartradition of research, we may not have all sorts of interesting layers on our personalraters. another example of seeing what qualitative evidence can actually mean to enrich the insightsof quantitative studies is this cross-sectional study from carpenter that indicates that lifesatisfaction is more strongly related to community

participation than to the impairment itselfand activity limitations. so in the article, carpenter really pleads for community participation,and that’s actually a good thing but then again, i found this study from newman andi included a short video or a fragment of it that actually states a lot of barriersand facilitators of people function in their own community that should be taken care ofwhen we intend to promote community participation. i’d like to look at this video with you.okay. at the end of this video, i just want to pick up on two points. the first one isthat the video really shows that we’re actually not limited to textual accounts in qualitativeresearch that we use a lot of our senses to actually reach out to what people want toexpress. the second thing is that it also

shows that we can address a different layerof knowledge here in community participation, and it’s probably a layer of knowledge thatquantitative reviews just can’t access. the next point i want to make is that in linewith what archie cochrane actually mentioned is that it would be great if we could summarizeor synthesize all the qualitative evidence that has been generated from different contextswhile still remaining sensitive to that context and bringing that together into some sortof new theoretical, practical insight. what i wanted to present next is a short definitionof qualitative evidence synthesis that we came up with in our cochrane guidance checkedaround dealing with qualitative research. actually, it is a process of summarizing qualitativeresearch findings by comparing and analyzing

textual, visual, or other sorts of researchevidence that might even be performance or dance-related type of evidence. it can bederived from multiple accounts or just one event, one phenomenon or a situation. theimportant thing is that it has to be reported in basic qualitative research studies. whyam i saying this? the term qualitative review or qualitative evidence synthesis is actuallyvery often misused. quantitative research is used actually for reviews in which statisticaltooling is just not an option to them, and they then summarize the information from thestudies in some sort of a narrative. this is actually not a qualitative synthesis becausethe basic ingredient of that synthesis is still quantitative. it remains quantitativeinformation. these narratives are then actually

very descriptive in nature. they don’t gobeyond the descriptives into a more profound level of interpretive understanding. you reallyneed to be able to distinguish between these two. now, what this qualitative evidence synthesiscan do is explore questions such as how do people experience a condition or a situation?i think i’ve just shown that through the examples we’ve discussed. why does an interventionwork or may not work, for whom? in what particular circumstances does it work or does not work?what are potential barriers and facilitators related to a program you're trying to implement,and what impact do specific barriers and facilitators have not only on the program but also on thepeople, their experiences, their behavior? the definition of qualitative evidence synthesisis actually very close to the definition of

synthesis displayed in the oxford englishdictionary. it states that synthesis is actually a process or a result of building up separateelements, especially ideas, into a connected whole, especially a theory or a system. soi think that there are three components in my definition of qualitative evidence synthesis.i always call it a systematic empirical inquiry into meaning. it is systematic in the sensethat qualitative evidence synthesis also has some sort of protocol or a starting bump.they’re planned. they’re ordered and structured. the process of conducting a qualitative evidencesynthesis may not be as linear as a quantitative review but what we do as an author is actuallyreconstruct that logic of science into some sort of linear report of what we’ve done.it’s empirical in the sense that it comes

very close the original intention of the word“empirical” which is that it depends upon the world of our experience. it builds onwhat we can capture with our senses, with all of our senses. it is by far an inquiryinto meaning because we try to develop a more complex picture of a phenomenon, and thatis what i call rich, deep, think, textured, insightful as well when i was talking aboutthe three little pigs and the questions that generated in my children. now that we knowwhat qualitative evidence synthesis is, it’s maybe interesting to look into how it canspecifically contribute to treatment effectiveness reviews. it contributes in many ways. we alreadylearned from the pregnancy and the hair growth example that it contributes to the understandingof heterogeneity between effects and between

individuals, but it can do more than that.it provides also a research-based context for interpreting and explaining of trial results.if we take up a set of different questions such as how to achieve change more effectively,how to improve our interventions, how to fit the subjective needs of our target group,or even the ones who are in charge of delivering the program. what other type of interventionsmight be needed to make it more successful? i think that overall would increase a lotof the quality of the interventions we are engaged in. it provides evidence on subjectiveexperiences of those involved in developing, delivering, receiving interventions, or evenliving with a particular condition or in a specific vulnerable environment, for example.it can also reveal the extent to which effective

interventions are actually adopted in policiesand practice. it engages a lot with questions related to implementation. this is actuallya short excerpt from our chapter 20 in the cochrane handbook on the role of qualitativeevidence into cochrane effectiveness reviews. as you can see, we identified four differentroles of qualitative evidence synthesis. it can inform a review by using evidence fromqualitative research to help define and refine questions. we are very used to develop ourown research questions and in the past before i engaged with qualitative evidence synthesis,i wasn’t at all too sensitive to the fact of whether or not my question was actuallyrelevant to those out in the field and those having to implement what we academics wereactually suggesting. by qualitative research,

you can actually probe people to refine yourquestion, make it a better match for them. we can also enhance reviews by synthesizingevidence from qualitative research that is identified while you’re actually lookingat evidence of effectiveness. a lot of the rcts we include in quantitative reviews, theyactually contain some implementation of process-related qualitative information that we could summarizebut that we tend to neglect too much. a lot of these studies may also have sibling studiesfloating around on the web that are close connected to the effectiveness question butnot actually incorporated in the same study. so we cannot enhance these reviews by actuallylooking at that evidence. we can extend them by searching specifically for evidence fromqualitative studies to address questions that

relate to the effectiveness reviews but arenot focused on effectiveness itself. for example, that we achieved the intended outcome, whynot? why didn’t we achieve it? one other influence is do we see that we’re not protocolingat files and how can we deal with them? how can we change the program so that it wouldbe sensitive to concerns of those reviews? or we can also supplement effectiveness reviewsby just synthesizing qualitative evidence within a stand-alone, complementary reviewand in these types of reviews, we addressed questions on aspects other than effectiveness.for example, how do people experience living with a particular condition. there has beena lot of drivers that are needed for mixing both strands of evidence. one of them is agreater recognition of the value of qualitative

research in the evidence policy movement.we also are faced in the past with a lot of empty reviews, stating like, “oh, we didn’tfind any rcts,” and we all know how hard it is to conduct an rct, for example, in aschool setting. you can’t just randomize pupils across interventions. you have to workwith groups. so there’s a lot of practical limitations that lead to these empty reviewswhere there are lots of evidence that could be taken into account. there’s also an increasingdemand for an incorporation of public perspectives and experiences in those reviews from fundersthat has sparked the mixed methods debate. the most simple interventions within cochraneat least, they have long been researched and we’re now facing more complex questions,interdisciplinary fields that can’t be accessed

through rcts alone. there’s a lot of interestgoing to be in issues of process and implementations to actually optimize programs. a lot of traditionsthat are growing in the primary research area, there’s a lot of mixed-method primary researchstudies coming up, and these are also used to motivate reviews to take mixed-method approach.there’s a lot more funding for these types of reviews as well, and the most dedicatedmethods groups that have long catered for qualitative evidence are actually now movinginto a more mixed-method type of thinking. i haven’t spoken about how different qualitativeevidence synthesis are from reviews of effectiveness. we know the questions they might ask, butthere are certain differences that are more on a conceptual level which i’d like tospeak to you about. i’ve stolen this metaphor

from david gough and james thomas. james thomasmight be one of the speakers coming up in a later phase of this series, but they positionedquantitative and qualitative reviews as a spectrum between aggregated versus configurativetypes of reviews. let’s start with the easy one that everyone is familiar with, whichis the aggregated type of metaphor. metaphors are really interesting because they make visualwhat we actually lack in capacity to do. so this is actually a metaphor, the pile of stonesfor meta-analysis for quantitative types of reviews, because what they actually do isidentifying individual studies and then pooling the results of each of these studies, buteach of these studies in the meta-analysis remains visible. they don’t tear them apart.they just create one overall measure of effect

across these studies. what they do is theyincrease the power of the measure there. i’m not sure whether you’ve been hiking in wales,for example, in the uk but there’s a lot of mist there. what they do for hikers isthey pile these stones up so that it makes it easy for people to actually find theirway. this is some sort of metaphor to say like you increase the power compared to justone stone lying there flat with an arrow on it. the next slide is actually an examplefrom cochrane of a quantitative bit from a mixed-method review, and this review actuallyevaluated the impact of lay health workers in primary and community health care for maternaland child health and the management of infectious diseases. what this review concluded as oneof the main conclusions is that lay health

workers can increase immunization uptake inchildren below the age of five years old. you can see the parallel with the pile ofstones here. you see all the individual studies on the left of your screen – the barnesuntil the rodewald study. then you see on the right-hand side, you see the effect measuredfor each of these individual studies. at the end of the plot, you can see this diamondthat actually shows that lay health workers have a positive impact on the maternal andchild health. if you look at the individual studies, the picture is much more confusing.not all of them are on the same side. not all of them have confidence integrals thatfall within the positive area. it’s only by pooling them that you get some sort ofclear picture of where the true effect may

lie. now, the second metaphor i’d like touse is one of a mosaic and this really links into the idea of configuration that is verycentral to qualitative evidence synthesis. what we do, we rearrange. we configure thefindings of primary studies in order to generate new theory or explore the salience of existingtheory in particular situations. what we actually do is piece together research knowledge fromdifferent contexts. as you can see in that picture of the mosaic, the individual studiesare no longer recognizable. we actually turn them into a new holistic understanding ofa particular phenomenon. it’s much more diffused what the actual role of one particularstudy is in these reviews. what is so important in qualitative evidence synthesis is thatyou are very sensitive to context because

what we do is we piece together research knowledgefrom different contexts and you can’t just pile that up. you need to remain sensitiveto where does this evidence come from and would it be generalizable to other countriesor not. just to give you one small anecdote on the role of context in our daily life,this is an anecdote i’ve stolen from one of my colleagues. cynthia, she’s from theu.s., and she was telling that she was phoning her husband, saying like, “honey, i’mrunning late. can you please put the chicken on the stove? i love you a lot,” and i hopehe does it. okay, she thought to herself. then she came home and what she found wasthe chicken on the stove but not particularly in the way she had meant it because she actuallymeant, “i’m late so i want you to cook

dinner for us tonight.” he had interpretedit as, “yes, i’ll just put the box on the stove and then it’s melted down themoment cynthia will arrive.” so there are lots of examples in our daily life where wecan see that context is very important. what strikes me the most is that in quantitativestudies, we fielded out context as much as possible. we create experimental studies insome sort of virtual laboratory environment, and so it’s remarkable that these streamsnot have been brought together much, much earlier in history because we know that wecan’t do the one without the other to create some sort of ease in understanding of phenomenon.so what you see here is the qualitative part of that mixed-method review produced by claireglenton on barriers and facilitators to the

implementation of these lay health workerprograms to improve access to maternal and child health care. what it actually addressedinstead of effectiveness of these lay health workers was the program acceptability, feasibility,and appropriateness. it looked into the lay health worker relationship with other healthprofessionals, and with their clients the patient flow process, service integration,socio-cultural conditions, and so on. the nice thing about it is that it gives a clearoverview of what people liked from the point of view from the lay health workers, fromthe point of view of the clients, and from the point of view of the professional workerswho needed to work with these lay health workers on a daily basis. what this information cancontribute to the whole is that it allows

you – and i’m fully aware that this isvery small for you to see but on the right-hand side, you see the overall outcome of the review,improved health outcomes among mothers and children and some of the more secondary outcomeslike better quality service including appropriateness of consultation services. this is actuallyevaluated through the quantitative one but the nice thing is that the qualitative synthesisidentified a lot of negative and positive moderators in that relation between lay healthworkers and clients or other health professionals. for example, one of the negative moderatorswas that health professionals were really concerned about their own loss of authorityabout the knowledge of these people and whether it would be enough. while on the positivemoderator side, people explained that they

really liked the lay health workers becauseof the fact that they had more time. they were really supportive, but they did feela lack of knowledge as well, so they actually found each other in that particular group.what you can do is you can bring in all that qualitative evidence in your flow chart ofyour review and it provides you with a much clearer understanding of what is actuallygoing on. i’m going to sum this up with some of the differences between that sortof meta-analysis and meta-synthesis stream of evidence. meta-analysis is actually tryingto accumulate knowledge. the studies that you include in such a review have to be strictlycomparable to be able to pool them statistically, and it aims at creating more power. thinkabout the pile of stones that i’ve shown

you earlier today. they do that through numericaldata. while the meta-synthesis bits is actually trying to make sense of data. so the studiesthat you include, they don’t have to address the complete similar account. it can be pickedfrom different context, even different target groups. they need to have some sort of basiccomparability on the level of the phenomenon or the topic of interest of your research,and they add value in content through interpretation. we can move on to some of the general approachesthat can be used in qualitative evidence synthesis. to me, that’s a very difficult thing todo. thus, it used to be simple in the quans bit where you have to choose between a randomor a fixed sort of model. to me, the approaches to qualitative evidence remind me of a bighuge circus tent that hosts all these different

approaches, and they’re very different interms of methodologies, perspectives, strategies they use. it is really a little bit more complexthan choosing between fixed or a random model in meta-analysis. qualitative evidence synthesisis actually just an umbrella term which encompasses many different approaches. in other words,there’s room for many different types of views about qualitative evidence synthesisthat can comfortably be fit underneath this big tent. one thing that helps you to choosebetween all these methods is think about what is the purpose of my qualitative evidencereview, and some of the purposes can be linked to certain types of qualitative evidence synthesis.if you’re interested in bringing together separate findings into some sort of interpretiveexplanation, you want to generate theory.

you want to bring some newness, some holisticpoint of view. you can’t choose, for example, meta-ethnography, and i’m very sure thatthis type of approach will return in some of the other lectures. it’s one of the mostcommonly used approaches. on the other hand, if you critically approach the literaturein terms of deconstructing research traditions or theoretical assumptions, if you want tocritique the work of others, then critical interpretive synthesis would be somethingyou should consider. other reviewers like to produce theories or models that are basedon their topic of interest and may involve, for example, interactive processes, contextualizedunderstanding and action. they would rather move into grounded theory on a meta-level.some data you encounter are more or less descriptive.

they don’t take too much for a lot of interpretation,and in that case the meta-analysis would be a potential option. i personally work a lotwith meta-aggregation. it’s an approach that mirrors the linear approach of quantitativereviews and summarizes evidence in order to develop lines of actions for practice andpolicy. there’s a lot of more complex review types that really try to unpick relationshipsbetween persons and environments that try to formulate patterns. for example, with suchintervention, what outcomes will i expect and how do think that links into differentpopulations, for example, on the level of gender or different settings – schools,hospitals, and so on? there are also approaches that bring together research of differentdesigns and paradigms. meta-narrative is an

approach that caters for quantitative as wellas qualitative basic research, for example. so that’s one area that would help you tochoose. another one is that you need to take some sort of stance epistemologically if youwant to choose a particular approach. as you can see, all of these approaches are somewheresituated between a realist perspective and an idealist perspective. the idealist perspectiveis much closer to interpretive science. the realist perspective is a bit closer to whatpositivists would believe in, in the sense that they believe that there is some sortof external truth out there that we can search for using qualitative evidence. while an idealisttype of research is really claiming that there is actually no shared reality that is independentof human construction. so that means that

every synthesis, every meta-level type ofproduct is actually colored through the lens of the review author. so depending on whereyou are, some of these approaches would link better to your personal position than others.that’s a different axe for you to choose, so there’s the purpose and the aim of yourreview. there’s the position that you have about what is knowledge and how can you generatethat? it leads a bit into difference between people who are rather on the inquiry sideof qualitative evidence or on the more empirical scientific side of generated qualitative evidence.so it links into that debate. then a few other axes you need to consider if you want to choosebetween approaches. that is what experience do i have in my team and what sort of resourcesdo i have? that’s very important to consider.

some of these approaches are very complexto execute and some require a lot more time than others. you can actually try to jot downall of the information on the nature of the research including the aims on the resourcerequirements. is it something that needs a lot of work? is it something that works comprehensiveor more purposefully? you can jot down elements on the nature of your research theme. whatexperience do i have? do i have quantitative researchers? and do i want to do qualitativesynthesis or do i have relapse? what that could move us along that interpretive sideof meta-analysis? there’s the nature of research of how much structure do i prefer?do i prefer a linear approach? am i comfortable with interrelated non-linear approaches throughevidence synthesis and so on? based on the

arguments between each of these categories,you can then choose the approaches that my colleagues are going to outline in the nextseries of lectures. this whole choosing process is actually outlined in one of the books weproduced with some of the colleagues of mine in the methods group. it’s available andwe also have a maillist from the cochrane qualitative implementation methods group wherepeople can actually post questions for discussion. if you’re stuck in choosing things, if you’restuck in definitions you don’t know how to approach things, sometimes people postquestions to us and we do our best to try and answer those as well. joann starks: well,thank you very much, karin. that was a wonderful presentation. i also want to thank everyonefor participating today. we hope you found

the session to be informative and that youwill join us for the next three webinars. as you see on the final slide, we have a linkto a brief evaluation form - and let me move to that slide. as you can see on the finalslide, we have a link to a brief evaluation form and we’d really appreciate your input.we will also be sending an email with the link for the evaluation. on this final note,i would like to conclude today’s webinar with a big “thank you” to our speaker,karin hannes, from myself, ann williams, and all of the staff at the ktdrr. we appreciatethe support from nidrr to carry out the webinars and other activities. we look forward to yourparticipation in the following sessions. thank you.