Selection Bias - DIME Wiki

Selection bias is when participants in a program (treatment group) are systematically different from non-participants (control group). Selection bias ... Navigation MainPageRecentChangesRandomPage Tools WhatlinkshereRelatedchangesSpecialpagesPrintableversionPermanentlinkPageinformation SelectionBias Jumpto: navigation, search Selectionbiasiswhenparticipantsinaprogram(treatmentgroup)aresystematicallydifferentfromnon-participants(controlgroup).Selectionbiasaffectsthevalidityofprogramevaluationswheneverselectionoftreatmentandcontrolgroupsisdonenon-randomly.Theonlyfoolproofwaytoavoidselectionbiasistodoarandomizedcontroltrial. Contents 1ReadFirst 2Guidelines 2.1Howtoavoidselectionbias? 2.2SelectionbiasinSampling? 3BacktoParent 4AdditionalResources ReadFirst Selectionbiasmeansthattreatmentandcontrolgroupsarenotcomparable,andthereforetheimpactevaluationisnotinternallyvalid. Guidelines Selectionbiascanbeeitherpositiveornegative.Forexample,anevaluationofanafter-schoolprogramforat-riskyouthcomparesthosewhovolunteeredfortheprogramtothosewhodidnot.Itislikelythatthevolunteersaremoremotivatedandeagerthanthosewhodidnot,whichmaymaketheprogramappearmoreeffectivethanitis.Ontheotherhand,comparingevaluatingtheprogrambycomparingoutcomesfortheparticipantsintheafter-schoolprogramto'average'studentsmayunderstatetheeffectoftheprogram,astheat-riskyouthlikelyperformworseonaverage. Thereisnoreliablewaytoestimatethesizeofselectionbias. Howtoavoidselectionbias? Thebestwaytoavoidselectionbiasistouserandomization.Randomizingselectionofbeneficiariesintotreatmentandcontrolgroups,forexample,ensuresthatthetwogroupsarecomparableintermsofobservableandunobservablecharacteristics. Itisimportanttorandomizebothattheleveloftreatmentandtohavearandomsampleofsurveyrespondents.Forexample,foranevaluationoftheimpactoftabletsintheclassroom,treatmentisrandomlyassignedbyclassroomandforthesurvey,arandomsampleofstudentsisdrawnperclassroom. Non-randomizedevaluationsattempttoavoidselectionbiasbymakingthecontrolgroupascomparableaspossible,typicallybymatchingonobservables.Themoredatathatisavailableformatching,themoreconvincingthisis. SelectionbiasinSampling? Selectionbiascanbeaproblemeveninrandomizedcontroltrials.Forexample: Highlevelsofattritionbetweensurveyrounds:therespondentsforthefollow-upsurveymaybesystematicallydifferent.Forexample,thiswouldbethecaseifwealthierhouseholdsaremorelikelytomigrateandthereforethesampleatfollow-upwouldbesystematicallypoorer. Highitemnon-response:missingdatacancreateworriesofselectionbiaswithinaparticularquestion.Forexample,ifhalfthesampleanswers'doesn'tknow'toaquestiononincome,thoserespondentswillbeexcludedfromtheanalysis.However,ifpeoplewhohavelowerlevelsofnumeracyorlesssystematicincomearelesslikelytoknow,thiscancreatebias. Surveymode:forexample,phonesurveyslimitthesetofrespondentstothosewhohaveaccesstoamobilephone.Ifthefullpopulationofinterestdoesnot,thiscouldbiasresponses(respondentsofhighersocioeconomicstatusaremorelikelytohavephones,butimpactsmaydifferbysocioeconomicstatus) BacktoParent ThisarticleispartofthetopicQuasi-ExperimentalMethods AdditionalResources listhereotherarticlesrelatedtothistopic,withabriefdescriptionandlink Retrievedfrom"https://dimewiki.worldbank.org/index.php?title=Selection_Bias&oldid=4531" Category:Quasi-ExperimentalMethods