A recent post by Kate Cronin-Furman at wronging rights explains how the numbers on rape in the Congo can mislead us. The problem she highlights is a general issue with relying on data from hospital admissions in order to measure health problems. You can legitimately claim that dataset is a representative sample of women who showed up at the hospital during your study period, but this masks the wide range of selection biases that drive who shows up. As Cronin-Furman notes, in the case of rape in the Congo this includes oversampling more-severe cases, and potentially also oversampling women who feel they have an incentive to go to the hospital.
Closer to my own research, hospital admissions data was also used historically to measure the prevalence of HIV. This was problematic because in poor countries hospital admissions are dominated by women seeking prenatal care (and sometimes prenatal care visits alone were used). Women who show up at prenatal clinics have one important characteristic in common as far as HIV is concerned: they have all had unprotected sex within the past 9 months. Again, it’s not technically wrong to describe your sample as comprising women who came to clinics for prenatal care – but it’s deeply misleading, because saying that doesn’t explain why we shouldn’t expect those women to be representative of the population as a whole. In the case of HIV prevalence this error was fairly harmful: we later had to go back and correct it, and that meant trying to explain that while the rate was lower that didn’t mean we were winning the fight against the epidemic. It also meant confronting a strain of thought that argues that people measuring HIV prevalence are misleading the public in order to overemphasize the threat posed by the virus, in order to gain more funding.
A similar sampling strategy was used in one of the only studies of the prevalence of dry sex in Malawi, which I previously wrote off without much discussion. The sampling bias is a bit harder to pin down in this case – I think studying sexually active individuals is totally reasonable here, for instance – but some points still stick out. The first is that this is a self-described study of Malawian women, and given the prevalence of transactional sex and concurrent partnerships in the country, it’s possible that men would have report dry sex at very different rates than women. The second is that these are women who expect to give birth soon, and so sex workers (who more commonly use birth control) may be underrepresented. Third, this study is in fact of Malawian women who went to Queen’s Hospital in Blantyre, which is to say that it is almost exclusively of urban women. In my own research I found that there was vast heterogeneity in sex practices across geography and ethnic group, with people from cities claiming they hadn’t even heard of behaviors that are common in the rural areas that are home to 87% of the country’s population. [These issues with sampling are wholly separate from my main issue with the Dallabetta et al. paper, which is that it focuses on the use of drying agents such as herbs or powders. My own experience is that very few Malawians use such agents, or have even heard of them. What is far more common is methods that involve using e.g. a towel to dry off the genitals during sex.]
The temptation to rely on data from hospital admissions is particularly high when doing research on healthcare poor countries, since it may be the only data source available. But it is in exactly these same circumstances that we should be particularly wary of such data, since the biases that it incorporates are exacerbated by low resources and limited access to formal health services.