Discarding efficacy?

Andrew Grove, former CEO of Intel, writes an editorial in Science:

We might conceptualize an “e-trial” system along similar lines. Drug safety would continue to be ensured by the U.S. Food and Drug Administration. While safety-focused Phase I trials would continue under their jurisdiction, establishing efficacy would no longer be under their purview. Once safety is proven, patients could access the medicine in question through qualified physicians. Patients' responses to a drug would be stored in a database, along with their medical histories. Patient identity would be protected by biometric identifiers, and the database would be open to qualified medical researchers as a “commons.” The response of any patient or group of patients to a drug or treatment would be tracked and compared to those of others in the database who were treated in a different manner or not at all.

Alex Tabarrok of Marginal Revolution (who is a big advocate for FDA reform, running this site) really likes the idea. I hate it. While the current system has some problems, Grove's system would be much, much worse than the current system. The biggest problem is that we would have no good data about whether a drug is truly efficacious, because all of the results in the database would be confounded by selection bias. Getting a large sample size and having subgroups tells you nothing about why someone got the treatment in the first place.

Would physicians pay attention to peer-reviewed articles and reviews identifying the best treatments for specific groups? Or would they just run their own analyses? I think there would be a lot of the latter, which is scary since many clinicians can’t even define selection bias or properly interpret statistical tests. The current system has limitations, but Grove's idea would move us even further from any sort of evidence-based medicine.

Other commenters at Marginal Revolution rightly note that it's difficult to separate safety from efficacy, because recommending a drug is always based on a balance of risks and benefits. Debilitating nausea or strong likelihood of heart attack would never be OK in a drug for mild headaches, but if it cures cancer the standards are (and should be) different.

Derek Lowe, a fellow Arkansan who writes the excellent chemistry blog In The Pipeline, has more extensive (and informed) thoughts here.

Update (1/5/2012): More criticism, summarized by Derek Lowe.

What does social science know?

Marc Bellemare wrote a post "For Fellow Teachers: Revised Primers on Linear Regression and Causality." Good stuff for students too -- not just teachers. The primers are PDFs on linear regression (6 pages) and causality (3 pages), and they're either 1) a concise summary if you're studying this stuff already, or 2) something you should really read if you don't have any background in quantitative methods. I also really enjoyed an essay by Jim Manzi that Marc links to, titled "What Social Science Does -- and Doesn't -- Know." Manzi reviews the history of experimentation in natural sciences, and then in social sciences. He discusses why it's more difficult to extrapolate from randomized trials in the social sciences due to greater 'causal density,' amongst other reasons. Manzi summarized a lot of research in criminology (a field I didn't even know used many field trials) and ends with some conclusions that seem sharp (emphasis added):

...After reviewing experiments not just in criminology but also in welfare-program design, education, and other fields, I propose that three lessons emerge consistently from them.

First, few programs can be shown to work in properly randomized and replicated trials. Despite complex and impressive-sounding empirical arguments by advocates and analysts, we should be very skeptical of claims for the effectiveness of new, counterintuitive programs and policies, and we should be reluctant to trump the trial-and-error process of social evolution in matters of economics or social policy.

Second, within this universe of programs that are far more likely to fail than succeed, programs that try to change people are even more likely to fail than those that try to change incentives. A litany of program ideas designed to push welfare recipients into the workforce failed when tested in those randomized experiments of the welfare-reform era; only adding mandatory work requirements succeeded in moving people from welfare to work in a humane fashion. And mandatory work-requirement programs that emphasize just getting a job are far more effective than those that emphasize skills-building. Similarly, the list of failed attempts to change people to make them less likely to commit crimes is almost endless—prisoner counseling, transitional aid to prisoners, intensive probation, juvenile boot camps—but the only program concept that tentatively demonstrated reductions in crime rates in replicated RFTs was nuisance abatement, which changes the environment in which criminals operate....

I'd note here that many researchers and policymakers who are interested in health-related behavior change have been moving away from simply providing information or attempting to persuade people to change their behavior, and moving towards changing the unhealthy environments in which we live. NYC Health Commissioner Thomas Farley spoke explicitly about this shift in emphasis when he addressed us summer interns back in June. That approach is a direct response to frustration with the small returns from many behavioral intervention approaches, and an acknowledgment that we humans are stubborn creatures whose behavior is shaped (more than we'd like to admit) by our environments.

Manzi concludes:

And third, there is no magic. Those rare programs that do work usually lead to improvements that are quite modest, compared with the size of the problems they are meant to address or the dreams of advocates.

Right, no pie in the sky. If programs or policies had huge effects they'd be much easier to measure, for one. Read it all.

Accompaniment

Paul Farmer has a piece in Foreign Affairs titled "Partners in Help". Much of it is a re-telling of stories and ideas Farmer has used before (to great effect, of course), focusing largely on the idea of 'accompaniment.' I especially like (and wish he would expand on) this ending section:

Another way of putting this is: Beware the iron cage. About 25 years ago, when I was a graduate student at Harvard, I bought a copy of sociologist Max Weber's collected works. It hurt my back and brain to even look at this giant tome, but his topic -- how the "iron cage" of rationality comes to suppress innovation -- remains relevant to this day. It occurs through "routinization," a process by which rationalized bureaucracies gradually assume control over traditional forms of authority. This is often a good thing: Rationalized procedures can improve efficiency and equity. (Atul Gawande made this insight the core of his "checklist manifesto.") When the World Health Organization launched its directly-observed therapy protocol for tuberculosis, many countries, such as Peru, made great strides against the ancient scourge.

But exceptional events -- black swans, in popular parlance -- expose the limits of this form of efficiency. When patients began falling ill with drug-resistant strains of tuberculosis, WHO guidelines suggested they be treated with the same first-line drugs as non-resistant patients. Yet treating patients with the very drugs to which their disease had developed resistance not only failed to help them; it enabled the worse strains to spread unchecked among patients' families and co-workers. This is the double-edged sword of routinization: Rationalized treatment protocols first helped health providers increase the effectiveness and reach of treatment but later prevented them from taking necessary steps to curb the spread of drug-resistant strains. Increases in bureaucratic efficiency can come at the price of decreased human flexibility. In other words, as institutions are rationalized, and as platforms of accountability are strengthened, the potential for accompaniment can be threatened, since it is open-ended, elastic, and nimble.

When the iron cage of rationality leads to a poverty of imagination, cynicism and disengagement follow. It is easy to be dismissive of accompaniment in a world in which arcane expertise is advanced as the answer to every challenge. But expertise alone will not solve the difficult problems ahead. This was the long, hard lesson of the earthquake: We all waited to be saved by expertise, but we never were. True accompaniment does not privilege technical expertise above solidarity, compassion, and a willingness to tackle what may seem insuperable challenges. It requires cooperation, openness, and humility; this concept may, I hope, infuse new vitality into development work.

Another way to help in Somalia

One of the best ways to address the severe acute malnutrition seen during famines -- like the one in Somalia now -- is a Ready-to-Use Therapeutic Food (RUTF). They're basically nutritionally fortified peanut butter manufactured to certain quality standards, and they're incredibly effective. Which brings me to MANA, or Mother-Administered Nutritive Aid (and yes, a Biblical reference). They're on a long list of things I've been meaning to write about, but my memory was jogged by their Somalia email blast. I'm a natural skeptic about start-up nonprofits, but over time they've won me over with their idea. The model is relatively simple: for now they're manufacturing RUTFs in the US and selling them to UNICEF and large NGOs that have established distribution networks. I like that aspect -- they're not trying to be all things to all men by distributing it themselves, as they've recognized that role is better done by others.

But the US manufacturing is just a stop-gap. For one, it's helping them learn the ropes on producing high quality RUTFs  and supplying these badly needed and under-produced goods to organizations with complicated purchasing requirements. Their end goal is to establish a self-sustaining (ie, profitable) manufacturing plant in Rwanda, and they're making progress on it.  A donation now will help them make more RUTFs and help them establish the Rwanda facility until it gets to a point where it no longer requires ongoing help.

One reason I think MANA is the right sort of group to establish such a business in Rwanda is that it's co-founded by Mark Moore, and he's well situated to work on both the problems of small enterprises in east Africa and international politics and supply chains. Like me, Mark is a Harding alum. He's a smart guy who spent ten years in eastern Uganda as a missionary (and started the Kibo Group development org), but he also has a Masters in development studies from Georgetown and served as Mary Landrieu's Africa specialist in the Senate. His work was the sort of evangelical aid I thought of when I read Dave Algo's recent post on how secular aid and development workers should be less hostile to good aid work done by evangelicals. Well, this it: in my opinion it's a smart business model run (an being an aspiring development professional, I'd welcome critical feedback in the comments as well) by people who can provide some necessary help to get things set up, and then step back out of the way. Once the facility is up and running in Rwanda it will mean more of our aid money can actually go into the east African economy as NGOs buy RUTFs from MANA and pay its local workers' wages.

Those who know me well or read this blog know that I have ambivalent feelings about Harding. I went there planning on being a medical missionary, and while I lost my faith I also made many friends, and my experiences there led me to my current interests in global health. So I have good things to say and bad things to say. One of the good things -- that I don't say enough -- is that there are a lot of incredibly sincere, hard-working people who come out of the school and do work that I couldn't find fault with if I tried. This is one of them, and I'm sure they'd appreciate your support.

Avoid immunization, go to jail. Eek.

Via Foreign Policy:

In Nigeria, avoiding a shot could mean going to jail

As Bill Gates unveiled his plan this week to rid the world of polio, health officials in the northern Nigerian state of Kano announced their own assault on the disease. "The government will henceforth arrest and prosecute any parent that refuses to allow health workers to vaccinate his child against child-killer diseases, particularly polio," said a health ministry official.

This news, which was announced at the outset of the government's four-day vaccination campaign targeting six million children, marks a shift in government policy toward immunization programs in the north of the country. Nigeria's polio vaccination program stalled for more than a year after Muslim leaders raised doubts over the inoculations' safety in the summer of 2003 -- resulting in bans issued by some northern state governments....

I'm not familiar with every vaccination law in the world, but this seems like a first to me. If not a first, at least an exception to the norm. I don't like this more coercive approach. If you have enough resistance to a policy that you feel you need to threaten jail time, then actually making that threat -- and following through on it -- seems likely to breed more resistance.

I think governments can and should both incentivize vaccination and make it difficult to avoid without a really good reason. Any government policy should make it easier to get vaccinated against childhood diseases than avoid vaccination, because having a fully-vaccinated population is a classic public good. I like the fact that most states in the US have opt-out provisions for religious objections to vaccination, but I also think that states should not design a policy such that getting that exemption is simpler -- in terms of time and money -- than getting a child vaccinated, as is the case in many states.

But threatening to throw parents in jail? Way too heavy-handed to me, and too likely to backfire.

Happy Hep Day

Today is the first ever WHO-sponsored World Hepatitis Day:

These successes and challenges are amplified because viral hepatitis is not a single disease. Hepatitis is caused by at least five viruses—including two spread by water or food contaminated with feces(hepatitis A and E) and three transmitted by blood and body fluids (hepatitis B, D, and C) during childbirth (from infected mother to child); through injecting drug use, needle sticks, or transfusions; or through sexual contact. Hepatitis B and C infections can cause cirrhosis of the liver and lead to liver cancer.

Today, more than 500 million persons worldwide are living with viral hepatitis and do not have adequate access to care—increasing their risk for premature death from liver cirrhosis and liver cancer. Each year, more than 1 million people die from viral hepatitis and millions of new infections add to this global burden of disease and death.

It is not, however, the first ever World Hepatitis Day – it’s just the first one recognized by WHO. Many of these international attention-raising events grow out of smaller things which pick up steam and eventually get official recognition from international organizations. It turns out that World Hepatitis Day has been going on for several years.

On a related note, did you know that Hep B is a cause of discrimination in China, and that there is a burgeoning carriers’ rights movement? I didn’t either until I started browsing the impressively worked out Wikipedia Hepatitis B page (some epidemiologist had a field day) and found that there’s an entire page for Hep B in China. An excerpt:

Discrimination

Hepatitis B sufferers in China frequently face discrimination in all aspects of life and work. For example, many Chinese employers and universities refuse to accept anyone who tests positive. Some kindergartens refuse admission to children who are carriers of the virus. The hepatitis problem is a reflection of the vast developmental gap between China's rural and urban areas. The largest problem facing Chinese people infected with HBV is that illegal blood testing is required by most employers in China.[17] Following an incident involving a Hepatitis B carrier's killing of an employer and other calls against discriminatory employment practices, China's ministries of health and personnel announced that Hepatitis B carriers must not be discriminated against when seeking employment and education.[18] While the laws exist to protect the privacy of employees and job seekers, many believe that they are not enforced.

"In the Hepatitis B Camp"

"In the Hepatitis B Camp" is a popular website for hepatitis B carriers' human rights in China. Its online forum is the world's biggest such forum with over 300,000 members. The website was first shut down by the Chinese government in November 2007. Lu Jun, the head of the rights group, managed to reopen the website by moving it to an overseas server, but the authorities in May 2008 began blocking access to the website within China, only 10 days after government officials participated in an event for World Hepatitis Day at the Great Wall of China. An official had told the head of the rights group, Lu Jun, at the time that the closure was due to the Beijing Olympic Games.[19]

(h/t to Tom)

Football epidemiology

In an attempt to prove Cowen's First Law -- "there is literature on everything" -- I enjoy highlighting unusual epidemiological studies (see tornado epidemiology, for one.) These studies may seem a bit odd until you start thinking like an epidemiologist: measurement is the first step to control. The latest issue of Pediatrics has a new study by Thomas et al. on the "Epidemiology of Sudden Death in Young, Competitive Athletes Due to Blunt Trauma." Some of the methods seem a bit sketchy, but that's kind of the authors' point as they note,

"without a systematic and mandatory reporting system for sudden cardiac deaths in young competitive athletes, the true absolute number of these events that occur in the United States cannot be known."

While this study is mostly concerned with the sudden deaths not caused by cardiac events, the same principle holds true: if anything, the problem is under-reported.

Thomas et al. use 30 years of data from the "US National Registry of Sudden Death in Young Athletes," looking at 1980–2009. Deaths in the database came from a variety of sources including LexisNexis searches, news media accounts assembled by other commercial search services, web searches, reports from the US Consumer Product Safety Commission and the National Center for Catastrophic Sports Injury Research, and direct reports from schools and parents.

Of the total deaths included in the study, about 261 were caused by trauma, or around 9 deaths per year. 57% of the 261 deaths were in a single sport, football. Notably, there were about four times as many deaths due to cardiac causes as to trauma.

In football they find defensive positions have more deaths than offensive positions, "presumably because such players commonly initiate and deliver high-velocity blows while moving toward the point of contact." While the majority of deaths were in defensive players, the single most represented position was running backs.

Why the focus on deaths in young athletes? The authors note by comparison that lightning causes about 50 deaths per year, and motor vehicle injuries case 12,000 deaths per year. (Aside: You can tell the authors don't work in injury prevention since they say "motor vehicle accident" rather than "injury" -- injury prevention researchers prefer the latter terminology because they believe "accidental" deaths sound unavoidable.) The authors explain their own focus by noting that these sudden deaths attract "considerable media attention, with great importance to the physician and lay communities, particularly given the youthful age and apparent good health of the victims."

In related news: "The Ivy League [announced that...] in an effort to minimize head injuries among its football players, it will sharply reduce the number of allowable full-contact practices teams can hold."

The battle for hearts and minds

A major difference between the public health approach and the beliefs and strategies underlying fields such as human rights or medicine is that public health concerns the prioritization of limited resources. There is a limited pie. Even if you believe that pie can be expanded (it can, at times), it cannot be expanded infinitely, and so at some point in the policy process someone has to make a decision about how to prioritize the resources at hand. This traditional public health approach overlaps with and gets blurred into human rights and medicine and politics such that the value judgments underlying different claims aren't always apparent. We have a certain number of interventions that are known to work -- they save lives and reduce suffering -- but we don't have enough resources to do all of those things in every place that needs them. If we choose option A, some people will be saved or helped, and some will die. If we choose option B, a different number of people will be saved or helped, and some other group of people will die. The discussion of who will be saved is often explicit, while the discussion of the opportunity cost, those who will not be saved is almost always lacking. Both groups are abstract, but the opportunity cost group is usually more abstract than the people you're trying to help. These are generalities of course, and in reality there is uncertainty built into the claims about just how many lives could be saved or improved with any one approach.

The problem is this: pretty much everything we do in global health is good. Sure, we can argue specifics and there are glaring examples to the contrary, but for the most part we all want to save lives, prevent suffering, and improve health. No one is seriously against successful interventions when they stand alone: no one thinks people with HIV shouldn't get antiretrovirals, or children with diarrhea shouldn't get oral rehydration therapy. Rather, they may oppose spending money on HIV instead of on childhood diarrhea (or in reality, vice versa). Who is comfortable with making an argument against preventing childhood burns? Being against treating horrific cancers? Any takers? So we all argue for something that is good, and avoid the messy discussions of trade-offs.

Thus, much of the conflict in the global health fields is about spending money on X intervention versus intervention or approach Y. Or, better yet, traditional and known intervention A versus new and sexy and unproven-at-scale approach B. I don't think I'd want to live in a world where all health decisions are made entirely by cost-benefit analysis, nor would I want to live in a world where all decisions on care and policy are made from a rights-based approach -- both approaches result in absurdities when taken to their extremes and to the neglect of each other. My impression is that most professionals in global health draw insight from both poles, so that individuals fall somewhere on a continuum and disagree more with others who are furthest away. The tension exists not just between differing camps but within all of us who feel torn by hard choices.

So the differences between the mostly utilitarian public health old-guard and the more recent crop of rights-driven global health advocates aren't always clear-cut, and they often talk right past each other ... or they just work at different organizations, teach at different schools and attend different conferences so they won't have to talk to each other. To some extent they're fundraising from different audiences, but they also end up advocating that the same resources -- often a slice of the US global health budget -- get spent on their priorities. These tensions usually simmer under the surface or get coated in academic-speak, but sometimes they come out. Which brings me to an anecdote to leaven my generalities:

A few months ago I was having a private conversation with a professor, one who leans a bit towards the cost-benefit side of the continuum with a dose of contrarianism thrown in for good measure. Paul Farmer came up -- I don't remember how. I paraphrase:

Resource allocation is the central dilemma in public health. Period. If people don't get that, they're not public health. Paul Farmer? Fuck Paul Farmer. He just doesn't get it.

You won't hear that in a lecture or in a public speech, but it's there. I've heard similar sentiments from the other side of the spectrum, those who see the number-crunching cost-benefiteers as soulless automatons who block the poor from getting the care they need.

These dilemmas are not going away any time soon. But I think being conscious of them and striving to be explicit about how our own values and biases shape our research and advocacy will help us to collectively reach a balance of heart and mind that makes more sense to everyone.

HIV/AIDS is one of the areas of global health where the raw passion of the heart most conflicts with the terrible dearth of resources we have to fight the demon. Decisions have ugly consequences either way you choose, and, rightly or wrongly, dispassionate research is often anything but. The recent news that pre-exposure prophylaxis (PrEP) can prevent HIV acquisition in sero-discordant heterosexual couples is huge in the news right now. Elizabeth Pisani (epidemiologist and author of The Wisdom of Whores) hits the nail on the head in this recent blog post. She notes that there are voices clamoring for widespread scale-up of PrEP -- treating the HIV negative partner -- but that PrEP prevents infection in 60% of cases while treating the HIV-positive partner cuts infection by 96%. Continuing:

That leaves us with the question: who should get PReP? Right now, there are not enough antiretrovirals to go around to treat all the sick people who need treatment. If we’re going to use them selectively for prevention, we should start with the most effective use, which appears to be early treatment of the infected partner in discordant couples. We could also give them to people who aren’t in a couple but who know that they’re likely to get around a bit and might want to stay safe without using condoms. That’s potentially a lot of people; it will stretch our purses. But more than that, it will stretch our political will.

So who is PReP for? We’ve got a better option for discordant couples. We’re not going to want to give it to randy adolescents. We know it works for gay men, but some of the countries where the trials took place would rather thump or jail gay men than protect their sexual health.[...] But I think we would be unwise to rush around talking about massive roll-out of PReP before we actually figure out who it works for in the real world.

Treating people with HIV is good. Preventing infection via treatment is good. Prevention infection via PrEP is good (assuming it doesn't breed more drug resistant strains and make it harder to treat everyone... but that's another story). But most voices in the debate have an agenda and are pushing for one thing above the rest. One of them -- or a balance of them -- is right, but you have to understand their values before that can be discerned. And I think many people in global health don't even think explicitly about their own values, such as the mix of cost-benefit and rights-based approaches they find most appealing. Rather, we all want to promote whatever we're working on that the moment. After all, it's all good.

CIA's despicable Pakistan vaccination ploy

Via Conflict Health, The Guardian reports that the "CIA organised fake vaccination drive to get Osama bin Laden's family DNA":

In March health workers administered the vaccine in a poor neighborhood on the edge of Abbottabad called Nawa Sher. The hepatitis B vaccine is usually given in three doses, the second a month after the first. But in April, instead of administering the second dose in Nawa Sher, the doctor returned to Abbottabad and moved the nurses on to Bilal Town, the suburb where Bin Laden lived.

Christopher Albon of Conflict Health writes:

If true, the CIA’s actions are irresponsible and utterly reprehensible. The quote above implies that the patients never received their second or third doses of the hepatitis B vaccine. And even if they did, there is absolutely no guarantee that the vaccines were real. The simple fact is that the health of the children of Abbottabad has been put at risk through a deceptive medical operations by the Central Intelligence Agency. Furthermore, the operation undermines future vaccination campaigns and Pakistani health workers by fueling conspiracy theories about their true purpose.

Albon notes that the Guardian's source is Pakistan's ISI... but this McClatchy story seems to confirm it via US sources:

The doctor's role was to help American officials know with certainty that bin Laden was in the compound, according to security officials and residents here, all of whom spoke only on the condition of anonymity because they feared government retribution. U.S. officials in Washington confirmed the general outlines of the effort. They asked not to be identified because of the sensitivity of the topic.

The sensitivity of the topic? No kidding. This is absolutely terrible, and not just because the kids originally involved might not have gotten the second round of vaccine (which is bad) or because it will make the work of legitimate public health officials in Pakistan even harder (which is very bad). Vaccines are amazing innovations that save millions of lives, and they are so widely respected that combatants have gone to extraordinary lengths to allow vaccination campaigns to proceed in the midst of war. For instance, UNICEF has brokered ceasefires in Afghanistan and Pakistan for polio vaccine campaigns which are essential since those are two of the four countries where polio transmission has never been interrupted.
I hope I'm not overreacting, but I'm afraid this news may be bad for the kids of Pakistan, Afghanistan, and the rest of the world. Assuming the early reports are confirmed, this plot should be condemned by everyone. If US officials who support global vaccination efforts are going to control the damage as much as possible -- though it's likely much of it has already been done -- then there need to be some very public repercussions for whoever authorized this or had any foreknowledge. What tragic stupidity: a few branches of the US government are spending millions and millions to promote vaccines, while another branch is doing this. The CIA is out of control. Sadly, I'm not optimistic that there will be any accountability, and I'm fuming that my own country breached this critical, neutral tool of peace and health. How incredibly short-sighted.


Update: In addition to the Guardian story, Conflict Health, and McClatchy stories linked above, this NYTimes article offers further confirmation and the CNN piece has some additional details. Tom Paulson at Humanosphere, Mark Leon Goldberg of UN Dispatch, Charles Kenny of CGD, and Seth Mnookin all offer commentary.


Randomized Medicaid. Also, working papers

Karen Grepin gives some background:

"Back in 2008, Oregon had a long waiting list of low-income adults wanting to enroll in its state Medicaid program. Given severely constrained resources it was not able to provide insurance to everyone who wanted it, so it decided to allocate eligibility to enroll into the program by lottery – it randomly assigned insurance eligibility – creating one of the most incredible opportunities to study the impact of health insurance. Period."

This study is the talk of the town amongst health / economics / research methodology wonks: here's some other commentary by Ezra Klein, David McKenzie of the World Bank, and William Savedoff of CGD.

One outstanding question to me is how on earth the Medicaid lottery happened in the first place. Yes, it was an environment of limited resources, which explains not making health insurance available to everyone who needed it in this particular timeframe. But why not choose to give the insurance to the poorest, oldest, sickest, etc? Did the officials who made that decision not have data to make such decisions? Did they think it would be less ethical? Did they think it would be more expensive to put the neediest (and most likely to use the service) on Medicaid? Did they anticipate the evidentiary value of their decision? I'm guessing it's some combination of the above, but I want to learn more.

A side note on working papers: all this talk is about a working paper (available here), which reminds me of Berk Özler's recent post on the World Bank's consistently excellent Development Impact blog criticizing the system of economics working papers. Özler cites as an example his own experience with a working paper on the role of conditionalities in cash transfer programs:

"Our findings in the March 2010 [working paper] suggested that CCTs that had regular school attendance as a requirement to receive cash transfers did NOT improve school enrollment over and above cash transfers with no strings attached. Our findings in the December 2010 version DID.... However, the earlier (and erroneous) finding that conditions did not improve schooling outcomes was news enough that it stuck. Many people, including good researchers, colleagues at the Bank, bloggers, policymakers, think that UCTs are as effective as CCTs in reducing dropout rates – at least in Malawi."

Ouch. This hits home in part because I remember reading that original working paper as part of a literature review for a grad school project on health impacts of conditional cash transfers (in which we designed a hypothetical community-based cluster-randomized trial in Bangladesh). I disseminated those results -- in part because they were counterintuitive -- to more than a few of my peers, but I didn't realize Özler had reversed his findings. If nothing else, we should take working papers that have not been through the full peer-review process with a large grain of salt.

The next step in tobacco control?

From The Guardian:

Iceland is considering banning the sale of cigarettes and making them a prescription-only product.

The parliament in Reykjavik is to debate a proposal that would outlaw the sale of cigarettes in normal shops. Only pharmacies would be allowed to dispense them – initially to those aged 20 and up, and eventually only to those with a valid medical certificate.

The radical initiative is part of a 10-year plan that also aims to ban smoking in all public places, including pavements and parks, and in cars where children are present. Iceland also wants to follow Australia's lead by forcing tobacco manufacturers to sell cigarettes in plain, brown packaging plastered with health warnings rather than branding.

Under the mooted law, doctors will be encouraged to help addicts kick the habit with treatments and education programmes. If these do not work, they may prescribe cigarettes.

I'll hazard a guess that this approach would not be popular in the United States.

From the front lines of public health

Rashida is a Peace Corps volunteer in Uganda teaching "teaching life skills/health/whatever at a local secondary school," as well as a blogger. She also happens to be in the same Hopkins global health Masters program as me (but a year or two ahead, so we've never met). Her latest post starts with this:


Since the kids are often shy around the muzungu (and in front of their classmates), I thought it would better to set up an anonymous questions box, where students can ask questions about health, etc. without having to ask them in front of everyone. Well, no one else seemed as excited about this idea as I was, so I thought the box would just be forgotten about, or maybe even stolen by a trouble-making student. Imagine my surprise when I came back to the school two days after setting up the box to find it overstuffed with questions. I was a bit overwhelmed by the volume of questions posed to me, so I told the students to let me take them home and prepare my answers for next week. Well, here are some of the questions that I got:
  • If you have sex during your menstruation, do you get pregnant?
  • Is it true that if you kiss someone who has HIV, you’ll also get HIV?
  • There are some boys who disturb me during my leisure time, but if I see them I feel like vomiting. What can I do, please help me?
  • Is it true that if young people play sex before menstruation begins you can still get pregnant?
  • How can I know when playing sex that sperm is coming through the penis?
  • Is it true that if you delay having sex you become an abnormal person?
  • Is it bad to practice homosexuality?
  • People usually tell us to have sex when we are still young in order to become perfect in sex. What is the meaning of perfect in sex?

There are quite a few more in the rest of the post,  and they just get more disturbing. Personally I wouldn't know where to start, and I greatly admire those who have the patience, courage, and tact to do this badly needed work. It must be especially difficult to do this sort of work as a foreigner.  Alas, I described this post -- and the sample questions -- to a friend who does sex ed in New York City and was told that the questions are remarkably similar to what you get asked here.

Measles is big this year

The CDC just put out a Health Advisory describing measles' big comeback. Though endemic transmission is the US has been interrupted, but importations keep happening when the unvaccinated population travels or come into contact with travelers:

The United States is experiencing a high number of reported measles cases in 2011, many of which were acquired during international travel. From January 1 through June 17 this year, 156 confirmed cases of measles were reported to CDC. This is the highest reported number since 1996. Most cases (136) were associated with importations from measles-endemic countries or countries where large outbreaks are occurring. The imported cases involved unvaccinated U.S. residents who recently traveled abroad, unvaccinated visitors to the United States, and people linked to these imported cases. To date, 12 outbreaks (3 or more linked cases) have occurred, accounting for 47% of the 156 cases. Of the total case-patients, 133 (85%) were unvaccinated or had undocumented vaccination status. Of the 139 case-patients who were U.S. residents, 86 (62%) were unvaccinated, 30 (22%) had undocumented vaccination status, 11 (8%) had received 1 dose of measles-mumps-rubella (MMR) vaccine, 11 (8%) had received 2 doses, and 1 (1%) had received 3 (documented) doses.

Measles was declared eliminated in the United States in 2000 due to our high 2-dose measles vaccine coverage, but it is still endemic or large outbreaks are occurring in countries in Europe (including France, the United Kingdom, Spain, and Switzerland), Africa, and Asia (including India). The increase in measles cases and outbreaks in the United States this year underscores the ongoing risk of importations, the need for high measles vaccine coverage, and the importance of prompt and appropriate public health response to measles cases and outbreaks.

Measles is a highly contagious, acute viral illness that is transmitted by contact with an infected person through coughing and sneezing. After an infected person leaves a location, the virus remains contagious for up to 2 hours on surfaces and in the air. Measles can cause severe health complications, including pneumonia, encephalitis, and death.

The message is simple: parents should vaccinate their children because not doing so has serious health effects not only on those children, but also on those who are unable to be vaccinated because they are either too young or have medical contraindications. If everyone who believed (wrongly) that vaccines are unsafe would move to one country (let's call it Unvaccinstan) then the choice would have fewer ethical pitfalls: you make a bad choice, and your kids might get sick. But as it is there are many people who simply can't get vaccinated -- kids with cancer for example, or kids in the window between when your maternal antibodies aren't that effective against measles but still interfere with the vaccine -- so the choice has much broader societal impact. I imagine that many of the parents who choose not to vaccinate -- who are often of higher educational status and more liberal politics -- view themselves  as virtuous; the reality is sadly the opposite.

Lead poisoning in China

It's a huge problem -- the Times calls it a Hidden Scourge:

Here, Chinese leaders have acknowledged that lead contamination is a grave issue and have raised the priority of reducing heavy-metal pollution in the government’s latest five-year plan, presented in March. But despite efforts to step up enforcement, including suspending production last month at a number of battery factories, the government’s response remains faltering.

At a meeting last month of China’s State Council, after yet another disclosure of mass poisoning, Prime Minister Wen Jiabao scolded Environmental Minister Zhou Shengxian for the lack of progress, according to an individual with high-level government ties who spoke on the condition of anonymity.

The government has not ordered a nationwide survey of children’s blood lead levels, so the number of children who are at risk is purely a matter of guesswork. Mass poisonings like that at the Haijiu factory typically come to light only after suspicious parents seek hospital tests, then alert neighbors or co-workers to the alarming results.

And relevant to my current work, which I hope to write about more soon.

Tornado epidemiology

The news out of Joplin, Missouri is heartbreaking, and it comes so quickly on the heels of the tornadoes that hit Tuscaloosa, Alabama. Central Arkansas, where I grew up, gets hit by tornadoes every spring, so I have plenty of memories of taking shelter in response to warnings. College nights with social plans ruined when we had to hunker down in an interior hallways. Dark, roiling clouds circling and the spooky calm when the rain and hail stop but the winds stay strong. Racing home from work to get to my house and its basement -- a rarity in the South -- before a particularly ominous storm hit. Neighboring communities were sometimes hit more directly by storms, and Harding students often participated in clean-up an recovering efforts, but my town was spared direct hits by the heaviest tornadoes. So what does epidemiology have to say about tornadoes? Their paths aren't exactly random, in the sense that some areas are more prone to storms that produce tornadoes. Growing up I knew where to take shelter: interior hallways away from windows if your house didn't have a basement or a dedicated storm shelter. I also knew that mobile homes were a particularly bad place to be, and that the carnage was always worst when a tornado happened to hit a mobile home lot.

But there is some interesting research out there that tells us more than you might think. Obviously and thankfully you can't do a randomized trial assigning some communities to get storms and others not, so the evidence of how to prevent tornado-related injury and death is mostly observational. What do we know? I'm not an expert on this but I did a quick, non-systematic scan and here's what I found:

First, the annual tornado mortality rate has actually gone down quite a lot over the last few decades. That says nothing about the frequency and intensity of tornadoes themselves, which is a matter for meteorologists to research. The actual number deaths resulting from tornadoes would probably be a function of the number of people in the US, where they live and whether those areas are prone to tornadoes, the frequency and intensity of the tornadoes, and risk factors for people in the affected area once the tornado hits.

This NOAA site has the following graph of tornado mortality where the vertical axis is tornado deaths per million people in the US (on a log scale) and the horizontal axis covers 1875 - 2008.

Second, many of the risk factors for tornado injury and death are intuitive and suggest possible interventions to minimize risk in tornado-prone areas. Following tornadoes in North and South Carolina in 1984, Eidson et al. surveyed people hospitalized and family members of people who were killed, along with uninjured persons who were present when the surveyed individuals were hurt. The main types of injury were deep cuts, concussions, unconsciousness and broken bones. Risk factors included living in mobile homes, "advanced age (60+ years), no physical protection (not having been covered with a blanket or other object), having been struck by broken window glass or other falling objects, home lifted off its foundation, collapsed ceiling or floor, or walls blown away." Some of those patterns might indicate potential tornado education interventions -- better shelters for mobile home residents, targeting alerts to older residents, covering with a blanket, and staying in interior hallways, to say nothing of building codes to make more survivable structures.

Third, some things are less clear, like whether it's safe to be in a car during a tornado. Daley et al. did a case-control study of tornado injuries and deaths in the aftermath of tornadoes in Oklahoma in 1999. They found higher risk of tornado death for those in mobile homes (odds ratio of 35.3, 95% CI 7.8 - 175.6) or outdoors (odds ratio of 141.2, 95% CI 15.9 - a whopping 6,379.8) compared to other houses. They found no difference in risk of death, severe injury, or minor injury among people in cars vs. those in houses. And they found that risk of death, severe injury, or minor injury was actually lower among those "fleeing their homes in motor vehicles than among those remaining." That's surprising to me, and contrary to much of the tornado-related safety warnings I heard from meteorologists and family growing up. I wonder if this particular study goes against the majority of findings, or whether there is a consensus based in data at all.

Fourth, our knowledge of tornadoes can be messy. One demographic approach to tornado risk factors (Donner 2007) is to look for correlations between tornado fatalities and injuries with rural population, population density, household size, racial minorities, deprivation/poverty, tornado watches and warnings, and mobile homes. Donner noted that "Findings suggest a strong relationship between the size of a tornado path and both fatalities and injuries, whereas other measures related to technology, population, and organization produce significant yet mixed results."

That's just a sampling of the literature on tornado epidemiology. The studies are interesting but relatively rare, at least from initial perusal. That's probably because tornado deaths and injuries are relatively rare in the US. Still, the storms themselves are terrifying and they often wreak havoc on a single community and thus generate more sympathy and news coverage than a more frequent -- and thus less extraordinary -- problem like car crashes.

Update: NYT has an interesting article about tornado preparedness, including some speculation on why the Joplin tornado was so bad.

Miscellany: Epidemic City and life expectancy

In 8 days I'll be done with my first year of graduate studies and will have a chance to write a bit more. I've been keeping notes all year on things to write about when I have more time, so I should have no shortage of material! In the meantime, two links to share: 1) Just in time for my summer working with the New York City Department of Health comes Epidemic City: The Politics of Public Health in New York. The Amazon / publisher's blurb:

The first permanent Board of Health in the United States was created in response to a cholera outbreak in New York City in 1866. By the mid-twentieth century, thanks to landmark achievements in vaccinations, medical data collection, and community health, the NYC Department of Health had become the nation's gold standard for public health. However, as the city's population grew in number and diversity, new epidemics emerged, and the department struggled to balance its efforts between the treatment of diseases such as AIDS, multi-drug resistant tuberculosis, and West Nile Virus and the prevention of illness-causing factors like lead paint, heroin addiction, homelessness, smoking, and unhealthy foods. In Epidemic City, historian of public health James Colgrove chronicles the challenges faced by the health department in the four decades following New York City's mid-twentieth-century peak in public health provision.

This insightful volume draws on archival research and oral histories to examine how the provision of public health has adapted to the competing demands of diverse public needs, public perceptions, and political pressure.

Epidemic City delves beyond a simple narrative of the NYC Department of Health's decline and rebirth to analyze the perspectives and efforts of the people responsible for the city's public health from the 1960s to the present. The second half of the twentieth century brought new challenges, such as budget and staffing shortages, and new threats like bioterrorism. Faced with controversies such as needle exchange programs and AIDS reporting, the health department struggled to maintain a delicate balance between its primary focus on illness prevention and the need to ensure public and political support for its activities.

In the past decade, after the 9/11 attacks and bioterrorism scares partially diverted public health efforts from illness prevention to threat response, Mayor Michael Bloomberg and Department of Health Commissioner Thomas Frieden were still able to work together to pass New York's Clean Indoor Air Act restricting smoking and significant regulations on trans-fats used by restaurants. Because of Bloomberg's willingness to exert his political clout, both laws passed despite opposition from business owners fearing reduced revenues and activist groups who decried the laws' infringement upon personal freedoms. This legislation preventative in nature much like the 1960s lead paint laws and the department's original sanitary code reflects a return to the 19th century roots of public health, when public health measures were often overtly paternalistic. The assertive laws conceived by Frieden and executed by Bloomberg demonstrate how far the mandate of public health can extend when backed by committed government officials.

Epidemic City provides a compelling historical analysis of the individuals and groups tasked with negotiating the fine line between public health and political considerations during the latter half of the twentieth century. By examining the department's successes and failures during the ambitious social programs of the 1960s, the fiscal crisis of the 1970s, the struggles with poverty and homelessness in the 1980s and 1990s, and in the post-9/11 era, Epidemic City shows how the NYC Department of Health has defined the role and scope of public health services, not only in New York, but for the entire nation.

2) Aaron Carroll at the Incidental Economist writes about the subtleties of life expectancy. His main point is that infant mortality skews life expectancy figures so much that if you're talking about end-of-life expectations for adults who have already passed those (historically) most perilous times as a youngster, you really need to look at different data altogether.

The blue points on the graph below show life expectancy for all races in the US at birth, while the red line shows life expectancy amongst those who have reached the age of 65. Ie, if you're a 65-year-old who wants to know your chances of dying (on average!) in a certain period of time, it's best to consult a more complete life table rather than life expectancy at birth, because you've already dodged the bullet for 65 years.

(from the Incidental Economist)

Modelling Stillbirth

William Easterly and Laura Freschi go after "Inception Statistics" in the latest post on AidWatch. They criticize -- in typically hyperbolic style, with bonus points for the pun in the title -- both the estimates of stillbirth and their coverage in the news media. I left a comment on their blog outlining my thoughts but thought I'd re-post them here with a little more explanation. Here's what I said:

Thanks for this post (it’s always helpful to look at quality of estimates critically) but I think the direction of your criticism needs to be clarified. Which of the following are you upset about (choose all that apply)?

a) the fact that the researchers used models at all? I don’t know the researchers personally, but I would imagine that they are concerned with data quality in general and would much preferred to have had reliable data from all the countries they work with. But in the absence of that data (and while working towards it) isn’t it helpful to have the best possible estimates on which to set global health policy, while acknowledging their limitations? Based on the available data, is there a better way to estimate these, or do you think we’d be better off without them (in which case stillbirth might be getting even less attention)? b) a misrepresentation of their data as something other than a model? If so, could you please specify where you think that mistake occurred — to me it seems like they present it in the literature as what it is and nothing more. c) the coverage of these data in the media? On that I basically agree. It’s helpful to have critical viewpoints on articles where there is legitimate disagreement.

I get the impression your main beef is with (c), in which case I agree that press reports should be more skeptical. But I think calling the data “made up” goes too far too. Yes, it’d be nice to have pristine data for everything, but in the meantime we should try for the best possible estimates because we need something on which to base policy decisions. Along those lines, I think this commentary by Neff Walker (full disclosure: my advisor) in the same issue is worthwhile. Walker asks these five questions – noting areas where the estimates need improvement: - “Do the estimates include time trends, and are they geographically specific?” (because these allow you to crosscheck numbers for credibility) - “Are modelled results compared with previous estimates and differences explained?” - “Is there a logical and causal relation between the predictor and outcome variables in the model?” - “Do the reported measures of uncertainty around modelled estimates show the amount and quality of available data?” - “How different are the settings from which the datasets used to develop the model were drawn from those to which the model is applied?” (here Walker says further work is needed)

I'll admit to being in over my head in evaluating these particular models. As Easterly and Freschi note, "the number of people who actually understand these statistical techniques well enough to judge whether a certain model has produced a good estimate or a bunch of garbage is very, very small." Very true. But in the absence of better data, we need models on which to base decisions -- if not we're basing our decisions on uninformed guesswork, rather than informed guesswork.

I think the criticism of media coverage is valid. Even if these models are the best ever they should still be reported as good estimates at best. But when Easterly calls the data "made up" I think the hyperbole is counterproductive. There's an incredibly wide spectrum of data quality, from completely pulled-out-of-the-navel to comprehensive data from a perfectly-functioning vital registration system. We should recognize that the data we work with aren't perfect. And there probably is a cut-off point at which estimates are based on so many models-within-models that they are hurtful rather than helpful in making informed decisions. But are these particular estimates at that point? I would need to see a much more robust criticism than AidWatch has provided so far to be convinced that these estimates aren't helpful in setting priorities.

Fact for the day

Astonishingly, a third of the wealthiest 20% of Indian children are malnourished, too, and they are neither poor nor excluded.

That's from the Economist last Thursday.

Wonder what it says about India, but also what it says about our measures of wealth and malnutrition.