False claims undermine good causes

Today is White Ribbon Day. It is an important symbolic event reminding us all to be aware of violence against women.

Domestic violence and family abuse are a scourge on all human societies. Events such as White Ribbon Day play an significant role in breaking down the shame and stigma which makes it so hard for individuals to seek help. I wholeheartedly support this aim. What I do not support, however, are dodgy statistics and false claims which belittle this good cause. On Monday, 25th November, 2014, SkyNews Australia published the following tweet:

This is a sensational claim that is easily fact-checked. Research institutions such as the Australian Institute of Health and Wellbeing (AIHW), the Australian Bureau of Statistics (ABS) and the Australian Institute of Criminology (AIC) keep records of causes of death, and rates of victimisation for people in this age group every year.

To fact-check SkyNews Australia’s claim, let’s break down the most recent data we have for causes of death for men and women under the age of 45 (see Table 2 in the AIHW summary). Keep in mind these statistics are for both men and women:

1. Suicide                                 2,769 deaths

2. Accidental poisoning        1,534 deaths

3. Transport collisions          1,388 deaths

4. Heart disease                        915 deaths

5. Breast cancer                         509 deaths

Death by homicide does not make the top 5, for men or for women.

ABS data tells us that on average, one woman takes her life via suicide each day. AIC data tells us that in 2012, 33 women died from homicide, nationally, while 67 men did. In contrast, 336 women aged 15 – 45 died from suicide. Our rates of suicide should be our national shame. Combined, suicide and drug overdose claim eighty people per week under the age of 45, a significant proportion of whom are women. But violence is the sensational social issue du jour, so we do not hear about it. In May, 2014, ABCNews ran a story which stated:

Domestic violence is the leading cause of death and injury in women under 45, with more than one woman murdered by her   current or former partner every week.

Yet almost one woman dies every day from suicide, and almost two from breast cancer. So how is domestic violence “the leading cause of death” for women in this age group? Where does this claim even come from?

Source of the Claim

The claim comes from a ten year old report by the Australian government body VicHealth  tabled for the World Health Organisation. In 2004, VicHealth teamed up with a group of women’s advocates for the purposes of quantifying the overall health burden inflicted upon women and more broadly, society, from domestic violence.

In quantifying the burden of disease, the researchers involved chalked up health problems of victims as direct outcomes of exposure to violenceSee the figure below.

Untitled

Health outcomes contributing to the disease burden of intimate partner violence include mental health issues (73% of the total disease burden), tobacco use at 14% and cervical cancer at 1%.

In calculating the total “health burden” of violence, the study’s authors came to the conclusion that intimate partner violence was the leading cause of preventable illness, disease and disability for women aged 15-44. How they came to this conclusion is difficult to gather due to the report’s opacity. Yet astonishingly, at some point in our national discourse, the claim that intimate partner violence is “a leading cause of disease burden” has been replaced by this:

It behooves us then to take a closer look at the source of this claim, in order to see if it stands up to scrutiny.

Methodological Concerns

The VicHealth report is based on what is known as a cross-sectional design. Data was taken from pre-existing reports and in their analysis, health variables and exposure to violence were measured at the same time. The most fundamental limitation to such a design is confusing correlation with causation

Such a design cannot tell us whether or not violence came before the onset of mental health problems, tobacco use or cervical cancer nor any other health outcome.

While it is highly likely that victims of violence do go onto develop mental health problems such as depression, anxiety and PTSD. It is also highly likely that individuals with pre-existing mental health conditions find themselves in circumstances where such victimisation occurs. Last year’s report from the Australian Longitudinal Study on Women’s Health said the following:

Women in their 20s and 30s who report intimate partner violence experience poorer mental health prior to intimate partner violence, suggesting an inter-connected relationship; that is, intimate partner violence affects mental health status and likewise mental health affects intimate partner violence. [6 pp 83]

The only way to prove causality is to prove that violence occurred at a point in time prior to the onset of mental health problems. The authors of the report have not done this. They also have not proven any causal link between violence and cancer, or tobacco use either. When referring to this limitation in their ‘technical report’, they stated simply that they “decided” violence preceded such health variables as cancer. Take a look at their reasoning in their own words —

A cross-sectional analysis is a weak design to examine the relationship between a risk factor and disease outcomes because it cannot indicate whether exposure to the risk factor preceded the health outcome, a necessary condition to prove causality. A longitudinal study design would be better suited to study this issue. Despite the large overall study size of ALSWH the number of women who newly reported intimate partner violence between the first and second survey was too small and the health status information too limited to examine temporality. However, we decided that a causal relationship between intimate partner violence and health outcomes was much more plausible than a health outcome being the cause of intimate partner violence. [5 pp 742-743]

“We decided”

Let’s take a look at who “we” is. For the VicHealth report, “prevalence data review and expertise” was overseen by Melanie Heenan, from the Australian Centre for the Study of Sexual Assault. The “Health impact data review and expertise ” was overseen by Jill Astbury, of the Key Centre for Women’s Health in Society. These researchers have dual roles as political advocates.

The authors of the report “decided” that intimate partner violence caused negative health outcomes. But they did not prove it. They did not rule out alternative explanations for the relationship between violence and negative health outcomes. And they did not attempt to temper their study’s conclusions in light of these serious methodological flaws. They also looked at female victims only, despite the fact that intimate partner violence is known to affect men at significant levels as well.

Political biases do not always undermine the quality of research, but they can and sometimes do. This study (which has not been replicated) contains major limitations. Published in a WHO newsletter, as opposed to a scientific journal, the report has never passed what is generally considered an acceptable standard of peer review. It is a government report, overseen by bureaucrats, funded by taxpayers. In short, it is an example of bad research performed for a political agenda. And now it is the basis for sensationalist false claims promoted in Australia by Sky and ABC News.

On White Ribbon Day, or any other day, we do not need false claims about the impact of intimate partner violence to know that it is a shocking thing, and a scourge on our society. We do not need to be told that domestic violence is the leading cause of death for women aged 15 – 45 in order to take it seriously.

The more false claims are publicised about violence against women, the more community cynicism will grow.

We do not do women any favours by producing bad research, and making exaggerated claims in their name.

***

Update

Reader Stu makes the following comment.

More data from the ABS here:

http://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/3303.02012?OpenDocument

From this we actually can break down the actual data to females between the ages of 15 and 44.

In all age subgroups (15-24,25-34,35-44), the top cause of death is suicide, although if you combine cancers into one group, cancer tops the 35-44 list. 

Overall the breakdown is similar to the gender-neutral one above – it is still suicide, followed by poisoning, then traffic accidents, then various subgroups of cancer and heart disease.

It’s hard to tell whether the false claims are deliberately dishonest or just carelessness, but they’re only harming the credibility of those who make them.

***

Further Links

ABC News – Domestic violence of epidemic proportions a ‘national emergency’: campaign groups

NSW Bureau of Crime Statistics and Research – New South Wales Recorded Crime Statistics 2013

NSW Bureau of Crime Statistics and Research – Trends and patterns in domestic violence assaults: 2001 to 2010

Carlson, M. D., & Morrison, R. S. (2009). Study design, precision, and validity in observational studiesJournal of palliative medicine12(1), 77-82.

The health costs of violence: Measuring the burden of disease caused by intimate partner violence: A summary of findings“, VicHealth, Carlton South, Australia, 2004

Holden L, Dobson A, Byles J, Loxton D, Dolja-Gore X, Hockey R, Lee C, Chojenta C, Reilly N, Mishra G, McLaughlin D, Pachana N, Tooth L & Harris M. “Mental Health: Findings from the Australian Longitudinal Study on Women’s Health.“, Report prepared for the Australian Government Department of Health & Ageing, June 2013.

Vos, T., Astbury, J., Piers, L. S., Magnus, A., Heenan, M., Stanley, L., … & Webster, K. (2006). Measuring the impact of intimate partner violence on the health of women in Victoria, Australia. Bulletin of the World Health Organization, 84(9), 739-744.

13 Comments

  1. Stu

    More data from the ABS here:

    http://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/3303.02012?OpenDocument

    From this we actually can break down the actual data to females between the ages of 15 and 44.

    In all age subgroups (15-24,25-34,35-44), the top cause of death is suicide, although if you combine cancers into one group, cancer tops the 35-44 list.

    Overall the breakdown is similar to the gender-neutral one above – it is still suicide, followed by poisoning, then traffic accidents, then various subgroups of cancer and heart disease.

    It’s hard to tell whether the false claims are deliberately dishonest or just carelessness, but they’re only harming the credibility of those who make them.

  2. Staffan

    Good work.

    It was even worse than I imagined, especially considering VicHealth sorts under the Department of Health. We had a similar study in Sweden a few years back, claiming that 46% of all women had been victims of domestic violence although this included verbal abuse. So a slight majority had never even been subjected to angry words of that nature.

    Disease burden is obviously worthless unless causality can be established, and most of it may well go in the other direction. But I don’t doubt that it can be a very important factor in certain cases, like for instance women who are stalked.

  3. etmalthusianism

    A case of never letting the facts get in the way of a good story or public hysteria. Feminism has had the effect of making all women scared of men and all men self self loathing and guilt burdened. A Hobbsian war of all against all, but to whose advantage? To whose tune is the fear drum being beaten? Feminists have have as cynical and ruthless as the most yellow tabloid.

  4. Kristina

    Great post Claire. Something I’ve been thinking about for a while now is the damaging repercussions non-experts can have writing articles or trying to dissect research they are simply unable or not taught how to interpret. Should there be higher standards of education for writers who write for the general public? Is there a greater need for academics/experts/scientists to make their research more accessible?

  5. Staffan

    Kristina: Sadly it is often the experts with plenty of academic credentials who make these simple mistakes. You may have heard of the large correlational study showing a link between milk consumption and heart disease. No causality established but the main author said he will now give up milk.

  6. genderneutrallanguage

    A semantics Quibble.

    White Ribbon day is a great day to try and break down the shame and stigma associated with Domestic violence.

    You are re-branding (well sticking with the incorrect branding) of Domestic Violence being “Violence against Women”. This bad branding has the devastatingly negative affect of increasing the shame and stigma associated with being a male victim of domestic violence. Calling Domestic Violence “Violence against women” is telling male victims of domestic violence, you can’t be both a man and a victim of domestic violence. There is little more shaming and stigmatizing than that.

  7. John Stuart Mill

    Not proving but assuming causality is only one of the issues here. The other big one is the issue of comorbidity. Comorbidity is the presence of multiple health outcomes as a consequence of one disease, risk factor, or combination of risk factors.

    The authors of the VicHealth study were responsible for the inclusion of intimate partner violence (IPV) in the Austrlalian Institute of Health and Welfare (AIHW) 2003 Australian Burden of Disease study. The chapter of this study that dealt with IPV was dealing with risk factors that affect health outcomes, the introduction of the chapter addresses the issue of comorbidity and it’s impact on burden of disease estimates (emphasis mine).

    Secondly, the causal paths between a number of related health risks and their eventual health outcomes can be complicated. For example, physical inactivity can lead to obesity, which can cause hypertension or high blood cholesterol, which can ultimately lead to cardiovascular disease. Most of the analyses presented in this chapter only measure the effect of a risk independent of the other exposures and irrespective of the risk’s place in a causal path. The important implication here is that such analyses are not additive. Using the example above, the burden attributable to physical inactivity is estimated to be 23.7% of total cardiovascular disease burden, while that for high body mass, high blood cholesterol and high blood pressure was 19.5%, 34.5% and 42.1% of cardiovascular disease, respectively (Table 4.2). The burden attributable to these health risks in combination, however, is not the sum of burden from each risk (that is, the combined burden is not 119.9%). This is because the combined effect of these risks has to be expressly calculated rather than derived from the
    addition of their individual effects. Ignoring shared causal paths in this example leads to obvious over-estimation of the combined effect.
    [1 pp 72-73]

    Considering that the majority of the IPV disease burden is attributable to anxiety and depression, it is interesting to see what the Burden of Disesease study has to say about the comorbidity of these two health outcomes.

    While the data sources have remained mostly the same as were used for the previous
    Australian burden study, there are a number of key methodological changes. First, we have grouped all anxiety disorders (panic, agoraphobia, social phobia, generalised anxiety disorder, obsessive-compulsive disorder, post-traumatic stress disorder and separation anxiety disorder) and the unipolar depressive disorders (major depression and dysthymia) that were previously modelled separately into a single disease category. This is based on the argument that the high degree of comorbidity and the similarity in psychological and drug treatment means that all these disorders can be considered as part of the same entity, with a continuum between mostly depressed to mostly anxious (for example (Andrews et al. 1990; Andrews & Slade 2002). The advantage of this approach is that it takes away some of the difficulties of dealing with the frequent comorbidity among these disorders. [1 pp 154-155]

    By not accounting for comorbidity, as well as not proving causality, the figures relating to the attributable disease burden are overestimated. This is further compounded by the reliance of this methodology in estimating the costs of IPV to the Australian economy such as in the 2004 Access Economics paper [2] and in the 2009 KPMG estimates [3].

    This not only potentially overestimates the amount of pain and suffering caused as a result of IPV but also overestimates the costs to the healthcare system by not recognising that people are often treated by the same healthcare practitioner for both anxiety and depression. The two conditions are often treated together by the same psychologist or psychiatrist and the cost of treatment is only borne once.

    You could make the argument that the authors of the VicHealth study were unaware of the issues surrounding comorbidity and mental health. However, a 2000 WHO report co-authored by Jill Astbury specifically looking at the comorbidity of women’s mental health issues, including in relationship to IPV, shows that this is not the case [4]. The issues were well known by this group of researchers, they just chose not to discuss them.

    Another thing in the VicHealth study technical paper is the claim that this was the first burden of disease study to include estimates on IPV. It isn’t, it was first included in the World Bank 1993 World Development Report a decade earlier, something that Jill Astbury can be shown to have known.

    If you want more details, send me an email.

    1. Begg, S., Vos, T., Barker, B., Stevenson, C., Stanley, L., & Lopez, A. (2007). Burden of disease and injury in Australia, 2003. Australian Institute of Health and Welfare AIHW.

    2. Access Economics – The Cost of Domestic Violence to the Australian Economy, 2004

    3. KPMG – The Cost of Violence against Women and their Children, 2009

    5. Astbury, J., & Cabral, M. (2000). Women’s mental health: an evidence based review. Geneva: World Health Organization.

  8. Stu

    Three guesses which Daily Life columnist just repeated this mistake in a piece for The Age?

    Okay, okay, one guess. I promise it’s all you need.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s