QLD police will use AI to 'predict' domestic violence before it happens. Beware the unintended consequences

The Queensland Police Service (QPS) is expected to begin a trial using artificial intelligence (AI)^[1] to determine the future risk^[2] posed by known domestic violence perpetrators.

Perpetrators identified as “high risk” — based on previous calls to an address, past criminal activity and other police-held data — will be visited at home by police before domestic violence escalates, and before any crime has been committed.

It is necessary to find better ways to improve safety for women subjected to domestic violence. However, using AI technology in this context may have unintended consequences — and the proposed plan raises serious questions about the role of police in preventing domestic violence incidents.

The approach relies on an algorithm that has been developed from existing QPS administrative data (QPRIME). All statistical algorithms must assess risk based on available data^[3], which in turn means they are only as good as the data underpinning them.

Experts who criticise the use of data-driven risk assessment tools^[4] in policing point to the lack of transparency in the specific kinds of data analysed, as well as how predictions based on these data are acted upon.

Because of how police operate, the key data most consistently captured are information about past situations police have been called to, and criminal history data.

Using this information to train an AI algorithm could reinforce existing biases in the criminal justice system. It could create an endless feedback loop between police and those members of the public who have the most contact with police.

In Australia, they are Aboriginal and Torres Strait Islander people^[5]. It is not difficult to imagine that under this new regime Aboriginal and Torres Strait Islander people will be visited more by police^[6].

QPS representative Ben Martain^[7] has said police won’t be able to charge someone they door-knock for a future suspected offence.

He also said for the pilot, attributes of ethnicity and geographic location were removed before training the AI model. But despite this, it seems likely Aboriginal and Torres Strait Islander people will continue to be disproportionately targeted, since they are over-represented across all kinds of police contact.

Read more: NSW Police want access to Tinder's sexual assault data. Cybersafety experts explain why it's a date with disaster^[8]

Introducing risk

The aim of such AI-based strategies in policing is to prevent or reduce crime, through an assessment of the risk of future offending. In theory, this means police would intervene early to stop a crime from occurring in the first place.

However, with this approach there are risks police may create crime. An unprompted police door-knock would be unwelcome in most households — let alone one where police have previously attended to carry out searches or make arrests.

In this “preventative” program, perpetrators and the victims they live with may be nervous, agitated or even angry at the police intrusion at their home for no apparent reason.

A visited person might use offensive language^[9] or refuse to provide their name. It would not be surprising if this led to charges.

Such charges might lead the visited person to become even more nervous, agitated or angry, and then they may find they are charged with assault and resisting police. This is popularly known as the “trifecta”^[10], wherein a person who has otherwise not offended is ultimately charged with offensive language, resisting arrest and assaulting police.

The standard powers in the police toolbox are to arrest and charge. With QPS’s proposed plan, there is an obvious risk of widening the net of criminalisation for both perpetrators, as well as victims who may be misidentified^[11] as perpetrators. For instance, sometimes victims who have used violence in self-defence have been arrested instead of the perpetrator.

Bringing further harm to victims

The role of the victim in such a program is also of concern. Any program that deepens surveillance of perpetrators also deepens surveillance of victims^[12].

Victims do not always want police to intervene in their lives. In some cases, this form of proactive policing might feel like an extension of control, rather than help. What happens when police visit and discover a high-risk perpetrator and victim are living together again?

Victims may fear child protection^[13] authorities will get involved and feel obliged to cover up the fact they are still with the perpetrator. And once a victim has been pressured to lie, they may be reluctant to call the police the next time they do need police intervention.

In some cases, the perpetrator or victim may decide not to take the safety advice of police officers who visit. It is not clear what police might do in a situation where they ask a perpetrator to leave, or try to take a victim to safety, but they refuse.

The mission of any domestic violence intervention should be to restore power to victims. But we know interventions do not assist all women (or men) equally. Structural inequalities^[14], including race and class, mean interventions are experienced differently by different people.

Will a victim have a say in whether police engage in proactive policing of their perpetrator? Should they have a say?

Read more: Police access to COVID check-in data is an affront to our privacy. We need stronger and more consistent rules in place^[15]

Are there safer options?

In the context of risk assessment^[16], many experts argue women often (although not always) have a strong sense of when they are at heightened risk.

Family court-ordered contact visits can be one of those moments of high risk. Yet in these situations women often report police refusing^[17] to help keep them and their children safe. How is the voice of the victim factored into risk assessment with this tool?

One particular concern is whether police are really equipped to intervene in circumstances where there is no crime. QPS representative Ben Martain^[18] said when perpetrators are “not at a point of crisis, in a heightened emotional state, or affected by drugs or alcohol” — they are “generally more amenable to recognising this as a turning-point opportunity in their lives”.

But police themselves have questioned their role in domestic violence circumstances^[19] — instead highlighting the potential role social workers may have, in their place.

It is not clear whether police are the best-positioned service to intervene when there is no identified disturbance. Queensland already has information-sharing protocols^[20] involving teams tasked specifically with responding to people involved in high-risk domestic violence relationships. These teams include community-based support workers.

This may be a better path for intervention during those critical periods of calm.

References

1. ^{^} artificial intelligence (AI) (www.theguardian.com)
2. ^{^} future risk (papers.ssrn.com)
3. ^{^} on available data (papers.ssrn.com)
4. ^{^} data-driven risk assessment tools (www.brennancenter.org)
5. ^{^} Aboriginal and Torres Strait Islander people (www.proquest.com)
6. ^{^} visited more by police (www.sentencingcouncil.qld.gov.au)
7. ^{^} Ben Martain (www.theguardian.com)
8. ^{^} NSW Police want access to Tinder's sexual assault data. Cybersafety experts explain why it's a date with disaster (theconversation.com)
9. ^{^} offensive language (www.alrc.gov.au)
10. ^{^} “trifecta” (humanrights.gov.au)
11. ^{^} victims who may be misidentified (www.anrows.org.au)
12. ^{^} deepens surveillance of victims (link.springer.com)
13. ^{^} child protection (rcfv.archive.royalcommission.vic.gov.au)
14. ^{^} Structural inequalities (www.unswlawjournal.unsw.edu.au)
15. ^{^} Police access to COVID check-in data is an affront to our privacy. We need stronger and more consistent rules in place (theconversation.com)
16. ^{^} risk assessment (dfvbenchbook.aija.org.au)
17. ^{^} police refusing (www.crimejusticejournal.com)
18. ^{^} Ben Martain (www.theguardian.com)
19. ^{^} circumstances (www.theguardian.com)
20. ^{^} information-sharing protocols (dfvbenchbook.aija.org.au)