Australia’s Medicare system holds health and medical data relating to more than 27 million people, making news of a hack involving an OpenAI agent in June particularly alarming. While the Australian government responded strongly to the breach, experts have questioned whether anyone can ultimately be held legally responsible for what happened

Prime Minister Anthony Albanese disclosed the incident while in New York campaigning at the United Nations this month for stronger international regulation of artificial intelligence (AI), as recalled in an article by The Conversation on this issue.

According to early reports, the OpenAI agent had been instructed to research publicly available medical data when it gained access to Medicare systems. The agent reportedly accessed private statistical information, including billing patterns, rather than individual medical records, through an old Australian government website containing Medicare statistics.

Albanese said the agent had found a way around privacy safeguards and “didn’t accept ‘no’ for an answer”.

The incident represents a significant escalation in a recent series of troubling cases involving AI systems operated by OpenAI, Anthropic and Google, all of which have reportedly been found to have hacked multiple websites.

AI agents are essentially advanced autonomous chatbots. Unlike conventional chatbots, however, they can be given complex objectives and allowed to pursue them with limited human guidance. They can browse websites, download information and execute code without continuous human supervision.

That combination of greater capability and limited oversight is central to why AI agents have repeatedly been involved in hacking websites. It also makes such behaviour difficult for their developers to prevent completely. Giving agents more capabilities makes them more useful, but at the same time provides them with tools that can potentially be used to compromise computer systems.

Although personal medical information was apparently not accessed in the Medicare incident, its seriousness is difficult to overstate. The breach also appears to have gone unnoticed by Medicare itself until OpenAI informed the Australian government on September 10, around three months after the incident occurred.

OpenAI had discovered the breach in August while reviewing the model’s activity. The company then notified the government by sending an email to a public government mailbox.

Legal grey zone

The delay between the hack and OpenAI discovering it raises another question: how many other actions by AI agents may have gone undetected?

The way AI companies have disclosed such incidents, as well as the language used to describe them, points to a broader problem: public discussion and legislation have struggled to keep pace with the development of autonomous AI agents.

People often think about AI systems as though they were human, partly because of the conversational way in which people interact with them. But treating AI in human-like terms can also obscure responsibility for decisions made by the companies that develop and operate these systems.

Australian Deputy Prime Minister Richard Marles described the Medicare incident as “unauthorised” but “unintended”, while OpenAI spokesperson Drew Pusateri called it a “misaligned model”.

Such descriptions can shift attention away from the human decisions and systems that ultimately produced the outcome. AI agents do not operate independently of people: they are software systems designed by humans and deployed and operated by humans.

Determining who should bear legal responsibility for an action carried out by an AI agent, however, presents a significant challenge.

In a speech earlier this year, New South Wales Chief Justice Andrew Bell addressed this issue, arguing that Australian law currently makes one point clear: AI agents cannot themselves be held responsible for their actions.

That does not necessarily mean liability automatically rests with the person who used the AI system. Legal scholars have pointed out that criminal law generally requires a certain level of deliberate intention from the person responsible for an offence.

For example, if someone instructs an AI agent to collect health statistics and the agent independently hacks a government server to complete the task, the person may not have possessed the deliberate intent required for a criminal conviction. At the same time, the AI agent cannot be charged because it has neither legal personhood nor human intentionality.

Ultimately, AI agents are commercial products developed and deployed by billion-dollar corporations. And while Chief Justice Bell has argued that “regulatory lag is inevitable” as AI develops, he has also warned that such a gap cannot be sustained in an era of rapidly advancing technology.

By Nazrin Sadigova

Source: caliber.az