Category Archives: Big Data

AI hallucinations and the law

Posted on by
1

Several years ago, I blogged about the role of technology within the legal profession. One development I noted was the nascent use of AI to help test the merits of a case before it goes to trial, and to assess the likelihood of winning. Not only might this prevent potentially frivolous matters coming to trial, it would also reduce court time and legal costs.

More recently, there has been some caution (if not out and out scepticism) about the efficacy of using AI in support of legal research and case preparation. This current debate has been triggered by an academic paper from Stanford University that compared leading legal research tools (that claim to have been “enhanced” by AI) and ChatGPT. The results were sobering, with a staggering number of apparent “hallucinations” being generated, even by the specialist legal research tools. AI hallucinations are not unique to legal research tools; nor to the AI tools and the Large Language Model (LLMs) they are trained on, as Stanford has previously reported. While the academic paper is awaiting formal publication, there has been some to-and-fro between the research authors and at least one of the named legal tools. This latter rebuttal rightly points out that any AI tool (especially a legal research and professional practice platform) has to be fit for purpose, and trained on appropriate data.

Aside from the Stanford research, some lawyers have been found to have relied upon AI tools such as ChatGPT and Google Bard to draft their submissions, only to discover that the results have cited non-existent precedents and cases – including in at least one high-profile prosecution. The latest research suggests that not only do AI tools “imagine” fictitious case reports, they can also fail to spot “bad” law (e.g., cases that have been overturned, or laws that have been repealed), offer inappropriate advice, or provide inaccurate or incorrect legal interpretation.

What if AI hallucinations resulted in the generation of invidious content about a living person – which in many circumstances, would be deemed libel or slander? If a series of AI prompts give rise to libelous content, who would be held responsible? Can AI itself be sued for libel? (Of course, under common law, it is impossible to libel the dead, as only a living person can sue for libel.)

I found an interesting discussion of this topic here, which concludes that while AI tools such as ChatGPT may appear to have some degree of autonomy (depending on their programming and training), they certainly don’t have true agency and their output in itself cannot be regarded in the same way as other forms of speech or text when it comes to legal liabilities or protections. The article identified three groups of actors who might be deemed responsible for AI results: AI software developers (companies like OpenAI), content hosts (such as search engines), and publishers (authors, journalists, news networks). It concluded that of the three, publishers, authors and journalists face the most responsibility and accountability for their content, even if they claimed “AI said this was true”.

Interestingly, the above discussion referenced news from early 2023, that a mayor in Australia was planning to sue OpenAI (the owners of ChatGPT) for defamation unless they corrected the record about false claims made about him. Thankfully, OpenAI appear to have heeded of the letter of concern, and the mayor has since dropped his case (or, the false claim was simply over-written by a subsequent version of ChatGPT). However, the original Reuters link, above, which I sourced for this blog makes no mention of the subsequent discontinuation, either as a footnote or update – which just goes to show how complex it is to correct the record, since the reference to his initial claim is still valid (it happened), even though it did not proceed (he chose not to pursue it). Even actual criminal convictions can be deemed “spent” after a given period of time, such that they no longer appear on an individual’s criminal record. Whereas, someone found not guilty of a crime (or in the mayor’s case, falsely labelled with a conviction) cannot guarantee that references to the alleged events will be expunged from the internet, even with the evolution of the “right to be forgotten“.

Perhaps we’ll need to train AI tools to retrospectively correct or delete any false information about us; although conversely, AI is accelerating the proliferation of fake content – benign, humourous or malicious – thus setting the scene for the next blog in this series.

Next week: AI and Deep (and not so deep…) Fakes

 

 

 

 

Whose side is AI on?

Posted on by
Reply

At the risk of coming across as some sort of Luddite, recent commentary on Artificial Intelligence suggests that it is only natural to have concerns and misgivings about its rapid development and widespread deployment. Of course, at its heart, it’s just another technology at our disposal – but by its very definition, generative AI is not passive, and is likely to impact all areas of our life, whether we invite it in or not.

Over the next few weeks, I will be discussing some non-technical themes relating to AI – creativity and AI, legal implications of AI, and form over substance when it comes to AI itself.

To start with, these are a few of the questions that I have been mulling over:

– Is AI working for us, as a tool that we control and manage?  Or is AI working with us, in a partnership of equals? Or, more likely, is AI working against us, in the sense that it is happening to us, whether we like it or not, let alone whether we are actually aware of it?

– Is AI being wielded by a bunch of tech bros, who feed it with all their own prejudices, unconscious bias and cognitive limitations?

– Who decides what the Large Language Models (LLMs) that power AI are trained on?

– How does AI get permission to create derived content from our own Intellectual Property? Even if our content is on the web, being “publicly available” is not the same as “in the public domain”

– Who is responsible for what AI publishes, and are AI agents accountable for their actions? In the event of false, incorrect, misleading or inappropriate content created by AI, how do we get to clarify the record, or seek a right of reply?

– Why are AI tools adding increased caveats? (“This is not financial advice, this is not to be relied on in a court of law, this is only based on information available as at a certain point in time, this is not a recommendation, etc.”) And is this only going to increase, as in the recent example of changes to Google’s AI-generated search results? (But really, do we need to be told that eating rocks or adding glue to pizza are bad ideas?)

– From my own experience, tools like Chat GPT return “deliberate” factual errors. Why? Is it to keep us on our toes (“Gotcha!”)? Is it to use our responses (or lack thereof) to train the model to be more accurate? Is it to underline the caveat emptor principle (“What, you relied on Otter to write your college essay? What were you thinking?”). Or is it to counter plagiarism (“You could only have got that false information from our AI engine”). If you think the latter is far-fetched, I refer you to the notion of “trap streets” in maps and directories.

– Should AI tools contain better attribution (sources and acknowledgments) in their results? Should they disclose the list of “ingredients” used (like food labelling?) Should they provide verifiable citations for their references? (It’s an idea that is gaining some attention.)

– Finally, the increased use of cloud-based services and crowd-sourced content (not just in AI tools) means that there is the potential for overreach when it comes to end user licensing agreements by ChatGPT, Otter, Adobe Firefly, Gemini, Midjourney etc. Only recently, Adobe had to clarify latest changes to their service agreement, in response to some social media criticism.

Next week: AI and the Human Factor

Victorian Tech Startup Week – Pitch Night

Posted on by
Reply

As part of the recent Victorian Tech Startup Week, Silicon Beach Melbourne and YBF Melbourne hosted the city’s first in-person pitch night for over a year (thanks to the 3 lock-downs we have had in that time). Compered by Karen Finch of Legally Yours, and supported by OVHcloud, the esteemed judges for the evening were Farley Blackman (YBF), Yian Ling Tan (OVHcloud) and David Hauser (Silicon Beach).

The usual Silicon Beach rules applied – Round One featured 90-second pitches from each founder (and no slide decks), from which the judges shortlisted 3 startups for Round Two. The Round One presentations in order of appearance were (as usual, website links are embedded in the names):

TwistedXeros.com

Using “emotional phase shifting to accelerate personal growth and transformation through Insight, Manifestation and Neuroscience”, the impetus for this startup came about from the founder’s own experience. Designed to help overcome certain mental health issues associated with anxiety, the founder claims his technique can help practitioners overcome events such as panic attacks within 6 seconds (as opposed to 600 seconds with traditional CBT methods). Had been accepted into the Founders’ Institute, then COVID came along.

The Leaf Protein Co.

There is a growing demand for plant-based foods, both as a source of sustainable protein, and in response to the increased prevalence of food-based allergies (e.g., gluten and soy). Add concerns about GMOs, unsustainable agriculture and climate change, the founder is looking to develop a scalable process for extracting specific types of leaf protein, including arid-climate plants and Australian natives such as saltbush to counter soil salination. Currently seeking funding to pay for a CSIRO pilot to scale the protein extraction.

E-Toy Library

Essentially a toy-lending app, that provides an end-to-end process (source, distribute, cleanse, circulate) via a subscription model. In trials, already secured 50 customers and over 100 subscribers. Estimates there is a $2.4bn toy market in Australia – but it wasn’t clear how much of this market the founders aim to capture.

Kido Paint

This app aims to bring childrens’ drawings to life, using AI/ML to scan a photo of the drawing, and convert it into an animated 3-D digital file that can be rendered within the app using augmented reality.

Thorium Data

Using the oft-heard tag line “data is the new oil”, this B2B solution is designed to help companies organise, manage and extract more value from their data. It does this by resolving issues of data inconsistency, privacy, risk and governance. It also derives and assigns numerical factors to to individual datasets to assess the “value” of this data, and uses indices to benchmark that value.

QuestionID

This product feels like a cross between a wiki for academic research papers, and an open text search tool to find answers within the wiki database. I know from experience that repositories of published research reports (especially refereed and peer reviewed papers) are highly structured and tagged, with the emphasis being on classification, authorship and citation. Often, you sort of need to know in advance the rough answer to the question you want to pose. Significant resources are already allocated to maintaining and commercialising these existing databases, so I’m not sure how QuestionID will deal with IP and other rights associated with these reference resources.

HiveKeepers

HiveKeepers is designed to support beekeepers by helping them to establish and maintain healthier hives, and enhance their own livelihoods at a time when the industry is facing numerous challenges. At its core is a smart phone app that monitors IoT sensors (temperature, weather, weight, motion, sound, etc.) attached to the hive itself. Over time, the data will enable predictive analytics. With the launch of its MVP, HiveKeepers has already attarcted 700 customers globally.

Round Two

The three finalists selected from Round One were KidoPaint, LeafProtein and HiveKeepers. Each founder made a longer pitch, and then answered questions from the judges:

Kido Paint – The Q&A discussion centred on the commercial model (B2B/C, gift cards, in-app vouchers), the file conversion process (turnaround time can be 24- 48 hours), options for scaling, and getting the right price pint for user prices. So it’s not an instant result (which may disappoint some impatient younger users), and the 3-D rendering and animation is somewhat limited to the imagination of the AI/ML algorithms used in the conversion process.

LeafProtein – There was a further discussion on the approach to producing sustainable and allergen free plant proteins. For example, the attraction of pereskia is two-fold – a higher protein ratio, and an arid climate plant. Also, the aim is to counter mono-culture and GMO crops. A D2C brand has been launched (using small-scale production processes), while the CSIRO project is to designed to scale protein extraction, as well as develop an emulsifier for use in the food industry.

HiveKeepers – The founder talked more about the need to address climatic and environmental impact on hives. Having benefited from support from the La Trobe University and SVG Thrive AgriFood accelerator programs, this startup is seeking funding for product development – current price point is $105 USD per smart hive per annum. While the industry is seeing a 2% growth in new hives, it is also suffering significant hive losses due to parasites and diseases.

The overall winner on the night was LeafProtein.

Next week: From R&D to P&L

Open Banking and the Consumer Data Right

Posted on by
Reply

While most of Australia has been preoccupied by things such as Covid-19 lock-downs, border closures, which contestant got eliminated from Big Brother/Masterchef, and which federal politician went to an NRL game (and depending on which State you live in), the ACCC has implemented the first phase of the Consumer Data Right regime (aka Open Banking).

The TLDR on this new regulation, which has been several years in the making, can be distilled as follows:

Banks can no longer deny customers the right to share their own customer data with third parties.

So, in essence, if I am a customer of Bank A, and I want to transfer my business to Bank B, I have the right to request Bank A to share relevant information about my account to Bank B – Bank A can no longer hold on to or refuse to share that information.

Why does this matter? Well, a major obstacle to competition, customer choice and product innovation has been the past refusal by banks to allow customers to share their own account information with third party providers – i.e., it has been an impediment to  customer switching (and therefore anti-competitive), and a barrier to entry for new market entrants (and therefore a drag on innovation).

Of course, there are some caveats. Data can only be shared with an accredited data recipient, as a means to protect banking security and preserve data privacy. And at first, the CDR will only apply to debit and credit cards, transaction accounts and deposit accounts. But personal loans and mortgages will follow in a few months. (And the CDR is due to be extended to utilities, telcos and insurance in coming years – going further than even the similar UK Open Banking scheme.)

Although I welcome this new provision, it still feels very limited in application and scope. Even one of the Four Pillar banks couldn’t really articulate what it will actually mean for consumers. They also revealed something of a self-serving and defensive tone in a recent opinion piece:

“Based on experience in other markets, initial take up by consumers is likely to be low due to limited awareness and broader sensitivities around data use.”

Despite our fondness for bank-bashing (and the revelations from the recent Royal Commission), Australians are generally seen as being reluctant to switch providers. Either because it’s too hard (something that the CDR is designed to address), or customers are lazy/complacent. In fact, recent evidence suggests existing customers of the big four banks are even more likely to recommend them.

For FinTechs and challenger brands, the costs of complying with some aspects of the CDR are seen as too onerous, and as such, act as another impediment to competition and innovation. Therefore, we will likely see a number of “trusted” intermediaries who will receive customer data on behalf of third party providers – which will no doubt incur other (hidden?) costs for the consumer.

Full competition will come when consumers can simply instruct their existing bank to plug their data into a product or price comparison service, to identify the best offers out there for similar products. (Better still, why not mandate incumbents to notify their existing customers when they have a better or cheaper product available? A number of times I have queried the rate on an existing product, only to be offered a better deal when I suggested I might take my business elsewhere.)

Recently, my bank unilaterally decided to change the brand of my credit card. Instead of showing initiative by offering to transfer my existing subscriptions and direct debits to the new card, the bank simply told me to notify vendors and service providers myself. If I didn’t request the change of card, why am I being put to the inconvenience of updating all my standing orders?

For real innovation, we need banks and other providers to maintain a unified and single view of customer (not a profile organised by individual products or accounts). Moreover, we need a fully self-sovereign digital ID solution, that truly puts the customer in charge and in control of their own data – by enabling customers to decide who, what, when, why and for how long they share data with third parties. For example, why do I still need 100 points of identity with Bank B if I’m already a client of Bank A?

Finally, rather than simply trying to make money from managing our financial assets, banks and others have an opportunity to ensure we are managing our financial data in a more efficient and customer-centric way.

Next week: Counting the cost of Covid19