Much of the public commentary has treated this as an embarrassment: the policy meant to govern AI was itself undermined by AI. As a senior lecturer in cyber law, including the regulation of AI, I argue that framing this episode as an embarrassment obscures what needs to be examined. It misses the main point of what is at stake.
by 
Nomalanga Mashinini, University of the Witwatersrand
South Africa’s first attempt to establish a binding artificial intelligence (AI) policy framework came to an abrupt halt just 16 days after it was officially gazetted.
On 10 April, the Department of Communications and Digital Technologies published the Draft South Africa National Artificial Intelligence Policy for public comment.
Journalists checked the references and found that they contained fabrications. These fell into two categories: academic journals that do not exist; and real journals in which the referenced research articles were never published.
Such fabrications are typical of a known generative AI problem called hallucination.
Withdrawing the draft, the communications minister was frank: the problem was not a technical glitch but a failure of oversight. Generative AI was used without proper human verification of the sources, compromising the credibility and integrity of the document.
Much of the public commentary has treated this as an embarrassment: the policy meant to govern AI was itself undermined by AI.
As a senior lecturer in cyber law, including the regulation of AI, I argue that framing this episode as an embarrassment obscures what needs to be examined. It misses the main point of what is at stake.
The hallucinated citations reveal two specific failures. Epistemic integrity (the assurance that research has been conducted through reliable, ethical and repeatable methods that any reader could verify) was absent. So was information integrity (the public’s reasonable expectation that information from an authoritative source can be trusted).
The policy was not equipped to govern either of these failures, and has now itself demonstrated both. This matters because generative AI can be harmful, and its harms are not limited to fake references, but also include fake images, fake videos, fake voices, and the weaponisation of people’s likenesses through deepfakes.
Hallucinations are a known problem of generative AI, the category of AI that produces text and images (audio and visual media) through tools like ChatGPT and Grok.
Hallucinations happen when an AI system, in trying to fulfil a task, produces content that sounds convincing but is inaccurate or entirely fabricated. They are a growing problem:
South Africa’s policy was based on responsible AI governance. Responsible AI needs accountability, transparency, and explainability. These are non-negotiable conditions, echoed by the Organisation for Economic Co-operation and Development principles and the Smart Africa AI Blueprint that the policy draws on.
These governance principles are not just for AI system designers. They bind any institution that uses AI, including use in the production of public documents. The policy failed all three in its own production. The department has some serious questions to answer on all these fronts.
This is an opportunity for the department to gain the trust of South Africans and demonstrate resilient and responsible governance in action.
Accountability calls for a comprehensive explanation of the extent to which the non-existent sources have affected the policy. The department should not proceed to revision without meeting the standards that the revised policy will propose for others.
Transparency demands disclosure. Which sections of the policy are materially affected by the fake sources? Which tool was used? By whom? At which stage of drafting or compiling public submissions did they enter the policy? Was AI used to generate the literature review, the founding values, the synthesis of public comments, or all the above?
The department has not told us.
Explainability demands that we can trace reasoning. The hallucinated sources appear in the reference list, but without a full disclosure from the department, the public cannot know which parts of the policy they were used to support, or how deeply they shaped its foundational priorities and values.
The public comment sections, by contrast, have a verifiable record of where the information came from.
Explainability requires that we can trace what shaped the normative framework of the policy. Without a section-by-section review that informs the public which parts of the policy were affected and to what extent, by the policy’s own standards, the department will have failed both the transparency and explainability requirements.
The retracted policy rightly recognised AI as a tool for inclusive economic growth, capacity development and human rights protection. It also acknowledged that it is a “point of departure” and that sector-specific approaches will be needed.
What must change is how generative AI is treated, both in the production of policy documents and in the mandates the policy creates for synthetic media, such as deepfakes.
These are not problems to be sorted out later at sector level. They are public trust cross-cutting challenges that require their own regulatory logic and governance mechanisms built on cross-sectoral cooperation. The revised policy must incorporate them as a structural pillar, not as a subcategory of innovation governance, but as a problem the state is already living with.
This means designating a specific mandate holder for synthetic media and information integrity. Existing regulatory bodies already hold overlapping jurisdiction over digital content, identity harms, and information distribution.
What is missing is an agreed framework on definitions, remedies, and the steps to be taken when generative AI is used to spread misinformation and disinformation through fake sources and synthetic media.
Mandating that is not a question of creating new institutions. It is a question of political will and policy design.
Acknowledgements: After drafting this article myself, I used Claude to improve the readability of the piece. I personally drafted, verified and reviewed all the substance and sources referenced in it. I take full responsibility for the contents of this article.
Nomalanga Mashinini, Senior Lecturer, University of the Witwatersrand
This article is republished from The Conversation under a Creative Commons license. Read the original article.
The bit about the policy being undone by AI-generated fake research is wild, but honestly it just shows how easy it is to produce convincing-looking content now. I’ve played around with an AI Doll Creator for fun, turning photos into custom dolls, and even that simple tool makes you realize how fast AI can fabricate something that looks real. If we can’t tell the difference in a toy app, policy writers definitely need better checks. AI Doll Creator