Editorial by Mireille Hildebrandt

Privacy and data protection cannot be taken for granted. They must be organised and embedded in technical systems that would otherwise expose our dependencies and vulnerabilities. The integration of fundamental rights protection in models, applications and critical infrastructure requires basic and detailed understanding of the abilities and the limitations of these systems. Blind rejection or naïve embracing of generative AI will not do. Lawyers must get their act together and develop a genuine interest in asking the right questions and in learning how to test the behaviour and contest the decisions of AI models, products and services. This will require keen attention to their theoretical underpinnings, such that their reliability and fairness can be (con)tested, if needs be in a court of law. 

Legal contestability and scientific falsifiability

At the LSTS/BPH Research Day of 24 June 2025, I discussed the link between legal contestability and scientific falsifiability. The resulting paper can be found at the repository of the Open Science Foundation (SocArXiv) (to be published in the proceedings of the 2025 LSTS Research Day). The paper argues that at the end of the day, in scientific research, nothing is as practical as good theory. It opens with a reminder of how adversarial expert evidence can lead to new scientific methodologies, referring to Science at the Bar by Sheila Jasanoff, who, as I explain in the paper,

demonstrated how the contestability that is key to the adversarial trial turned out to be key to scientific practice. Notably with regard to DNA fingerprinting, ‘[i]n an effective display of boundary work, DNA fingerprinting was originally represented by its proponents as a taken-for-granted technique belonging only to the fields of molecular genetics and molecular biology’, Due to adversarial expert testimony in a range of court cases, culminating in the O.J. Simpson case, it turned out that without integrating population genetics DNA, fingerprinting was unreliable and in point of fact biased against black defendants.

I concluded that

[p]erhaps it is time to test the reliability of so-called AI models, deployed in a whole range of sectors and applications, based on appropriate scientific methodology, if needs be in a court of law.

The GDPR right to an explanation and cross-disciplinarity

As many have asserted, the deployment of AI systems to make decisions that impact individuals and/or public infrastructures, such as healthcare, education, housing, insurance, finance and employment, often falls within the scope of the GDPR and should be tested against its legal requirements. At the same time, the AI Act and the Digital Services Act (DSA) have become relevant for the protection of fundamental rights such as privacy and data protection, instigating a whole series of legal obligations for the providers and deployers of AI systems, making sure that their output is contestable in a court of law when needed.

The GDPR’s right to an explanation in the case of automated decision-making has already given rise to a subdomain in computer science, called ‘explainable AI’, demonstrating the point made by Jasanoff above. In the above mentioned paper, I argue that the type of explanations that are at stake, do not solve the problem of scientific falsifiability, suggesting that we may need a type of legal contestability that would have major impact on scientific practice, contributing to the falsifiability of scientific theories that inform AI systems or models.

I would claim, that such an undertaking is not a matter of interdisciplinary legal scholarship, but hinges on the cross-disciplinary engagement of legal scholars and legal practitioners, who are willing to understand the intricacies of ‘the other discipline’ insofar as relevant, notably in the context of an adversarial or contradictory procedure. The idea here is not that other disciplines impose their assumptions and methods on legal practice or legal scholarship (as happened with ‘law and economics’), but, on the contrary, that the findings of other scientific practices are put to the test in a court of law. In that sense, law is not merely one of many different scientific disciplines, but first and foremost a dedicated system of checks and balances, meant to compensate power asymmetries and unearth inconvenient truths, if necessary.

When I applied for my ERC Advanced Grant on ‘Counting as a Human Being in the Era of Computational Law’ (COHUBICOL, 2019-2024), I proposed a dedicated methodology to bridge between legal research and computer science, and between legal practice and the development of legal technologies. Instead of advocating an ‘interdisciplinary’ method, I presented a ‘cross-disciplinary’ approach. This was part of the targeted impact of the project on both law and computer science, inviting a genuine inter-esse between both disciplines and both practices, thus achieving a key innovation of legal method.

Traduire, c’est trahir

Those familiar with translation between different languages know that traduire, c’est trahir, meaning that translating is necessarily transformative, fitting the text that must be translated in the context of another language and thus in another ‘world’. This is how nuances, connotations and even denotations get ‘lost in translation’, while simultaneously new linguistic and real-world associations will emerge, wittingly or unwittingly. Those who can read the translation but not the original will not notice what was lost, nor what was gained. And while such transformations may be enriching, confusing or both, the translation will begin a life of its own, unhindered by an awareness of what it does to the original meaning.

This is not necessarily a problem, unless a genuine understanding is needed, for instance when translating a contract, law or deed. Especially where power asymmetries reign, it is important to ensure that a translation respects the implied consequences of a text, for those vulnerable to manipulation. When translating legal text corpora into digital data, or legislation into executable code, legal principles may get lost, while efficiency gains may be won.

In scientific as well as in professional practices, a dedicated practice will have its own disciplinary language, requiring the relevant disciplinary literacy. In law, for instance, privacy has a specific meaning that matters because the violation of privacy has a legal effect. A computer scientist, however, may define privacy with the goal of formalising the concept such that they can translate it into executable code; this results in another meaning. This is one of many reasons why ‘trusting’ compliance software may not result in actual compliance. When deploying such software, we’d better engage in ‘constructive distrust’, foregrounding the contestability that is key to the law in our dealings with automation and software compliance.

The ‘ordinary meaning’ of words may actually be irrelevant when it comes to fluency in a disciplinary language, because it is built on sets of domain-specific assumptions and focused on specified goals. A disciplinary practice owes its findings to detailed methodologies, meaning that terminology is defined in a specific and detailed way that differs from everyday speech. The assumptions, goals and methodologies form a complex interrelated ‘whole’ that requires training and in-depth study to master the level where such findings count as the findings of a specific discipline or practice.

Trying to do computer science based on the assumptions, goals and methodologies of the law, or vice versa, would be useless if not ridiculous. Trying to practice law, based on the assumptions, goals, and methodologies of computer science, would denaturalise legal practice and destabilise the checks and balances that protect against undesirable dependencies. This is precisely why legal scholars who engage in cherry-picking findings from other disciplines, whose internal controversies and methodological disagreements we cannot understand without in-depth study, may disrupt both the study and the practice of law.

Against the colonisation of legal method: legal certainty, justice and instrumentality

Though ‘interdisciplinarity’ and the finding of a ‘common language’ to bridge disciplinary boundaries sound like laudable endeavours, they easily end up by betraying either both disciplines in an ‘interdisciplinary soup’, or by betraying one discipline that is ‘colonised’ by the other. Let me explain.

Terminology in law is part of a web of meaning, while meaning in law refers to the legal effect that is at stake. That web starts with legislation and/or case law and is constantly refabricated in doctrine or restatement, that weaves together what may otherwise seem incoherent. The need for coherence is instigated by the idea of legal certainty that demands that those who share jurisdiction must be able to foresee the consequences of their actions, because otherwise they will not be able to act, in the end. If ‘privacy’ is defined differently, depending on the whims of whoever should protect it, with courts asserting such self-serving behaviour, we face arbitrary decision-making and protection becomes a scam. At the same time the need for coherence should not be confused with logical consistency, because the law cannot be reduced to coherence.

There is also justice, that forms the vanishing point of modern positive law, requiring that equal cases are treated equally while unequal cases should be treated unequally to the extent of their inequality. Justice as equal treatment requires discernment, to decide which elements or dimensions of a specific case are relevant for the decision on equality. Discernment, however, is both more and less than logic, asking for experience and judgement, which is not a matter of calculation, but something more subtle and more complex.

Finally, the law has goals for which it serves as an instrument, such as high-quality healthcare, access to education, economic prosperity, access to justice, fair treatment and the ability to contest legally relevant decisions. Again, logic cannot do the job here, because these goals may be incompatible in concrete situations, requiring reasonably motivated choices based on legislation, prior case law, doctrinal reasoning and discernment of what takes precedence and why. Law is an argumentative practice. It cuts knots when parties disagree, and though it is not about a final judgement in the religious sense, it is meant to provide a pragmatic form of closure to enable people to move on. However, it demands contestation, hesitation, consideration and delay, before cutting that knot. Such contestation and consideration enables refining the norm that should hold things together for future cases.

In the context of data-driven AI, we should assert that legal certainty, justice and the instrumentality of law are not a matter of stochastic regularities but of overlapping elements, grounded in a normative framework that cannot and should not be reduced to probabilistic distributions in legally relevant training data.

Cross-disciplinarity

Cross-disciplinarity respects the assumptions of e.g. computer science, notably the assumption that real-world phenomena can be reduced to digital data and/or inferred from them and, and the assumption that in the case of machine learning, the distribution of training data is the same as that of future data. Computer scientists should be aware that these assumptions do not fly but nevertheless contribute to the development and deployment of AI systems that – due to their extraordinary computing power – may be able to predict behaviours, e.g. of the weather, generate text based on stochastic relations between word-tokens and enable robotic systems to adapt to their physical environment. Computing systems work with computational models of reality and, as the saying goes: ‘all models are wrong but some are useful’.

For computer science research, the assumptions invoked are necessary, and as long as we are aware ofsuch assumptions, the limitations of building on  there need not be a problem. In this context, cross-disciplinary research by lawyers would imply that they learn to engage with computer science, based on the assumptions, goals, and methodologies of computer science, without, however, integrating them into their own discipline. The latter would imply that we can now ‘solve’ law, or better understand law, due to the use of computer science methods in law. Instead of allowing our discipline to be colonised by computer science, we can nevertheless learn from the output of computational technologies, while taking a critical stance, based on the assumptions, goals, and methodologies that are key to the law.

We may learn how to safeguard confidentiality in an online environment, how to obtain reasonably reliable summaries of legal text corpora, how to detect different types of bias in draft judgments or legislative proposals, how to enhance search in legal databases and how to enable and secure storage of legal documents and communication between actors within the administration of justice. Taking a critical stance, we can put the computational methods to the test by exploring alternative models or systems, asking a series of relevant questions, comparing outputs, and requiring sufficient transparency and falsifiability.

A Typology to test and contest automation in law

In the context of the COHUBICOL project, we developed a Typology of Legal Technologies, where we demonstrated the salience of digging deeper into the assumptions and implications of these technologies, based on a proper understanding of their computational background – in relation to what we expect of legal method in terms of legal certainty, justice, and instrumentality, including procedural fairness and contestability.

Some of the legal technologies we researched concerned compliance software, which offers a way for data controllers and/or providers and deployers of AI systems to outsource compliance with their legal obligations. An interdisciplinary legal mindset that is fine with importing convenient outputs from other disciplines, based on an uncritical trust in the scientific robustness of such outputs, should be avoided.

We must learn, as lawyers, to dig deeper into the claims made on behalf of compliance software, while reminding data controllers, providers, and deployers that compliance at the technical level is key but cannot be outsourced to a technology. Not only because technologies lack the kind of agency that is required for accountability, but also because compliance software, supposedly debiased AI models, and allegedly fair automated decision systems must be contestable and tested on a running basis to make sure they are reliable. This requires a cross-disciplinary legal mindset that fosters curiosity towards the assumptions, goals, and methodologies of computer science, thus finding out what adversarial questions must be raised when the contestability of AI models, compliance software, and automated decision systems is at stake.