Addison Cameron-Huff, Cryptocurrency Lawyer

An alternative title for this blog post could be: Assumptions About Law That Surprise Engineers (And Some Lawyers).

Over the many years I've been a lawyer, I've had a lot of conversations with engineers and businesspeople who think that lawyers are tricking them about how complicated law is. Or, they think that software can do it better. Although both of these things may be true, they're not very important in understanding why there's no computer that can spit out the answers to legal questions. The dream that everyone has is to know the law. Why is that so hard? Isn't that already the case? This blog post seeks to answer that question, distilling down some of the many conversations I've had with engineers over the years.

Who Am I?

Who am I to answer this question? I'm a lawyer with a decade of experience who's also a programmer and the developer behind the world's largest search engine for international law (global-regulation.com, which I started with Dr. Nachshon Goltz back in 2014 - he runs it today). I also started and sold a business that provided near real-time monitoring for keywords for the Ontario and federal governments. In short: I've thought a lot about these problems and I too would like a computer that knows the law. But this dream is sadly quite far from reality. This is shocking to many people, who assume this already exists and it's merely a matter of assembling what exists into a nice UI.

How Programmers Imagine The Law

People who aren't lawyers generally assume that the law works something like the diagram below.

The truth is something much closer to the diagram at the top of this post.

How Decisions Are Made: If You Could Know The Raw Materials

People cannot know the raw materials that make up the law. This is a bold statement, which I explain further down in this article. But for the sake of argument, let's assume that's not true and it would be possible for a business to amass all of the required legal rules. Even if this could be done, a legal machine could not be built that would accurately align with reality. It woudl fail to predict the outcomes of the cases it's presented with. Here I mean cases in the very broadest sense, which is the legal decisions that happen countless times in the day between government and the governed. There are also of course legal decisions that happen between private actors, within the context of government law, but this article assumes government-person as the dominant interaction of interest.

Why couldn't the law be known? There are dozens and dozens of reasons, many of which are a surprise to people who haven't had much interaction with legal decisionmaking:

1. Laws in modern countries are far more than just statutes and regulations. They include various kinds of government guidance documents, procedural rules, and similar sorts of quasi-legal documents.

2. The meanings of words change over time. Sometimes to achieve certain outcomes, but also due to changes in society that affect meanings. Laws are rarely rewritten to account for society moving on.

3. Discretion is rampant and adds a high degree of randomness.

4. Adjudicators are not very good or consistent. Not everyone gets the Supreme Court of Canada when they need a decision to be made. Most people can't afford to appeal, and the vast majority of legal decisions that affect people are so expensive to challenge that there isn't even an opportunity to complain.

5. Law is incredibly complicated. Sometimes the people who pass the laws don't even understand what rules they're making. Laws are frequently poorly drafted and full of ambiguity (whether intentional or not).

6. Many laws are never enforced, so the real law may be something other than what a layperson might think. To know what law is actually in force and alive is sometimes a trickier exercise than people might think. In some cases there can be several different lines of legal thinking that could apply to the same problem.

7. Courts are usually poorly resourced. In some countries they receive significant investigatory assistance, and in others they're largely left to sort it out for themselves based on what the parties present. The methods differ greatly by jurisdiction, but in all countries, there are limited resources for adjudication, which leads to mistakes (or derogations from the law).

8. There's no comprehensive data that connects laws with outcomes. Court decisions are just one artifact. The overall connections between laws and the adjudicators is often not known to the participants. Sometimes court files are locked up, other times they're not online, and in other cases the actual outcome is simply unknown.

9. Out of court settlements or agreements are common, but usually not known to those outside of the settlement.

10. Law is often a tool of politics. Even when justice is blind, the people who administer it have their own biases. It's almost impossible to escape. This can be seen in the different decisions made about the same facts, even given the same law, in different decades. Gay people were once prosecuted throughout Canada and now there are rainbow flags flying at government buildings. Marijuana users were once arrested by the tens of thousands in Canada every year and now the Ontario government is one of the world's largest wholesalers. Real life is full of examples of changes in politics and society that are hugely influential.

11. The people within the legal system who interpret the laws are often not very skilled in the subject matter or have deficiencies in their education about the rules themeselves. Imperfection is rampant. Since these people are a part of how the law is understood, there is variance in even saying what the law is.

12. Decisions are not neutral outputs of the machine of law. The people who make up the machinery are often keen to see certain purposes achieved, and they bend existing rules to mean what they'd like.

13. What is called a "test" by a court is usually just a rule of thumb. It's rare to find legal tests that can be actually followed from start to finish.

14. Facts are very often contested. Adjudictors have limited information available to them and this will cause some decisions to be wrong. Even a perfect machine that can fully know the law would make errors because of this problem. What seems like a simple fact may well be an opinion, and it's common that no one knows what really happened or even how things are right now. Courts regularly admit statistical evidence that is wrong or take judicial notice of incorrect facts. A court in Ontario recently took judicial notice that toilets flush in the opposite direction on the other side of the world (this is a common myth), and they did so in the context of criticizing other judges for not applying judicial notice properly!

The above are just a few examples of factors that lead to a departure from what a machine might say the law means. So long as humans are in the mix, and so long as law is not available in the form of concrete rules, the dream of a law machine will remain a dream.

Unknowable Law

The law is not actually knowable. It's a legal fiction that it is, and that's a fiction that many have bought into. When I made Global-Regulation with my business partner, we often found potential customers puzzled that they would even have to pay for search a service. They would usually say that Google has that. We'd then ask them to try searching for a Vietnamese law on Google or try to search French law in English and they'd soon get the point. But many people continue to imagine that law is easily available when it isn't. They've never tried and they're confident in their assumption. Surely it must be the case! It is not the cas.e

What do I mean by unknowable law? There are many ways that law can be unknowable:

1. Simply not online. This is less of a problem for statutes and regulations in advanced countries, but in many places it's a big problem. And even in advanced countries, many older laws are not online.

2. Laws online are not always (or even often) current. They're frequently consolidations, which may be only infrequently updated so the online law is out of date.

3. Pronouncements about parts of law being in force are not associated with the law in a timely fashion, or even in a data-oriented fashion.

4. Court decisions are frequently not available online. Some higher courts publish their decisions but many lower courts don't. And the business of most people is found in the lower courts.

5. Government decisions, guidelines, and quasi-legal documents are frequently not organized or published in a standardized fashion.

6. Every country in the world has dedicated significant portions of the law to private, often foreign institutions. That sounds unbelievable! But it's true. These are called technical standards and they frequently require a fee to read. So the public law will reference the technical standard but reading that standard requires paying a fee. This is private law. It's rampant in areas the public is usually not very familiar with, but are very important (e.g. building standards).

7. Law portals made by the government sometimes block search engines from indexing content (like CanLII does), or are structured in such a way that search engines can't index the content.

8. Inconsistent data formats make it impossible to extract data in an automated way (even using LLMs like GPT4). Data is not standardized to any international standard, so even when a parser is written for one jurisdiction, the fields can't be mapped to other places. That fuzziness makes the law unknowable from a programmatic perspective, even if not from a human perspective.

9. The law is generally not clear. This is usually seen as an advantage by the human participants within the system. The degree to which rules really are rules, or are rules that have clear boundaries, varies greatly by law and legal system. Programmers expect that law works like programmatic rules but they're usually more like guidelines.

11. Administrative boards and other types of non-court adjudicators are usually shrouded in secrecy and attempt to not make their decisions known. This is cheaper for them, and that's certainly a motivation, but also they're not required to do it usually by law. More cynically, many sources of law (whether boards or otherwise) know that if people can't see their decisions then they can't criticize those decisions. Data is often simply unavailable, especially for boards that may seem to be of less importance to people high up in government (but that doesn't mean that their decisions are not profoundly important to the people affected, such as workplace injury claims or labour law violations).

12. Some laws only come into force in part, and the parts that are in force can only be known by looking in a different place to look for various pronouncements about which part is in force. This fracturing of data sources, and a lack of digital cross-referencing, makes it very difficult to know what the status of the law is on any given day.

13. It's rare that governments publish APIs or make the law actually available in a digital format. So complex crawlers need to be written to extract the information from the government. This makes the data functionally unavailable, even if it might be technically possible to obtain. And in some cases the government blocks crawlers that aren't Google or Microsoft, so independent organizations or researchers can't actually access the laws.

14. Municipal laws are often very difficult to obtain, or impossible.

15. Some laws are so vast in scope that they're difficult for a computer to understand. For example, municipalities in Canada frequently have hundreds of thousands of zoning rules that apply to properties. In some cases that are special rules for specific addresses. In Toronto this is consolidated as online map, with every property in the city viewable, and the many overlapping zoning rules displayed - but even the map doesn't include every rule. It's common that governments disclaim their online law sites by saying that the information may not be accurate! For example, the screenshot below shows the disclaimer from the Prince Edward Island government about how even the list of what laws exist in PEI is not up-to-date, shouldn't be relied upon, and is only current to two months ago.

Between paywalled technical standards, unpublished tribunal decisions, partly-in-force laws, out-of-date laws online, unavailable court decisions, and unorganized government announcements or guidelines - there's no end to the problems of knowing what exactly the law is. And that's before getting into more academic debates about what counts as law. Even the parts of the law that are unqestionably law such as acts or regulations, can be inaccessible, depending on the jurisdiction. The above are just some of the reasons why knowing the law is not possible in modern Canada, and it's even worse in other places. Until laws are published online in a timely fashion, and comprehensively so, this will continue to be the case.

To Know The Law

To know the law is still impossible in 2023. And a comptuer that tries to explain the law will always suffer from this knowledge problem, and the inherent difficulties in guessing at how imperfect humans would decide on imperfect facts. People are getting closer every day to knowing the law and being able to provide accurate guesses of how an adjudicator would decide. But this still doesn't exist.

It's fair to point out that humans also can't predict the outcomes of adjudicators, but they at least have the benefit of accurate mental models of the various non-law factors that are taken into consideration. Experience can substitute for knowledge, and may very well generate better guesses as to how a decision will go. Or even whether a decision is even possible, given that the vast majority of legal problems are simply unaddressed. There are far more laws on the books than there are judges or investigators. Knowing what might result in actual consequences is a skill that humans will be better at than computers for quite some time.

Academic Commentary

There is an extensive body of academic literature about the rule of law and how decisions are made. There has always been debate about whether or not law is really a set of rules that are followed, or a system for framing power relations. In a law review article from 1995 (The Myth Of The Rule Of Law), law professor and philosopher John Hasnas argues that law is indeterminate to a high degree. He then goes on to ask how it can be that if the law doesn't determine the outcome, so many outcomes can be aligned?

The stability of the law derives not from any feature of the law itself, but from the overwhelming uniformity of ideological background among those empowered to make legal decisions. Consider who the judges are in this country. Typically, they are people from a solid middle- to upper-class background who performed well at an, appropriately prestigious undergraduate institution; demonstrated the ability to engage in the type of analytical reasoning that is measured by the standardized Law School Admissions Test; passed through the crucible of law school, complete with its methodological and political indoctrination; and went on to high-profile careers as attorneys, probably with a prestigious Wall Street-style law firm. To have been appointed to the bench, it is virtually certain that they were both politically moderate and well-connected, and, until recently, white males of the correct ethnic and religious pedigree. It should be clear that, culturally speaking, such a group will tend to be quite homogeneous, sharing a great many moral, spiritual, and political beliefs and values. Given this, it can hardly be surprising that there will be a high degree of agreement among judges as to how cases ought to be decided. But this agreement is due to the common set of normative presuppositions the judges share, not some immanent, objective meaning that exists within the rules of law.

There is surely something to Professor Hasnas's view. Depending on one's political persuasion and life expereinces, it may seem more or less true that the law is indeterminate. But if you take any reasonably complicated hypothetical case to a skilled lawyer you will find that their answer often lacks certainty. Clients are frustrated by that and often believe a better lawyer would know the answer, but the truth is that there are often many answers, and the ultimate disposition depends as much on the messy reality of human lives as it does on the organized writings that make up The Law.