Whetstone of Witte, complexity and programming

This article is about using large language models (LLMs) to write code, but gets there indirectly.  First it talks about what makes things, such as programming and maths, hard and what doesn’t via a couple of historical things.  Next, I will expand on complexity – how much is baked into some activity, and how much comes along for the ride via how we happen to do things in the real world.  Then I will go into problems and their solutions – how common are the problems and how many different solutions are there to them?  These lay the foundation for how LLMs and code do

Equations using words or symbols

In 1557 Robert Recorde published a book called The Whetstone of Witte. (Witte is just a different way of spelling wit.)  Among other things, it’s the first book in the English language to use the equals sign.  Page FF.i.v has some algebra problems that are introduced with the following text:

To many modern readers it’s hard to understand.  Not only is the font hard to read, but the spelling is a bit wobbly and there are shortenings and punctuation idiosyncrasies.  Decoded into a more modern form:

Nowbeit, for easy alteration of equations. I will propound a few examples, because the extraction of their roots may the more aptly be wrought. And to avoid the tedious repetition of these words is equal to I will set as I so often in work use, a pair of parallels, or Gemowe lines of one length, thus: ===== because no 2 things can be more equal. And now mark these numbers.

14z + 15y = 71y

Note that by modern standards the plus and equals signs are both very wide.  Gemowe comes from the same root as Gemini, meaning twins.

Before the equals sign was available, authors had to write is equal to, which could be in the Latin aequales, sometimes abbreviated to aeq.

Code using words or symbols

The next historical thing is COBOL, short for COmmon Business Oriented Language. It’s a programming language first designed in 1959, which was intended to be read and even written by non-programmers.  A lot of money still flows through COBOL code running on bank mainframes, so it’s still an important language.

One unusual feature of COBOL is its use of words where other programming languages would use symbols.  If I wanted to work out a total cost, where I had some number or quantity of things, and a rate per thing (called unitRate below), in most languages I would write something like this:

totalCost = quantity * unitRate

In COBOL I would write it out in prose, like this:

MULTIPLY quantity BY unitRate GIVING totalCost

There is no equals sign or asterisk.

What makes programming and maths hard

I find that the extract from The Whetstone of Witte (WoW) and COBOL make an interesting pair.  In WoW, writing out the words each time is tedious, and the equals sign is not only easier but also clear because its form reflects its function (two equal lines show that two things are equal).  With COBOL, words are meant to be less alien or confusing to the target audience than the equivalent symbols would be, even though this would be more typing i.e. more tedious.

Unfortunately, surface-level things like whether you use words or symbols to represent maths aren’t the only barriers to people understanding programs and being able to write them.  There are other issues, such as:

• What the program should do is understood in only a high-level and fuzzy way.
• Different people have different views of what the program should do.
• Given a computer’s lack of common sense, all kinds of edge cases need to be thought of and the details worked through.
• There are important issues that cut across lots of the program in ways that are hard to build and keep track of.  These are things like security, performance, usability, reliability and so on.

Similarly, people who find maths hard don’t usually find all their troubles go away if they write is equal to rather than using an equals sign.  It’s the concepts and techniques that cause much of the problem, not the way they are expressed.

Complexity

We come now to the issue of complexity.  Here I mean this in the normal sense of the word, rather than the more computery way that describes how quickly the difficulty of a task grows as the task grows.  When you look at some code or text, it can be hard to understand.  Where does that difficulty come from?

I find it helpful to divide the difficulty into two groups, in a way I first read described by Fred Brooks in his book The Mythical Man Month.  The first group is essential complexity, which is complexity that is built into a problem.  The second group is accidental complexity, which is complexity that at least in theory doesn’t have to be there.

Imagine I have some text that describes how to tie your shoelaces.  This text is clear and easy to understand.  I then edit the text so that it has lots of irrelevant asides, unclear sentences, unnecessarily long and obscure words etc.  The first version of the text has minimum accidental complexity, and however much essential complexity there is in the task of tying shoelaces.  The second version has the same amount of essential complexity (because it’s still talking about tying shoelaces), but more accidental complexity.

When you look at some code or some text and it’s hard to understand, there are two extremes to the reasons why.  At one extreme it’s like a hedge made of brambles, where there’s lots of wicked essential complexity.  At the other extreme it’s more like a hedge of candyfloss, where a small amount of essential complexity is puffed up with a lot of accidental complexity.

This is another way of describing the issues of the previous sections – whether you use words or an equal sign is part of the accidental complexity of maths and programming.  There’s a core lump of essential complexity that will always be there.

Problems and their solutions

There are some problems that occur fairly often and have reasonably common solutions.  The kinds of thing I mean are: have 2kg of freshly-made puff pastry, or a list of numbers sorted in ascending order.

This means that when LLM builders harvest data that will train the LLM, they are likely to find some problems more often than others, and some solutions more often than others.  This is likely to increase the LLM’s performance on those problems.

There are other kinds of problems that are common, but don’t have a single common solution.  This is things like: have a system for organising my To Do list.  There’s a core that most people would expect – a way of entering and editing tasks, viewing a list of them, and marking them as complete.

However, beyond that it quickly gets much fuzzier.  Depending on whom you ask it could include:

• Optional deadlines for tasks
• Notifications when deadlines are closed or have passed – to email, Slack, Teams etc.
• Optional tags, colours, groups etc. for tasks
• Filter tasks by their tag, colour, group etc.
• Create tasks by sending an email address to a magic address
• Offline mode
• Etc.

Adding everything up

While some of the difficulty in writing code is due to the accidental complexity, much of the difficulty is the essential complexity.  The essential complexity is often the human stuff – the arbitrary rules that people have decided are important.  It’s hard enough for humans to understand and translate this into code, even with the benefit of common sense, native understanding of human language etc.

Your meaning of To Do list might not be the same as mine.  Should an LLM create just yours, just mine, or something that’s a union of both?  Could this union still be as easy to use as just yours or mine?

LLMs can be good at learning common patterns, then spotting when they’re reoccurring and so apply them again.  But they don’t understand much beyond the symbols they were trained on, so the essential complexity of the bit of human life being modelled is often beyond them.