Literate Programming

The advent of Large Language Models (LLMs) in software developer's everyday lives offers possibilities which can transform the entire field. This, however, is not restricted to Computer Science. For the use as tools in the natural and engineering sciences, mathematical models and algorithms need to be transformed into instructions executable by computers, and therefore also the way we solve research problems in these fields will inevitably change.

In my opinion LLMs offer the chance of increasing the quality of software. We must, however, be very careful not to fall into the trap of using these new tools to produce software faster (or even worse: produce more software) rather than improving the code quality. Apart from this chance and risk, there is a chance even more important - the unique chance to improve the quality of software development itself. In the light of research dependent on algorithm and code development this means the chance to not only increase the quality of research results but to increase the quality of the research process. One possible way to do that is based on the concept of literate programming. In the following, I will present my thoughts and give an illustrative example.

The term "Literate Programming" was coined by Donald Knuth. For the discussion here, let us focus on a quote from his paper

D. E. Knuth, Literate Programming
The Computer Journal, Volume 27, Issue 2, 1984, Pages 97–111 https://doi.org/10.1093/comjnl/27.2.97

"Let us change our traditional attitude to the construction of programs. Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do."

Knuth's idea was to write computer code and its documentation in the same document, in an order easily understandable by humans. Before compiling, the literate program was pre-processed in two steps called weaving (generation of the human readable documentation) and tangling (generation of the computer readable instructions) from the same source (the literate program).

By using LLMs as the "translator from human language to computer instructions" one can go a step beyond the original idea of literate programming. In fact, if the LLM is capable of understanding the task it shall solve in the language humans would use to explain the task to other humans, we can omit the part of writing source code within the literate program. In the following, I will present this approach by means of an example.

An example for literate programming: Construction of matrix groups

While writing my PhD thesis, I worked a lot on finite groups, in particular on finite subgroups of U(3), the unitary group in three dimensions. These groups are finite groups of unitary 3 x 3-matrices. The simplest unique characterisation of such a group is by giving a set of generators, i.e. a set of elements of the group which generates all group elements by multiplication. What I wanted to do numerically is, given a set of unitary 3 x 3-matrices, determine the order of the generated group and check whether the representation (defined by the matrices) is irreducible. For my thesis (for the paper on the finite subgrous of U(3) of order smaller than 512) I used this program to verify the group order and irreducibility of representations generated by the computer algebra system GAP (Groups, Algorithms and Programming).

In the following, a literate program to solve this problem is presented. I wrote it in markdown, you can find the markdown code here. Converting the markdown file to html using pandoc (pandoc -f markdown --mathml -t html5 -o output.html Groups.md) allows to display it without the need for a markdown extension: Literate program: Checking the order of a matrix group. Note that for all parts of the literate program I also specified unit tests to be implemented. This helps when checking whether the code generation was correct (and of course also helps in detecting errors in the literate program itself). The markdown file was translated to computer source code in different languages by an LLM (OpenAI GPT 4 Turbo). I tried translation to three different languages:

• Python: This translation went very smooth, and almost no asking for corrections was necessary. The generated code [download code] worked out-of-the-box and also the unit tests [download tests] passed. The result and run time of the main program is:

  The order of the group is 648.
  The representation is irreducible: True.
  Time taken for computation: 1971.2705 seconds.

• Julia: This translation needed some interaction, but was still very easy, despite me having worked with Julia only very little before. Here you can look at the generated Julia code [download code]. Executing it gives:

  The order of the group is: 648
  The representation is irreducible: true
  Time needed for the computation: 14.103000164031982 seconds

  As you can see, the generated Julia code is more than 100 times faster than the python code. I suspect that this at least partly is due to the more efficient check

  function isInList(matrix, list; tol=1e-9)
      for item in list
          if all(abs.(matrix - item) .< tol)
              return true
          end
      end
      return false
  end

  compared to the corresponding generated python code

  def is_in(matrix, matrices):
      return any(matrix.are_numerically_equal(m) for m in matrices)

  which checks all list elements instead of immediately returning when an equal element has been found.
• Rust: Also this translation needed some interaction, but also required not too much interaction with the LLM to make it correct the errors. Here you can view the generated Rust code [download code]. Executing it gives:

  The order of the group is 648
  The representation is irreducible: true
  Time needed is: 13.5238412s

To summarize: All three code generations worked well. The time needed to generate the code was much less than writing the literate program. And writing the literate program did not take longer than it would have needed to write the code (for one of the languages) directly. But the literate program may serve as the basis to generate code for any language and it is much better understandable by human beings than the computer code. This makes programming a more enjoyable task and allows to concentrate on the actual problem to be solved rather than how to implement it in a specific programming language.