deriv/LOG.md

# Development Log

## Initial Project Idea and Planning

I decided to make a basic symbolic differentiation engine.  The
essence of this is that you can provide mathematical expressions as
input, and the program can compute the derivative of them.  For the
sake of simplicity, it will be command-line based, so its input and
output will both be text.  An example interation might look something
like:

```
> 2x^2 + 3x
4x + 3
> x^3 - x^2
3x^2 - 2x
```

### Architecture

The program will have three major components: the reader, which will
take user input and parse it into a convenient internal representation
(generally an AST of some kind); the solver, which will transform the
input expression into an output expression; and the printer which will
format output expressions back to human-readable text.  This is
essentially a standard architecture for a simple interpreter.

![Top-level architecture diagram](./top-level-architecture.png)

The reader will be composed of two passes: a simple state machine
lexer to tokenise user input and a parser to build an AST from the
tokens.  For simplicity here, I will only be supporting basic latin
characters in UTF-8 (i.e. ASCII-compatible text).

Since I am only implementing differentiation, the solver can be
relatively simple, consisting of a pattern-matching loop with an
accompanying ruleset.

The printer is the simplest of the three components and will consist
only of a procedure to walk an AST and emit a serialisation of it
equivalent to the input language.

In addition to the major components, a harness will be needed to tie
them together and handle errors.  This will be in the form of a simple
REPL (Read Eval Print Loop).

### Implementation and Testing Strategy

I'll begin by implementing the reader and printer, as this will allow
me to get to a working end-to-end demo as quickly as possible by
initially skipping the solver and simply printing the input
expressions unmodified.  This will then enable easy and fast iteration
when implementing the solver logic.

I will be writing a suite of unit tests to accompany every module of
the system, following a TDD (Test-Driven Development) methodology.
This system is quite simple and well-defined, so the advantage of TDD
(as opposed to writing tests after-the-fact) is probably relatively
minimal here, especially since it will not be being modified or
maintained after its initial development and will not be being worked
on by multiple people, but the practise will be beneficial regardless
in tightening the feedback loop of the development process and
ensuring that the code is structured to be amenable to testing.  If I
have time I'll also implement some automated integration tests to
verify end-to-end functionality.

I'll be using Rust as the implementation language, as I am fairly
familiar with it and it is well-suited to such tasks.  In addition,
its strong type system and borrow checker are useful in catching many
classes of bugs at compile time and it has unit testing support
provided in the base language, which is convenient for doing TDD.

### LLM / AI Usage

I will not be using any LLM coding assistants for this project.

## Reader

### Lexer

I first implemented the lexer with a very simple finite state machine
consisting of two states, `Init` and `Integer`.  This was
straightforward as the input language is very simple.

### Parser

Since the input grammar consists of terms, parentheses and infix
operators with varying precedence, Pratt parsing is the logical choice
for the implementation of a parser.

I find the usual way of handling associativity in precedence climbing
parsers (wherein a small amount is added to the binding power on one
side) somewhat inelegant, so I decided to implement a simple algebra
for binding powers that handles associativity using custom comparison
semantics.

Implementing the parser itself was straightforward as I have
implemented Pratt parsing many times before (though not in Rust
specifically).

To tie together the lexer and parser, and to provide a convenient
means of decoupling other components from them for testing purposes, I
then implemented the Rust standard library's `FromStr` trait, which is
for types that can be parsed from a string.

## Printer

For expression printing, a simple recursive tree-walking algorithm,
dispatching on the various kinds of expression was sufficient.
Satisfyingly, I found I was able to use the same binding power algebra
I defined for the parser in deciding when to use parentheses, and thus
got correct associativity handling effectively for free, while also
keeping all associativity-handling logic well-factored in one place.

The only complication was in the handling of products, as here there
was some ambiguity as to when to use explicit or implicit
multiplication (that is `x*y` or `xy`).  Products of symbols with
other symbols and integers with symbols are more naturally expressed
with implicit multiplication, but of course this couldn't be used for
products of integers with other integers as this would result in
incorrect serialisations (such as "12" for the product of 1 and 2).  I
ultimately settled on an approach where the right-hand operand is
first printed into a string, and an explicit multiplication sign is
used whenever that serialisation begins with a numerical character
(either a digit or a minus sign).  This nicely handled all of the
cases I could think of and systematically avoids any erroneous
serialisations.

The printing logic could then be packaged up in an implementation of
the Rust standard library's `Display` trait, which is for types that
have a textual representation.

## REPL

At this point, with the reading and printing implemented, it made
sense to create the REPL logic and demo binary so that a full flow
through the program could be demonstrated.  I first created an
`Evaluator` trait to capture the future behaviour of the evaluator,
and then wrote a generic REPL module that's able to work with any
expression type implementing the `FromStr` and `Display` traits.

I was then able to create a demo program by defining a dummy evaluator
that simply returns the input expression unmodified, and running a
REPL using that, connected to stdin and stdout.  This demonstrated a
full round trip through the parsing and printing procedures.

## Evaluator

Rather than trying to handle everything in one complex evaluator, I
decided to implement a multi-pass evaluator.  This essentially means
that the expressions run through multiple different passes that apply
different transformations to them, thereby keeping the complexity of
each pass more managable.  This is a common method used when
implementing compilers and interpreters.

### Normalisation

The first preliminary pass I implemented was a normalisation pass.
This transforms things like `x - y` into `x + (-1)y`, and `x/y` into
`xy^(-1)`.  The main benefit of this is to make it easier for the
later differentiation pass as it will only have to handle sums,
products and exponents (not differences and quotients).  I decided to
leave quotients of integers as-is since these will be untouched by the
differentiation process anyway, and "1/2" reads far better than
"1*2^(-1)".

### Constant Folding

The purpose of this pass is to combine constants together as much as
possible, such as transforming `1 + 2*3` into `6`, or `x + 1 + 2` into
`x + 3`.  This is mostly a nice-to-have rather than an essential
feature, though it will make the output much more readable.

Implementing this pass did get a little hairy, as the binary tree
structure I used for the AST made handling cases like `x + 1 + 2`
(which is represented in the AST as `(x + 1) + 2` quite complex.  The
module implementing this therefore ended up being somewhat messy,
especially since Rust's move/borrow semantics made flattening trees of
sums and products more complex than it would otherwise be.  However, I
decided to move on rather than spend a great deal of time tidying this
up since, as previously mentioned, this was more of a nice-to-have
feature rather than core functionality.

### Evaluator Composition

I then implemented a mechanism to compose evaluators together, and
build a pipeline of arbitrarily many of them.  This was simple to do
since I was implementing the `Evaluator` trait for each pass, so
supporting composition simply meant creating a struct containing two
evaluators, and implementing the `Evaluator` trait for this such that
the evaluators were run in sequence on any input expression.

To combine more than two evaluators, I created a `pipeline` macro that
constructs nested compositions of evaluators with Rust's `macro_rules`
system.