10 KiB
Development Log
Initial Project Idea and Planning
I decided to make a basic symbolic differentiation engine. The essence of this is that you can provide mathematical expressions as input, and the program can compute the derivative of them. For the sake of simplicity, it will be command-line based, so its input and output will both be text. An example interation might look something like:
> 2x^2 + 3x
4x + 3
> x^3 - x^2
3x^2 - 2x
Architecture
The program will have three major components: the reader, which will take user input and parse it into a convenient internal representation (generally an AST of some kind); the solver, which will transform the input expression into an output expression; and the printer which will format output expressions back to human-readable text. This is essentially a standard architecture for a simple interpreter.
The reader will be composed of two passes: a simple state machine lexer to tokenise user input and a parser to build an AST from the tokens. For simplicity here, I will only be supporting basic latin characters in UTF-8 (i.e. ASCII-compatible text).
Since I am only implementing differentiation, the solver can be relatively simple, consisting of a pattern-matching loop with an accompanying ruleset.
The printer is the simplest of the three components and will consist only of a procedure to walk an AST and emit a serialisation of it equivalent to the input language.
In addition to the major components, a harness will be needed to tie them together and handle errors. This will be in the form of a simple REPL (Read Eval Print Loop).
Implementation and Testing Strategy
I'll begin by implementing the reader and printer, as this will allow me to get to a working end-to-end demo as quickly as possible by initially skipping the solver and simply printing the input expressions unmodified. This will then enable easy and fast iteration when implementing the solver logic.
I will be writing a suite of unit tests to accompany every module of the system, following a TDD (Test-Driven Development) methodology. This system is quite simple and well-defined, so the advantage of TDD (as opposed to writing tests after-the-fact) is probably relatively minimal here, especially since it will not be being modified or maintained after its initial development and will not be being worked on by multiple people, but the practise will be beneficial regardless in tightening the feedback loop of the development process and ensuring that the code is structured to be amenable to testing. If I have time I'll also implement some automated integration tests to verify end-to-end functionality.
I'll be using Rust as the implementation language, as I am fairly familiar with it and it is well-suited to such tasks. In addition, its strong type system and borrow checker are useful in catching many classes of bugs at compile time and it has unit testing support provided in the base language, which is convenient for doing TDD.
LLM / AI Usage
I will not be using any LLM coding assistants for this project.
Reader
Lexer
I first implemented the lexer with a very simple finite state machine
consisting of two states, Init and Integer. This was
straightforward as the input language is very simple.
Parser
Since the input grammar consists of terms, parentheses and infix operators with varying precedence, Pratt parsing is the logical choice for the implementation of a parser.
I find the usual way of handling associativity in precedence climbing parsers (wherein a small amount is added to the binding power on one side) somewhat inelegant, so I decided to implement a simple algebra for binding powers that handles associativity using custom comparison semantics.
Implementing the parser itself was straightforward as I have implemented Pratt parsing many times before (though not in Rust specifically).
To tie together the lexer and parser, and to provide a convenient
means of decoupling other components from them for testing purposes, I
then implemented the Rust standard library's FromStr trait, which is
for types that can be parsed from a string.
Printer
For expression printing, a simple recursive tree-walking algorithm, dispatching on the various kinds of expression was sufficient. Satisfyingly, I found I was able to use the same binding power algebra I defined for the parser in deciding when to use parentheses, and thus got correct associativity handling effectively for free, while also keeping all associativity-handling logic well-factored in one place.
The only complication was in the handling of products, as here there
was some ambiguity as to when to use explicit or implicit
multiplication (that is x*y or xy). Products of symbols with
other symbols and integers with symbols are more naturally expressed
with implicit multiplication, but of course this couldn't be used for
products of integers with other integers as this would result in
incorrect serialisations (such as "12" for the product of 1 and 2). I
ultimately settled on an approach where the right-hand operand is
first printed into a string, and an explicit multiplication sign is
used whenever that serialisation begins with a numerical character
(either a digit or a minus sign). This nicely handled all of the
cases I could think of and systematically avoids any erroneous
serialisations.
The printing logic could then be packaged up in an implementation of
the Rust standard library's Display trait, which is for types that
have a textual representation.
REPL
At this point, with the reading and printing implemented, it made
sense to create the REPL logic and demo binary so that a full flow
through the program could be demonstrated. I first created an
Evaluator trait to capture the future behaviour of the evaluator,
and then wrote a generic REPL module that's able to work with any
expression type implementing the FromStr and Display traits.
I was then able to create a demo program by defining a dummy evaluator that simply returns the input expression unmodified, and running a REPL using that, connected to stdin and stdout. This demonstrated a full round trip through the parsing and printing procedures.
Evaluator
Rather than trying to handle everything in one complex evaluator, I decided to implement a multi-pass evaluator. This essentially means that the expressions run through multiple different passes that apply different transformations to them, thereby keeping the complexity of each pass more managable. This is a common method used when implementing compilers and interpreters.
Normalisation
The first preliminary pass I implemented was a normalisation pass.
This transforms things like x - y into x + (-1)y, and x/y into
xy^(-1). The main benefit of this is to make it easier for the
later differentiation pass as it will only have to handle sums,
products and exponents (not differences and quotients). I decided to
leave quotients of integers as-is since these will be untouched by the
differentiation process anyway, and "1/2" reads far better than
"1*2^(-1)".
Constant Folding
The purpose of this pass is to combine constants together as much as
possible, such as transforming 1 + 2*3 into 6, or x + 1 + 2 into
x + 3. This is mostly a nice-to-have rather than an essential
feature, though it will make the output much more readable.
Implementing this pass did get a little hairy, as the binary tree
structure I used for the AST made handling cases like x + 1 + 2
(which is represented in the AST as (x + 1) + 2 quite complex. The
module implementing this therefore ended up being somewhat messy,
especially since Rust's move/borrow semantics made flattening trees of
sums and products more complex than it would otherwise be. However, I
decided to move on rather than spend a great deal of time tidying this
up since, as previously mentioned, this was more of a nice-to-have
feature rather than core functionality.
Evaluator Composition
I then implemented a mechanism to compose evaluators together, and
build a pipeline of arbitrarily many of them. This was simple to do
since I was implementing the Evaluator trait for each pass, so
supporting composition simply meant creating a struct containing two
evaluators, and implementing the Evaluator trait for this such that
the evaluators were run in sequence on any input expression.
To combine more than two evaluators, I created a pipeline macro that
constructs nested compositions of evaluators with Rust's macro_rules
system.
Differentiator
The differentiation pass itself was very easy to implement given the simplications made by first having a normalisation pass. I also had decided to only support constant exponents, which simplifies things too. In the end, it ended up being more-or-less a fairly direct translation of the sum rule and product rule into code, along with the power rule and chain rule combined for handling exponents.
Simplification
The output of the differentiation pass was correct but contained a lot
of noise such as terms multiplied by zero and other similar things. I
therefore implemented a basic simplification pass that applied various
identities such as x + 0 = x, 0x = 0 and 1x = 1. Expression
simplification is famously a very complex problem, but I kept this
pass quite limited in scope and only implemented the most high-payoff
features.
Canonicalisation
To make the output a little easier to read, the final evaluation pass
I implemented was a limited multiplication reordering canonicalisation
pass that ensured that products had the constants first. This is
still incomplete as the re-ordering only takes place within binary
products (so x*4 becomes 4x, but x*(y*4) would become x*4y not
4xy), and terms in a sum were not re-ordered at all. However, I
decided full canonicalisation would be excessively complex for this
project and decided to leave it there, as the output was acceptably
readable at this point anyway.
Wrapping up
The final pipeline I ended up with was:
- Normalisation
- Differentiation
- Constant folding
- Simplification
- Constant folding again
- Canonicalisation
I then created an evaluator function to wrap this up in the eval
module, to allow for a simple interface to the whole library.