Finextra webcast – registration here.

]]> http://mdavey.wordpress.com/2009/07/21/ria-matrix-webcast/feed 0 http://blog.lab49.com/archives/3011 http://blog.lab49.com/archives/3011#comments Sun, 19 Apr 2009 04:02:56 +0000 Kalani Thielen http://blog.lab49.com/archives/3011 With the spreading popularity of languages like F# and Haskell, many people are encountering the concept of an algebraic data type for the first time.  When that term is produced without explanation, it almost invariably becomes a source of confusion.  In what sense are data types algebraic?  Is there a one-to-one correspondence between the structures of high-school algebra and the data types of Haskell?  Could I create a polynomial data type?  Do I have to remember the quadratic formula?  Are the term-transformations of (say) differential calculus meaningful in the context of algebraic data types?  Isn’t this all just a bunch of general abstract nonsense?

We’ll investigate these questions, and perhaps demystify this important concept of functional languages.

Algebraic Expressions at the Level of Types

To understand the concept of algebraic data types, simply stated, we need to unify the set of concepts that we associate with algebra, and the set of concepts that we associate with data types.

In an algebraic expression, we might encounter a humble lone variable:

x

In a data type expression, we might encounter a humble lone type:

int

Algebraic expressions can also be compositions of other expressions, so for example we might see:

x*x

And similarly for data types:

int*int

To explain: the product of two types denotes the type of pairs of those types.  In some languages, this might instead be written as pair<int,int> but as we’ll soon see, it’s useful to think of this as a product.

Of course, the algebraic expression above could be simplified to x**2 (“x squared”) and so you could also write int**2.

Now, in an algebraic expression you might also see:

x+x

And the same concept exists for data types:

int+int

The sum of two types denotes the type of variants which can have values either of the left type or the right (but not both).  The sum in the above type might seem superfluous, because whether a value of the above sum type has the left or the right type, it’ll be an int.  However, there’s some additional useful information in knowing whether the left or right was chosen.

Consider, for example, a data type representing the balance of a bank account:

type balance = int + int

We might interpret values on the left as positive balances, and values on the right as negative balances.  So if we wrote a function to describe a balance for screen-reading, it might look like:

string describe(balance b) {
   if (b.is_left()) {
      return “The bank owes you $” + b.left();
   } else {
      return “You owe the bank $” + b.right();
   }
}

For many people, sum types are less obvious than product types.  It may help to consider that, just as pair<int,int> corresponds to product types, boost::variant<int,int> corresponds to sum types.  C-style unions also serve a similar function.

Now, we can use these type-composition functions to make arbitrarily complex expressions:

(int*int*string)+(date*(string+int))

Just as we could use them to make arbitrarily complex algebraic expressions:

(x*x*y)+(z*(y+x))

But there is perhaps a simpler algebraic concept to consider:

1

What does the natural number “one” mean at the level of types?  This is called the “unit type” and it has only one value, written ().  It doesn’t do much good to have a value of the unit type — you already know what it must be.  However, it can become useful in composition, for example if you consider the simple identity:

1+1=2

Now “two” becomes the type of bits (or boolean values) — because a value of this type could either be the (trivial) unit value on the left, or the one on the right.  This view may help to understand variant types better, consider:

int+int = 2*int

In other words, the sum type of two ints is the same thing as the product type of a boolean value and a single int (the boolean value tells you whether the int is the one on the left or the right of the sum).

Furthermore, as you might expect:

1*int = int

In other words, having a pair of a unit value and an int is the same as having an int (the unit value adds no information here — you already know what it must be).

Now there’s one other natural number that deserves special consideration:

0

This is a special kind of type — it’s known as the “void” type (slightly different than the void type in programming languages like C, you’ll have to forgive this overloading of terms).  Unlike the unit type, which has a single value, the void type has no values.  It’s impossible to construct a value of this type.

That means, for example, that:

1+0=1

Because if you have a sum type with unit on the left and void on the right, no values of this type can possibly be on the right (again, it’s impossible to construct a value of the void type) — so it’s safe to assume that you can simplify this sum type to just be the unit type.

By a similar argument:

1*0=0

Because to construct a pair, you have to construct both of its members — if you can’t construct one member of the pair (again, it’s impossible to construct a value of the void type) then you can’t construct the whole pair.

Now, these types that we’ve covered so far are quite useful, and it’s hopefully pretty clear how they map to simple algebraic expressions.  However, it might be pointed out that there are more complex types, and more complex algebraic expressions to consider (even though the expressions above can get quite complex).

For example, how might the type of a list of integers be described (a list<int>) as an algebraic expression?  To help motivate this answer, consider the possible lists of integers we might produce.  For example, a list of integers could be an empty list, or it might be just one integer, or it might be two, or three, etc.  Each “or” here should be thought of as a sum, so that (with the unit type representing the trivial empty list) we come to a type represented by an infinite sum:

1+int+int**2+int**3+…+int**n

Now, as an algebraic expression we might be happy with this, or we might convert it to the “big-sigma” notation.  However, in the world of types, this kind of infinite expansion is described with a recursive type.  A recursive type associates the expansion with a symbol and consolidates the type to an expression where that symbol may be used.  For the case of our list of ints, this is written:

μX.(1+int*X)

This can be pronounced “mu X, one plus int times X.”  Forget the “mu” if Greek symbols bother you; all it really means is “look out, here comes a recursive type.”  The most important idea is that the type should be thought of as the infinite expansion you get when you repeatedly substitute the entire recursive type for X (the recursion variable).  For example, performing that substitution once in the above type will produce:

1+int*(μX.(1+int*X))

and again:

1+int*(1+int*(μX.(1+int*X)))

Using the normal distributive properties of * and +, this can be reduced to:

1+int+int**2*(μX.(1+int*X))

And in the limit that we perform this substitution continually, we will produce the same infinite sum that we expected for this list of ints.

Although this formalism may seem somewhat removed from the language of types in F#, you can quickly recover it if you replace the keyword “type” for “mu,” “=” for “.”, “|” for “+”, recognize that empty constructors are constructors on the unit type, and ignore constructor names.  For example:

type IntList = Nil | Cons of int*IntList

becomes:

μIntList.(1+int*IntList)

Again, the type of lists of integers that we have already identified (modulo the choice of “X” or “IntList” for the recursion symbol).

This machinery of recursive types takes some getting used to, but it can produce the same infinite series that we expect in algebra, and it’s very convenient for recursive types with multiple branches of recursion, for example the type of binary trees of integers:

μX.(int+X*X)

In other words, a binary tree is either a leaf (storing an int) or it’s a branch (storing a pair of binary trees).

It seems like we can describe a very rich category of types with these tools (for more detail, see “the bible of type theory”).  What’s more, our existing knowledge of algebra can be ported to this world of types, where we find many familiar structures.  But this formalization of types goes even deeper, and there are more concepts left to unify.

∂ for Data

In the world of functional programming, destructive mutation is discouraged or even banned outright.  From a formal perspective, an update should produce a new copy of the original value, with both versions available for some later use.  This view has simplified many otherwise thorny problems, and it makes programs much simpler to reason about.

We have previously discussed ways to recover mutation-like functions, inspired by the most famous data structure of its kind: the Zipper.

In 2001, Conor McBride made a remarkable discovery about algebraic data types, and the concept of a zipper, which he elaborated four years later (along with coauthors Abbot, Ghani, and Altenkirch).

Central to this discovery is the concept of a one-hole context.  A one-hole context can be thought of as a data type “with a hole in it.”  This is a concept closely linked to the concept of mutation, as a value can be “updated” by “plugging in” that hole.

Consider as a very simple example, a 3-tuple of integers.  Its type is int*int*int, or int**3.  Now, we might have a value of this type:

(98, 76, 54)

We might want to “update” one of these values, or at least single one out for updating.  We can do this by “poking a hole” in the data structure where a particular value goes.  In other words, we might choose to update the first, second, or third components of this 3-tuple (using “_” to represent a hole):

(_,  76, 54)
(98, _,  54)
(98, 76, _)

These three cases represent the only ways that we could “poke a hole” in this 3-tuple data, and we can capture that idea in a corresponding data type.  Because there are three ways to poke a hole, the type should be a three-way sum.  Each possibility leaves two integers (in different configurations), and so we can pretty straightforwardly deduce this type of one-hole contexts:

(int*int)+(int*int)+(int*int)

In other words, the first int*int corresponds to the type of “(_, 76, 54)“, the second int*int corresponds to the type of “(98, _, 54)“, and the third int*int corresponds to the type of “(98, 76, _)“.

It’s useful to simplify the type of this expression, and so we might shrink it to (explicitly naming squares):

int**2+int**2+int**2

and then combining like terms:

3*int**2

In other words, the type of one-hole contexts of 3-tuples of ints is one of three int-pairs.

Now here’s an odd thing, we started out with the type:

int**3

and taking its one-hole context we derived the type:

3*int**2

That looks an awful lot like the derivative of differential calculus!

In fact that’s exactly what McBride discovered, that the derivative of an algebraic data type is its type of one-hole contexts!  Extended to recursive types (see McBride’s paper for more details) we can use the derivative to mechanically produce functions for updating and iterating over arbitrary data structures.

Further Reading

Predating McBride’s work, Andre Joyal explored the concept of combinatorial species (built on the concept of generating functions — a formalism similar in purpose to algebraic data types), where the use of the derivative was also discovered.

McBride is also the co-creator of the programming language Epigram, a programming language with a powerful theory of types.

In the next article, we’ll look at implementing this notion of differentiation of data types with a brief Haskell program.

Lab49’s Marc Jacobs and I were out at Microsoft’s Mix09 conference last week and, in conjunction with Zoom-In Online, we produced a number of blog postings & podcasts.

We produced a bunch of stuff, so here are all the podcasts in one place for your easy consumption.  As you will see & hear there’s a lot to keep up on.  But there was so much great information and exciting new developments that I’m sure we didn’t cover it all!

• Christian Schormann, Group Program Manager for Expression Blend -  http://edgecastcdn.net/0005A0/blog/mix09_ChristianSchormann.mp3
• James Pratt, Senior Product Manager for Internet Explorer – http://edgecastcdn.net/0005A0/blog/mix09_JamesPratt.mp3
• John Lam, Program Manager, Dynamic Language Runtime – http://edgecastcdn.net/0005A0/blog/mix09_JohnLam.mp3
• Johnny Lee, Researcher, Microsoft Applied Sciences – http://edgecastcdn.net/0005A0/blog/mix09_JohnnyLee.mp3
• Kevin Gjerstadt, Principal Group Program Manager, WPF – http://edgecastcdn.net/0005A0/blog/mix09_KevinGjerstad.mp3
• Lou Carbone, Founder & CEO of Experience Engineering – http://edgecastcdn.net/0005A0/blog/mix09_LouCarbone.mp3

Enjoy listening to these.

Also you should check out Marc’s blog postings here.

I switched from C++ to Java and then Java to C# so I feel somewhat qualified to speak on the topic of switching languages.

Here my recommendations to guide your language studies:

• Leverage existing strengths.
• Attack “cognitive noise”.
• Consider contributing to an open source project.

Leverage existing strengths

You’re still a great coder. Then the only issue is finding way to manifest that greatness on this new platform. Identify some of those techniques and capabilities from C++ that you’re awesome in, and find the “.net way” of doing the same thing. By “.net way,” I mean finding something appropriate for the platform instead of doing a literal port of C++ techniques to C#. In all likelihood, you have some “cash cow” knowledge that made you valuable in the C++ world. If you were an STL god, then go through the .net generic collections and find their limitations and strengths.

For example, if you’re wondering whether to learn IL, you should ask yourself, “did I get into the machine code when I worked with C++?”  If you were good at disassembling and/or stepping through code with gdb, windebug or what-have-you, then you should study the .net way of the same. It served you well as a C++ coder – so it’s a cash cow. It has a high likelihood of serving you well in C#. You tend to ask yourself the same questions in C# as you do in C++. “I wonder what that looks like in IL?”

For other features (like attributes and serialization) that don’t have direct C++ equivalents, find out why they’re useful and how you have accomplished similar things in the past. If you find that the feature is useful for one of your cash cows, then study in-depth. Otherwise, study it only when a real business case might find it useful.

Attack cognitive noise

As far as I can tell, the biggest obstacle to a new language is not the language. It is just getting into the “zone” – being effective, more signal, less noise. A lot of coding is spent addressing noise elements. For example, an inordinate amount of time can be wasted on naming stuff, access control, organizing directories, deployment units, unit testing, navigating between source files, getting documentation, and trying to do things the way I used to. In the beginning, most of my code-test-debug cycle is spent thinking about noise items. Just navigating the IDE is a significant cognitive burden.

When you notice this cognitive noise, attack it forcefully. Your goal is to find ways to reduce the cognitive load. Give yourself a time limit, say, an hour to find a better way.

For example, if you keep mistyping “interface,” you can go off into a text editor and type it a thousand times.

Consider contributing to an open source project

When I made the transition from Java to C#, I joined the Spring.NET project. I had been using Spring in Java and was going to miss the framework dearly. I learned a tremendous amount from all the developers. Just reading the code addressed most of the cognitive noise issues. This worked well because (1) I knew a lot about Spring Java, and (2) Spring.NET had spectacular developers on it.

You may have used some open source C++ stuff, e.g. boost. Some of those have inspired open source ports of those projects. For example, the BGL (Boost Graph Library) inspired QuickGraph.

Observing other more experienced developers on the platform is a great way to learn.

Jason started with an application that one would write in a “traditional” way, with button clicks handled by event-handlers in the code-behind that then updated other parts of the UI. Using WPF data-binding, Commands, and Unity, he transformed it, piece by piece, in a much more manageable, encapsulated, readable, and testable M-V-VM design. It was awesome.

It was so awesome, in fact, that after the presentation Jason recorded the demo for all to see here.

Check it out. It’s the most practically instructive explanation of WPF design I’ve seen.

UPDATE: I thought I should mention that while Jason’s presentation is geared towards WPF, the patterns he describes are very applicable for Silverlight as well. There are a few things to take note of, though:

1) Jason creates ViewModels that are DependencyObjects. For some (apparently undocumented) reason, DataContexts in Silverlight cannot be DependencyObject descendants. That means that you need to implement ViewModels as INotifyPropertyChanged (an option also available for WPF that Jason mentions in his video). This doesn’t fundamentally affect the pattern, but should be noted.

2) Unity, which Jason uses for dependency injection, is available for Silverlight right now as part of pre-release Composite WPF & Silverlight (Prism) v2. You can download it here. (I haven’t been able to find Unity for Silverlight as a separate download.)

3) Silverlight does not support bindings for Commands out of the box. It does, however, expose the ICommand interface. With just a little bit more work using an “Attached Behavior” approach, we can get the kind of Commanding that Jason uses in Silverlight as well. Check out the description here.

Second, keep in mind that there are three important APIs which are inter-related: JAX-WS, JAXB, and StAX. Once you understand how these fit together, everything else falls into place more easily.

JAX-WS

Let’s begin our journey with the latest Sun standard for creating and consuming web services: JAX-WS, which stands for Java API for XML Web Services. This standard was introduced in 2004. You can ignore JAX-RPC, since JAX-WS replaces it.

There are three noteworthy implementations of JAX-WS. The first is from Sun, and is called JAX-WS RI for the JAX-WS Reference Implementation (they always had a way with names). The second and third are both from the Apache Group and are called Axis2 and CXF. You can ignore Axis1, XFire, and Celtix, since they are all obsolete. There is also a web service framework called Spring-WS, but it’s not JAX-WS compliant.

So if you are creating web services in Java, the first order of business is to to choose an implementation to work with, and unless you have a reason not to, you should probably stick to one that complies with JAX-WS, which means either JAX-WS RI, Axis2, or CXF.

Related to these is an open source project from Sun called Web Services Interoperability Technologies (WSIT), previously known as Project Tango. This is an implementation of several web service standards (WS-SecurityPolicy, WS-ReliableMessaging, and so on). Metro is an open source web service stack which is a combination of JAX-WS RI and WSIT (so it’s actually a reasonable fourth option).

JAX-WS is oriented around SOAP web services, but many programmers are now using the REST approach. Sun is coming out with the JAX-RS API to support that, but it’s not quite ready yet.

JAXB

Web service development requires mapping between XML and Java objects. JAXB is the Sun API for that (also referred to as JAXB2 since the latest version is the important one). There are two noteworthy implementations: JAXB-RI (Sun’s reference implementation) and JaxMe (the unfortunately named contribution from Apache). JaxMe is in the incubation stage and is not formally part of Apache yet. There are many other interesting and popular XML/Java mapping frameworks, but most of them are not compliant with JAXB. Examples include Castor (from Codehaus), JiBX (a spectacularly fast open source implementation), and XMLBeans (a flexible implementation from Apache).

A recurring source of confusion is that in the past, Sun was less clear about the distinction between APIs and reference implementations, so people would take JAXB to mean both, and you would often see online articles like “Which is better: JAXB or JiBX?” But today developers should always try to use the JAXB API, which will enable a choice of compliant implementations such as JAXB-RI or JaxMe with minimal or no code changes.

StAX

For Java code that needs to read and write large XML documents quickly without necessarily mapping them to objects, there is the Streaming API for XML (StAX). There are several implementations of this API too. There is the Sun Java Streaming XML Parser called SJSXP (another snappy name from Sun), the Woodstox open source implementation which is excellent, and the StAX reference implementation from Codehaus which is referred to simply as StAX (unfortunately perpetuating the confusion between APIs and implementations). Xerces is a streaming XML processing library which used to be part of the Apache project, and work was underway to make it StAX compliant, but that was dropped.

Putting it all together

Web services need to process XML, sometimes mapping it to and from Java objects (e.g. for creating proxy objects and an RPC-like experience), and sometimes processing it directly (e.g. for streaming results when high performance is needed). Therefore Sun designed the JAX-WS API to rely on the JAXB API, which makes perfect sense; any JAX-WS compliant web service implementation should therefore be able to use any JAXB compliant mapping library. Other relationships between these APIs are up to individual implementations. For example, JAX-WS RI supports the StAX API, so you can use any StAX implementation for streaming. CXF also supports the StAX API, as well as a host of Java/XML mapping options including JAXB (allowing the use of any JAXB compliant implementation), XMLBeans, Castor, and JiBX. Yes, they are heroes.

So if you get confused, just ask yourself clarifying questions like: Does this Java web service library support JAX-WS? Which JAXB compliant Java/XML mapping implentation shall I use? Which is better for processing streaming XML? Woodstox or the StAX reference implementation?

If you’re still confused, then just accept the recommendations I promised not to make: Use CXF for your web services (which complies with JAX-WS), JAXB-RI for your Java/XML mapping (which complies with JAXB), and Woodstox for streaming (which complies with StAX).

Philosophical Preliminaries

We first got into trouble in our interpreter when we introduced compound expressions.  Primitive Number and Symbol values were still fairly easy to construct (and should be even simpler in our S-Expression form), but constructing a program to sum three numbers was still difficult.  It required a box-pointer diagram like this:

Further, we said that this corresponded to an S-Expression: (+ 1 (+ 1 1)).

This example demonstrates that we have three important cases to consider when parsing expressions: symbols (as “+” in the above example), numbers, and (trickiest of all) lists of expressions (everything between parentheses, broken by spaces).

Here we can significantly simplify the problem of parsing lists by flattening the representation.  In this case, that means recognizing that parsing (+ 1 (+ 1 1)) can be viewed as parsing (+ 1 b) where b is the result of parsing (+ 1 1).  So we need to be able to set aside a parsed-list mid-way, and resume it later with the result of a nested parsed-list.

Now we can use the “(” character as an indication that a new list is beginning, and “)” that the most current list has ended.

Finally, we have a strategy for parsing S-Expressions.  To parse an expression, we ignore characters up to the first non-whitespace character.  If it’s numeric, process the rest of the input up to a non-numeric character as data for a Number value, construct the Number, and return it.  Likewise (but for alphabetic characters), construct Symbol values.  If a “(” character is encountered, push a new Pair value onto the stack, then proceed to parse sub-expressions by allocating a new Pair for each one, linked to the end of the list on the top of the stack.  Finally, when a “)” character is found, pop the Pair from the top of the stack and either insert it in the outer list (if the stack isn’t empty), or return it as the final result.

This informal description essentially captures a push-down automaton.  By explicitly defining the set of states we can be in, rules for transitioning from one state to the next, and rules for constructing terms from the strings captured between states, we can completely define our S-Expression parser.

Actual Code

Just as we did last time, we’ll start with the basic type signature of the function we aim to define:

Value* ReadSExpression(std::istream& input);

This says that if we can provide a source of text (whether from the console, a file, the network, etc), the ReadSExpression function will produce a Value usable by our interpreter.

Using the general strategy of defining a push-down automaton outlined above, we ought to be able to create a class to encapsulate that process, which should be straightforward to use in our final implementation of ReadSExpression by the end of this article.

Following this trail of thought, we can immediately begin by making explicit the set of states we expect to be in:

enum SExprReadState
{
    LookingForValue,
    ReadingAtom
};

Here we’ll collapse the states for reading a Symbol or a Number into the ReadingAtom state (once we’ve read the “atom” we’ll determine which type to use).

Now we’ll need some machinery for understanding these states and the various transitions between them that we may define.  I’ve prepared this machinery ahead of time (which can be safely ignored until the end, when we’ve got to compile the whole thing) in a class, StreamFSM:

    StreamFSM.hpp

To use this class, we’ll need to provide it with a character type (char should be good enough, unless we’ll need to parse UNICODE), a state type (our SExprReadState above), and the implementation type (the type of the S-Expression reader we intend to define).  We can conveniently forward-declare the whole bundle like this:

class SExprReader;
typedef StreamFSM<char, SExprReadState, SExprReader> SERBase;

Now we can begin to define the actual SExprReader class.  Here we’ll inherit from the StreamFSM to import the basic machinery, which lets us define our initial state (LookingForValue), the procedures to invoke when reading characters in any of our defined states, and the procedures to invoke when transitioning between states:

class SExprReader : public SERBase
{
public:
  SExprReader() : SERBase(LookingForValue), output_value(0)
  {
   AddStateHandler(LookingForValue, &SExprReader::LookForValue);
   AddStateHandler(ReadingAtom,     &SExprReader::ReadAtom);

  // [...]

This is the most important part of our parser, so it’s worth going over the exact meaning of the code above.  First, the SExprReader class inherits the parsing machinery from SERBase.  In its constructor, SExprReader must give SERBase its initial state (here that initial state is LookingForValue).  It also introduces the variable output_value — which we’ll see later is the accumulator for the final parse result.

The rest of the constructor here uses AddStateHandler to map states to procedures that should be invoked for each input character (while that state is active), and AddTransitionHandler to map a pair of meta-states to a procedure that should be invoked when a transition between the two meta-states occurs.  A meta-state can be the symbol AnyState (which stands in for any of our SExprReadState values), StartOfStream (the state that’s set when we first begin parsing), EndOfStream (the state that’s set after we’ve reached the end of input), or any of our standard SExprReadState values.

In English, these state and transition mappings can be read as: When in the LookingForValue state, call the LookForValue procedure.  When in the ReadingAtom state, call the ReadAtom procedure.  When transitioning between the ReadingAtom state and any other state, call the AddAtom procedure.  When transitioning between any state and the end of input, call the ValidateState procedure.

Continuing to fill out this class, we’ll need local data for building up parsed expressions:

private:
    typedef std::pair<Pair*, Pair*> ListState;
    typedef std::stack<ListState>   NestedExprs;
    NestedExprs nested_exprs;
    // [...]

Here the nested_exprs stack will keep track of lists within lists, and the ListState type will hold the initial Pair node of a linked list, and the final node of the list (this is important because we’ll append new nodes to the final node, but we’ll return or insert the initial node).

Now, since it will be the first procedure to run while parsing, let’s fill out the LookForValue procedure:

void LookForValue(const char& c)
{
    switch (c)
    {
    case ‘(‘:
        nested_exprs.push(ListState(0, 0));
        break;
    case ‘)’:
        if (nested_exprs.size() > 0)
        {
            ListState ne = nested_exprs.top();
            nested_exprs.pop();
            AddToResult(ne.first);
        }
        else
        {
            throw std::runtime_error(“Unexpected ‘)’.”);
        }
        break;
    default:
        if (!is_whitespace(c))
        {
            PutBack(c);
            SetState(ReadingAtom);
            break;
        }
    }
}

Again, there are a lot of things being done here, so let’s take them case by case.

If we receive the ‘(‘ character, we push a new ListState onto the stack (as described earlier).  Here we make the choice to push nil on the stack initially — this will allow us to use the (arguably more elegant) notion that nil should be equivalent to the empty list: ().  We’ll achieve this by only allocating Pair nodes for the list as they’re needed (i.e.: as we read actual Values within the list), so that if none are needed (i.e.: we read “()“) the result will be the null pointer, nil.

Next, if we receive the ‘)‘ character, we should complete the active list.  If nested_exprs is empty, that means there is no active list (so we raise an error).  Otherwise, we pop off the state for the active list, and use the AddToResult procedure (we’ll examine this later) to either set the final parse result or accumulate this newly-completed list into the next active list.  Notice the symmetry with the ‘(‘ case here, where we’ll call AddToResult(nil) if the active list has had nothing accumulated into it.

Finally, if we receive any character that isn’t whitespace (given the trivial is_whitespace function), we’ll put that character back (as the PutBack procedure makes the character available for the next state-handler) and set the state to ReadingAtom, which brings us directly to the ReadAtom state-handler:

void ReadAtom(const char& c)
{
    if (is_whitespace(c) || c == ‘(‘ || c == ‘)’)
    {
        PutBack(c);
        SetState(LookingForValue);
    }
}

This state-handler is simple enough.  If we get a character that’s either whitespace or a command to begin/end an active list, the current atom must be ending.  Hopefully this jives with the notion that a number should only consist of a contiguous sequence of numeric characters, and a symbol should only consist of a contiguous sequence of alpha-numeric characters.

If we do get a character that indicates the end of the current atom, we just put it back, and switch back into the LookingForValue state (whose handler we’ve already covered).

You may wonder how the actual parsed text turns into a Value that’s either returned or inserted into the active list.  That’s the job of the transition handlers, which we’ll examine now.

void AddAtom(elem_iter begin, elem_iter end)
{
    std::string val(begin, end);
    if (is_numeric(val))
        AddToResult(allocate<Number>(from_string<double>(val)));
    else
        AddToResult(allocate<Symbol>(val));
}

Here our SERBase machinery gives our transition handlers the sequence of characters recognized between being in the first state and transitioning to the second.  We use it to construct an std::string, which is used either to construct a Number (given the trivial is_numeric function), or a Symbol.  Here we also use the allocate<T> function to allocate a garbage-collected value of type T, which in the last article we decided to defer to a later article (and which we’ll again defer to a later article).  Finally, we use the same AddToResult procedure as we used earlier to add this value (in either case) to the overall parse result. 

Additionally, if the from_string function doesn’t seem trivial, we can use some existing machinery provided by the STL to define it as easily as:

template <typename T>
    T from_string(const std::string& s)
    {
        T ret;
        std::istringstream ss(s);
        ss >> ret;
        return ret;
    }

We have one other transition handler to consider, which really just makes sure that we don’t finish parsing our input if we haven’t closed off all active lists:

void ValidateState(elem_iter begin, elem_iter end)
{
    if (nested_exprs.size() > 0)
        throw std::runtime_error(“Expected ‘)’.”);
}

Now, we’ve only left one procedure to define in order to complete our SExprReader class:

void AddToResult(Value* v)
{
    // if we’re not accumulating values on an active list
    //   return this value as the result value
    // otherwise add this value to the end of the active list
    if (nested_exprs.size() == 0)
    {
        output_value = v;
        Detach();
    }
    else
    {
        NestedExpr& ne       = nested_exprs.top();
        Pair*       cur_pair = allocate<Pair>(v, (Value*)0);

        // if we’ve already allocated the initial node,
        //   append the new node as the final node
        // else set the initial and final node to the new node
        if (ne.first != 0)
            ne.second->tail(cur_pair);
        else
            ne.first = cur_pair;

        ne.second = cur_pair;
    }
}

Finally we’ve wrapped the trickiest part of the whole problem into this procedure.  The purpose of this procedure is to add a single value to the result — if there are no active lists, the value should be the final result but if there is an active list, the value should be appended to the end of it.  Here that’s achieved essentially with two cases.

In the first case, when nested_exprs is empty that means that there are no active lists to append to.  So here we’ll set output_value to the input Value and use SERBase‘s Detach procedure to stop processing input.  This is the most basic case, and in typical circumstances this is where the complete parse bottoms out.

In the next case, however, we’ve got to append the input Value to the active list.  Because we were a little clever in not allocating the initial node before any values are added to the list, we need to account for that decision here.  First we allocate a new list node containing just the input value.  Then, if the active list has no initial node set, we’ll make our new node the initial node (this will be the start of the list, and it will be used as the value for the entire list).  If the active list does have an initial node, we’ll append our new node to the active list’s final node.  In either case, this new node will be the new final node for the active list.

Now we just need to make the final result accessible:

Value* value() const
{
    return output_value;
}

And we’ll need to close off the definition of this SExprReader class:

};

Now we can bring this all together in a simple function that makes use of this class:

Value* ReadSExpression(std::istream& input)
{
    SExprReader reader;
    reader.Run(input);
    return reader.value();
}

There we have it!  We now have an easy way to parse S-Expressions from any input source (a file, the console, the network, etc), which can be used in conjunction with the Eval function we defined last time!  It’s so easy to use with Eval, in fact, that (provided we make a simple function PrintExpression to serialize a Value to an output stream) we can concisely define REPL, the procedure for running a Read-Eval-Print-Loop:

void REPL(std::istream& i, std::ostream& o, EnvironmentFrame* e)
{
    while (i)
    {
        try
        {
            // read
            Value* inexp = ReadSExpression(i);

            // eval
            Value* outexp = Eval(inexp, e);

            // print
            PrintExpression(outexp, o);

            // loop
            output << std::endl;
        }
        catch (std::runtime_error& e)
        {
            o << “*”               << std::endl
              << “** ” << e.what() << std::endl
              << “*”               << std::endl;
        }
    }
}

Which could be used just as easily to enable an RPC-like session (where input and output streams are network connections) as it could to interact with the console.  In the (maybe more typical) example of console use, all that’s necessary to run a REPL session is the following call (assuming that your root EnvironmentFrame is prepared in the variable root_env):

REPL(std::cin, std::cout, root_env);

Clearly we’ve come a long way toward making very simple what may have once seemed hopelessly complex!

In the next article, we’ll look at how to make it even more convenient to bind functions to our interpreter, and we’ll add some advanced functions that take us beyond the simple arithmetic we’ve done so far into list processing.

This series of articles on practical programming language theory (PLT) will be an answer to the above refrain. In this first article, I’ll show just how simple it is to make an interpreter for a basic functional language using any modern C++ compiler.

Philosophical Preliminaries

What is an interpreter? There are many different ways to view an interpreter, but (since we’re choosing to implement a particular one) we’ve got to choose a consistent way to think about it. Generically, an interpreter has some input (a program), and some output (what the program computes). We can view interpretation as the progressive refinement of our program, up to a fixed point. For example, imagine the program 1+1+1. The first step in a refinement of this program will either be 2+1 or 1+2, and in this case that choice is arbitrary. Either way, the final result will be 3, which can be refined no further (evaluation reaches a fixed point in that 3 evaluates to 3, which evaluates to 3, …).

All that’s necessary to make this view match our intuition of programming is to add abstractions. An abstraction, as its name implies, is an expression with a single value abstracted out of it. For example, with the expression 1+2, we may want to abstract away the number 1. This is written λx.x+2. Finally, we’ll need to apply values to abstractions, which requires substituting the actual value where the abstraction’s variable appears in its expression. For example, to regain our original 1+2, we can write the application ((λx.x+2) 1) — substituting 1 for x yields 1+2.

This perspective is elaborated colorfully in these famous videos, based on Harold Abelson and Gerald Sussman’s foundational book Structure and Interpretation of Computer Programs.

This view of interpretation follows the tradition of the λ-calculus, in contrast to the view in Turing machines. A useful rule of thumb in thinking about these two views is that the λ-calculus closely matches the way people traditionally think (and write) about calculation, while Turing machines closely match the way real physical computers are organized and manufactured.

Actual Code

Now that we’ve chosen a theoretical model of computation, we’ve got to get our hands dirty and write some code to make it work. There are many problems to start with in building a real interpreter, but a good first approximation is to start with the type signature:

Value* Eval(Value* program, EnvironmentFrame* env);

This says that the Eval function will take a Value and an EnvironmentFrame (a map of variables -> values, which accounts for the substitutions necessary when applying abstractions) and return a Value — the fixed point of evaluating the input program. We should expect that the following examples of the Eval function should be defined:

Eval(1, {}) = 1
Eval(x, { x -> 42 }) = 42
Eval(1+1+1, { + -> machine-code }) = 3
Eval(((λx.1+x) 2), { + -> machine-code }) = 3

Having covered the essential boundary of the problem, let’s refine it further to fill out these Value and EnvironmentFrame types, and provide a full definition of this special Eval function.

Data Types

In the example definitions of Eval above, we actually define the function over four different data types. In the first case we interpret a number, in the second case we interpret a symbol (the variable x), in the third case we interpret a compound expression representing the function associated with the symbol “+” applied to three numbers, and in the fourth case we apply an abstraction (itself a kind of function) to a value. Somehow we’ll have to capture all of these cases in the Value data type. Classes in the C++ language give us a method of describing each of these cases, inheriting from the most generic case — which we can use to define the empty base type:

class Value
{
};

Having set this much down, we can try to cover every case we considered above. First, numbers can be defined as:

class Number : public Value
{
public:
    Number(double v) : val(v) { }
    double value() const { return val; }
private:
    double val;
};

This effectively “boxes” the primitive double type. Moving on, we can similarly represent symbols:

class Symbol : public Value
{
public:
    Symbol(std::string v) : val(v) { }
    std::string value() const { return val; }
private:
    std::string val;
};

Having put those two cases to rest, we have to deal with the case of compound values. We could approach this in a number of ways, but perhaps the simplest is to view any case of compound values as a pair of values. In the case of a compound computation (as in the third example of Eval above), the head of the pair will represent the function to apply to the values represented by the tail of the pair. Following from this, the tail can be another pair, which allows us to chain together several values (or compound values) up to some final value: nil, the empty pair. For example, the expression 1+1+1 above can be represented in constructor form as:

Pair(+, Pair(1, Pair(Pair(+, Pair(1, Pair(1, nil))), nil)))

This type of structure is commonly visualized in box-pointer diagrams like this:

Here, each box points to a value, and each box-pair represents a Pair (itself a value), and the image represents nil.

This structure can also be written as an S-expression: (+ 1 (+ 1 1)).

With all of that said, the actual definition of this pair type is very simple:

class Pair : public Value
{
public:
    Pair(Value* h, Value* t) : hd(h), tl(t) { }
    Value* head() const { return hd; }
    Value* tail() const { return tl; }
private:
    Value* hd; Value* tl;
};

Finally, we need to define the data type of functions, so that we can make the above program (with its references to a “+” function) meaningful and so that we can generally define abstractions.

class Function : public Value
{
public:
    virtual Value* Invoke(Pair* args, EnvironmentFrame* env)=0;
};

This leaves us with an interface that all functions must satisfy. It’s important to define this interface, because we’ll implement it differently for primitive functions than we will for abstractions. In either case, when a “function” is invoked, it will be given an argument list (a linked list of values, as described above) of unevaluated expressions (later we’ll see why it’s important that they be unevaluated). Because the argument list is unevaluated, we will also need to provide the EnvironmentFrame (the record of variable/value mappings).

To fill this out, let’s look at implementing the addition function required by the 1+1+1 program:

class AddFunction : public Function
{
    Value* Invoke(Pair* args, EnvironmentFrame* env)
    {
        Number* n1 = (Number*)Eval(args->head(), env);
        Pair* arg_tail = (Pair*)args->tail();

        Number* n2 = (Number*)Eval(arg_tail->head(), env);

        return allocate<Number>(n1->value() + n2->value());
    }
};

Later we’ll see how primitive functions can be defined more concisely, but for now it’s useful to stop here and understand just what the class is doing.First of all, it’s a subclass of Function, so it provides an Invoke method. Next it calls Eval (the interpreter function we have yet to define) on the head of the argument list, casting the result to a Number. Of course we could be safer here, perhaps using dynamic_cast to ensure that it’s safe to treat the value as a number, but for the purposes of this exercise we’ll keep it simple (and unsafe).

Next, the tail of the argument list is read (and cast to a Pair). Because the argument list is effectively a linked list, this is similar to (say) incrementing a std::list<Value*>::iterator. Having loaded the next node in the argument list, we again evaluate the head of this node, to acquire the second argument.

Finally, we return a new Number value made of the sum of the first two numbers. Ignoring the allocate<T> function for now (which should allocate a garbage collected value of type T), we’ve now completely come to terms with how a primitive function can be defined for this interpreter.

We now have almost enough detail worked out to implement Eval, the actual interpreter. Before we do that, we should fill out the EnvironmentFrame type, which keeps track of variable-name to Value substitutions:

class EnvironmentFrame
{
public:
    EnvironmentFrame(EnvironmentFrame* p) : parent(p) { }

    void define(const std::string& var, Value* val)
    {
        substitutions[var] = val;
    }

    Value* lookup(const std::string& var)
    {
        ValMap::iterator vi = substitutions.find(var);

        if (vi != substitutions.end())
            return vi->second;
        else if (parent != 0)
            return parent->lookup(var);
        else
            throw std::runtime_error(“Undefined: ” + var);
    }
private:
    typedef std::map<std::string, Value*> ValMap;
    EnvironmentFrame* parent;
    ValMap substitutions;
};

This type effectively just maps variable names to values, with the small distinction that it defers to an outer EnvironmentFrame if the variable isn’t immediately defined. Later we’ll look at implementing abstractions, where this ability to define hierarchies of substitutions is critical.  With this type defined, we can create the root environment frame to hold our addition function:

EnvironmentFrame* root = allocate<EnvironmentFrame>(0);
root->define(“+”, allocate<AddFunction>());

Now we’ve come far enough to finally define Eval, the actual interpreter.

The Eval Function

Value* Eval(Value* e, EnvironmentFrame* env)
{
    if (Symbol* s = dynamic_cast<Symbol*>(e))
        return env->lookup(s->value());
    else if (dynamic_cast<Pair*>(e) == 0)
        return e;

    Pair* p = (Pair*)e;
    Function* fn = (Function*)Eval(p->head(), env);

    return fn->Invoke(p->tail(), env);
}

If the expression is a compound value, it’s interpreted as a function application. As we saw previously, the head of this compound expression must evaluate to a Function value. Having produced a Function value then, all that’s left is to call its Invoke implementation with the tail of the compound expression (the function’s argument list).

In a sense, that’s all there is to writing an interpreter. Although this explanation may appear lengthy, there’s very little code involved. All that’s left for us to do now is to “fill out” the interpreter with useful functions (since presently the only function we can apply is addition).

Additional Functions

In most “real world” programs, we’ll need some form of conditional statement. Currently our interpreter doesn’t support this, but it’s easy enough to add as a Function value.

class IfFunction : public Function
{
public:
    Value* Invoke(Pair* args, EnvironmentFrame* env)
    {
        // extract function arguments
        Value* test_exp = args->head();
        args = (Pair*)args->tail();
        Value* cons_exp = args->head();
        args = (Pair*)args->tail();

        Value* alt_exp = args->head();

        // if the condition is true, evaluate the
        // consequent else evaluate the alternative
        if (Eval(test_exp, env) != 0)
            return Eval(cons_exp, env);
        else
            return Eval(alt_exp, env);
    }
};

Now we can use this utility to create a function that allows programmers to define new variables:

class DefineFunction : public Function
{
public:
    Value* Invoke(Pair* args, EnvironmentFrame* env)
    {
        Symbol* var = (Symbol*)nth(args, 0);
        Value*    v = Eval(nth(args, 1), env);

        env->define(var->value(), v);
        return 0;
    }
};

Here it’s important that we not evaluate the first argument, since it should be the name of the variable we want to define (and since it isn’t already defined, if we evaluated it we’d get an error!). The second value (the value to be associated with the variable) should be evaluated though, and so it is. Finally, we update the environment with this new variable and (arbitrarily) return.  Just to fill out our interpreter a little more, we can also add more arithmetic-related functions: one to multiply two numbers and another to check two numbers for equality:

class MultiplyFunction : public Function
{
public:
    Value* Invoke(Pair* args, EnvironmentFrame* env)
    {
        Number* n1 = (Number*)Eval(nth(args, 0), env);
        Number* n2 = (Number*)Eval(nth(args, 1), env);

class EqFunction : public Function
{
public:
    Value* Invoke(Pair* args, EnvironmentFrame* env)
    {
        Number* n1 = (Number*)Eval(nth(args, 0), env);
        Number* n2 = (Number*)Eval(nth(args, 1), env);

Here, the result of EqFunction is meant to match the condition value of IfFunction — so the value returned to represent “true” is arbitrary, but the value for “false” must be nil. Now that we’ve collected some important functions, we should add them all to our root environment:

root->define(“if”,     allocate<IfFunction>());
root->define(“define”, allocate<DefineFunction>());
root->define(“*”,      allocate<MultiplyFunction>());
root->define(“=”,      allocate<EqFunction>());

We’ve now assembled a pretty respectable interpreter and runtime environment, but we’ve still got one critical feature to define.

Abstractions

To complete the goal we set out for ourselves initially, we’ve still got to implement abstractions. Our implementation should support the intuition we’ve developed that the application of an abstraction just substitutes the arguments for the parameters in the body of the abstraction. Luckily, in the EnvironmentFrame type we have already implemented a method of mapping variable names to values. Also, because we have implemented this substitution method by recording variable/value mappings in an EnvironmentFrame, we’ll need to capture the active EnvironmentFrame in the implementation for an abstraction.

To motivate this notion of capturing an EnvironmentFrame in an abstraction, consider the case of λx.λy.x+y. Here we have an abstraction that returns an abstraction! The inner abstraction refers to a variable bound by the outer abstraction, and the clearest way to make that work in our interpreter is to give the outer abstraction’s EnvironmentFrame to the inner abstraction.

Now we should be able to define the Abstraction type:

        return r;
    }
private:
    Pair* arg_names;
    Pair* body;
    EnvironmentFrame* edef;
};

This is probably the most complicated piece of code we’ve written in this interpreter! Still, if we go through it line-by-line we shouldn’t find anything particularly surprising. First, an Abstraction must be constructed with an argument list (a list of Symbols representing parameter names), a body (a list of expressions to be evaluated when the abstraction is applied), and an EnvironmentFrame (the active substitutions when the Abstraction is created).

Next, a new EnvironmentFrame is created to hold the variable/value mappings for the parameters of the Abstraction. It is then filled by iterating over both the list of parameter names and the list of arguments (expressions applied to the Abstraction). Each argument is evaluated and associated with its parameter name.

Finally, once the EnvironmentFrame for the call has been filled, it is used to evaluate each expression in the body (with the result of evaluating the last expression in the body used as the return value for Invoke).

To come to grips with this important aspect of our interpreter, let’s consider the case of ((λx.x+2) 1), which we considered early on. First, we’ll create Abstraction((x), ((+ x 2)), root). Here, (x) is just the parameter list, ((+ x 2)) the (one-element) list of expressions in the body, and root is the EnvironmentFrame in use when the Abstraction is first created (this should contain the definition of + which we use in the body). Then we apply the abstraction to the argument list (1). This produces an EnvironmentFrame for the call that looks like { x -> 1 { + -> PlusFunction } } (the nested mapping is meant to represent the contents of the parent EnvironmentFrame). Finally, the body is evaluated in this EnvironmentFrame, yielding a sequence of evaluations that looks as follows:

Eval((+ x 2), { x -> 1 { + -> PlusFunction } })
PlusFunction->Invoke((x 2), { x -> 1 { + -> PlusFunction } })
3

All that’s left is to provide a way for the programmer to construct these Abstraction values at runtime:

And we’ll need to put this function into the root EnvironmentFrame, so that it’s available to use. Traditionally, this function is written “lambda” (the less-convenient way of saying “λ“) so we’ll (arbitrarily) stick with that:

root->define(“lambda”, allocate<MakeAbstractionFn>());

Bringing it all together

We now have a minimalist interpreter for a Scheme-like programming language. There are still additions we may want to make, and plenty of supporting tools, but at this point we can already write very sophisticated programs. For example, we can construct a program to compute factorials:

Pair* program =
allocate<Pair>
(
  allocate<Symbol>(“define”),
  allocate<Pair>
  (
   allocate<Symbol>(“factorial”),
   allocate<Pair>
   (
    allocate<Pair>
    (
     allocate<Symbol>(“lambda”),
     allocate<Pair>
     (
      allocate<Pair>(allocate<Symbol>(“x”), 0),
      allocate<Pair>
      (
       allocate<Symbol>(“if”),
       allocate<Pair>
       (
        allocate<Pair>
        (
         allocate<Symbol>(“=”),
         allocate<Pair>
         (
          allocate<Symbol>(“x”),
          allocate<Pair>
          (
           allocate<Number>(0),
           0
          )
         )
        ),
        allocate<Pair>
        (
         allocate<Number>(1),
         allocate<Pair>
         (
          allocate<Pair>
          (
           allocate<Symbol>(“*”),
           allocate<Pair>
           (
            allocate<Symbol>(“x”),
            allocate<Pair>
            (
             allocate<Pair>
             (
              allocate<Symbol>(“factorial”),
              allocate<Pair>
              (
               allocate<Pair>
               (
                allocate<Symbol>(“+”),
                allocate<Pair>
                (
                 allocate<Number>(-1),
                 allocate<Pair>
                 (
                  allocate<Symbol>(“x”),
                  0
                 )
                )
               ),
               0
              )
             ),
             0
            )
           )
          ),
          0
         )
        )
       )
      )
     )
    ),
    0
   )
  )
);

Yikes! That’s horrible! That will correctly construct a program to define the factorial function (which our interpreter will correctly evaluate), but nobody would want to write programs if they had to be written like that!

Using the S-Expression form, this program can be represented much more concisely:

(define factorial
(lambda (x)
  (if (= x 0)
   1
   (* x (factorial (+ x -1)))
  )
)
)

That’s certainly much closer to what we humans would prefer to write! The job of translating the convenient shorthand of S-Expressions into the explicit in-memory construction required by our interpreter is the responsibility of a parser. We’ll look at how to write an S-Expression parser in the next installment.

• Foundational Calculi for Programming Languages
• Introduction to Lambda Calculus
• Making [Erlang] (Joe Armstrong’s PhD thesis)

The “foundational calculi” paper introduces both the lambda and pi calculus. It explains Church numerals, booleans, and data structures purely in terms of lambda expressions. It’s written by Benjamin Pierce, whose book Types and Programming Languages is one of the holy books of programming language theory.

Joe Armstrong’s thesis is especially interesting for this little insight into the politics of software development at large companies:

In February 1998 Erlang was banned for new product development within Ericsson—the main reason for the ban was that Ericsson wanted to be a consumer of software technologies rather than a producer.

They reversed this position a short time later, much to the relief of their bottom line. This case especially highlights the central role that niche programming languages (formalized industry jargon) play in realizing the dreams of businesses.