In my recent post I discuss how our systems could become even less coupled when we pass functions – “Verbs” – around, instead of passing entire Dependencies.

Examples are given in CoffeeScript and C# using MEF.

Read more here …

This can also be done a line at a time in the PowerShell command line REPL (Read Eval Print Loop). Meaning, you can create different arrays and multidimensional arrays at the command line and then type and call the Excel functions.

Plus, these are only 4 of the many Excel functions available.

If you want to find out the other Excel math functions available, type this line at the command line (run the first three lines of code up to the WorksheetFunction first):

$wf | Get-Member -MemberType method | Out-GridView

# Calling Excel Math Functions From PowerShell            

$xl        = New-Object -ComObject Excel.Application
$xlprocess = Get-Process excel            

$wf   = $xl.WorksheetFunction
$data = 1,2,3,4
$m    = ((1,2,3),(4,5,6),(7,8,9))            

Write-Host -ForegroundColor green Median
$wf.Median($data)            

Write-Host -ForegroundColor green StDev
$wf.StDev($data)            

Write-Host -ForegroundColor green Var
$wf.Var($data)            

Write-Host -ForegroundColor green MInverse
$wf.MInverse($m)            

$xl.quit()
$xlprocess | kill

# Results
Median
2.5
StDev
1.29099444873581
Var
1.66666666666667
MInverse
-4.5035996273705E+15
9.00719925474099E+15
-4.5035996273705E+15
9.007199254741E+15
-1.8014398509482E+16
9.00719925474099E+15
-4.5035996273705E+15
9.00719925474099E+15
-4.5035996273705E+15

Grab the Code

I put the code up on my GitHub PowerShell Repository.

PowerShell inside Visual Studio 2010

That’s right, PowerShell inside Visual Studio 2010. Interesting possibilities. He also demoed (code that won’t will be shipped) where he worked Visual Studio like a puppet on strings from the PowerShell console. Again, interesting possibilities.

NuPack is a free, open source developer focused package management system for the .NET platform. I’ve blogged on NuPack and Power Console. PowerConsole has been re-released as part of NuPack.

• Check Available .NET Packages without NuPack Installed
• NuPack – Open Source Package Manager for .NET – Includes a PowerShell Interface
• Visual Visual Studio Keymaps using PowerShell
• Accessing Visual Studio’s Automation API From PowerShell
• PowerShell and F# Consoles in Visual Studio 2010
• PowerConsole – PowerShell Integrated with Visual Studio 2010

DM F#, Clojure, and Scala are all fairly new and popular languages this decade, the former two with striking resemblance to OCaml and Lisp, respectively, and the lattermost being more original in syntax. In what way do these languages represent forward thinking in language design, or how do they fail to build upon lessons learned in more venerable languages like Haskell?

KT It’s a common story in the world of software that time brings increasingly precise and accurate programs.  Anybody who grew up playing games on an Atari 2600 has witnessed this firsthand.  The image of a character has changed from one block, to five blocks, to fifty blocks, to (eventually) thousands of polygons.  By this modern analog of the Greeks’ method of exhaustion, Mario’s facial structure has become increasingly clear.  This inevitable progression, squeezed out between the increasing sophistication of programmers and the decreasing punishment of Moore’s Law, has primarily operated at three levels: the images produced by game programs, the logic of the game programs themselves, and finally the programming languages with which programs are produced.  It’s this last group that deserves special attention today.

The lambda calculus (and its myriad derivatives) exemplifies this progression at the level of programming languages.  In the broadest terms, you have the untyped lambda calculus at the least-defined end (which closely fits languages like Lisp, Scheme, Clojure, Ruby and Python), and the calculus of constructions at the most-defined end (which closely fits languages like Cayenne and Epigram).  With the least imposed structure, you can’t solve the halting problem, and with the most imposed structure you (trivially) can solve it.  With the language of the untyped lambda calculus, you get a blocky, imprecise image of what your program does, and in the calculus of constructions you get a crisp, precise image.

Languages like F#, Scala and Haskell each fall somewhere in between these two extremes.  None of them are precise enough to accept only halting programs (although Haskell is inching more and more toward a dependently-typed language every year).  Yet all of them are more precise than (say) Scheme, where fallible human convention alone determines whether or not the “+” function returns an integer or a chicken.  But even between these languages there is a gulf of expressiveness.  Where a language like C is monomorphically-typed (and an “int_list” must remain ever distinguished from a “char_list”), F# introduces principled polymorphic types (where, pun intended, you can have “’a list” for any type ‘a).  Beyond that, Haskell (and Scala) offer bounded polymorphism so that constraints can be imposed (or inferred) on any polymorphic type – you can correctly identify the “==” function as having type “Eq a => a -> a -> Bool” so that equality can be determined only on those types which have an equivalence relation, whereas F# has to make do with the imprecise claim that its “=” function can compare any types.

No modern programming language is perfect, but the problems that we face today and the history of our industry points the way toward ever more precise languages.  Logic, unreasonably effective in the field of computer science, has already set the ideal that we’re working toward.  Although today you might hear talk that referential transparency in functional languages makes a parallel map safe, tomorrow it will be that a type-level proof of associativity makes a parallel reduce safe.  Until then, it’s worth learning Haskell (or Scala if you must), where you can count the metaphorical fingers on Mario’s hands, though not yet the hairs of his moustache.

DM One might assume that increased “precision” in a language would come at the cost of increased complexity in concepts and/or syntax. In a research scenario, the ability to solve the halting problem certainly has its merits, but is that useful for modern commercial application development, and, if so, does it justify the steeper learning curve?

KT It’s absolutely true that languages that can express concepts more precisely also impose a burden on programmers, and that a major part of work in language design goes into making that burden as light as possible (hence type-inference, auto roll/unroll for recursive types, pack/unpack for existential types, etc).

However — to your point about the value of that increased precision – I would argue that it’s even more important in commercial application development than in academia.  For example, say you’ve just rolled out a new pricing server for a major client.  Does it halt?  There’s a lot more riding on that answer than you’re likely to find in academia.  And really, whether or not it halts is just one of the simplest questions you can ask.  What are its time/space characteristics?  Can it safely be run in parallel?  Is it monotonic?  In our business, these questions translate into dollars and reputation.  Frankly I think it’s amazing that we’ve managed to go on this long without formal verification.

DM Most dynamic languages have some air of functional programming to them, while still being fundamentally imperative. Even more rigorous imperative languages like C# are picking up on a more functional style of programming. The two languages you mentioned earlier, Cayenne and Epigram, are both functional languages. Are we moving toward a pure functional paradigm, or will there continue to be a need for imperative/functional hybrids?

KT What is a “functional language”?  If it’s a language with first-class functions, then C is a functional language.  If it’s a language that disallows hidden side-effects, then Haskell isn’t a functional language.  I think that, at least as far as discussing the design and development of programming languages is concerned, it’s well worth getting past that sort of “sales pitch”.

I believe that we’re moving toward programming languages that allow programmers to be increasingly precise about what their programs are supposed to do, up to the level of detail necessary.  The great thing about referential transparency is that it makes it very easy to reason about what a function does — it’s almost as easy as high school algebra.  However, if you’re not engaged in computational geometry, but rather need to transfer some files via FTP, there’s just no way around it.  You’ve got to do this, then this, then this, and you’ll need to switch to something more complicated, like Hoare logic, to reason about what your program is doing.  Even there, you have plenty of opportunity for precision — you expect to use hidden side-effects but only of a certain type (transfer files but don’t launch missiles).

But maybe the most important tool in programming is logic itself.  It’s a fundamental fact – well known in some circles as the “Curry-Howard isomorphism” – that programs and proofs are equivalent in a very subtle and profound way.  This fact has produced great wealth for CS researchers, who can take the results painstakingly derived by logicians 100 years ago and (almost mechanically) publish an outline of their computational analog.  Yet, although this amazing synthesis has taken place rivaling the unification of electricity and magnetism, most programmers in industry are scarcely aware of it.  It’s going to take some time.

I think there’s a good answer to your question in that correspondence between logic and programming.  The function, or “implication connective” (aka “->”), is an important tool and ought to feature in any modern language.  As well, there are other logical connectives that should be examined.  For example, conjunction is common (manifested as pair, tuple, or record types in a programming language), but disjunction (corresponding to variant types) is less common though no less important.  Negation (corresponding to continuations consuming the negated type), predicated quantified types, and so on.  The tools for building better software are there, but we need to work at recognizing them and putting them to good use.

Anybody interested in understanding these logical tools better should pick up a copy of Benjamin Pierce’s book Types and Programming Languages.

DM Many frameworks have a one or more DSLs to express things as mundane as configuration to tasks as complicated as UI layout, in an effort to be more express specific kinds of ideas more concisely. Do you see this as an expanding part of the strategy for language designers to increase expressiveness in code? Is it possible that what we consider “general purpose languages” today will become more focused on marshalling data from one DSL to another, or will DSLs continue to remain a more niche tool?

KT I guess it depends on what you mean by “DSL”.  Like you say, some people just have in mind some convenient shorthand serialization for data structures (I’ve heard some people refer to a text format for orders as a DSL, for example).  I’m sure that will always be around, and there’s nothing really profound about it.

On the other hand, by “DSL” you could mean some sub-Turing language with non-trivial semantics.  For example, context-free grammars or makefiles.  Modern programming languages, like Haskell, are often used to embed these “sub-languages” as combinator libraries (the “Composing Contracts” paper by Simon Peyton-Jones et al is a good example of this).  I think it’s likely that these will continue as valuable niche tools.  If you take monadic parser combinators for example, it’s very attractive the way that they fit together within the normal semantics of Haskell, however you’ve got to go through some severe mental gymnastics to determine for certain what the space/time characteristics of a given parser will be.  Contrast that with good old LALR(1) parsers, where if the LR table can be derived you know for certain what the space/time characteristics of your parser will be.

On the third hand, if a “DSL” is a data structure with semantics of any kind, your description of a future where programs are written as transformations between DSLs could reasonably describe the way that compilers are written today.  Generally a compiler is just a sequence of semantics-preserving transformations between data structures (up to assembly statements).  I happen to think that’s a great way to write software, so I hope it’s the case that it will become ever more successful.

In PowerShell Version 1 that cmdlet is not available so you need to use the one of the Load* static methods found in System.Reflection.Assembly.

Why?

For example, in PowerShell, if you want to encode a Url you can use the UrlEncode method found in the .NET HttpUtility Class.

If you fire up PowerShell and try to call the static method UrlEncode found in System.Web.HttpUtility you’ll get this error.

[System.Web.HttpUtility]::UrlEncode("this is a test")

Unable to find type [System.Web.HttpUtility]: make sure that the assembly containing this type is loaded.

You can solve this by loading the System.Web .NET assembly. This is equivalent to being in the Visual Studio IDE, adding a reference and then adding a using statement.

Using Reflection.Assembly in PowerShell Version 1

[Reflection.Assembly]::LoadFile( `
  'C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\System.Web.dll')`
  | out-null            

[System.Web.HttpUtility]::UrlEncode("this is a test")

Using Add-Type in PowerShell Version 2

Add-Type -AssemblyName System.Web
[System.Web.HttpUtility]::UrlEncode("this is a test")

Additional Reflection.Assembly usage

[void] [Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms")
[reflection.assembly]::Load("mscorlib.dll") | Out-Null
[void][system.reflection.assembly]::loadfrom("c:\windows\system32\inetsrv\microsoft.web.administration.dll")

Additional Add-Type usage

Add-Type -AssemblyName "System.Windows.Forms"
Add-Type -AssemblyName "mscorlib.dll"
Add-Type -Path "c:\windows\system32\inetsrv\microsoft.web.administration.dll"

Compiling C# on the fly in Version 2

Add-Type -TypeDefinition @"
public class Test
{
    public static int Add(int n1, int n2)
    {
        return n1 + n2;
    }
}
"@

If you change the signature of the Add method you will get this error:

When compiling code on the fly or loading assemblies they cannot be unloaded.

.NET assemblies can’t be unloaded from a session (it’s a .NET thing, not a PowerShell thing) therefore DLLs can’t be unloaded from a session. This means that you can’t update a DLLs once it’s been loaded. We can’t even update the assembly on disk because the file is locked when the assembly is loaded.

- Bruce Payette Windows PowerShell in Action, Second Edition

Another way to use the –Path parameter on Add-Type

You can point to a C# source code file directly.

Add-Type –Path c:\test\Hello.cs

Add-Type is not limited to compiling only C#

You can inline these languages using the –Language parameter CSharp, CSharpVersion3, VisualBasic, JScript.

Notice, F# is not supported. Here is a post How to add an F# type to a PowerShell session that inlines F#.

NOTE: This uses the F# DLLs shipped with .NET 4.0

Add-TypeFSharp -TypeDefinition @"
module MyModule

let Add a b = a + b;;
"@                        

1..10 | % { [MyModule]::Add( $_, $_*2 ) }

Finally

There are additional parameters that the Add-Type cmdlet supports. You can generate a DLL file for the assembly with the specified name in the location, specify compiler parameters, specify assemblies which your code depends on and use wild cards when loading assemblies.

Check out the online help for Add-Type.

In PowerShell Version 1 that cmdlet is not available so you need to use the one of the Load* static methods found in System.Reflection.Assembly.

Why?

For example, in PowerShell, if you want to encode a Url you can use the UrlEncode method found in the .NET HttpUtility Class.

If you fire up PowerShell and try to call the static method UrlEncode found in System.Web.HttpUtility you’ll get this error.

[System.Web.HttpUtility]::UrlEncode("this is a test")

Unable to find type [System.Web.HttpUtility]: make sure that the assembly containing this type is loaded.

You can solve this by loading the System.Web .NET assembly. This is equivalent to being in the Visual Studio IDE, adding a reference and then adding a using statement.

Using Reflection.Assembly in PowerShell Version 1

[Reflection.Assembly]::LoadFile( `
  'C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\System.Web.dll')`
  | out-null            

[System.Web.HttpUtility]::UrlEncode("this is a test")

Using Add-Type in PowerShell Version 2

Add-Type -AssemblyName System.Web
[System.Web.HttpUtility]::UrlEncode("this is a test")

Additional Reflection.Assembly usage

[void] [Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms")
[reflection.assembly]::Load("mscorlib.dll") | Out-Null
[void][system.reflection.assembly]::loadfrom("c:\windows\system32\inetsrv\microsoft.web.administration.dll")

Additional Add-Type usage

Add-Type -AssemblyName "System.Windows.Forms"
Add-Type -AssemblyName "mscorlib.dll"
Add-Type -Path "c:\windows\system32\inetsrv\microsoft.web.administration.dll"

Compiling C# on the fly in Version 2

Add-Type -TypeDefinition @"
public class Test
{
    public static int Add(int n1, int n2)
    {
        return n1 + n2;
    }
}
"@

If you change the signature of the Add method you will get this error:

When compiling code on the fly or loading assemblies they cannot be unloaded.

.NET assemblies can’t be unloaded from a session (it’s a .NET thing, not a PowerShell thing) therefore DLLs can’t be unloaded from a session. This means that you can’t update a DLLs once it’s been loaded. We can’t even update the assembly on disk because the file is locked when the assembly is loaded.

- Bruce Payette Windows PowerShell in Action, Second Edition

Another way to use the –Path parameter on Add-Type

You can point to a C# source code file directly.

Add-Type –Path c:\test\Hello.cs

Add-Type is not limited to compiling only C#

You can inline these languages using the –Language parameter CSharp, CSharpVersion3, VisualBasic, JScript.

Notice, F# is not supported. Here is a post How to add an F# type to a PowerShell session that inlines F#.

NOTE: This uses the F# DLLs shipped with .NET 4.0

Add-TypeFSharp -TypeDefinition @"
module MyModule

let Add a b = a + b;;
"@                        

1..10 | % { [MyModule]::Add( $_, $_*2 ) }

Finally

There are additional parameters that the Add-Type cmdlet supports. You can generate a DLL file for the assembly with the specified name in the location, specify compiler parameters, specify assemblies which your code depends on and use wild cards when loading assemblies.

Check out the online help for Add-Type.

Here is a quick trivial example that illustrates this:

The Tuple class used in the above snippet, represents a notion of an ordered set of elements (this will be available in .net 4.0 as part of the core framework). As usual, peeking into the above snippet using Reflector shows how the environment is shared as seen below:

Looking into the reflector output shows how both the Increment and Decrement functions are closed over the same environment. This can be tested by invoking the SharedClosure() as follows:

The output, as expected, will be 11, 12, 11, 10. The interesting thing to note in this case is that even if the value of i is changed after the closure has been created the output will remain the same. This can be verified by changing the value of i after line 2. This is because only a copy of i was passed to the SharedClosure() method and any change in i will not impact the closure. But if the seed being passed to the SharedClosure() was a reference type, the closure will be affected by any change in the seed.

In conclusion, the ability to share the environment between multiple functions using closures is a very powerful feature but could also lead to some nasty side effects when handled incorrectly. In future posts, I will describe how this feature can be applied to achieve some interesting results.

A closure, by definition, captures the environment in which the function was created and makes it available over multiple invocation of the function. A simple example of a closure in C# is as follows:

When the above snippet get executed, the value that gets printed is 25 instead of 1. In this case, the lambda is bound to a variable which is accessible outside the closure and whose value could be changed outside the closure. This is because in C#, the closure captures the reference to i and not its value. Analyzing the generated output using Reflector provides detailed insight into how closures are implemented in C#.

Going further and analyzing the generated output using ILDASM should give an indication how the actual value of i is getting updated.

Looking at line numbers 13 and 21 shows how the value of i is updated resulting in an output of 25 when the action is invoked.

Written mostly in F# via Don Syme

A  Todo-List in XML

XML is really just a tree serialized to a human readable form. Another way to express that is, XML is a way to store information in a hierarchical form.

<todo name="housework">
    <task priority="high">Clean the house.
    <task priority="medium">Wash the dishes.
    <task priority="medium">Buy more soap.
</todo>            

Parsing XML in PowerShell

Here we use PowerShell’s accelerator [xml] with a here-string to create an XmlDocument. In the end we use dot notation to print out the tasks.

$xml = [xml] @"
<todo name="housework">
    <task priority="high">Clean the house.
    <task priority="medium">Wash the dishes.
    <task priority="medium">Buy more soap.
</todo>
"@                        

$xml.todo.task            

# Results
priority #text
-------- -----
high     Clean the house.
medium   Wash the dishes.
medium   Buy more soap.

Corresponding PowerShell Version

Looks like this with the supporting PowerShell function described below.

New-TodoList "housework" {
    New-Task high "Clean the house."
    New-Task medium "Wash the dishes."
    New-Task medium "Buy more soap."
}

How To Make It Executable

Metaprogramming is the term used to describe the activity of writing programs that create or manipulate other programs.

PowerShell scriptblocks are at the center of most of the metaprogramming techniques.

– “PowerShell In Action”

The PowerShell function New-TodoList takes a scriptblock as one of its parameters. We use the call operator “&” to invoke the scriptblock passed in the $code parameter. The New-Task function creates and returns a PSObject with three properties Name, Priority and Task.

Note: The inner function is able to “see” the $name variable in the outer functions parameter list.

Function New-TodoList ($name, [ScriptBlock] $code) {            

    Function New-Task ($priority, $task) {
        New-Object PSObject -Property @{
            Name     = $name
            Priority = $priority
            Task     = $task
        }
    }            

    & $code
}

Putting It Together

You can look at this in few ways. We have used PowerShell and scriptblocks to construct a macro/domain specific language (DSL) to transform our Todo-List data.

New-TodoList "housework" {
    New-Task high "Clean the house."
    New-Task medium "Wash the dishes."
    New-Task medium "Buy more soap."
}            


Name      Priority Task
----      -------- ----
housework high     Clean the house.
housework medium   Wash the dishes.
housework medium   Buy more soap.


 

The PowerShell functions here demonstrate one way to generate output. This example chose to present the data as an array of PSObjects. The data emitted could take .many shapes, for example nested hash tables.

The POWER You Get That You Can’t Get With XML

Jeffrey Snover, creator of PowerShell, pointed out this key aspect. By using PowerShell scriptblocks in our little language, it gives us the full power of PowerShell so we can do following, something we can’t get with XML.

New-TodoList "housework" {
    New-Task high "Clean the house."
    New-Task medium "Wash the dishes."
    New-Task medium "Buy more soap."            

    ForEach ($item in “Soap”,”Beer”,”Doritos”,”Salsa”,”Microsoft Stock”) {
        New-Task medium “Buy more $item.”
    }            

}            

# Results
Name      Priority Task
----      -------- ----
housework high     Clean the house.
housework medium   Wash the dishes.
housework medium   Buy more soap.
housework medium   Buy more Soap.
housework medium   Buy more Beer.
housework medium   Buy more Doritos.
housework medium   Buy more Salsa.
housework medium   Buy more Microsoft Stock.

Quick Summary

XML has been used for a long time to create DSLs. A couple examples are ANT and NAnt which are specific to building projects.

Using DSLs to solve problems results in much more compact, maintainable, flexible programs. PowerShell helps take this abstraction further. Compare writing NAnt in pure C# and not using XML.

Additional Reading

“The Nature of Lisp” – Slava Akhmechet http://www.defmacro.org/

Because why anyone would want to write VBA in the Visual Studio macro editor is beyond me.

I agree!

Kurt lays out  the “how to” for injecting the Visual Studio’s DTE object (Development Tools Extensibility) into the F# console. He also provides the link to Visual Studio .NET Automation Object Model.

PowerShell PowerConsole

Makes accessing the Automation API simple. It brings the Visual Studio DTE automation object to your finger tips. Check this post for downloading PowerConsole – PowerShell Integrated with Visual Studio 2010.

$DTE.Solution.Projects | ForEach {$_.Fullname}

PS> $DTE.Solution.Projects | ForEach {$_.Fullname}

C:\PoShScripts\NYC Code Camp\CSharp\WPF+PoSh\Simple\Simple.csproj

C:\PoShScripts\NYC Code Camp\CSharp\WPF+PoSh\PowerShellExtensions\PowerShellExtensions.csproj

C:\PoShScripts\NYC Code Camp\CSharp\WPF+PoSh\HostAndAddVariables\HostAndAddVariables.csproj

No problem

Add-Type has a parameter CodeDomProvider and you can pass the FSharpCodeProvider to it (see the function Add-TypeFSharp).

Add-TypeFSharp lets you inline F# code.

Add-TypeFSharp -TypeDefinition @"
module MyModule

let Add a b = a + b;;
"@            

1..10 | % { [MyModule]::Add( $_, $_*2 ) }

Things TODO

• Install .NET 4.0, VS 2010 installs this and the latest F# bits
• Download and install The F# PowerPack
• Update the registry so PowerShell can work with .NET 4.0

Updating the registry

This is a bit tricky, thanks to the help of the PowerShell MVP community (Shay, Oisín, Marco and Vadims)on getting this to work.

Before you are allowed to edit this entry, you need to change the permissions on that key and take full ownership.

Then change the RuntimeVersion value, under the following key, to v4.0.30319.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\PowerShellEngine

Function Add-TypeFSharp

Function Add-TypeFSharp {
    param(
        [string]$TypeDefinition
    )            

    Add-Type -Path `
        "C:\Program Files\FSharpPowerPack-1.9.9.9\bin\FSharp.Compiler.CodeDom.dll"            

    $provider = New-Object `
        Microsoft.FSharp.Compiler.CodeDom.FSharpCodeProvider            

    Add-Type -TypeDefinition $TypeDefinition -CodeDomProvider $provider
}

Notes

• Investigate using a PowerShell Proxy command for Add-Type and extend the Language parameter
• Enhance Add-TypeFSharp to take an F# file in addition to a string

The PowerConsole also supports IronPython and IronRuby.

It’s not often that you get to spearhead the creation of a major new Microsoft API and it feels great to have seen this through from beginning to end

As I mentioned last month, I want to learn a functional programming language.  Lucky for me, Dr. Erik Meijer (and his electric t-shirt) gave a course on Haskell in October and its posted on Channel 9.

I watched the first lecture over a week ago but haven’t had time to do the homework till now.

Here it is in C#.  Its quicksort written as he described it in Haskell.  It doesn’t feel that different from the programming I usually do, but it is recursive (which I generally avoid), so maybe that’s something.  At least I did my homework, now I can go on to the second lecture .

As long as I was making a program, I figured I’d expand my WPF horizons and used an attached behavior for the button.  Of course its MVVM which is just a habit now.

Attached Behaviors

It wasn’t obvious how to link the static attached behavior to the instance properly.

A guy on StackOverflow recommended getting the DataContext from the sender in the event:

var fxElem = sender as FrameworkElement;

var vm = (VM) ((FrameworkElement)args.OriginalSource).DataContext;

Most other explanations of attached behaviors are limited to working with the control itself and not the ViewModel:

http://eladm.wordpress.com/2009/04/02/attached-behavior/
http://www.codeproject.com/KB/WPF/AttachedBehaviors.aspx

You can download the code here.

Here’s the homework code:

public  IEnumerable<IComparable>

    Sort(IEnumerable<IComparable> listToSort)

{

    if (listToSort == null)

        throw new InvalidOperationException

            ("Cannot sort a null list.");

    if (listToSort.Count() == 0)

        return listToSort;

    var result = new List<IComparable>();

    var start = listToSort.First();

    //smaller on left

    result.AddRange(Sort(listToSort.Skip(1)

        .Where(e => e.CompareTo(start) <= 0)));

    //val in middle

    result.Add(start);

    //larger on right

    result.AddRange(Sort(listToSort.Skip(1)

        .Where(e => e.CompareTo(start) > 0)));

    return result;

}

Here’s some highlights of the attached behavior class:

Hooking up the button in XAML is simple:

<Window x:Class=”WpfApplication1.Window1″

xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”

xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”

xmlns:local=”clr-namespace:WpfApplication1″

Title=”Window1″ Height=”129″ Width=”300″>

<Button local:SortStarter.SortBehavior=”True” >Sort</Button>

]]> http://goldmanalpha.wordpress.com/2009/12/26/functional-homework/feed 0 http://goldmanalpha.wordpress.com/2009/11/13/functionallanguage/ http://goldmanalpha.wordpress.com/2009/11/13/functionallanguage/#comments Fri, 13 Nov 2009 16:51:42 +0000 Aaron Goldman http://goldmanalpha.wordpress.com/?p=136 ]]>

There are a lot of people @Lab who know things like Erlang, Haskell, Python, R, etc., etc., etc.  I’m actually considering learning Lisp, but I need a project.  I “learned” php for a project over the summer, but I didn’t really like it since its so much like classic ASP (really messy).  Maybe I’ll learn F# instead…  That shouldn’t be such a strech since I built an ML compiler in grad school (F# is based on ML).  I wonder what kind of project I’d be interested in would benefit from a functional language…  I suppose I’ll have to read up on the language and maybe that will give me some ideas.

Its funny, doing WPF, my umpteenth technology, is just not enough.

]]> http://goldmanalpha.wordpress.com/2009/11/13/functionallanguage/feed 0 http://feedproxy.google.com/~r/DevelopmentInABlink/~3/mRs4u15KFU4/ http://feedproxy.google.com/~r/DevelopmentInABlink/~3/mRs4u15KFU4/#comments Mon, 26 Oct 2009 02:36:16 +0000 Doug Finke http://dougfinke.com/blog/index.php/2009/10/25/how-to-use-mongodb-from-powershell-and-f/ Dwight Merriman, founder of 10gen,  presented today, at Lab49 where I work, about his new database platform:

Mongo (from "humongous") is a high-performance, open source, schema-free document-oriented database. MongoDB is written in C++ and offers the following features:

• Collection oriented storage: easy storage of object/JSON -style data
• Dynamic queries
• Full index support, including on inner objects and embedded arrays
• Query profiling
• Replication and fail-over support
• Efficient storage of binary data including large objects (e.g. photos and videos)
• Auto-sharding for cloud-level scalability

PowerShell

Dwight pointed out the way different languages could to talk to MongoDB and C# was one of them using a community developed driver.

Following the post Experimenting with MongoDB from C#, MongoDB was up and running and I was storing and retrieving data in no time.

F#

Looks like Matthew Podwysocki used the same reference and has MongoDB working with F#.

Some highlights of the code are:

• Foundational material on F# async programming, and samples including:
  • Fetching web pages in parallel
  • The Bing translator sample using async programming
  • A Graphical, asynchronous twitter client in 50 lines
  • A Twitter feed example that uses F# first class reactive event processing to process the stream of feeds
  • An asynchronous web crawler
  • Processing images in parallel, using a mixture of I/O and CPU parallelism
• An updated version of the famous Interactive DirectX demo
• Lots of great micro samples on foundational topics such as
  • object oriented programming
  • language oriented programming
  • function composition
  • design patterns
  • discriminated unions
  • records
  • units of measure

via F# in VS2010

The conference took place over a three-day period.  The first and third days were broken down into four sessions.  The second day ended early, after Dr. Pierce’s presentation, and only had two sessions.  Each session consisted of two to four presentations around a common theme.  This structure worked well enough, and I’ll provide summaries in the same order.

Domain-Specific Functional Languages

First was a presentation on PICOBIT, a compact compiler and virtual machine for a Scheme language suitable for use with microcontrollers.  The Scheme compiler was a fairly straightforward mapping to C.  However, the authors also wrote another compiler for the subset of C that their first compiler produced.  In this C-language, recursion is disallowed, and function-callers pass parameters by modifying their callee’s variables directly.  There was some skepticism about this approach (versus the use of a standard C compiler), although the published performance numbers were reasonable.

Next was a presentation on “Kansas Lava,” a project from the University of Kansas which aims to embed a low-level “circuit” language in Haskell, so that either correct VHDL code can be generated, or the circuit can be tested by the Haskell evaluator.  Here, most of the interesting work in the presentation involved the type-system machinery needed to track the size (in bits) of values as they’re produced by non-trivial compositions of other sized values.  There were no interesting circuit optimizations present, and in fact the speaker demonstrated that even trivial optimizations (e.g.: “1 and X” => X) weren’t done.  Beyond the basic use-case for dependent types, the speaker also sang the praises of applicative functors in his application (twice over a win for Conor McBride).

On a similar note, the next speaker presented his work on Bluespec SystemVerilog, a commercial compiler with a Haskell-like system of Hindley-Milner type inference plus multiple-parameter type classes with functional dependencies.  Also like Kansas Lava, much of the difficult work (and the speaker’s most emotional subject) involved inching toward dependent types with type-level arithmetic and the inevitable difficulties of reasoning about arithmetic expressions.  The speaker was presented with an appeal to resort to standard “computer algebra” systems, but in fairness it’s pretty awkward to prove involved propositions about arithmetic in his type system.  There was also some interest in moving to a legitimate dependent-type system.

Finally, the last presentation in this session was called “iTasks for End-users,” which covered a web-based “task-management” application similar to FogBugz, Trac, or Mingle.  The novel aspect of this system is that the heavily Javascript-based web forms were derived automatically from the structure of the types of records representing those forms, and that the complex client/server interaction and state management involved in processing forms could be written in a straightforward monadic style.  The speaker conceded that most people found the subject of task-management unrewarding, and there was widespread agreement that he was right.

Multi-core Functional Languages

The first presentation in this session was one of Jane St. Capital’s “Summer of Code” winners – a concurrent garbage collector for O’Caml.  The O’Caml runtime system by default has a garbage collector that is particularly hostile to concurrency.  This project used the sensible approach of allocating level-one heaps distinctly to each thread in a process, and compacting to a shared heap when necessary.  Global synchronization happened only in the case where collection/compaction against the shared heap was invoked.  Overall the reported performance numbers looked promising, and the project seemed worthwhile (if you have to use O’Caml).  Unfortunately, the speaker’s accent was too difficult to understand (and ours for him, as he wasn’t able to parse audience questions) so a few good questions about performance edge-cases went unanswered.

Next was a presentation on a coordination language called “S-Nets.”  The language had explicit forms for the parallel and sequential composition of programs, as well as some type-directed dispatching logic that would decide when and how inputs would be presented to programs in these networks.  Because it was an explicit coordination language, complex concurrency scenarios translated to fairly complex expressions, but it may make a reasonable shell language, or a target for Hume (the next presentation).

Finally, the session ended with a presentation of “Hume,” a system for automatically partitioning a program across multiple cores by a “cost-directed” application of meaning-preserving program transformations, the whole body of which the authors have named the “box calculus” (“box” in this case refers to an individual process or thread of execution).  The presenter’s animation of these program transformations was compelling, although slightly dampened by the fact that Hume’s language was limited to non-recursive programs.

Functional Programming and Education

First up in this session was Andy Gill (also the presenter for the Kansas Lava project) discussing his “Chalkboard” library for writing efficient 2D image-rendering programs.  His system was inspired by Conal Elliott’s Vertigo library, a very influential and straightforward (if not inefficient) system for representing images as their sampling functions.  It qualifies as “education-related” in that it has been used to introduce problems of basic graphics programming to undergraduate students.

Next up was a presentation on “Holmes,” a system for detecting fraudulent homework submissions by a search against historical submissions.  The system worked by applying a series of heuristics, ranging from simple token list edit-distance comparison to similar functions on the control-flow-graph representation of two programs.  A project like this might generally be considered depressing, but it energized the audience like few others.

Finally, the session ended with a presentation appealing to attendees to make formal logic (as applied to proving software correctness) a required course in its own right (and not a sidebar in a general “discrete math” course).  Some statistics were used to make this case, showing that the subject can reasonably be taught to undergraduate students, and briefly surveying the results from schools that do treat the subject adequately.  No clear curricula were presented (in fact, the presenter demonstrated that the two most popular remotely-applicable textbooks only mentioned the subject of logical software verification briefly).  Some schools have taken the Coq’Art book as a starting point, although the speaker didn’t address that (it may be too early for judgment).

Parallel Programming and Reactive Programming

The “pHood” tool for analyzing the performance of parallel functional programs (especially Haskell programs) was presented in this final session of the first day.  Based on a similar tool (named “Hood”) for sequential programs, pHood allows the transparent insertion of some monitoring-annotations (presumably driven by some very cautious use of ‘unsafePerformIO’) to determine how effectively your program is making use of parallelism.

Next was a presentation on “fusion-equipped parallel skeletons,” an interesting (even useful) technique for realizing implicit data-parallelism, simultaneously eliminating redundant work by fusion.  Fusion is typically known as a compiler technique that saves work by “fusing” compositions of functions to eliminate the production of redundant intermediate structures (possibly the simplest example is the identity: map f (map g X) = map (f . g) X).  Impressively (or horrifically, depending on your political persuasion) this was achieved in C++ by a novel application of expression templates (effectively, compile-time representations of C++ parse trees realized by clever template functions, and functions operating on those trees).  For commercial users of C++, this is unquestionably a valuable technique to study.

”Liftless Functional Reactive Programming” was the subject of the next talk.  Like the “Chalkboard” talk before it, this project also traces back to the work by Conal Elliott (and others on FRP generally).  Here the speaker developed an FRP system in Scala, and sought to avoid some of the tediousness of other Haskell-based FRP systems by avoiding “lifting” – the mechanical process of mapping functions on pure values to functions on values in the context of an FRP signal.  The solution involved the manipulation of a hidden stack of widget actions, maybe a deflating approach to FRP enthusiasts (especially concurrency-minded ones), but probably effective for any users of Scala.

Finally, the last presentation of the session (and the first day) was entitled “Eventless Reactivity From Scratch,” and featured another FRP system (this one in Haskell).  The speaker briefly surveyed a few well-known FRP systems before explaining his own.  Key to this system is applicative binding (e.g.: his first example defines a lifted sum on signals using the applicative ‘liftA2’ function) and time-based signal functions (hence “eventless reactivity”).  A novel (to my eyes) use of the “mdo” syntax for recursive monadic binding within this system allowed a particularly elegant definition of the sine and cosine functions as integrals of each other.  Overall this presentation functioned as an accessible overview of FRP methods in Haskell, and as a clear roadmap for interested developers.  As with the other presentations on this subject, Conal Elliott’s FRP work was also highlighted.

Compilation

The second day began with a very interesting presentation on “compiling SaC for the ‘MicroGrid’ architecture.”  SaC (or “single-assignment C”) is a popular language among researchers in parallel languages, and in this case it was used to realize data-parallel programs on a novel hardware platform called the “MicroGrid.”  This architecture features a massive collection of processors, each capable of evaluating many threads of execution concurrently.  As with several other projects of this sort, it’s nested data-parallelism that poses unique challenges to compiler-writers, and although this architecture went some way toward alleviating those problems, in at least one example the presenter demonstrated sub-optimal distribution at the second level of nested parallelism (e.g.: the first pass distributes across three processors, and from there to 200 threads each).  This project is definitely an interesting work-in-progress.

”Arity raising” was the next topic; the main objective here is to optimize for large register sets by mechanically breaking one argument of a function into multiple arguments (the constituent parts of the argument that are actually used in the body of the function or, recursively, its callees) with the hope that better register usage can be achieved.  As newer processors feature ever larger register sets, the argument for this particular optimization will grow more compelling.  In this case the optimization was applied to a compiler for an ML-language, which made for a good testbed in any case (ML programs probably rely on language-level constructs for composing data more than most other languages).

A restricted form of dependent types, called “Symbiotic Expressions” by their presenters, made for the last presentation in this session.  The “expressions” meant here are actually index variable and array bounds constraints, which are “symbiotic” in that they are associated with AST nodes in the intermediate form of a SaC program.  A few associated optimizations were discussed, including an “algebraic with-loop folding” technique for flattening nested data-parallel loops.

Language Features

Here we received an interesting presentation titled “A Calculus for Coercive Subtyping.”  In fact, the aim of this project is to implement a dependently typed lambda calculus with implicit coercion (as a means of achieving subtyping).  The talk was very dense with typing rules, but overall the approach seemed amenable to the general system of qualified types (translated to an intermediate calculus with explicit witnesses of subtyping/overloading) developed by Mark Jones (though he didn’t study dependent type systems in that work).  Working in a dependently typed lambda calculus also adds some complication in the valid dependency-pairings admitted – i.e.: the presenters defined a lambda cube of overloading parallel to the standard one.  Unfortunately, the pace of the presentation made it difficult to follow, and the audience presented no questions.

A powerful application of “stream fusion” in Haskell was demonstrated next.  Stream fusion is a special case of the general principle of fusion (discussed earlier), which, with the GHC compiler, can be achieved with “rewrite rules” – directives that tell the compiler how certain terms can be simplified (and only valid due to the inherent referential transparency of Haskell terms).  Here the objective was to provide an efficient representation for Unicode strings, allowing (as with regular strings in Haskell) the normal list-functions (e.g.: map, filter, foldl, etc).  To make this work, the standard list functions must be defined in terms of streams rather than lists, and dual stream/unstream functions must be defined on data-types (in this case, on arrays of Unicode characters) in order to give them the expected interface (and to convert from that interface back to the array form).  In this way, a function removing numbers and converting to uppercase on Unicode strings can be expressed as “unstream . (map toUpper) . stream . unstream . (filter isNotDigit) . stream” and the fusion step here will be the application of the rewrite rule “stream (unstream x) = x” (which has the effect of eliminating intermediate structures, as desired).  In all, this was an ideal presentation.

“General contract boundaries” were discussed in the next presentation, in the context of PLT Scheme.  As the first audience member at question-time observed, there was a significant amount of overlap between this project and the general principle of type systems.

Finally the session ended with another gem, an embedding of a bidirectional language in Haskell.  This approach contrasted with the invited presentation by Ben Pierce (to follow) by defining a data-type for bidirectional functions, and combinators on such functions mirroring many standard functions (map, filter, etc).  Perhaps the greatest benefit from this approach, aside from the usual benefits from embedding in Haskell, was that reverse functions could be generated with their forward data trivially.  The bidirectional editor GUI was much less compelling, but it was good enough to demonstrate the basic principles.

How to Build Your Own Bidirectional Programming Language

This invited talk began with a summary of the kind of problem that bidirectional programming aims to solve: in essence, the “view-update” problem familiar to users and developers of relational databases.  As a running example, the presenter used an XML representation of a simple database of composers, and a projection (or “view”) of a subset of that data as simple newline-delimited text.  It should be possible then to update the newline-delimited view and have the original XML representation updated as well.  In contrast to the previous presenter’s solution to the same problem, this one was built as its own self-contained language: Boomerang.  Its primitives are finite-state transducers, just the simplest bidirectional transformations on strings (describable by regular expressions).  A tool inspired by this language is apparently used by RedHat to unify configuration of all sorts of small /etc/ configuration files.  Other applications are immediately possible as well.  For example, as my colleague Ken Overton pointed out, sharing structure representations between multiple languages (e.g.: Java/C++) can be made much simpler, without the intermediate canonical representations that many people resort to.

Theorem Provers and Testing

On this, the third and final day, the first session began with a presentation on “user-interface driven theorem prover development.”  Unfortunately, the subject of the talk was not the theorem prover itself, but just its GUI.  In essence, the presentation was about Intellisense for theorem provers (and thanks to the Curry-Howard isomorphism, the “for theorem provers” part can be dropped).  The presenter digressed at some length on the subject of highlighting sub-expressions.  I don’t remember the word “zipper” mentioned, but that was his approach, in essence.

Next up was a presentation on hygienic macros in the ACL2 theorem prover.  Actually this talk again was only incidentally related to theorem provers.  ACL2 admits a subset of Common Lisp, where higher order functions are strictly disallowed.  To get around this limitation, users of ACL2 use macros heavily (to generate first order functions from macros embodying higher order functions).  That would have been frustrating enough, but their macro system was unhygienic (it allowed macro substitution to capture variables from its enclosing scope) and so this presentation concerned the cleaning up of that mess.  In essence, this is the resolution of the same old “dynamic scope” bug under a different guise.

Finally the last presentation in this session was another gem, and (although strictly speaking it was about “testing”) this presentation came the closest to actually discussing a technique for proving theorems.  Its subject was “unifying QuickCheck and SmallCheck,” which requires some background explanation.  QuickCheck is a novel system for testing Haskell programs (and Erlang, if you need it) by requiring only the specification of straightforward logical propositions (e.g.: “reverse (xs++ys) = (reverse ys)++(reverse xs)”), after which it will randomly search the space of the quantified variables of the proposition (that’s the “xs” and “ys” variables of the last expression) until it finds a configuration that contradicts the proposition.  Having then found a contradiction, it simplifies the offending configuration until it’s as simple as possible, while still demonstrating the contradiction.  SmallCheck is a similar system, except that rather than randomly searching for a contradiction it will exhaustively search for a contradiction (up to a given size).  This presentation concerned the technical details of bringing both of these systems together into one.

Incrementalization, GADTs, Generic Programming

“Incrementalization” of datatype-centric libraries was the focus of the first presentation in this series.  As the presenter defined it, datatype-centric libraries are those where a small set of datatypes are constructed and summarized or translated only within the context of that library.  “Incrementalization” of such a library entails the addition of some accumulating data in the constructors of the relevant datatypes, and the rewriting of “deep” recursive functions over such structures (for example, recursively counting the nodes of a binary tree) to “shallow” functions of the accumulators added to those datatypes (for example, adding the count of children to a binary tree’s constructor, so that counting them is trivial and incrementally building the binary tree also incrementally computes this count from nested binary tree nodes).  This approach of “incrementalizing” libraries amounts to an embedding of attribute grammars in Haskell, which the presenter briefly touched on.

The next presenter introduced the Clean language’s “Dynamic” type (for runtime type checking) and then gave a presentation on simulating Clean’s “Dynamic” with GADTs in Haskell.  His analysis suggested that Clean’s dynamic type-information consumption is dual to an analogous type-information construction inherent in the use of GADTs.

A language (“Qube”) for efficient “rank-generic” array programs formed the basis for the next and final presentation in this session.  This presentation covered the application of dependent typing (in a limited context) to type-check “shape analyses” on array operations.  This is a subject covered in bits and pieces by several different approaches, but it has long seemed that dependent types provide an ideal way to synthesize these many disparate techniques, so I was especially happy to find this presentation.  The generalized treatment of matrix multiplication (for matrix, vector, and scalar cases in each configuration) is particularly amenable to this treatment, as are many other important functions like the inner product and so on.  The presenters covered the type-safe implementation of these functions in Qube, and demonstrated performance numbers on massively parallel hardware that were in most cases almost identical to hand-written C.  All in all, this was one of the most interesting presentations of the entire conference for those of us working in the financial sector.

Evaluation Models and Environments

Leading this session was a presentation on “Call by Effect.”  The presenter initially made the standard argument in favor of effect systems – higher order functions (like map, filter, etc) only need to be written once, and are then polymorphic in both computation types and effect types (in contrast to Haskell where there’s a pure map function and a monadic ‘mapM’).  Following that, the presenter defined “call by effect” as a method of achieving lazy evaluation when pure code is evaluated, and eager evaluation in the presence of effects.  As far as evaluation order goes, this approach allows a language with effect types to reduce terms in the same way as Haskell (lazily when possibly, eagerly when necessary) although, strictly speaking, Haskell offers more than two options.  With its ability to describe effects without special-purpose type-system extensions, Haskell still seems closer to the ideal, but its lifting functions and monad transformers are awkward enough to make effect system research like this worthwhile.  Maybe some day the two will meet somewhere in the middle.

The next presentation was titled “Very Lazy Evaluation: A New Execution Model for Functional Languages.”  It concerned a system similar to abstract machines like the SECD or STG machines, suitable as a target for compilers of functional languages (and specifically those with lazy evaluation).  The approach seemed to me to be a fairly straightforward stack machine, but further details in the post-conference paper may demonstrate more subtle features.

Finally, the last presentation in this session seemed like essentially a small FRP system for mobile devices in Scheme.  Like the other Scheme presentations, there was a fair amount of audience skepticism about basic points of practice (ad-hoc type systems being developed, unclear/inconsistent semantics of low-level OS interaction, etc).  The presenter showed a couple of amusing projects built with this system (one great video was of a group of developers shaking their phones to produce music).

Implementation and Benchmarking

”Record combinators” were the subject of the first presentation in this last session of the conference.  This presentation was also fairly dense in implementation details, but the general approach seemed to be to embed a language of record extensions/reflection in Haskell.  Here I was again reminded of Mark Jones’s work in qualified types – where qualified types are shown to unify Haskell-style type classes, O’Caml-style structural subtyping, and type systems with record constraints.  The full details were a little difficult to follow in the presentation setting, but (like the example of the inner-product function for rank generic dependent array programming) there were a few examples given that helped motivate the audience to want such a system.  These seemed to boil down to wanting to treat records generically in one case (say, sending any serializable record as a message across a network) while simultaneously verifying code against the specific structure of the record in most other cases (as when a specific field is requested from a record).  I have not personally done much work with these kinds of systems, but it seems like a interesting subject to explore.

The design and implementation of higher-order modules in SML/NJ was covered next.  Modules are ML’s principle method of organizing programs, along with functors (ML’s name for module-dependent modules).  SML/NJ also includes support for “higher-order” modules (functor-dependent modules, or modules that define functors).  The SML/NJ documentation has some useful examples (under the heading “higher-order modules”).  This presentation explored where the design of higher-order modules has worked well in SML/NJ, and clarified some issues of its static semantics for those of us who don’t work often with ML modules.

Finally, the session (and the conference overall) ended with a presentation on a project called “Unibench,” which aims to be a universal tool for benchmarking applications.  Unibench is a web application that can run several builds of arbitrary programs (where builds are organized along the usual OS/compiler groups), and automatically apply ad-hoc benchmarking scripts to keep track of each program’s performance profile over time.  Several audience members pointed out that the Unibench tool had a lot in common with build-server systems, and in fact it seemed to be a primitive version of TeamCity.

Summary

Overall the conference had several very interesting presentations, some of which have a direct impact on those of us working in the financial sector.  I hope that I haven’t mischaracterized any of the presentations (if so, I hope that the aggrieved parties will add comments here).  It was very interesting to get a chance to put faces to the names of several people whose work I’ve only encountered before in print.  At the end of the last day, I was left with a lot of ideas to think about.

Generally, I thought it was worth 3 days of my life and saw some things that look pretty promising. It was also nice to get a window on the types of problems that the field is grappling with right now. Not surprisingly, there were several presentations on concurrency/parallelism; some were attempts to provide more syntax for explicitly declaring concurrent programs, but what I find more promising were the presentations of data parallelism where the language or library figures out what can be processed in parallel and optimally does it for you. Thus your programs address the problem you’re actually solving rather than concurrency arcana.

What did surprise me (as an outsider) was the struggles to deal with reactive programming models — eventing, callbacks, GUIs. The sorts of things that are pretty trivial when you don’t mind keeping state (I include callback-functions here) are very complicated without. Another surprise (in a good way) was the wealth of possibilities in the FL-universe for testing: theorem provers to provide logical underpinnings and automatic testing tools that can generate tests that unit-testing people write for ourselves. These tools are light-years ahead of our typical unit-testing environment. Not only can they save us the time of writing the tests, they’ll do a better job of coming up with all the corner cases.

One other observation: the way to find out who you should have been paying attention to is not to look at who’s presenting — look at the citations from all the presenters and which names keep popping up. For this conference the name Conal Elliott seemed to crop up in quite a range of presentations.

Niels Bohr

Enumerables and Enumerators, oh my!

Enumerables and Enumerators are great; big sacks of data we can iterate through using a well-established method. Just in case anyone isn’t up on their GOF, the pair looks a bit like this:

interface IEnumerable<T>

{

    IEnumerator<T> GetEnumerator();

}

interface IEnumerator<T> : IDisposable

{

    bool MoveNext();

    T Current { get; }

}

So we have our “sack”, our Enumerable of type T, where we can obtain a disposable reference to the Enumerator (also of type T), which we can use as a sort of pointer into the sack to move about and grab elements out. This is of course exactly what the compiler does when you use the foreach construct in C#:

foreach (var item in myEnumerable)

{

    Console.WriteLine(item);

}

// Is equivalent to:

using(var myEnumerator = myEnumerable.GetEnumerator())

{

    while(myEnumerator.MoveNext())

    {

        var item = myEnumerator.Current;

        Console.WriteLine(item);

    }

}

This bit of background is necessary in order to discuss the really interesting bit:

  What is the opposite of an Enumerable?

Part the second, where we talk about funky math

Long ago in days of yore, when computer scientists, philosophers and mathematicians all hung out at the same bars discussing grand ideas, people with names like Russell, Church, and Turing spoke of a new way of describing “stuff” that was governed entirely by the unyielding rules of math and logic. Finally, people the world over could talk about apples and oranges, and even state in precise terms how apples and oranges were both indeed fruit, even though an apple was not an orange, nor an orange an apple!

Ok, that may have been a bit much – rather than go into a deep discussion about type theory and lambda calculus, I shall refer you, humble reader, to the magic of the internets, where you shall find descriptions so as to make your brain melt into a goo. Alternatively, you could talk to this guy.

Instead, in the hopes of keeping at least some of your attention, we shall jump right in with trying to express the above Enumerable/Enumerator pair using a very simple “type language.” Looking first at the IEnumerable<T>, we see that it has one method (we’ll call it a “function”):

IEnumerator<T> GetEnumerator();

In our simple type language, we shall describe it thusly:

     () -> ( () -> T )

The above can be read as: “A function that returns a thing that contains a function that returns something of type T”. It can also be read as “A function that returns an enumerator that contains a function pair that can return something of type T”.

Of course, it’s quite a bit more complicated than this – since we are implicitly changing the state of the enumerator whenever we call MoveNext, we get into some funky stuff relating to that cringe-worthy word “Monad”, but we’re going to glaze over all that for now. Just think of it as described above.

Ok, now that is settled, we’re going to do something really funky; we’ll take this:

     () -> ( () -> T )

And turn around all the arrows, producing this (why this is a valid thing to do is related to mystical terms like covariance and contra-variance, which I won’t begin to try to describe here):

     () <- ( () <- T )

It’s a little easier to read left to right, so reordering:

     (T -> ()) -> ()

We can read this as: “A function ‘A’ that takes another function ‘B’ that takes a thing of type ‘T’ and returns nothing, with function ‘A’ also returning nothing”. Interesting…that looks a lot like the definition of a callback:

void Foo(Action<T> callback);

If we perform that same type of “type reversal” to all the members of IEnumerable and IEnumerator, we get definitions a bit like these:

interface IBackwardsEnumerable<T>

{

    void Subscribe(

        IBackwardsEnumerator<T> callbackLikeThing

    );

}

interface IBackwardsEnumerator<T>

{

    void OnDone(bool done);

    void OnValue(T value);

}

But wait, that’s not quite right…the earlier definition for the enumerator had IDisposable in there somewhere, plus there was the implicit possibility of an exception being thrown by Current.Get (in the case where we’ve MoveNext’ed past the end of the enumerable). We still need to account for all the inputs and outputs when we do our reversal (henceforth called the dual), so how do we account for these? Well, it doesn’t make a lot of sense to have our callback thingy be disposable; but we can make the registration of that callback disposable…

interface IBackwardsEnumerable<T>

{

   IDisposable Subscribe(IBackwardsEnumerator<T> callbackLikeThing);

}

interface IBackwardsEnumerator<T>

{

   void OnDone(bool done);

   void OnError(Exception error);

   void OnValue(T value);

}

Wait a second…this is starting to look a lot like an Asynchronous method call:

void AsyncMethod(T initialParameter,

   Action<T> callback,

   Action<Exception> fault,

   Action cancel);

The “callback” is what gets invoked when we have done some work (OnValue(T) above), the fault handler is invoked if we have a problem (OnError(Exception) above), and we have a way of stopping or cancelling the call in mid-process (OnDone(bool=true) above).

The similarities are not unusual – we’ve gone from a blocking-read type of operation (pulling values from an Enumerable collection) to a non-blocking write type of operation (asynchronously pushing values into a sort of stream).

Cool. So what the heck does all of that mean?

Observations on Observation

The Observer pattern (read your GOF!) is defined as:

“A software pattern in which an object, called the subject, maintains a list of its dependents, called observers, and notifies them automatically of any state changes, usually by calling one of their methods”

Looking back at the dual of the Enumerable pattern above, we can see that what we have derived is nothing more than the Observer pattern, in a slightly altered form. And so, we’ll call it as such:

interface IObservable<T>

{

    IDisposable Subscribe(IObserver<T> observer);

}

interface IObserver<T>

{

    void OnDone();

    void OnError(Exception error);

    void OnNext(T value);

}

The only change (other than renaming some things) is to remove the redundant Boolean value passed into OnDone – this is because the only time “OnDone” really matters is when that Boolean value is true. It doesn’t make a whole lot of sense to keep telling someone watching you work “I’m not done yet, I’m not done yet” when you can simply say “I’m done now”.

If this looks strangely like the way one would attach an event handler to a control – say, a button click – that’s because it’s basically the same thing, since a multi-cast delegate is pretty much an observer pattern condensed into a neat little package.

So we’ve proved, albeit in a rushed fashion, that the Observer pattern is the dual of the Enumerator pattern. We’ve also shown that it’s virtually identical to an asynchronous method call. We can also infer that it’s very similar to the way we would normally attach event handlers.

Whoop-de-do… What is the point?

Ever hear of LINQ?

Coda: QNIL

So LINQ does all its work with Enumerables. Using that same reasoning we went through above, a couple of very clever folks over at Microsoft created what is essentially “a dual of LINQ” – I rather like the idea of calling it QNIL (pronounced “Kyu-nil”), but as they are much smarter than I, I’ll use their terms and call it Rx – short for “Reactive framework”, but don’t ask me where they got the “x” from.

Introducing Rx LINQ to Events

Why is this cool? Observe, and I shall dazzle you:

Doing drag-and-drop is a pain. Every single time you implement it, you need to catch the initial mouse down event, set some internal flag to know it’s in a “dragging state”, attach some MouseMove handler to be able to determine where it’s being drug to, and then waiting for the next MouseUp event to come along so you can unhook your move handler, reset your internal state, and finally you’re done.

And you have to re-implement the same damned logic each time.

Let’s think about what we’re really trying to do. If Visual Studio understood English (Visual Studio 2014, perhaps?), I would say to it:

“If a user clicks on my control, listen for any mouse move events until the next mouse up event, and give those events to me so I can move my control where the user has moved the mouse.”

With Rx, it might be something like:

IObservable<Event<MouseEventArgs>> draggingEvent =

    from mouseLeftDownEvent in

        ctl.GetMouseLeftDown()

    from mouseMoveEvent in

        ctl.GetMouseMove()

        .Until(ctl.GetMouseLeftUp())

    select mouseMoveEvent;

draggingEvent.Subscribe(

(moveEvent) =>

{ ctl.MoveTo(moveEvent.X, moveEvent.Y);})

And that, dear reader, is just plain cool.

Still, many good ideas are destroyed (or exalted) when the time comes to make a real program out of them.  So in this article, we will write that program, and in the process shore up this notion of a derivative by defining it over any recursive type.

Defining the Derivative

In his paper first exploring this topic, Conor McBride determines the six laws for differentiating data types, roughly as follows:

This table is slightly more complicated than the rosy picture painted by the last article.  Before getting into the details of this definition, it’s worth pointing out one of the important things done here with recursive types.  We’ll need to distinguish between a recursive type, μ y.F , and its one-step unrolling, F|y=S (see the left column of rules 5 and 6).  Our previous definition of recursive types still applies, but instead of informally saying that a recursive type defines an infinite expansion by repeated substitution, we capture a single-step of substitution in this new form.

Also, this point was slightly glossed over in the last article, it should be pointed out that for typical data-types, we’re actually interested in the partial derivative.  In the definition above, ∂x(T) means the partial derivative of T with respect to x, and ∂y(T) means the partial derivative with respect to y.  Likewise, when we use ∂x(T) we are poking an x-hole in T (i.e.: we’re finding one x in T to single out).

The first four rules are just the same rules as in Calculus, but it’s instructive to informally consider what they mean in the context of types.  The first rule says that poking an x-hole in just the single variable x produces the unit type (this also tells us that holes have the unit type, perhaps a subtle point that was not made in the last article).  The second rule says that poking an x-hole in any type where x is not mentioned produces the void type (i.e.: you can’t poke a hole in an x that isn’t there).  The third rule says that if you have a type that could either be an S or a T, then poking an x-hole in this type produces a type that’s either an S with an x-hole or a T with an x-hole.  And the fourth rule says that poking an x-hole in a pair of an S and a T is either an S with an x-hole and a T (left intact), or it’s an S (left intact) and a T with an x-hole — in other words, in a pair, the hole is either in the first or the second value.

In other words, rules 3 and 4 are just the normal sum and product rules of Differential Calculus (respectively) and rules 1 and 2 are the regular base cases of differentiation (for the differentiated variable, or a constant — respectively).

So what explains rules 5 and 6?  These are two sides of the plain old chain rule.  It’s (arguably) easiest to make this case for rule 6, so let’s repeat it for reference:

Now consider the definition of the chain rule for partial derivatives:

For z = f(u, v)
∂z/∂x = (∂f/∂u)*(∂u/∂x) + (∂f/∂v)*(∂v/∂x)

This definition looks almost identical to rule 6, except that in this case the left side of the sum (∂x(F)|y=S) is just ∂f/∂x.

The connection is a little more obscure in rule 5:

Here we actually have the same structure as in rule 6, except that we see “z” where the chain rule tells us to expect ∂v/∂x.  Informally, this anomaly is resolved by recognizing that “z” (representing the infinite expansion of the entire recursive type) is in fact ∂v/∂x.

Let’s try a non-trivial example to develop some intuition for this method before we set out to write the differentiation program.  Consider the recursive type of lists of integers:

μ X.(1+int*X)

And we will want to poke an int-hole, so we’re going to find:

∂int(μ X.(1+int*X))

By rule #5 (and given that “y” from rule #5 matches “X” in our expression, and “F” from rule #5 matches “1+int*X” in our expression), this becomes:

μz.((∂int(1+int*X)|X=μX.1+int*X)+(∂X(1+int*X)|X=μX.1+int*X)*z)

Yikes!  There’s a lot going on there, but perhaps we can break this problem into easier pieces by a couple of choice substitutions.  Since this is just a recursive type with a sum in it, let’s choose a couple of variables to represent both sides of the sum:

μz.(U+V*z)
where
   U = ∂int(1+int*X)|X=(μX.1+int*X)
   V = ∂X(1+int*X)|X=(μX.1+int*X)

Now we can evaluate U and V independently.  First let’s consider U:

∂int(1+int*X)|X=(μX.1+int*X)

By rule #3 (the sum rule) this becomes:

(∂int(1)+∂int(int*X))|X=(μX.1+int*X)

Now we have two derivatives to evaluate.  First the one on the left becomes 0 (by rule #2) and we can eliminate it:

∂int(int*X)|X=(μX.1+int*X)

Then by rule #4 (the product rule) this last derivative becomes:

(∂int(int) * X + int * ∂int(X))|X=(μX.1+int*X)

Now we’re almost done.  Here we have one more sum, and the derivative on the left becomes 1 (by rule #1), leaving a lone X:

(X + int * ∂int(X))|X=(μX.1+int*X)

And the derivative on the right becomes 0 (by rule #2), which means that the right side of the sum can be eliminated.  Therefore we have solved “U”:

U = X|X=(μX.1+int*X)

Now, since we’re done, we could substitute for X here to eliminate the redundant context:

U = μX.(1+int*X)

In other words, U is a list of ints.  Now we must solve V (the trickier side of the chain rule, where we switch to poking an int-list hole rather than an int-hole):

∂X(1+int*X)|X=(μX.1+int*X)

By rule #3, this becomes:

∂X(1)+∂X(int*X)|X=(μX.1+int*X)

Here the left side of the sum is eliminated (by rule #2) and the right side is elaborated (by rule #4) becoming:

(∂X(int) * X + int * ∂X(X))|X=(μX.1+int*X)

Now the left side of the sum is eliminated (by rule #2), and the right side just simplifies to int (by rule #1), so that we’ve also now solved for V:

V = int|X=(μX.1+int*X)

In fact, we don’t need this trailing context information anymore (since “X” doesn’t appear in the simple type variable “int”, we don’t need to remember how to substitute “X”), so we can simply say:

V = int

Now, substituting U and V back into our original solution for the derivative of a list of ints, this becomes:

μz.((μX.(1+int*X))+int*z)

To help us make some sense of this answer, it’s useful to observe that the following identity holds:

μ a.(T+S*a) = list<S> * T

We know this to be true in the normal case of lists — because in that case T = 1 (and list<S>*1 = list<S>).  But if we apply this identity to the answer we’ve derived, we can conclude that the derivative of a list of ints is a pair of lists of ints (which makes sense — the first list is the prefix up to the hole, and the second list is the suffix following the hole).  In fact, it’s a good thing that we derived this answer, because we’ve already made use of it!

Now, assuming that we’re satisfied with this differentiation procedure, all that’s left is to actually implement it in a real program.

A Brief Haskell Program

The first thing that we need to do is set down a data structure that represents types.  The circularity of this definition may seem odd (we’re defining a data type for structures that represent data types), but because we’ve already covered in detail the forms that types can take, it shouldn’t be very surprising:

data Type =
   Ty String
| Sum (Type, Type)
| Prod (Type, Type)
| Fix (String, Type)
| FApp (Type, String, Type) deriving Eq

Stated plainly, a type is a type variable (e.g.: int, string, date, unit, void, etc), or the sum of two types, or the product of two types, or a recursive type, or the one-step unrolling of a recursive type.

For convenience, we can define a function to display these type structures conveniently (and consistent with the notation we’ve developed so far):

instance Show Type where
show (Ty “unit”)     =”1″
show (Ty “void”)     =”0″
show (Ty v)          =v
show (Sum  (t1, t2)) =(show t1)++”+”++(show t2)
show (Prod (t1, t2)) =(show t1)++”*”++(show t2)
show (Fix  (v,  ty)) =”mu “++v++”.”++(show ty)
show (FApp (t, x, s))=”["++(show t)++"|"++x++"="++(show s)++"]“

This way, if we enter an interesting type at the Haskell prompt (like the recursive type of lists of integers), we should see it in the shorthand we’ve developed:

Prelude> Fix (“X”, Sum (Ty “unit”, Prod (Ty “int”, Ty “X”)))
mu X.1+int*X

Finally, we can define a “derivative” function on types with respect to a particular type-variable by explicitly translating each of the 6 rules we’ve considered above.  First rule #1:

derivative (Ty v) x | x == v = Ty “unit”

Then rule #2 (a type other than x):

derivative (Ty v) x = Ty “void”

Then rule #3 (the rule for sums):

derivative (Sum  (t1, t2)) x = Sum (derivative t1 x, derivative t2 x)

And rule #4, for products:

derivative (Prod (t1, t2))   x = Sum (Prod (derivative t1 x, t2), Prod (t1, derivative t2 x))

Rule #5 (assuming for the moment that we have a function unique_name to choose a unique type-variable name that does not appear in a particular type expression):

derivative rty@(Fix (v, ty)) x =
   let z = unique_name rty in
   let dt_dx = derivative ty x in
   let dt_dv = derivative ty v in
     Fix (z,
      Sum (
        FApp (dt_dx, v, rty),
        Prod (FApp (dt_dv, v, rty), Ty z)))

And finally rule #6:

derivative (FApp (t, y, s))  x =
    let dt_dx = derivative t x in
    let dt_dy = derivative t y in
    let ds_dx = derivative s x in
       Sum (FApp(dt_dx, y, s), Prod (FApp (dt_dy, y, s), ds_dx))

Now, as a question of correctness, we have finished this problem — we have a Haskell program that can compute derivatives of types (modulo the definition of this unique_name function).  However, as in the normal differential calculus, the terms we produce can be a little messy — with lots of 1*X and X+0 hanging around.  There can also be unnecessary recursive types (where the recursion variable is not mentioned in the body of the recursive type), and those should be eliminated as well.

The simplest approach is probably to define the single-step simplification of a type expression:

simplify (Sum  (Ty “void”, x))= simplify x
simplify (Sum  (x, Ty “void”))= simplify x
simplify (Sum  (x, y))        = Sum (simplify x, simplify y)
simplify (Prod (Ty “void”, x))= Ty “void”
simplify (Prod (x, Ty “void”))= Ty “void”
simplify (Prod (Ty “unit”, x))= simplify x
simplify (Prod (x, Ty “unit”))= simplify x
simplify (Prod (x, y))        = Prod (simplify x, simplify y)
simplify (Fix  (v, ty))       | not (v `member` names ty) = ty
simplify (Fix  (v, ty))       = Fix (v, simplify ty)
simplify (FApp (t, y, s))     | not (y `member` names t) = t
simplify (FApp (t, y, s))     = FApp (simplify t, y, simplify s)
simplify x                    = x

Then, given that we can perform a single step of simplification to a term (keep in mind, one 0*X becoming 0 could propagate into a new 0*Y, as in the expression Y*(0*X)), we can completely simplify a term by the fixed-point of this process:

fixed_point f x | x == (f x) = x
fixed_point f x              = fixed_point f (f x)

csimplify x = fixed_point simplify x

Also, because the way we display these types loses some grouping information (the type “int*(string+date)” is actually printed “int*string+date”, which is actually a different type!) we should probably flatten types (e.g.: converting “int*(string+date)” to “int*string + int*date”):

flatten ty = distribute Nothing ty
where
  distribute Nothing t@(Ty _) = t
  distribute (Just lty) t@(Ty _) = Prod (lty, t)
  distribute mlty (Sum (t1, t2)) = Sum (distribute mlty t1, distribute mlty t2)
  distribute mlty (Prod (t1, t2)) = distribute (Just (distribute mlty t1)) t2
  distribute Nothing (Fix (v, ty)) = Fix (v, distribute Nothing ty)
  distribute (Just lty) t@(Fix (v, ty)) = Prod (lty, distribute Nothing t)
  distribute Nothing (FApp (t, x, s)) = FApp (distribute Nothing t, x, distribute Nothing s)
  distribute (Just lty) t@(FApp (f, x, s)) = Prod (lty, distribute Nothing t)

Finally we can package all of these niceties on top of the “derivative” function into a single definition:

d ty x = flatten (csimplify (derivative ty x))

For convenience, I’ve prepared a script — please rename it to dtype.hs, somehow our weblog software thinks that *.hs files are insecure — which contains all of these definitions (along with several utility functions, like unique_name, which are necessary but not vital to the understanding of this differentiation procedure).  Provided that you have GHC installed, you can load the script like this:

> ghci dtype.hs

We can then test it out on two types whose one-hole contexts we’ve already made use of.  First the derivative of a list:

Prelude> d (Fix (“X”, Sum (Ty “unit”, Prod (Ty “int”, Ty “X”)))) “int”
mu a.[X|X=mu X.1+int*X]+int*a

Also we can take the derivative of a binary tree:

Prelude> d (Fix (“X”, Sum (Ty “int”, Prod (Ty “X”, Ty “X”)))) “int”
mu a.1+[X+X|X=mu X.int+X*X]*a

Again, using the list identity and the observation that X is the type of binary trees of ints, this tells us that the derivative of a binary tree is a list of a binary tree or a binary tree (either the left or right branch passed on the path from the hole in the tree to the root of the tree).

We’ll investigate these questions, and perhaps demystify this important concept of functional languages.

Algebraic Expressions at the Level of Types

To understand the concept of algebraic data types, simply stated, we need to unify the set of concepts that we associate with algebra, and the set of concepts that we associate with data types.

In an algebraic expression, we might encounter a humble lone variable:

x

In a data type expression, we might encounter a humble lone type:

int

Algebraic expressions can also be compositions of other expressions, so for example we might see:

x*x

And similarly for data types:

int*int

To explain: the product of two types denotes the type of pairs of those types.  In some languages, this might instead be written as pair<int,int> but as we’ll soon see, it’s useful to think of this as a product.

Of course, the algebraic expression above could be simplified to x**2 (“x squared”) and so you could also write int**2.

Now, in an algebraic expression you might also see:

x+x

And the same concept exists for data types:

int+int

The sum of two types denotes the type of variants which can have values either of the left type or the right (but not both).  The sum in the above type might seem superfluous, because whether a value of the above sum type has the left or the right type, it’ll be an int.  However, there’s some additional useful information in knowing whether the left or right was chosen.

Consider, for example, a data type representing the balance of a bank account:

type balance = int + int

We might interpret values on the left as positive balances, and values on the right as negative balances.  So if we wrote a function to describe a balance for screen-reading, it might look like:

string describe(balance b) {
   if (b.is_left()) {
      return “The bank owes you $” + b.left();
   } else {
      return “You owe the bank $” + b.right();
   }
}

For many people, sum types are less obvious than product types.  It may help to consider that, just as pair<int,int> corresponds to product types, boost::variant<int,int> corresponds to sum types.  C-style unions also serve a similar function.

Now, we can use these type-composition functions to make arbitrarily complex expressions:

(int*int*string)+(date*(string+int))

Just as we could use them to make arbitrarily complex algebraic expressions:

(x*x*y)+(z*(y+x))

But there is perhaps a simpler algebraic concept to consider:

1

What does the natural number “one” mean at the level of types?  This is called the “unit type” and it has only one value, written ().  It doesn’t do much good to have a value of the unit type — you already know what it must be.  However, it can become useful in composition, for example if you consider the simple identity:

1+1=2

Now “two” becomes the type of bits (or boolean values) — because a value of this type could either be the (trivial) unit value on the left, or the one on the right.  This view may help to understand variant types better, consider:

int+int = 2*int

In other words, the sum type of two ints is the same thing as the product type of a boolean value and a single int (the boolean value tells you whether the int is the one on the left or the right of the sum).

Furthermore, as you might expect:

1*int = int

In other words, having a pair of a unit value and an int is the same as having an int (the unit value adds no information here — you already know what it must be).

Now there’s one other natural number that deserves special consideration:

0

This is a special kind of type — it’s known as the “void” type (slightly different than the void type in programming languages like C, you’ll have to forgive this overloading of terms).  Unlike the unit type, which has a single value, the void type has no values.  It’s impossible to construct a value of this type.

That means, for example, that:

1+0=1

Because if you have a sum type with unit on the left and void on the right, no values of this type can possibly be on the right (again, it’s impossible to construct a value of the void type) — so it’s safe to assume that you can simplify this sum type to just be the unit type.

By a similar argument:

1*0=0

Because to construct a pair, you have to construct both of its members — if you can’t construct one member of the pair (again, it’s impossible to construct a value of the void type) then you can’t construct the whole pair.

Now, these types that we’ve covered so far are quite useful, and it’s hopefully pretty clear how they map to simple algebraic expressions.  However, it might be pointed out that there are more complex types, and more complex algebraic expressions to consider (even though the expressions above can get quite complex).

For example, how might the type of a list of integers be described (a list<int>) as an algebraic expression?  To help motivate this answer, consider the possible lists of integers we might produce.  For example, a list of integers could be an empty list, or it might be just one integer, or it might be two, or three, etc.  Each “or” here should be thought of as a sum, so that (with the unit type representing the trivial empty list) we come to a type represented by an infinite sum:

1+int+int**2+int**3+…+int**n

Now, as an algebraic expression we might be happy with this, or we might convert it to the “big-sigma” notation.  However, in the world of types, this kind of infinite expansion is described with a recursive type.  A recursive type associates the expansion with a symbol and consolidates the type to an expression where that symbol may be used.  For the case of our list of ints, this is written:

μX.(1+int*X)

This can be pronounced “mu X, one plus int times X.”  Forget the “mu” if Greek symbols bother you; all it really means is “look out, here comes a recursive type.”  The most important idea is that the type should be thought of as the infinite expansion you get when you repeatedly substitute the entire recursive type for X (the recursion variable).  For example, performing that substitution once in the above type will produce:

1+int*(μX.(1+int*X))

and again:

1+int*(1+int*(μX.(1+int*X)))

Using the normal distributive properties of * and +, this can be reduced to:

1+int+int**2*(μX.(1+int*X))

And in the limit that we perform this substitution continually, we will produce the same infinite sum that we expected for this list of ints.

Although this formalism may seem somewhat removed from the language of types in F#, you can quickly recover it if you replace the keyword “type” for “mu,” “=” for “.”, “|” for “+”, recognize that empty constructors are constructors on the unit type, and ignore constructor names.  For example:

type IntList = Nil | Cons of int*IntList

becomes:

μIntList.(1+int*IntList)

Again, the type of lists of integers that we have already identified (modulo the choice of “X” or “IntList” for the recursion symbol).

This machinery of recursive types takes some getting used to, but it can produce the same infinite series that we expect in algebra, and it’s very convenient for recursive types with multiple branches of recursion, for example the type of binary trees of integers:

μX.(int+X*X)

In other words, a binary tree is either a leaf (storing an int) or it’s a branch (storing a pair of binary trees).

It seems like we can describe a very rich category of types with these tools (for more detail, see “the bible of type theory”).  What’s more, our existing knowledge of algebra can be ported to this world of types, where we find many familiar structures.  But this formalization of types goes even deeper, and there are more concepts left to unify.

∂ for Data

In the world of functional programming, destructive mutation is discouraged or even banned outright.  From a formal perspective, an update should produce a new copy of the original value, with both versions available for some later use.  This view has simplified many otherwise thorny problems, and it makes programs much simpler to reason about.

We have previously discussed ways to recover mutation-like functions, inspired by the most famous data structure of its kind: the Zipper.

In 2001, Conor McBride made a remarkable discovery about algebraic data types, and the concept of a zipper, which he elaborated four years later (along with coauthors Abbot, Ghani, and Altenkirch).

Central to this discovery is the concept of a one-hole context.  A one-hole context can be thought of as a data type “with a hole in it.”  This is a concept closely linked to the concept of mutation, as a value can be “updated” by “plugging in” that hole.

Consider as a very simple example, a 3-tuple of integers.  Its type is int*int*int, or int**3.  Now, we might have a value of this type:

(98, 76, 54)

We might want to “update” one of these values, or at least single one out for updating.  We can do this by “poking a hole” in the data structure where a particular value goes.  In other words, we might choose to update the first, second, or third components of this 3-tuple (using “_” to represent a hole):

(_,  76, 54)
(98, _,  54)
(98, 76, _)

These three cases represent the only ways that we could “poke a hole” in this 3-tuple data, and we can capture that idea in a corresponding data type.  Because there are three ways to poke a hole, the type should be a three-way sum.  Each possibility leaves two integers (in different configurations), and so we can pretty straightforwardly deduce this type of one-hole contexts:

(int*int)+(int*int)+(int*int)

In other words, the first int*int corresponds to the type of “(_, 76, 54)“, the second int*int corresponds to the type of “(98, _, 54)“, and the third int*int corresponds to the type of “(98, 76, _)“.

It’s useful to simplify the type of this expression, and so we might shrink it to (explicitly naming squares):

int**2+int**2+int**2

and then combining like terms:

3*int**2

In other words, the type of one-hole contexts of 3-tuples of ints is one of three int-pairs.

Now here’s an odd thing, we started out with the type:

int**3

and taking its one-hole context we derived the type:

3*int**2

That looks an awful lot like the derivative of differential calculus!

In fact that’s exactly what McBride discovered, that the derivative of an algebraic data type is its type of one-hole contexts!  Extended to recursive types (see McBride’s paper for more details) we can use the derivative to mechanically produce functions for updating and iterating over arbitrary data structures.

Further Reading

Predating McBride’s work, Andre Joyal explored the concept of combinatorial species (built on the concept of generating functions — a formalism similar in purpose to algebraic data types), where the use of the derivative was also discovered.

McBride is also the co-creator of the programming language Epigram, a programming language with a powerful theory of types.

In the next article, we’ll look at implementing this notion of differentiation of data types with a brief Haskell program.

Can’t we all just get along?

(An answer inside, plus the continuation of our exploration of type-classes.)

Yes We Can

We just need to be liberal with how we define “mutation.”  In the practice of most programming languages, “mutation” means destructive updates — so that when we perform an update on a data structure, the previous state of that structure is lost.  However, when most people think of mutation (and when they consider why it’s useful to them) they typically think of an update that takes a (short) constant amount of time — and this becomes most useful in the aggregate of a lot of little mutations applied incrementally.  When you tell them that a functional update preserves previous states by creating an updated copy, a little cash-register in their brain goes nuts calculating the expense of (say) making three consecutive updates.

But we can perform a purely functional update in constant time.  As a trivial example, consider a pair of values.  We can update a pair either by updating its first element or its second; in Haskell:

update_left  (_, y) x = (x, y)
update_right (x, _) y = (x, y)

Although these functions don’t destructively update the structures they work on, they do produce updated structures in a constant amount of time.  So where does the expense of this functional method get out of hand?

Consider a function to update the nth element of a list.  This kind of function is easy enough to write:

update_nth []     _ _ = []
update_nth (_:xs) 0 v = v:xs
update_nth (x:xs) n v = x:(update_nth xs (n – 1) v)

This does work correctly, but it’s a good example of the kind of argument that a mutation-loving programmer could justifiably make against Haskell.  Because, if you need to update just the 70th, 90th, and 120th elements of the list, you wind up stepping through the first 69, 89, and 119 elements at each stage (for a total of 69+89+119=227 visits).  However, in an incremental C/C++/Java program we can just increment an iterator and update it, meaning that for the equivalent program we’d only need to visit 119 elements (of course, the distance between these two expenses gets even greater with longer lists, and with more updates).

Where did we go wrong?

If purely functional updates to tuples are efficient, perhaps we can make our list look like a tuple.  This basic insight led to the introduction of the Zipper, an important kind of data-type.

To see how this solves our problems for lists, we need to consider the concept of a one-hole context.  A one-hole context represents a “point of interest” in a data structure.  So for instance, with a list, a one-hole context should represent the list elements we’ve visited up to our point of interest, and what remains after our point of interest.

For example, to focus on the 3rd element of this list:

[1,2,3,4,5]

Our one-hole context should look like this:

([2,1], [4,5])

Notice that the first list is in reverse order, because it represents the elements in the order that we visited them.

Now with this data type we’ve almost settled on the perfect representation.  All that we need to do is add the value of interest, and we’ll have a complete representation of the list with an element in focus and in just the right form for purely functional updates.  Again, with the 3rd element of the list [1,2,3,4,5] selected:

([2,1], 3, [4,5])

Now we can write a general function to update this iterator value:

update (prefix, _, suffix) x = (prefix, x, suffix)

We can also write functions to increment this iterator (again, a constant-time operation):

forward (_,  _, [])   = error “Iterator at end.”
forward (px, x, s:sx) = (x:px, s, sx)

And also to decrement the iterator:

backward ([], _, _)    = error “Iterator at beginning.”
backward (p:px, x, sx) = (px, p, x:sx)

Finally, we have a method to efficiently and incrementally walk a list, updating arbitrary parts of it in constant time in a purely functional way.  Of course, there is an added expense (hopefully one that we can amortize away) if we need to reconstruct the “real” updated list from our iterator:

close (px, x, sx) = (reverse px) ++ (x:sx)

Beyond Lists

At this point, we may be able to win over some mutation-loving programmers.  However, the point will quickly be made that in the real world we use data structures other than lists.  Binary trees, n-ary trees, directed graphs, and many other kinds of structures crop up in all sorts of problems.

Once again, type-classes are here to save us.  We just need to take a step back, and recognize that to apply this principle of purely-functional-incremental-update-able iterators to arbitrary “container types” we need a couple of suitable type-classes.  We’ll need to begin our script with the following line:

{-# LANGUAGE MultiParamTypeClasses, FunctionalDependencies, FlexibleInstances, UndecidableInstances, TypeSynonymInstances #-}

First, our representation of iterators should encapsulate everything that we’ve learned here.  We should be able to increment and decrement them.  We should be able to check their current value, and update them.  For convenience, we should also be able to determine when we’re at the end (or beginning) of the iterator’s extent:

class Iterator b a | b -> a where
   bos      :: b -> Bool
   eos      :: b -> Bool
   forward  :: b -> b
   backward :: b -> b
   value    :: b -> a
   update   :: b -> a -> b

Here, the type “b” represents an iterator and “a” its “element type” (hence the functional dependency “b -> a” saying that b derives a).

Finally, we’ll need to convert back and forth between a container and its iterator:

class Iterator b a => Iterable f b a | f -> b, b -> f where
   open  :: f a -> b
   close :: b -> f a

This type-class captures the critical observation that every container-type has a unique iterator type (“f -> b”) and that every iterator type derives a unique container type (“b -> f”).

Now, as we’ve already spent the first half of this article addressing it, we can put lists and their iterators into this type-class:

instance Iterable [] ([a], a, [a]) a where
   open (x:xs)       = ([], x, xs)
   close (px, x, sx) = (reverse px) ++ (x:sx)

instance Iterator ([a], a, [a]) a where
   bos ([], _, _)           = True
   bos _                    = False
   eos (_, _, [])           = True
   eos _                    = False
   forward  (px,   x, s:sx) = (x:px, s, sx)
   backward (p:px, x, sx)   = (px, p, x:sx)
   value  (_, x, _)         = x
   update (px, _, sx) x     = (px, x, sx)

With this structure in place, it is now possible to write functions that perform incremental updates to arbitrary data structures.  So far we’ve only shown that this can be done for lists.  However, if we follow the lesson of the Zipper we can also put (say) binary trees into the Iterable type-class as well.

Recall our definition of binary trees:

data Tree a = Leaf a | Branch (Tree a) (Tree a)
   deriving (Show, Eq)

Now, to represent a point in a binary tree (its one-hole context) we need to record the converse branches at each step of the path we followed from the root of the tree (similar to the Deutsch-Schorr-Waite algorithm for constant-space garbage collection).  Along with that one-hole context representation, we’ll store another binary tree (the focused tree) to complete our iterator type.

type TreeIter a = (Tree a, [Either (Tree a) (Tree a)])

To aid us in this representation of a binary tree’s iterator, we will need a few utility functions.  First, to move up one level in the tree:

up (r, (Left  l):p) = (Branch l r, p)
up (l, (Right r):p) = (Branch l r, p)

Also, to descend to the first leaf (either by following all left or right branches):

descendl c@(Leaf _, p)   = c
descendl (Branch l r, p) = descendl (l, (Right r):p)
descendr c@(Leaf _, p)   = c
descendr (Branch l r, p) = descendr (r, (Left  l):p)

Notice here that the “up” and “descend” functions cancel each other out (“up” removes a node from the list of converse branches, and “descend[l|r]” adds one at each step).

Also, to make it slightly easier to check the converse branches in our paths, we can define a couple of simple utilities:

is_right (Left _)  = False
is_right (Right _) = True
is_left            = not . is_right

Now we have enough to easily put Tree, our type of binary trees, into the Iterable type-class:

instance Iterable Tree (TreeIter a) a where
   open t                 = descendl (t, [])
   close (t, [])          = t
   close (r, (Left  l):p) = close (Branch l r, p)
   close (l, (Right r):p) = close (Branch l r, p)

instance Iterator (Tree a, [Either (Tree a) (Tree a)]) a where
   bos (_, p)                   = all is_right p
   eos (_, p)                   = all is_left p
   forward    (t, (Right t’):p) = descendl (t’, (Left t):p)
   forward  c@(t, (Left  t’):p) = forward (up c)
   backward   (t, (Left  t’):p) = descendr (t’, (Right t):p)
   backward c@(t, (Right t’):p) = backward (up c)
   value  (Leaf x, _)           = x
   update (_, p) x              = (Leaf x, p)

Phew!  That’s it; we can now incrementally iterate over lists and trees, updating arbitrary elements in a purely functional way, in a constant amount of time.

Iterable implementations for other container types can be derived in a straightforward way.

A Few Extensions

I should start off by mentioning that the features of type-classes we’ll cover here are not part of the Haskell98 standard, but they’re regularly used and supported by GHC.  Provided that you’re using GHC, the code that follows in this article will work if you place the following line at the head of its file:

{-# LANGUAGE MultiParamTypeClasses, FunctionalDependencies, FlexibleInstances #-}

Subtyping

In F# as well as Haskell, tuples are an important and commonly-used tool.  They are used to define composite types, joining an arbitrary (but finite, statically known) number of types together.  As we saw in the last article, they’re a convenient form for representing vectors.  However, in Haskell (and in F# for that matter) you may notice an odd thing about the standard functions provided for tuples: fst and snd.

Prelude> :t fst
fst :: (a,b) -> a
Prelude> :t snd
snd :: (a,b) -> b

These functions, fst and snd, are used to extract the first and second components of a 2-tuple (respectively).  Unfortunately, if we happen to be using 3-tuples, or 4-tuples (or n-tuples, for n > 2) we’re out of luck.  In those cases, we can’t even extract the first and second members without having to create something like:

fst3 (x,_,_) = x
fst4 (x,_,_,_) = x
… etc …

This is pretty silly.  This is clearly a case where some overloading would benefit us, so that we wouldn’t need to define the “fst” function with a different name for each possible size of tuple.  What we really want is something with a type signature like:

first :: Pair p a b => p -> a

Here we’re seeing a type-class constraint unlike the comparatively simple ones in the last article.  In essence, what “Pair p a b” must mean here, is that for some pair type “p” a type “a” and a type “b” can be extracted.  In the case of the “first” function we’ll just extract the “a” type, so the type of the “second” function must be:

second :: Pair p a b => p -> b

This scheme might sound reasonable, but how is the Haskell compiler supposed to know that the “p” type really does derive the “a” and “b” types?  For that, we need functional dependencies.  We can simply declare this dependency when we make the “Pair” type-class and Haskell’s type-checker will respect the restriction across all members of the type-class:

class Pair p a b | p -> a, p -> b where
   first  :: p -> a
   second :: p -> b

When filling out instances, we won’t have to mention these dependencies again.  So for example, if we wanted to define “first” and “second” over 2, 3, and 4-tuples:

instance Pair (a, b) a b where
   first  (x, y) = x
   second (x, y) = y

instance Pair (a, b, c) a b where
   first  (x, y, z) = x
   second (x, y, z) = y

instance Pair (a, b, c, d) a b where
   first  (x, y, z, w) = x
   second (x, y, z, w) = y

Now it’s true that we’re still filling out each and every definition, like we had done with fst3, fst4, etc.  However, we have at least gotten far enough to avoid having to know the size of the tuple when we’re using it!  It would be a strong language indeed that didn’t require this boilerplate, but it’s easy enough to generate it and forget it.

Of course we don’t have to stop at “first” and “second” (but we do have to stop somewhere).  We might also want to define “third” on any tuple as big or bigger than a triple.  Of course, a triple should already be a pair (i.e.: if third is defined on a tuple, first and second had better also be).  We can express this requirement with a type-class constraint, as well as the functional dependency required to claim that the triple’s type can produce yet another type (but we don’t need to re-declare the functional dependencies inherited from the Pair type):

class Pair p a b => Triple p a b c | p -> c where
   third :: p -> c

After that, it’s just as easy (if not a little tedious) to put more tuples into this type-class as well:

instance Triple (a, b, c) a b c where
   third (x, y, z) = z

instance Triple (a, b, c, d) a b c where
   third (x, y, z, w) = z

Points and Colored Points

It requires a bit of initial work (whether manual or automatic) to set this up, but once our generic tuple functions are in place we can then use them to achieve structural subtyping without any other work from the language.

For example, we may make a distinction between points and colored points:

data Color = Red | Green | White | Yellow | Blue

makePoint x y = (x, y)
makeCPoint x y c = (x, y, c)

However, either kind of point can have a distance (from the origin) defined:

distance :: (Pair p Float Float) => p -> Float
distance p = sqrt ((first p)**2 + (second p)**2)

To Be Continued

It’s fairly remarkable that type-classes (with these extensions) allow the recovery of subtyping in a language that otherwise wasn’t designed to support it.

In the next article we’ll continue investigating applications of multi-parameter type-classes with functional dependencies, to address a generic approach to incremental iteration and constant-time updates for arbitrary Haskell containers.

APL/KDB programmers don’t have the benefit of a type-checker, but they do have some sophisticated overloading of arithmetic types.  Here we’ll see how we can recover APL-style arithmetic in Haskell, properly typed, and discover an important generalization that should further the argument that type-classes would be a valuable addition to F#.

A Simple Point

Type-classes in Haskell are used to represent bounded polymorphism.  This is in contrast to “parametric polymorphism,” the idea that a function can be defined over all types.  In fact, we don’t need type-classes to define “bounds” on parametric polymorphism — we could just supply proof of the type being within bounds every time that proof is required (this was the moviephone-F# program of the last article — always passing the right “add” function when it was needed).  Type-classes simply save us from that burden.

Just as type-class bounds apply to the types of functions, they can also apply to the types of type-class instances.  For example, consider the definition of a simple 2D point:

instance (Num a, Num b) => Num (a, b) where
   (a1, a2) + (b1, b2) = (a1+b1, a2+b2)
   (a1, a2) – (b1, b2) = (a1-b1, a2-b2)
   (a1, a2) * (b1, b2) = (a1*b1, a2*b2)
   abs (av, bv)        = (abs av, abs bv)
   signum (av, bv)     = (signum av, signum bv)
   fromInteger x       = (fromInteger x, fromInteger x)

This restriction (following the keyword “instance” introducing a type-class instance) says that a pair of values (a, b) can be treated like a number, provided that a can be treated like a number and b can be treated like a number.  The rest of the definition defines arithmetic over these pairs in a fairly straightforward way.

Prelude> (2, 3.5) + (1, 0.2)
(3, 3.7)

An important point here is that the ‘fromInteger’ function “lifts” integers into any numeric type.  So we can demonstrate expressions similar to what we might see at the start of a linear algebra course (multiplying vectors by scalars):

Prelude> 2 * (3, 3.5)
(6, 7.0)

Extending this definition to 3D points is just as simple:

instance (Num a, Num b, Num c) => Num (a, b, c) where
   (a1, a2, a3) + (b1, b2, b3) = (a1+b1, a2+b2, a3+b3)
   (a1, a2, a3) – (b1, b2, b3) = (a1-b1, a2-b2, a3-b3)
   (a1, a2, a3) * (b1, b2, b3) = (a1*b1, a2*b2, a3*b3)
   abs (av, bv, cv) = (abs av, abs bv, abs cv)
   signum (av, bv, cv) = (signum av, signum bv, signum cv)
   fromInteger x = (fromInteger x, fromInteger x, fromInteger x)

A Less Simple Point

KDB also defines arithmetic over arbitrary-length lists, and we will do that here as well.  However, it’s worth first covering “higher-kinded type-classes” or type-classes defined over type constructors.

What is a type constructor?  Speaking roughly, a type constructor is something which, when applied to a type, produces a new type.  For example, in F# “list” is a type constructor because it isn’t until you supply it a type, “int list”, that you have a real type.  In Haskell, this distinction is made by the “kind system” — allowing a more concise definition of a type constructor.  A “basic type” (e.g.: Int, Float, Int -> Float, [Char], etc) has kind *, so a “type constructor” (e.g.: [], ((,) Int), Maybe, (Either Int), etc) has kind *->*.  At the GHCI prompt, you can check a type’s kind with the “:k” command:

Prelude> :k Int
Int :: *
Prelude> :k []
[] :: * -> *
Prelude> :k (,)
(,) :: * -> * -> *
Prelude> :k ((,) Int)
((,) Int) :: * -> *

Probably the most natural example of a higher-kinded type-class is Haskell’s Functor class.  It contains just the fmap function, and looks something like this:

class Functor f where
   fmap :: (a -> b) -> (f a -> f b)

Here it’s clear that “f” must have kind *->*, because it is used in the definition of the fmap function to construct two types.  What’s more, this Functor instance “lifts” a function (from some a to b) to a new function over a and b in the f container (analogously to the definition of a functor in category theory).

For lists, this definition is exactly the same as the “map” function that everyone is familiar with:

instance Functor [] where
   fmap f [] = []
   fmap f (x:xs) = (f x):(fmap f xs)

However, lists are not the only kind of data structure that we use in functional programs, so that if we have a simple binary tree:

data Tree a = Leaf a | Branch (Tree a) (Tree a)

We can define a similar Functor instance:

instance Functor Tree where
   fmap f (Leaf x) = Leaf (f x)
   fmap f (Branch l r) = Branch (fmap f l) (fmap f r)

And, importantly, in both cases fmap will preserve the structure of the container it operates on, only transforming elements.

Prelude> fmap (1+) [1,2,3,4]
[2,3,4,5]
Prelude> fmap (1+) (Branch (Branch (Leaf 1) (Leaf 2)) (Branch (Leaf 3) (Leaf 4)))
Branch (Branch (Leaf 2) (Leaf 3)) (Branch (Leaf 4) (Leaf 5))

With this knowledge, we can define arithmetic over type-constructors very easily, as long as the type-constructor is “stretch-zippable” and its elements are numbers:

instance (SZip f, Num a, Show (f a), Eq (f a)) => Num (f a) where
   xs + ys       = fszip (+) xs ys
   xs – ys       = fszip (-) xs ys
   xs * ys       = fszip (*) xs ys
   abs xs        = fmap abs xs
   signum xs     = fmap signum xs
   fromInteger i = fszunit (fromInteger i)

This definition isn’t complete without an explanation of SZip (“stretch-zippable” — which assumes that a Functor is also defined — as evidenced by the use of fmap in abs and signum).  The SZip type-class is similar to Functor‘s fmap, except that it maps a binary function pair-wise over two sequences.  Unlike Haskell’s standard zip function, it then extends the final element of the shorter sequence over the remainder of the longer sequence (hence the “stretch” in “stretch-zippable”).  It also defines a function, fszunit, to put a single element in a stretch-zippable container (although this function is common enough to probably warrant its own type-class):

class Functor f => SZip f where
   fszip :: (a -> b -> c) -> f a -> f b -> f c
   fszunit :: a -> f a

The definition for lists is fairly straightforward:

instance SZip [] where
   fszip f [] _          = []
   fszip f _ []          = []
   fszip f (x:xs) [y]    = (f x y):(map (\ x -> f x y) xs)
   fszip f [x] (y:ys)    = (f x y):(map (f x) ys)
   fszip f (x:xs) (y:ys) = (f x y):(fszip f xs ys)
   fszunit x             = [x]

Here we get the expected pair-wise addition of lists, for example:

Prelude> [1,2,3] + [4,5,6]
[5,7,9]

As well as scalar arithmetic distributing over a list:

Prelude> 2 + [1,2,3]
[3,4,5]

This latter fact comes from the application of fszunit via fromInteger, which converts 2 into [2]:

[2] + [1,2,3]

By the definition of fszip, this maps (2+) over [1,2,3], which explains why we produce [3,4,5].

Now, we can also define an SZip instance for our binary trees (for which we’ve already provided the necessary Functor instance):

instance SZip Tree where
   fszip f (Leaf x) (Leaf y) = Leaf (f x y)
   fszip f (Branch l1 r1) (Branch l2 r2) = Branch (fszip f l1 l2) (fszip f r1 r2)
   fszip f x (Branch l r) = Branch (fszip f x l) (fszip f x r)
   fszip f (Branch l r) x = Branch (fszip f l x) (fszip f r x)
   fszunit x = Leaf x

At this point, we’ve out-KDBed KDB, because all of our numeric overloads combine to produce a beautiful result:

Prelude> 2 * (Branch (Leaf [(1,2,3), (4,5,6)]) (Leaf [(7,8,9), (10,11,12)]))
Branch (Leaf [(2,4,6),(8,10,12)]) (Leaf [(14,16,18),(20,22,24)])

Hopefully the value of this kind of overloading, over types and type-constructors, is clear.  In the next article, I’ll show how type-classes can also be used to define relations between types (and/or type-constructors).

Yet, in the world of F# (and OCaml in a slightly different case), we encounter some logical anomalies:

> (+);;
val it : (int -> int -> int)
> 3.2 + 5.4;;
val it : float = 8.6

Which is the truth here?  Initially we’re told that the “+” operator expects two int values, but it seems to happily accept two floats.  Worse yet, if we expect to rely on this magical addition, our hopes are soon dashed:

> let twice x = x + x;;
> twice 9.4;;
  ——^^^
stdin(2,5): error FS0001: This expression has type
    float
but is here used with type
    int.

Our only chance for generic arithmetic (over ints, floats, vectors, etc) is to explicitly pass the necessary arithmetic functions around:

> let twice fn x = fn x x;;
val twice : (‘a -> ‘a -> ‘b) -> ‘a -> ‘b
> twice (+) 1;;
val it : int = 2
> twice (+) 1.2;;
val it : float = 2.4
> twice (fun (a,b) (c,d) -> (a+c,b+d)) (1,2.3);;
val it : int * float = (2, 4.6)

And although this works, it has a kind of Jerry Seinfeld “why don’t you just tell me the function you want to call?” feel to it.  There must be a better way.

Implicit, Ad-Hoc Polymorphism

This problem is all the more frustrating, because we can resolve it so easily in C++:

int    twice(int    x) { return x+x; }
double twice(double x) { return x+x; }

20 years ago, Phil Wadler and Stephen Blott developed an effective solution to this problem, in How to make ad-hoc polymorphism less ad hoc.  The result has been arguably one of the most significant advances made by the programming language Haskell.  It’s a tool that could be put to good use if it was included in F#.

In investigating this aspect of Haskell, it’s instructive to open a GHC prompt and ask for the “+” operator’s type:

Prelude> :t (+)
(+) :: (Num a) => a -> a -> a

This type signature tells us that the “+” operator will, for any type ‘a’, take two ‘a’ values as input and produce an ‘a’ value as output, provided that ‘a’ represents a “number.”  Likewise, if we use it in our own functions:

Prelude> let twice x = x + x
Prelude> :t twice
twice :: (Num a) => a -> a

We find that the same kind of type signature persists, and that we can safely use it on any number:

Prelude> twice 1
2
Prelude> twice 3.4
6.8

What’s going on here?  Essentially, our code is still translated to something like the moviephone-esque F# example above, except that the compiler decides which actual arithmetic functions to use at the last possible moment (when the type ‘a’ above is decided, however distant that is from the innermost function’s use of the “+” operator) by a kind of “moviephone-transform” of your program (more detail on this transform in a future post).  This added constraint “Num a =>” says that ‘a’ belongs to the “Num” type-class, and this type-class membership is defined by the programmer on an ad-hoc basis.

For example, suppose that we’ve defined a new type of number — like the Peano numbers:

data Peano = Zero | Succ Peano deriving Eq

Note that this type definition is almost identical to the kind we would write in F#, except that the “deriving” piece at the end specifies that the compiler should automatically put this type into the “Eq” type-class (allowing Peano values to be compared for equality).

In order to treat these structures as numbers (to put the “Peano” type into the “Num” type-class), we need to define each function required by the Num type-class:

instance Num Peano where
   p + Zero              = p
   p + (Succ x)          = (Succ p) + x
   p – Zero              = p
   Zero – x              = Zero
   (Succ p) – (Succ x)   = p – x
   p * Zero              = Zero
   p * (Succ x)          = (p * x) + p
   abs p                 = p
   signum Zero           = 0
   signum p              = 1
   fromInteger 0         = Zero
   fromInteger n | n > 0 = Succ (fromInteger (n – 1))

Here, we’ve implemented the “+”, “-”, and “*” operators by the standard definitions of Peano numbers.  The rest of the functions aren’t very interesting, but are required to make this Num Peano instance consistent with all uses of numbers in Haskell.

We can then test this definition in a fairly straightforward way, opening a GHC prompt with the above type/instance initialized (to do this, put the above definitions in a file, peano.hs, and then run: ghci peano.hs):

Prelude> let one = Succ Zero
Prelude> let two = Succ one
Prelude> let three = Succ two
Prelude> (one + two) == three
True

Our existing functions, defined only over instances of the Num type-class, will also work with this new form of number:

Prelude> (twice one) == two
True

If this kind of first-order ad-hoc polymorphism were added to F#, it would go a long way toward restoring some of the honest (logically consistent) conveniences of languages like C++ (or Haskell, for that matter).  But that’s not all that type-classes can do.  In the next article, I’ll show how they also subsume more language features, like inheritance and sub-typing, in an equally rigorous and logically consistent way.

If you write software for humans, or if you yourself are a human, read on for a simple, classical solution to this ancient problem, expressed concisely in F#.

The first problem is recognizing that there is a problem

Yes, your parents were right.  You can’t be helped if you don’t know that you need help.  That’s why it’s important, right off the bat, to be able to determine that a word is actually misspelled before you try to correct it.  We’ll see, shortly, that correcting a misspelled word is only a slightly different problem than recognizing that it is misspelled.

To determine whether or not a word is spelled correctly, we need to consult a dictionary.  Paper dictionaries typically sort all known words in alphabetical order, forcing humans to validate spelling by a form of binary search (better at least than a linear search!), which incurs a log(N) cost for each lookup (with N being the number of words in the dictionary).  That might be acceptable just for validation, but when we search for spelling correction options that will translate to a series of lookups — enough to make it worthwhile to choose a different structure to represent our dictionary.

Enter the prefix-tree, or trie.  Tries store dictionaries efficiently by grouping all strings with a common prefix.  This defines a recursive structure, so that  to verify that a word is in the dictionary represented by a trie, we find the sub-trie associated with the first character of the word and then we verify that the remaining suffix of the word is in the sub-trie (recursively).

So, for example, the dictionary consisting of the following words:

Hello
Help
world
work
Jimmy
Jim

Would produce the following trie structure:

Here, edges N steps from the root represent the Nth character of some test string, and large circles represent valid words in the dictionary (of course, not all prefixes of valid words should be valid words themselves!).  And determining whether or not a word is in the dictionary just takes an amount of time proportional to the length of the word itself, rather than the size of the dictionary.

Just a short distance to the solution

Now we have an efficient method to determine whether or not a particular string is in our dictionary, but how can we use this structure to search for “similar” words once we determine that a word isn’t valid?  As my friend Doug Finke has pointed out, the critical definition of “similarity” is the edit-distance (or Levenshtein distance).

The edit-distance between two strings is defined as the minimum number of edits necessary to translate one into the other.  Within this definition, an “edit” means inserting a character somewhere within the string, modifying a character, or deleting a character.  For our purposes (though this is easily generalized) when we determine that a word is misspelled, we’ll search for all words in the dictionary that are at most an edit-distance of 1 from the erroneous word.

So, suppose that we had the same dictionary as above, and we were presented with the word “Jimky.”  Put aside for the moment that we know the obvious correction.  If we want to find all strings in the dictionary that are an edit-distance of 1 from “Jimky,” we can derive those from the definition of edit-distance itself:

Inserts
?Jimky
J?imky
Ji?mky
Jim?ky
Jimk?y
Jimky?

Updates
?imky
J?mky
Ji?ky
Jim?y
Jimk?

Deletes
imky
Jmky
Jiky
Jimy
Jimk

Here, “?” means “any valid character” (in other words, each string with a “?” above actually represents a set of strings — one string for each valid replacement of “?”).

This might seem to present an expensive proposition, possibly testing (and constructing!) a large number of strings for even moderately-sized words.  However, we can rest a bit easier in recognizing that several of these generated strings share a common prefix (e.g.: consider “Jim?ky” and “Jim?y” sharing the prefix “Jim”).  Luckily for us, we already have a data structure ideally suited for finding multiple words with a common prefix — the trie — which we can use to group all of these alternatives together and accept or reject them in bulk.

There is very little left now to bringing our solution together.  We can simply walk the dictionary-trie with our erroneous word.  Prior to taking each step, we consider what words we could find if we were to insert a character at the current level, replace the first character and search for the remaining suffix at the current level, or delete the first character and search for the remaining suffix from the current level — then we follow the current character in this word to the next level of the trie, and continue this process for the remainder of the string.

This method saves us some potential pain.  We don’t need to explicitly construct all strings that could possibly be an edit-distance of 1 from our test string.  Also, we have the exact information we need at each point that we decide to substitute valid characters for “?” (i.e.: at a given level in a trie, we always know what outgoing edges there are and what their labels are).

All that’s left now is to implement it.

A little F#

The first place to start is in representing a trie as a data-type.  A reasonable definition could be:

type (‘k,’a) trie = TNode of (‘a option * Map<’k,(‘k,’a) trie>)

Here, the type-variables ‘k and ‘a represent the base key type and element type (respectively).  A trie is just an associative map, and as with other associative maps we’re just defining the same kind of key and element types (except here, our actual key will be a sequence of the base key type).

We won’t really care much about ‘a (the element type) for this problem, and we’ll set ‘k to char.  It’s worth noting, though, that the value stored in a trie node is actually an ‘a option because we won’t store values in proper prefixes of valid strings (those will have just the option value None).

The edge/sub-trie relation is saved in a Map — and any type of associative map would do here.  In fact, in principle this edge/sub-trie map could itself be another trie (useful if our key sequence type is a list of strings, or a list of lists of strings, etc…).  This method of defining a type in terms of a (usually weaker) form of itself is termed “data-structural bootstrapping” (by at least one influential book on the subject of purely functional data structures).

Next, to be sufficiently general in defining a trie, we will want to abstract away the actual method of iterating over the key sequence and extracting keys.  An easy method of packaging that abstraction in F# is a record of functions, like:

type (‘ks, ‘k) iter_module =
   {
      eos  : (‘ks -> bool);
      head : (‘ks -> ‘k);
      tail : (‘ks -> ‘ks)
   };;

Here ‘ks represents a type that stores a sequence of our key elements, and ‘k represents the type of our key elements.  The functions in this iterator module should be familiar — eos determines whether or not a key-sequence is at its end, head gets the current key-value in a key-sequence, and tail increments the key-sequence iterator.

But for this spelling-correction problem we don’t need to be so general.  We already know that our base key type is char, and so we can define an iterator over a sequence of chars straightforwardly as a pair of a string (the sequence of chars) and an int (the current position in that sequence).  All that’s left then is to fill out a module definition that assumes this representation:

let siterm : (string * int, char) Trie.iter_module =
{
  eos  = fun (s, i) -> i >= String.length s;
  head = fun (s, i) -> s.[i];
  tail = fun (s, i) -> (s, i+1)
}
;;

We’ll assume the generic iterator module when implementing tries, and we’ll come back to this instance when we get to implementing spelling correction.  For now it’s useful to keep in mind as a justification for the key sequence iterator abstraction.

Now, we can define some simple functions over the nodes of a trie.  First, to extract the option value associated with a node:

let node_value = function
| TNode (ov, _) -> ov
;;

Similarly, to extract the map representing connections from the current trie node to sub-tries:

let node_map = function
| TNode (_, m) -> m
;;

To determine whether a trie is empty, and the standard empty trie:

let is_empty tn = Map.is_empty (node_map tn);;
let empty_trie  = TNode (None, Map.empty);;

And to find the sub-trie (if it exists) from a given node, along a specific edge:

let find_subtrie tn k =
 try
  Map.find k (node_map tn)
 with
  Not_found -> empty_trie
;;

Now, the first critical function defined on a trie is to update it (here using the key sequence iterator module and maintaining the assumption that common string prefixes will be shared):

let add m tn ks v =
 let rec upd tn’ ks’ =
  if (m.eos ks’) then
   TNode (Some v, (node_map tn’))
  else
   let k    = (m.head ks’) in
   let ov   = node_value tn’ in
   let tn” = upd (find_subtrie tn’ k) (m.tail ks’) in
    TNode (ov, Map.add k tn” (node_map tn’))
 in
  upd tn ks
;;

Similarly, to lookup the option value associated with a string:

let lookup m tn ks =
 let rec lv tn’ ks’ =
  if (m.eos ks’) then
   node_value tn’
  else
   let k = (m.head ks’) in
    lv (find_subtrie tn’ k) (m.tail ks’)
 in
  lv tn ks
;;

Finally, we can determine whether or not a specific word is valid in the trie by a simple use of the lookup function:

let mem m tn ks =
 match (lookup m tn ks) with
| Some _ -> true
| None   -> false
;;

Now, stepping outside of this trie definition, we can concern ourselves with implementing the spelling correction method that we worked out earlier.  First, we’ll need a couple of simple utilities.  One to determine the suffix of a string (represented by our string*int iterator):

let suffix (s, i) = String.sub s i (String.length s – i);;

And another, which computes the set of words defined by a trie node if we follow any valid character from that node, and then try to follow a specific char sequence:

let strings_with_suffix pfx trie wi words =
 let accumulate_paths c trie’ ws =
  if (mem siterm trie’ wi) then
   Set.add (pfx ^ (string c) ^ (suffix wi)) ws
  else
   ws
 in
  Map.fold accumulate_paths (node_map trie) words
;;

In other words, for a prefix P and a suffix S, this function finds P?S (substituting for “?” all valid characters that can follow P).

Finally we can write out our real spelling correction function, taking into account the insertions, replacements, and deletions that yield suggestions a maximum edit-distance of 1 from our test word:

let suggestions trie word =
  let rec paths_around_char pfx trie wi words =
   if (siterm.eos wi) then
    strings_with_suffix pfx trie (“”, 0) words
   else if (is_empty trie) then
    words
   else
    let c      = siterm.head wi in
    let wi’    = siterm.tail wi in
    let trie’  = find_subtrie trie c in
    let ins_ws = strings_with_suffix pfx trie wi words in
    let rep_ws = strings_with_suffix pfx trie wi’ ins_ws in
    let del_ws = if (mem siterm trie wi’) then
                  Set.add (pfx ^ (suffix wi’)) rep_ws
                 else
                  rep_ws
    in
     paths_around_char (pfx ^ (string c)) trie’ wi’ del_ws
  in
   paths_around_char “” trie (word, 0) Set.empty
;;

Finishing touches

To make this spelling corrector useful, some additional supporting code should be written to package it as a command, taking arguments for a dictionary and plaintext file.  We’ll also need some basic utilities for reading files and splitting them into words.  Additionally, we should provide the option to produce nice graphviz diagrams for arbitrary tries (as was used to produce the initial trie diagram in this article):

let graphviz ovstr kstr tn =
 let rec nodes p tn’ =
  let scs k = nodes (p ^ “_” ^ (kstr k)) (find_subtrie tn’ k) in
   let nshape = if is_terminal tn’ then “circle” else “point” in
   (” ” ^ p ^ ” [shape=\"" ^ nshape ^ "\" label=\"" ^ (ovstr (node_value tn')) ^ "\"];\n”) ^
   (String.concat “” (List.map scs (path_heads tn’)))
 in
 let rec connections p tn’ =
  let
   scs k =
    let sn = p ^ “_” ^ (kstr k) in
    (” ” ^ p ^ ” -> ” ^ sn ^ ” [ label = \"" ^ (kstr k) ^ "\"];\n”) ^ (connections sn (find_subtrie tn’ k))
  in
   String.concat “” (List.map scs (path_heads tn’))
 in
“digraph g {\n” ^ (nodes “root” tn) ^ (connections “root” tn) ^ “}\n”
;;

You can download the complete source as a Visual Studio 2008, F# console project here:

scheck.zip (5 KB)

In the attached project, edit the “dict” and “text” files to taste, or change the project’s default arguments to generate trie-graphs from the dictionary file.