Author Archive

IPC systems announced Unigy (for “Unified Synergy”) a new SOA-based unified trading communications and applications platform. The platform attempts to unify and normalize all trade communications hardware and software, providing services and API’s upon which to build collaborative applications.

This unification of all trade communications — email, voice, etc. — can allow trading firms to potentially analyze market sentiment by tracking, recording and analyzing their phone calls.

Last week I mentioned how the CFTC’s proposal for “Swap Exchange Facilities” moved forward, but a possibly more important non-event happened: the proposal to limit commodity speculation via position limits did not go to a vote, again.  Read more...

The Dodd-Frank “Swap Execution Facilities” proposal was released by a 4-1 vote yesterday. Proposal details are here, but the skinny is:  Read more...

• Centralized, open/transparent limit order book
• Participants are allowed to post “indicative” quotes, so less liquid derivs are afforded some cover (this was the source of the single “no” vote)
• Allows RFQ process, as long as the request is sent to 5 or more market makers
• Provides for transaction tiering:
  1. Tier 1 transactions are not block trades and exhibit material transaction volume;
  2. Tier 2 transactions are not block trades and do not exhibit material transaction volume;

Use of Sentiment Analysis in financial applications has been on the rise for a while now, particularly since Paul Tetlock’s important paper on the subject in 2007. Several data providers can be found which offer streaming Sentiment Analysis feeds, but do they use the same dictionary Tetlock used? There are some problems with that dictionary’s categorization for many of those words as they relate to finance. Words like “tax”, “depreciation”, “floor”, even “foreign” are all negs in the Harvard dictionary, but in financial contexts have no such meaning.  Read more...

I spent part of last week attending IFL 2009; as I don’t really consider myself a ‘real’ functional programmer, what I have to say is kind of an outsiders perspective.

Generally, I thought it was worth 3 days of my life and saw some things that look pretty promising. It was also nice to get a window on the types of problems that the field is grappling with right now. Not surprisingly, there were several presentations on concurrency/parallelism; some were attempts to provide more syntax for explicitly declaring concurrent programs, but what I find more promising were the presentations of data parallelism where the language or library figures out what can be processed in parallel and optimally does it for you. Thus your programs address the problem you’re actually solving rather than concurrency arcana.  Read more...

I just ran across Microsoft Security Engineering Team’s newly released code-security/crash analyzer !Exploitable, this could be a really huge boost to developers if it’s done well. Anybody taken this out for a spin yet?

I’ve been test-flying a few datagrid/datafabric products lately and had a nice opportunity to try them out on gig-ethernet and infiniband. The thing I noticed with all distributed-data systems is that synchronous-replication is always a bottleneck, because you’re forced to wait on the replication-ack before proceeding with the next operation. Because of this, gigabit-ethernet’s latency puts a ~5k ceiling on any throughput rates you can achieve.

For any grid vendor it’s crucial to rollout modern, ultra-low latency interconnect support then, as its latency characteristics blow away any Gig-E numbers. For real-time price publishers, for algo-trading, latency is the key issue, more than throughput.

At a client-site I was privy to an impressive IBM presentation on their

Websphere MQ Low Latency Messaging platform (LLM), with some pretty exorbitant claims: 8-million messages per second on gig-ethernet, and 21-million per second on infiniband(!).

I’ll be interested to see if it stands up to scrutiny.

Built on the Apama platform, it’s called the Detica Market Surveillance Accelator. Designed as a sift, to survey all trades booked in a system for bad and/or illegal behavior, detect errors, and to aid in development of a more proactive support model.

In light of the Société Générale debacle, I expect to see more of these in development. Apama strikes me as a near-ideal match for this kind of application.

CEP engines try to keep state-management out of the developers’ hands (or, if you prefer, out of your way), letting you concentrate on just the events. Inevitably, though, we end up needing just a little more state than we have in the current event stream; or we have an event that’s needed by several monitors (e.g., benchmark prices); or the event data’s needed in several monitors, but doesn’t actually trigger any action.

Enter the cache-monitor: a common Apama pattern we’ve used in our projects. It’s a simple pattern, but I’m posting it to illustrate Apama’s programming model which seems to be not common knowledge. What throws people for a loop at first is that in order to retrieve a value from a cache-monitor, we mimic a function call from typical procedural languages by routing a Request-event to that monitor, and listening for the matching Reply-event routed back to us by the cache. To see what I mean, consider the 3 snippets below.  Read more...

The idea is to respond to incoming price events, but always process the latest event, skipping intermediate events. Here’s the way you can express that in Apama’s monitorscript language.

You can stage processing of an external event by caching an internal-version of the event, and enqueuing a ‘process-this-price’ event (putting it at the back of the queue). As more 3Y prices come in, they overwrite the cached event, and enqueue more ‘process-this-price’ events. When the ‘process-this-price’ events are reached, the latest price will be removed from the cache and the internal-version routed as the ‘real’ price event. Subsequent ‘process-this-price’ events result in no events because the 3Y was already removed from the cache. Here’s the code:  Read more...

This morning fellow 49′er Doug and I spoke with a colleague in Citibank who spent friday at an absolutely free boot camp for NVidias GPU development architecture. Wish we’d known about it beforehand, if nothing else for the free lunch.

There’s a lot of activity around building development architectures to take advantage of the compute power on Graphics Processing Units.  Read more...

… Dysfunctional Programming? Alright it’s just a joke, no mailbombs please.

I do think that it’s fair to say that our industry has gradually grown more and more impatient with side-effect-driven programming models. That’s not to say that classical OO-designs will all be failures, they can and do succeed often enough — successes being directly proportional to the project’s unit-testing commitment. I think the power of coupling state with logic slides into hopelessly error prone apps a lot sooner than most of us expect.  Read more...

In a little test-timer I wrote, the following pattern:


if( myDictionary.ContainsKey(mykey) )
myValue = myDictionary[myKey];


took about twice the time that the following would take:


Foobar myValue;
if( !myDictionary.TryGetValue(mykey, out myValue) )
// add the key here if so desired


The Problem
Microsoft’s Composite UI Application Block (CAB) attempts to instantiate and inject services on your behalf, but it often fails when you have services depending upon other services.  Read more...

Jeffrey Richter has posted an article about a ‘ReaderWriterGate’ he’s written in response to an interesting concurrency problem. Read the article of course; but in order to comment on it I will summarize a bit (so fair warning: spoilers below).

The problem was a web server with a typical scenario: data shared by many and frequent readers, with a few rare writers (think catalog order system). Textbook case for a ReaderWriter lock which Mr. Richter has addressed in the past. In this particular case, though, the infrequent writes took a long time. With ReaderWriter locks, that queues up lots of readers, then unleashes them on the scheduler en masse as soon as the write is finished. This results in excessive context-switching.  Read more...

The BCLTeam’s blog mentions some possible additions to the .NET Math class. Among them vector maths, complex numbers and many trig additions, factorials, and permutations. Although it’s folly to say “no, thanks, don’t want anymore useful tools”, I wonder about the cost-benefit from Microsoft’s perspective. When I asked myself what applications I would use these features for, the larger issue of whether .NET would be my first choice for those apps loomed. In most of the apps that I think of calcing reverse-hyperbolics, or multi-variable linear regression, it seems to me that performance is first order importance, whereas .NET elevates maintainability, extensibility and rapid-development over performance.  Read more...

I’ve recently been working in someone’s code that uses Regex quite a bit, and wanted to make a simple observation: I contend that in 99% of the regex usage I’ve seen, the pattern /.*/ is used where /.*?/ is intended.

When Regex processes .*, it shifts in the entire input then backtracks until it finds the pattern that follows your .*. Most of the time, though, the coder clearly is thinking in left-to-right terms. An example:


string input = "<blah>here's some stuff</blah>other>otherstuff</other>";
foreach(string pattern in new string[] {">.*<", ">.*?<"})
{
    Match m = Regex.Match(input, pattern);
    Console.WriteLine(m.Value);
}
 Read more...

If you’re like me, you can’t get enough of good old printf debuggery. Thankfully, those friendly folks at Microsoft kept even bad engineers like us in mind when dreaming up features for VS.2005; they’ve added a variation on breakpoints called “TracePoints”. These don’t actually break when you reach them, instead they allow you to output logging info to the console or run macros. You can set these tracepoints to continue or to break, which makes me wonder if I’ll ever use breakpoints at all anymore. I haven’t delved into the macros at all, so if anybody has more info on that, please feel free to chime in.  Read more...

In this Krzysztof Cwalina post, he mentions the changes to System.Collections in .NET framework 2.0 (yes, 2.0, not 3.0, therefore very prosaic and boring). Of course we know about all that new generics stuff which allows things like List<T>, Collection<T> and Dictionary<TKey,TValue>, which are great things, but further down the list he mentions a new SortedDictionary type:

SortedDictionary<TKey,TValue> does not have a corresponding non-generic collection. It’s similar to SortedList and SortedList<TKey,TValue>, but it’s implemented as a balanced tree (red-black tree). Insertions are on average much faster, but some lookups are slightly slower and memory consumption is larger.  Read more...