Grid

I’ve been test-flying a few datagrid/datafabric products lately and had a nice opportunity to try them out on gig-ethernet and infiniband. The thing I noticed with all distributed-data systems is that synchronous-replication is always a bottleneck, because you’re forced to wait on the replication-ack before proceeding with the next operation. Because of this, gigabit-ethernet’s latency puts a ~5k ceiling on any throughput rates you can achieve.

For any grid vendor it’s crucial to rollout modern, ultra-low latency interconnect support then, as its latency characteristics blow away any Gig-E numbers. For real-time price publishers, for algo-trading, latency is the key issue, more than throughput.

Since EC2 appears to be the topic of the week, its worth looking at Scalr. Scalr provides the “elastic” to enable EC2 to provide a dynamic cloud. Google (BigTable) and Microsoft (SQL Server Data Services) both appear to be chasing Amazon (S3), but neither appear to be close to EC2 - curious given Microsoft’s […]

Check out this excellent CodeProject submission named "Legion", which is a grid computing ("cycle stealing") framework in Silverlight, by Daniel Vaughan.

Full disclosure: uses graphing component by Lab49’s own Andre de Cavaignac

I wrote an article recently for GRIDToday. It talks about the promise of virtualization approaches like Amazon’s EC2 to allow for dynamic provisioning of compute resources.

Lab49’s Marc Jacobs was quoted in GridToday, today.

Amazon are pioneering a rent-a-grid service, called EC2 which looks extremely interesting. I hope to spend some time playing with it this weekend - sounds great for distributed Erlang testing!