Open Sharding Protocol

This week we introduced the Open Sharding Protocol (OSP), an open source protocol designed specifically for plug-compatible database drivers which support the full capabilities of database sharding. OSP is the foundation of the dbShards 3.x architecture, something I’ll be talking about a lot at the Cloud Computing Expo West, Nov. 7-10 and in the coming months as we continue to introduce leading advances in dbShards technology.
There are many approaches to database sharding, but they all agree on one principle: all modern scalable database architectures (RDBMS, NoSQL, “NewSQL”) rely on scalable, horizontal partitioning to accomplish ultra-high performance. As you probably know, we have been in the sharding game for many years, and early research we performed proved that a “share-nothing/shard-everything” architecture is unbeatable when it comes to overall performance and throughput. This is even more powerful with our Relational Sharding approach.
Now we have culminated this research and experience with the introduction of OSP, making it available to anyone interested in implementing an OSP-compatible client architecture. We’ve also open-sourced foundation components and specific OSP drivers too (starting with MyOSP for MySQL®).
At the heart of OSP is an extremely efficient communications layer based on Google® Protocol Buffers (http://code.google.com/apis/protocolbuffers/), the same wire-protocol that Google uses internally for many of its systems.
The primary advantage of OSP is its support of plug-compatible drivers, allowing for direct interaction between the application tier and the database tier. The OSP drivers themselves are “ultra-thin” meaning that they can directly access shared capabilities available in the OSP client that manages sharding logic and functionality. This direct client approach is seamless, and supports a powerful array of database sharding capabilities including global writes, relational sharding, joins and “Go Fish” parallel Queries. OSP provides a great deal of implementation flexibility, excellent performance, and compatibility when compared to other architectures. OSP supports several efficient inter-process communication mechanisms, as well as standard network communications when needed, enabling important features in the database sharding client architecture without compromising performance.
As part of this announcement, we are also introducing no-charge dbShards/Basic edition. This edition is a client-only version of dbShards, and is completely free for users of MySQL® in the Amazon AWS cloud environment. It includes dbShards/Tools (command-line) to support all phases of the database sharding process, and the same high-performance dbShards/Client technology included in dbShards/Enterprise. The private beta –kicks off this week at the 2011 Cloud Computing Expo West, and with this release we expect to broaden the accessibility of database sharding to a much wider audience.
Why did we introduce this free version? Because we have always provided dbShards/Client at no-charge with our Enterprise edition, and with awareness of database sharding in the database community on the rise, we wanted to make our technology available to as wide an audience as possible. Of course we’ll also offer our advanced database sharding expertise, implementation support and monitoring to those who wish to add that capability, but use of the free edition itself is — just as it says above – totally free.
(Reprint posts: from http://www.dbshards.com/blog/2011/11/open-sharding-protocol/)