For all of its obvious success, I still think that NoSQL is underutilized (or, at least, misunderstood) in the enterprise. Some of this can be explained by inertia… the classic relational model has ruled the roost for decades, and certainly isn’t going away anytime soon. Enterprises have significant skillset and infrastructure investments in the care and feeding of relational data and are loathe to give up on those very quickly.
The popularity of relational databases is not based on a single overarching factor. As Martin Fowler points out in his brief but excellent introductory book “NoSQL Distilled”, there are a few things going for it that made relational the dominant data technology of the last few decades. First and foremost is the SQL query language itself. Although the various relational engines support different dialects and extensions of the SQL language, there is a core subset underlying virtually all of them; this makes it easier to learn “the language” and be at least minimally productive in SQL Server vs. Oracle vs. MySQL, etc.
The other big win for relational is the set of common features… transactions, schema creation and maintenance, backup and restore, etc. Again, each specific product has its own take on these, but ultimately they tend to converge on a common set of concepts and capabilities that make it relatively straightforward to orient yourself as a developer or an admin. Relational databases are a mature market and a relatively known commodity, both of which made them (and, frankly, still make them) excellent choices upon which to build enterprise software.
So its upon this backdrop that NoSQL tries to make inroads into the enterprise. Certainly this is an ongoing story, and certainly there are already successes. Amazon and Azure have successful “NoSQL as a service” offerings. Products like MongoDB, Redis, CouchDB, and Cassandra are popular choices, too. If we broaden our definition of NoSQL to include big data technologies like Hadoop, then certainly there’s tons of interesting work happening there.
Surveys will occasionally tout significant movement toward NoSQL in the enterprise, but I tend to be skeptical of this. Based on my personal experience consulting for Wintellect, and anecdotally from others, I suspect there’s a significant subset of enterprises who have invested little in NoSQL technologies to date; certainly not in comparison to their ongoing investment in relational, even though they might very well benefit from a NoSQL approach, or a hybrid relational/NoSQL one.
Depending on who you ask and in which industry, I think enterprises as a whole are likely somewhere in the “Early Adopters” to “Early Majority” stage of the NoSQL Innovation Adoption Cycle:
(source: http://en.wikipedia.org/wiki/Technology_adoption_lifecycle)
This is good news for those who have yet to break their relational shackles and see what lies beyond the horizon. The NoSQL market is maturing, but still has a ways to go before the dust settles as in the relational world. The enterprise moves deliberately (most of the time) and old habits can be hard to break, but there’s still time to join the “majority” part of the curve and benefit both from the mistakes of the early adopters while still outpacing the laggards.
So far we’ve discussed the political and resource impediments; let’s consider some technical ones, too. A big hurdle for NoSQL adoption in the enterprise is that each NoSQL offering is an island unto itself. In essence, each NoSQL database offers a unique combination of scalability, performance, deployment footprint, maintenance capabilities, and read/write API. There is some conceptual synergy around the major types of NoSQL database offerings: graph vs. document vs. key/value, etc. But unlike relational with its unified model and relative commonality around featureset, the NoSQL world is a virtual Wild West of dozens of technologies, some of which are only tenuously related under the large and sometimes amorphous NoSQL umbrella. Many NoSQL proponents will point to this variety as a relative strength; and I won’t disagree with that. The wide variety of choices makes it likely you can, with persistence and skill, find the right tool for the job. But this variety also makes “the NoSQL market” confusing and difficult to sort out for the typical enterprise.
As I discuss in my WintellectNOW video “NoSQL For the Confused”, the lack of a unified query model is I think the greatest impediment to more widespread acceptance of NoSQL. Query is the gateway to any data storage technology; it’s the mechanism through which we become familiar, and then productive, and then masters of our data. Ease of query (and ease of learning to query) makes or breaks the experience.
And there’s the rub for NoSQL, to date. There is no standard mechanism to query all or even some NoSQL databases. Each offers its own API for accessing data within; each offers its own spin on indexing or (in some cases) transactional semantics. There are various movements afoot to change some of this, many of which follow the relational/SQL path and focus on defining a standard query language. And perhaps that is a solution in the long run.
However, I think near-term success lies instead (if anywhere) in the common API approach. I’m curious if a single API abstraction, with underlying implementations targeting popular NoSQL databases, might be capable of yielding a common but “useful enough” set of functionality? My head tells me the answer is likely “no” , but I think it still might be fun and instructive to try, even if it ultimately doesn’t work out. If anything I suspect I’ll be tripped up by the Law of Leaky Abstractions… the bane of “boil the ocean” middleware for time immemorial. It’s hard to abstract away the details of accessing NoSQL data into a common set of patterns while not losing the benefits inherent in any single, concrete, underlying implementation. That is… at best, it’s hard.
I’ve spent some time lately with node.js and mongoose, a MongoDB module for node. I really like the clean, straightforward model of mongoose, and I think it’d be interesting to bring a flavor of that to C# and .NET. I think I’ll start there, and see where I end up.
Stayed tuned for further developments.