Big Data: Online Gaming NoSQL Logging System

Details

The customer was embarking on a project to be the first in the industry to offer a state-run lottery sales and online Internet gaming product. The project required the implementation of a high performance service bus that could handle peak loads of up to 10,000 transactions per second. Because the system supports online internet gambling, regulations require that every transaction be logged, thus requiring a logging system that could keep pace with the service bus and log every transaction as it occurred in real time. The requirement was to create one log entry when the transaction started and one when it ended, therefore the persistence mechanism had to support up to 20,000 inserts per second. In addition to the high volume of operations that needed to be persisted, the logging system had to provide the following additional characteristics.

• High Availability & Automatic Fail-over. Every transaction had to be logged. If a transaction could not be logged, then the transaction could not occur and revenue would be lost. As such, the logging system needed to be highly available, using clustering technology that supports automatic fail-over if a node in the cluster goes down.
• Disaster Recovery. The logging service needed to support off-site disaster recovery via log-shipping, replication or mirroring.
• Scalability. The selected storage/database technology backing the logging service needed to scale as the transaction volume on the system scaled. Traditionally, database systems scale either vertically (bigger, faster machines) or horizontally (additional machines). Initial requirements needed to support 20,000 log entries inserted per second, and needed to allow for growth well beyond that.
• Durability. Every transaction logged must be committed and be permanent. This means that writes need to be committed to disk and/or replicated to other nodes before they can be considered durable. Lazy-writing or flushing of data to disk cannot be used, because a system failure would lose data in memory that has not yet been written to disk.

The objective was to select a persistence technology to meet the above requirements as well as implement a logging service that encapsulated the persistence technology and exposed the logging operations to the service bus.

Why Atmosera?

Atmosera is a recognized leader in Windows platform and database architectures. The customer turned to Atmosera for architecture and development assistance to ensure they had quality resources to lead their development efforts that needed to be completed in a tight timeframe.

Strategy

Atmosera performed extensive research to identify a storage technology that met the demanding logging needs of the project. This included the following:

• Analysis and benchmark testing on SQL Server, MongoDb, Hypertable, Riak, Cassandra, Couchbase, VoltDb, Redis, and CouchDb.
• Comparison of options based on performance, throughput, fault-tolerance, administration, community support, etc.
• Presentation of options and demonstration of the recommendation solution that exhibited the best combination of scalability, automatic failover and ease-of-use.

Extensive research and hands on analysis was performed to determine the best solution option for the business and technical requirements needed. Below are some of the options reviewed for the NoSQL portion of the system

• Microsoft SQL Server – SQL Server was tested to establish a baseline for performance on the Virtual Machine environment used during testing. SQL Server is generally scaled vertically and didn’t fit the requirement to easily scale as transaction volume scaled.
• MongoDb – MongoDb is a popular NoSQL choice and is extremely simple to setup and use in small clusters. Larger clusters can be more complicated to manage. Write performance scaled reasonably well but not linearly as nodes were added.
• Hypertable – Hypertable is based on Google’s Bigtable design and performed well and scaled linearly as nodes were added. Can be complicated to setup and operate, and did not support automatic failover at the time of testing.
• Redis – Extremely fast but requires all records to reside in memory limiting storage capacity and does not support failover.
• VoltDb – Requires all records to reside in the combined memory across all nodes which limits storage capacity.
• Riak – Riak is a NoSQL technology based on Amazon’s Dynamo. Easy to setup for the initial cluster and to scale the cluster. Has an architecture for which there is no single point of failure. Scaling up and scaling down is a very simple process.

Conclusion

With this project, Atmosera was able to bring to bear a NoSQL solution to meet the demanding requirements of our customer’s leading-edge online gaming product. The performance and fault-tolerance requirements of the logging system called for a careful analysis of the available options, a design that aligned with the needs of the solution and a flawless implementation that would provide the online gaming product with the necessary performance and reliability. A summary of the NoSQL solution benefits provided can be presented as follows:

• Easy to use and configure
• Simple to use REST interface in addition to .NET drivers
• No single point of failure
• Very easy to scale up and down
• Writes scale linearly with additional nodes
• Durability of writes is possible via replication and sync to disk

At the end of the project, the customer was able to be the first to the market to unveil a highly performing online gaming product that met the needs of their customers and complied with industry regulatory requirements.