Jul 31, 2015

MongoDB 3.0 WiredTiger is Big News for Multi Tenant Deployments

A database per tenant is a common practice in multi tenant systems.
Why? Well, first it s easier to implement as you don't need to record the tenant id in every document, and second, it easier to avoid security issues by compromising the tenant id.

MongoDB 3.0 Storage Engines
MongoDB introduced a new concept in version 3.0 (that is familiar for MySQL users): you can select your preferred storage engine. You can continue using the old MMAPv1 storage engine (that is still the default one for version 3.0) or select WiredTiger that offers compression, document level lock and better performance in some cases.

What is Wrong with MMAPv1
MMAPv1 creates for every new database a data file that its minimal size is ~70MB (and this one is being filled, a new file is created with a doubled size). 
This may not be an issue for large size databases, but if your system design is built on a large number of tenants, that many of them will not have more than few records (lets say trial tenants that decided not to go on w/ your system), you are going to have 70GB disk allocation for every 1,000 tenants. 
This behavior will result in another side affect: very high IO usage (mostly for read), that is due to the need to read every time a large file in order to update or insert very few rows.

WiredTiger Comes to Rescue
WiredTiger includes a compression method that can cut 85% of your storage needs in a large database (trust me that I saw this number in a 10TB billing system). That is great, but more important, it does not allocate this 70MB file per each database (it satisfies with two small files for indexes and data).
The result: for a 3,500 tenants system, the database was shrunk from 330GB to under 1GB... not to mention that IOPS that were dropped from 12,000 to 600...

Bottom Line
If you are using MongoDB for multi tenant system, WiredTiger can cut your storage needs in 99% and your IOPS in 95%. This can save a fortune.

Keep Performing,
Moshe Kaplan


Intense Debate Comments

Ratings and Recommendations