Aug 29, 2012

Who takes care of non Sharded tables in Sharding?

The Challenge
As you may remember, Sharding is about splitting a database according to a key. Usually it is according to user id, client id or a phone number.
One of the major questions when we implement Sharding is what to do with data dictionary tables? The data dictionary tables are those tables that include static data that is being used by all shards.

The Options
In general there are two options:

  1. Have a single copy of the data dictionary tables and ask the application to take care of the complex queries.
  2. Have multiple copies of the same table and create a data distribution solution that will update all of these table copies.

Data Dictionary/Static Tables Sharidng Solution

The Elegant Solution
Create a new database that will serve as a master to all shards.
This database will include all data dictionary tables, and will serve as the single write instance for all data dictionary tables.
Once a record will be updated at one of these data dictionary tables, it will instantly be replicated to the all of the shards and will be available.

Bottom Line
Sharding may seem hard, that with simple and elegant solutions you can achieve your own scale-able solution.

Keep Performing,
Moshe Kaplan


Intense Debate Comments

Ratings and Recommendations