amazon ec2 - How good is this MongoDB/EC2 setup?

Hi,
I wondered what was a good setup for AWS/MongoDB in terms of machines and the sizes of their disks.

Current setup

• 3 micro machines for the config servers, 1 mongos and arbiters. The 8Gb limit is almost reached. (and I ran the arbiters with --nojournal)


• per shard : a replica set of 2 machines m1.large with 8Gb for system + 20Gb for data


• everything is on EBS.

Questions

1. is 20Gb too big or too small ? Should I go with 100Gb for example ?


2. Am I supposed to inform mongodb about the 20Gb (or other) disk limit ?



3. Do you see anything wrong that I don't see ? Im new to mongodb and aws but Im an ok experienced SWE

Plan of use

My database should allow a 100 qps (mostly writes), and should grow up to 1Tb over the next 3 years. The plan is to add as many shards as needed, more or less manually (with scripts), when we see that more memory is needed on the database.

We will also run a few mapreduce over this and have some scripts that do aggregates with the data over the past 15 minutes, every 15 minutes.

We are a very small company, spending up to a few hundred $ per month on our servers would be ok but we can't go crazy on cash.

We hope that we won't have to manually take care of too many machine failures, manually taking care of things once a month would be fine.

Thanks for telling me what you think about that.

Thomas

Answer

First your specific questions:

is 20Gb too big or too small ? Should I go with 100Gb for example ?

This completely depends on your data requirements and how many documents you intend to insert. If you intend to have 5GB of documents then you should be fine, even with overheads for replication (oplog is 5% of free space) and storage (there is always an empty file pre-allocated for each database). If you plan to have 10-12GB of data (and remember you have to store indexes, journal, logs as well) then I would go for a larger disk.

Since you say you plan to grow to 1TB in a year then you will probably exceed 20GB inside a month and need to increase disk anyway, hence it will probably be easier to go for 100GB immediately. At 1TB in a year, assuming constant growth, that will only give you about 1 month of room (1TB per year ~= 83GB per month).

Am I supposed to inform mongodb about the 20Gb (or other) disk limit ?

No, there have been improvements in how it handles the situation but MongoDB will currently just use all available space until there is none left - you need to monitor your disk space independently.

Do you see anything wrong that I don't see ? Im new to mongodb and aws but Im an ok experienced SWE

Never use micro instances for anything in production - in particular do not use them for config servers. Your config servers are critical for the operation of a sharded cluster. But, no need to take my word for it - see page 6 of the updated Amazon whitepaper:

T1.micro instances are not recommended for production MongoDB deployments, including arbiters, config servers, and mongos shard managers.

Generally I would recommend reading through the whitepaper and following the guidelines therein - you'll find recommendations for Linux settings (readahead, hugepages etc.), storage, pIOPS and more. Also worth checking out are the Production Notes - some duplication, but it's updated more often than a whitepaper.

Finally, get some idea of your working set size for your database (per shard) - that will dictate how much RAM you need, which is really the key to selecting instance size on EC2 for MongoDB. You may have enough with 8GB, but if not you will see significant performance hits for hitting disk.