Cloud Backup Strategy on AWS
Always have a Cloud Backup plan.
Mila Kunis
So, you moved to the Cloud and you got yourself one or more server? You host your own data and potentially even client data? What if you got hacked today? Database corruption, File corruption, accidental deletion, all these nice things happen all the time.
Cloud Backup Its Not a Matter “If” but “When”
IRS Criminal Investigation
Hopefully, AWS offers lots of options when it comes to Cloud Backup and data redundancy. On the other hand, Online Cloud Backup is not free. EBS Snapshots and S3 Buckets are darn cheap, but costs accumulate since you pay by the GB.
What Are My Cloud Backup Options?
EBS Snapshots
Convenient, easy to generate, easy to automate, EBS Snapshots are however the most expensive Cloud Backup option from AWS.
EBS Snapshots = $0.05 per GB-month of data stored
50GB = $2.5 per month
Snapshots are critical for fast data recovery though, they are therefore necessary for your system volumes. Convenient, easy to generate, easy to automate, EBS Snapshots however can become quite expensive in the long run.
How much do EBS Snapshot cost?
$0.05 per GB-month of data stored
50GB = $2.5 per month
Can you automate them?
Somehow, through Lifecycle Manager
How far back in time can you go, for your $$$?
24h if you are cheap, years if your pockets are full
What if I want to restore last month backup?
You need to retain 30 snapshots in rotation.
Wait a minute. 30 Snapshots?
Yes, the maximum time between snapshots is 24h.
How much costs 30 Snapshots?
30 Snapshots of a typical 8GB Ubuntu system = 30 * $0.05 = $12 per month
Can you backup each week instead?
Not without aws cli scripts.
Can you automate the snapshots deletion?
Not without aws cli scripts.
Conclusion: You can live with only one snapshot in rotation every 24h per system volume, but for data volumes you certainly do not want that. Also, indeed Snapshots can be automated by aws cli, it makes the Snapshots console neither fish nor fowl. You don’t have this issue with S3 Buckets since you are forced to build scripts from the start.
What are these aws cli scripts you are mentioning?
Check this AWS documentation out
S3 Buckets
S3 Buckets are a much cheaper Cloud Backup alternative. Pretty much everything to know is said on AWS’ own wiki, but it’s a bit more complex. Depending on how frequently you want to access your data, you can go as low as $0.00099, as of 2020. Look at the prices below
| Storage pricing | |
|---|---|
| S3 Standard – General purpose storage for any type of data, typically used for frequently accessed data | |
| First 50 TB / Month That’s already half the cost of Snapshots | $0.023 per GB |
| Next 450 TB / Month | $0.022 per GB |
| Over 500 TB / Month | $0.021 per GB |
| S3 Intelligent – Tiering * – Automatic cost savings for data with unknown or changing access patterns | |
| Frequent Access Tier, First 50 TB / Month | $0.023 per GB |
| Frequent Access Tier, Next 450 TB / Month | $0.022 per GB |
| Frequent Access Tier, Over 500 TB / Month | $0.021 per GB |
| Infrequent Access Tier, All Storage / Month | $0.0125 per GB |
| Monitoring and Automation, All Storage / Month | $0.0025 per 1,000 objects |
| S3 Standard – Infrequent Access * – For long lived but infrequently accessed data that needs millisecond access | $0.0125 per GB |
| S3 One Zone – Infrequent Access * – For re-createable infrequently accessed data that needs millisecond access | $0.01 per GB |
| S3 Glacier ** – For long-term backups and archives with retrieval option from 1 minute to 12 hours | $0.004 per GB |
| S3 Glacier Deep Archive ** – For long-term data archiving that is accessed once or twice in a year and can be restored within 12 hours | $0.00099 per GB |
S3 Buckets are clearly cheaper. By default, my 50GB volume backup now costs me only $1.15, that’s 53% LESS than Snapshots. However, you cannot upload a Snapshot to S3. Even though they are hosted on S3, that’s in a separate network administered by AWS. All you can do is upload files. Remember dd? This smells like you need scripts!
How much do S3 Buckets cost?
It’s per GB.
Standard = $0.023 (50GB = $1.15/mo)
Standard _IA = $0.0125 (50GB = 62 cents/mo!)
Glacier = $0.004 (50GB = 20 cents/mo!!)
Glacier Deep = $0.00099 (50GB = 5 cents/mo!!!)
Can you automate them?
Yes. With scripts in bash, using the aws cli commands and a bit of cron.
How far back in time can you go for your $$$?
Since you can automate backup files expiration with scripts, it’s easy to keep the few you need and pay only for those.
What if I want to restore last month backup?
Restore the monthly or the weekly that you have setup. You could also retain 30 snapshots in rotation.
Wait a minute. 30 Snapshots?
Yes, for the price of an S3 Glacier, your 30 snapshots at $12/mo now cost less than a dollar! Access costs are negligible.
How much costs 30 Snapshots?
Standard: 30 * 8GB * $0.023 = $5.52/mo
Glacier: 30 * 8GB * $0.004 = $0.96/mo
Can you backup each week instead?
Why not.
Can you automate the snapshots deletion?
Yes.
What scripting language to use?
Scripts with bash or zsh for loafs like me (rely on the aws cli binaries), and Python for the courageous, or Ruby for the smart. Python and Ruby come with ssl and POST capabilities, which is needed to use the S3 API.
Wrapping up
More on S3 automation and its drawbacks in the next post! The goal being to sleep with your deaf ear up, a bit more efforts and challenge is needed the reach the Graal of the ultimate bargain price possible.
If you prefer convenience and money is not an issue, be my guest and go for hand-managed Snapshots in the console.
nice but where are the scripts?
i’m still undecided on the next move: dd filesystem to S3 or 7zip? maybe a mix of both