Key Takeaways
- Amazon S3 durability helps protect against infrastructure failures,but additional backup protection is needed for accidental deletion,misconfiguration,compromised credentials,and data corruption.
- Clumio helps protect S3 data with continuously updated,immutable,air-gapped copies that remain available even if the source AWS account is compromised.
- Protection groupsandpolicies let you define which S3 bucketsandobjects are protected,along with backup frequency,retention,storage tier,and recovery options.
- Clumio Backtrack helps provide point-in-time rollback for versioned S3 buckets,while SecureVault supports object-,prefix-,and bucket-level recovery from immutable backups.
- Testing restores before an incident helps teams become familiar with the recovery processandrespond more quickly when data is lostorcorrupted.
Amazon S3 is designed for 99.999999999% (11 nines) of data durability. Meaning that S3 objects should remain recoverable even if hardwareorinfrastructure components fail.
However,durability will only take you so far.
It doesn’t help if a file is accidentally deleted,a lifecycle rule is misconfigured,an identityandaccess management (IAM) credential is compromised,or someone runs a batch delete against the wrong S3 prefix.
In this post,I’ll walk through how Clumio protects Amazon S3 data and,more specifically,how to recover deletedorcorrupted files after an incident. I’ll cover setting up a protection groupandbackup policy,as well as findingandrestoring specific objects using the backup calendar,and performing point-in-time recovery using Clumio Backtrack.
How Clumio Protects S3 Data
Clumio is a SaaS-based,agentless backup service for AWS. For S3 specifically,rather than taking periodic snapshots of a bucket,it instead reads change events (object creations,updates,and deletions)andkeeps a continuously updated,air-gapped,immutable copy of your data. This is what makes it possible to recover cleanly even if the source account itself has been compromised.
Everything Starts with a Protection Group
A protection group is a named collection of oneormore S3 buckets. Buckets can be grouped based on tags,prefixes,storage classes,or manually selected. A policy is then attached to the protection group,and it’s the policy that defines the backup mechanisms that will apply to the group.
Creating a Protection Group
1) To create a protection group,clickProtect,Protection groups,S3 protection groups.
Creating an S3 protection group
2) Then clickCreate protection group,and give it a name that reflects the workload,like prod-customer-data. Select Automated protection to add buckets by account,region,or tag. This will apply to future buckets that meet the selection criteria as well as what exists today. After that,click Next.
Configuring automated protection
There’s also the option to select buckets manually,and after completing the selection,click Next.
Applying a Policy
TheDefault Policy performs monthly in-region backups with one-month retention,using SecureVault Standard,which creates a secondary,air-gapped copy of your objects. Restores typically complete in minutesandsupport instant access.
3) To create your own policy,clickCreate new policy,name the policy,and select the S3 asset type.
Creating a policy
Configuring Backup FrequencyandRetention Settings
4) Click the checkbox to enableSecureVault,and you have the option to configure the backup frequency in terms of the number of days between backups,or select continuous backups,to back up your data at frequent intervals throughout the day. This enables point-in-time recovery for any point in time during the day.
5) ChooseContinuous backup, and set the retention for one month.
6) Click the checkbox to enableBacktrack,for point-in-time rollbacks. Note that S3 buckets must be configured with versioning enabled in order for rollbacks to work.
7) UnderAdditional options,set the backup window to an appropriate time.
8)Backup destinationis set to in-region by default. Note that transfer charges will apply if you store backups in a different AWS region.
9) UnderSecureVault backup tier,select SecureVault Standardfor instant restores.SecureVault Archive is designed for data that will be retained for a long period,and should only be used when you can tolerate a 24- to 48-hour restore time.
10) Click the checkbox to enableThreat Scan,which is a service that scans every backup execution to help flag compromised filesandidentify safe,uncompromised restore points to prevent environment reinfection. After that,click Add,and Save.
Configuring the policy
11) Check that the new policy is selected for your protection group,then click Next.
12) UnderAdvanced options,you can choose which S3 storage classes to protect,whether you want all versions of an objectorthe latest version,includeorexclude certain prefixes,or include objects based on their creation date. Then click NextandCreate.
Advanced options
After that the new protection group will appear,and the first backup will be performed during the configured backup window.
Protection groups view
Verifying Your Backups
1) When you need to perform a restore,due to data being missingoridentified as corrupt,navigate to Protection groups,select the relevant S3 protection group to view the available backupsandClumio Backtrack Syncs. A blue dot in the calendar indicates that a successful backup exists for that day.
Viewing successful backups
2) View theBacktrack Syncson a per-bucket basis by selecting theBacktrack tab from inside the protection group,and verify that continuous backup has been functioning correctly.
Backtrack Syncs available for point-in-time restore
Restoring Data
In this example,I have some missing call transcripts. Each missing file has the prefix of transcript. I also have a file that is corrupted,named customers.csv. Luckily,I have a full backup as well as multiple point-in-timeBacktrack Syncsto restore from.
S3 shows only a single call transcript; the rest have been deleted accidentally.
1) From theRestore menu,select S3 buckets,and select the bucket that has missing files; then from the Backtrack dropdown,select Roll back.
Using Backtrack to roll back to a point-in-time
2) Select the bucket to roll back,choose the dateandtime,and then filter based on object key,sizeorS3 storage class. After that,click Previewto view the files available to roll back.
Selecting files to roll back to a point in time
3) Choose to roll back to the same source bucket,or specify a different bucket to roll back to. Then,click Copyto begin the rollback job.
4) View the progress of the rollback in theTasksmenu.
Viewing rollback progress
5)Next,to restore the corrupted customers.csv file,I’ll click on the previous night’s backup in the backup calendar,then select Restore. There’s a choice to instantly access a read-only copy,restore the objectorrestore multiple files based on bucket nameorprefix. 
Restoring from backup
6) SelectRestore object,then searchandselect the object to restore,and click Next.
Selecting a single object to restore
7) Select the version to restore,and clickNext.
Selecting a version to restore
8) Choose which AWS account,region,and bucket to restore to,and then begin the restore.
Set the destination before beginning the restore.
9) After the restore shows as successfully completed,the restored file should be available in S3.
Theobject is restored.
My Recommendation: Don’t Wait for an Incident
With a protection groupandpolicy in place,Clumio helps provide more than one way back from an S3 incident: Backtrack for near-instant,in-place,point-in-time rollback on versioned buckets,and SecureVault for restoring data from an immutable,air-gapped backup.
Both help support recovery at object,prefix,and bucket granularity,allowing you to choose the recovery approach that best fits the incident. And in both cases,recovery comes down to a few clicks rather than having to manually review multiple S3 versions.
Finally,the best time to test this is before you actually need it. Set up a protection group on your critical S3 buckets today,and run practice restores until they become second nature. Thenext time something disappears,you’ll already know exactly how to get it back.
To get started today,request a demoortwo-week free trial.
FAQs
Q: Why do Amazon S3 buckets need backup if S3 provides 11 nines of durability?A: S3 durability is designed to protect data against hardwareandinfrastructure failures,but it does not prevent logical data loss. Accidental deletions,misconfigured lifecycle rules,compromised IAM credentials,and unintended batch operations can still result in lostorcorrupted data.
Q: How does Clumio protect Amazon S3 data?A: Clumio uses S3 change events to help maintain a continuously updated,immutable,air-gapped copy of protected data. This approach helps provide an independent recovery path even when the source AWS accountordata has been compromised.
Q: What is an S3 protection group in Clumio?A: A protection group is a named collection of oneormore S3 buckets that share a protection policy. Buckets can be selected manuallyorautomatically based on criteria such as AWS account,region,tags,prefixes,or storage classes.
Q: What is the difference between Clumio BacktrackandSecureVault?A: Backtrack helps provide point-in-time rollback for S3 buckets with versioning enabled,making it useful for quickly reversing unwanted changes. SecureVault helps maintain an immutable,air-gapped backup that can be used to restore individual objects,prefixes,or larger sets of data.
Q: What is the difference between SecureVault StandardandSecureVault Archive?A: SecureVault Standard is designed for faster recoveryandhelps support instant access to protected data. SecureVault Archive is intended for longer-term retentionandis better suited to data for which a 24- to 48-hour restore time is acceptable.
Q: How can teams prepare for an S3 data recovery incident?A: Configure protection groupsandappropriate backup policies for critical S3 buckets before an incident occurs. Regularly test object restoresandpoint-in-time rollbacks so the recovery process is familiar when production data needs to be recovered.
Faye Ellis is Principal Training Architect – AWS at Pluralsight.
