Recovering from self inflicted data corruption – a summary

Of late I have been torturing myself about the question of - even if I build on top of a highly reliable storage service like Windows Azure Table Service do I still need to worry about backups, versioning, journals and such? The answer would seem to be, yes, I do. Mostly because even if the table store works perfectly, I’m still going to have bugs I introduced that are going to hork my data.

In fact what I specifically need to do is:

1. Lobby the Windows Azure Table Storage team to add undelete for tables so if I accidentally blow away one of my tables I have some hope (oh and ACL’s would be nice too)
2. Be very careful about how I update my schemas
3. Implement a command journal (and be clear about their limitations)
4. If time permits implement tombstoning
5. If I’m feeling really wacko implement my own versioning system on top of the table store (or just backups if I’m feeling only slightly wacko)
6. Put into place a realistic plan to take advantage of all these features while keeping in mind the limitations of these techniques.

The links in the previous text are to the other articles in this series that I wrote for my blog. Those articles are:

• Do I need to backup/journal my Windows Azure Table Store?
• The Limits of Command Journals
• Techniques to Ease Recovering from Self Inflicted Data Corruption
• Thoughts on implementing a command journal
• Tombstoning on top of Windows Azure Table Store
• The limits of recovering from application logic failures
• Implementing Versioning in Windows Azure Table Store