{"id":48,"date":"2022-01-05T22:32:54","date_gmt":"2022-01-05T22:32:54","guid":{"rendered":"https:\/\/blog.lottabytes.com\/?p=48"},"modified":"2022-01-05T23:48:46","modified_gmt":"2022-01-05T23:48:46","slug":"reason-n-to-test-backups","status":"publish","type":"post","link":"https:\/\/blog.lottabytes.com\/index.php\/2022\/01\/05\/reason-n-to-test-backups\/","title":{"rendered":"Reason N to Test Backups"},"content":{"rendered":"\n<p>A co-worker of mine has a reputation of saying that backups aren&#8217;t regularly tested aren&#8217;t actual backups.  While this seems a bit harsh, it&#8217;s absolutely correct.  I present to you (from my home environment) yet another reason why this is true&#8230;<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Broken-Backups-1024x87.png\" alt=\"\" class=\"wp-image-49\" width=\"838\" height=\"71\" srcset=\"https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Broken-Backups-1024x87.png 1024w, https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Broken-Backups-300x26.png 300w, https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Broken-Backups-768x65.png 768w, https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Broken-Backups.png 1210w\" sizes=\"auto, (max-width: 838px) 100vw, 838px\" \/><figcaption>File Differences<\/figcaption><\/figure>\n\n\n\n<p>As you can see from the above screenshot I have a file named <em>nc-data-backup-20211221.tar.gz<\/em> that has two different hashes on two different systems.  While this isn&#8217;t terribly surprising, <span style=\"text-decoration: underline;\">what is surprising is that it was successfully copied over from the remote system, validated and <strong>then<\/strong> experienced corruption while being written to disk.  <\/span>This wasn&#8217;t a one-time incident either&#8230; it was consistently repeatable over a dozen times as was trying to find the solution.<\/p>\n\n\n\n<p>Over the course of testing I was able to finally identify the problematic element (not actual root cause) as the Cache Mode on my QEMU\/KVM Disks.  <\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"602\" height=\"234\" src=\"https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Screenshot-from-2022-01-05-15-24-34.png\" alt=\"\" class=\"wp-image-51\" srcset=\"https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Screenshot-from-2022-01-05-15-24-34.png 602w, https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Screenshot-from-2022-01-05-15-24-34-300x117.png 300w\" sizes=\"auto, (max-width: 602px) 100vw, 602px\" \/><\/figure>\n\n\n\n<p>When using the default caching mode (writeback) I am experiencing 100% failure rates on large file transfers.  I haven&#8217;t isolated the actual root cause of this behavior, however, I did test will all the available cache modes as shown below.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"98\" src=\"https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Default-Cache-1024x98.png\" alt=\"\" class=\"wp-image-50\" srcset=\"https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Default-Cache-1024x98.png 1024w, https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Default-Cache-300x29.png 300w, https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Default-Cache-768x74.png 768w, https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Default-Cache-1536x147.png 1536w, https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Default-Cache.png 1901w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Those tests were mostly intended to give me an idea of the performance impact I would see using other caching modes as well as if I would experience continued errors.  Below are the results.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"589\" height=\"121\" src=\"https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Screenshot-from-2022-01-05-15-26-57.png\" alt=\"\" class=\"wp-image-53\" srcset=\"https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Screenshot-from-2022-01-05-15-26-57.png 589w, https:\/\/blog.lottabytes.com\/wp-content\/uploads\/2022\/01\/Screenshot-from-2022-01-05-15-26-57-300x62.png 300w\" sizes=\"auto, (max-width: 589px) 100vw, 589px\" \/><\/figure>\n\n\n\n<p>So, there you have it.  Had I not tested my restore process on a new VM, in the event of an actual failure I would have had a high risk of experiencing total data loss of all my important documents, photos, etc.  Still a far cheaper price to pay than an enterprise failure, but an important lesson none-the-less.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A co-worker of mine has a reputation of saying that backups aren&#8217;t regularly tested aren&#8217;t actual backups. While this seems a bit harsh, it&#8217;s absolutely correct. I present to you (from my home environment) yet another reason why this is true&#8230; As you can see from the above screenshot I have a file named nc-data-backup-20211221.tar.gz &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/blog.lottabytes.com\/index.php\/2022\/01\/05\/reason-n-to-test-backups\/\" class=\"more-link\">Read more<span class=\"screen-reader-text\"> &#8220;Reason N to Test Backups&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[10,11,8,9],"class_list":["post-48","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-backups","tag-disaster-recovery","tag-home","tag-nextcloud"],"_links":{"self":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts\/48","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/comments?post=48"}],"version-history":[{"count":3,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts\/48\/revisions"}],"predecessor-version":[{"id":56,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts\/48\/revisions\/56"}],"wp:attachment":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/media?parent=48"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/categories?post=48"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/tags?post=48"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}