The observation

CD-ROM are annonced as being able to live a century-long life. CD-R are supposed to be nearly as sturdy. However, it seems that experience is not fully supporting these claims.

While, in my collection of 600 albums, I have many audio CD which were bought in the 80’s and that spent part of their time in the gloves compartment of a -sometimes- overheated small car, none ever failed me. 20 years already and no random failure. I’d guess a century is a reasonable target.

On the other hand, my experience with CD-R tells me that they should not be used for storage. And I mean “any kind of storage”.

In the last four years I burnt around 500 silver discs for various archival purposes. I have been using several CD writers from different brands (Toshiba, Teac, etc.) and various interface technologies (IDE, SCSI, etc.) The empty discs where coming from all kinds of brands. So, I’d say I have a fairly good statistical idea of this kind of market and it is depressing: CD-R media is not worth it.

Around 20% of all discs are unreadable just a few months (maybe a year) after writing.

It would be re-assuring – in a sense – if I could say that some brands are better than others. Actually, I can’t. No-name dirt-cheap blank media is just what you should expect: Nothing survives after a year.

The origin

Even if it is difficult to know for sure, I have been trying to identify the most plausible source of this fast decay. The technical literature usually points to two dangers.

The first one is heat. High temperatures tends to deteriorate the sensitive surface. So, you are advised to keep your discs in a temperate room environment (for really long term, some corporations go to the extremes of near freezing lows in controlled humidity).

The other enemy of your data is light. This, I could easily confirm with some discs I forgot on the corner of an old stack of paper. Direct light will transform the sensitive surface to the point that the shadow of objects is visible. Bad, bad, bad! So avoid light at all costs. It may be as easy as storing discs in closed cupboards instead of an open shelf (this was my own mistake). Don’t be fooled by the plastic cases: They reduce the amount of light (and UV light) received by the CD, but they are not enough.

Software solutions?

When things went so wrong that you are starting to get errors from the discs you read, what should you do? I am unsure. There are many products on the market that claim that they can read the data from your failing media. Possible candidates are:

• CDcheck
• CD/DVD Diagnostic

Unfortunately my experience seems to prove that when the disc is still without errors, these products are (obviously) useless; Later, I never suceeded in recovering lost data. It seems that the best ones repetitively read the same sector applying statistic laws to discard errors and find the right data. Unfortunately, this is not very different from what all modern CD drives do silently when they discover a first error. So, you are not informed early of the failures of the media. When you hear about them (when an error is reported), it means that the siuation is already out of control. It means that tools like CDcheck should be used preventively. If reading a disc is still feasible but request some retries, it is time to restart from a fresh copy and dump the old failing disc . If you are willing to maintain this kind of long term commitment…

The real solution (my solution)

My preference goes to something else. I decided that the combination of rapidly failing optical media and ugly prophylactic procedures was too much for me.

Currently, I am commited to purely magnetic storage: Hard disc drives are not much more expensive than CD-R and they have an excellent longevity (this is even better if you allow the disc to sleep when unused for long periods of time): More than 100,000 hours between failures is not uncommon and this is more than 11 years of permanent use.

Even better

Despite this sturdiness, hard discs are not eternal. A power spike can be fatal, electronic components are prone to rare but random failures, plain bad luck is a fact of life.

In my eyes, RAID technology is the way to go. Already, a number of recent motherboards include either RAID-0 or RAID-5 technology that allow to use several disc at the same time to improve reliability at the cost of some lost storage space. I’ll provide more about this in later writings.

I had explored several on-line options like:

1. Amazon Cloud Drive, which is now quite expensive when you send it several tera-bytes.
2. OVH Cloud Archive, that I failed to configure (even with the help of the OVH excellent support team); I had to look at OVH Cloud which is actually priced very similarly to ACD (the price structure is a bit complex, though probably a bit cheaper in the long run).
3. Hubic, which decided to close its doors to new customers.

So, the only solution left was elsewhere: Archiving to an external USB hard drive, but alternating between two physical disks to allow always keeping one off-site (far from fire and flood).

The configuration is not very complicated but requires a bit of attention and of patience.

Pre-requisites  on Synology DSM

The main advantage of Synology is their comprehensive interface very consistent from one NAS model to another (DSM). So, this will essentially apply to all Synology configurations, large or small.

The general idea is to backup one directory from the Synology server to the USB3 external drive. You need the “Hyper Backup” package installed. It is perfectly apt to backups toward a variety of options (I previously used it with Amazon Drive (now too expensive to my taste) and with the now-dead hubiC).

A simple recommendation: Don’t apply versionning on the NAS itself (it tends to eat a ton of space up and appears difficult/impossible reconfigure later on). On the other hand, you can ask that from Hyper Backup which does it perfectly well. Eat space on low cost external storage rather than on high-cost primary storage.

Configuration

Format the hard disk drive(s)

I have chosen 8 tera-byte disks from Seagate. At first, I was looking at the unusual (but cheap) Seagate Archive 8TB. but they have now been replaced with Seagate Barracuda 8TB 5400rpm (which have about the same read speed but a much better write speed, for a similar cost).

Let’s start with connecting the hard drive (in its USB3 enclosure which has been selected as UASP-capable for best performance) to the Synology. It will be immediately recognized and mounted by DSM as USB Disk 1 (or USB Disk 2, depending on the USB port you used). It is highly recommended to format it with a single EXT4 partition.

In the DSM Configuration Panel, under the “External Devices” category, you will select the USB Disk 1 and click on the “Format” button. then, chose the default options.

Progress will be slow (half an hour is usual) but with no special issue to expect.

Name the hard disk drive

This is not very critical, but much more comfortable to name the disk (Data1 and Data2, if you want; For me, it will be Sciurus vulgaris 1 and 2).

Just after formatting, in the DSM Configuration Panel, under “Shared Folder”, select the usb_share_1 directory (the number may vary), then click on the “Edit” button to access a few options including disk naming.

I recommend (your choice) some options:

1. Hide the directory in “Network locations” and “Hide sub-directories” to limit the attack surface of a potential intruder.
2. Encryption: It may come later.
3. Access control: Limit it to the only user which is running Hyper Backup (Nobody else is supposed to directly access your archive).

Configuration of Synology HyperBackup

As I decided to use HyperBackup, everything will be quite simple from this point. In HyperBackup, click on  +, then “Data backup task”.

You will need to designate a backup on an external drive with “Local folder & USB” in the dialog window shown here.

You will then need to describe your backup(s) from several dialogs that are more or less explicit. First, you must point to the destination (the external USB disk):

Then, you will be invited to point to the directory (possibly several of them) that will be backed up. Then,  HyperBackup may offer to backup the parameters from some of the applications you installed on your Synology NAS (I chose not to do this, but it’s really your choice), before arriving to the main settings dialog:

The most critical is certainly client-side encryption (at the bottom of the dialog) which will allow password-protecting the backup data set. As we are intending to leave the external USB disks on a distant shelf, I strongly recommend to protect it against prying eyes. But, be sure not to forget the password, of course…

Then, you will have the opportunity to set the versioning parameters (to be able to travel back in time in your backups to recover one of the successive version backups of a given file). I chose the SmartRecycle parameter with 64 versions (nine months of protection if I apply a daily backup). But you may want to experiment a little in order to make your own decision.

Don’t forger to do this twice: One task per hard disk drive will allow HyperBackup to keep backing up when disks move out or come back from cold storage, depending on which is stored away (each month, maybe).

Finally…

Never forget that a backup only works if you can use its restore feature. It is best to spend a few hours every 6 months to restore a few files and check that everything is still working.

Of course, the best is to test the whole installation in the first hours after setting it up (at least, if you want to write down the restore procedure; You may need it some time in the future…)

I just doubled the memory size made available to the CrashPlan virtual machine, thanks to these indications. The application immediately restarted OK. Instead of exploding after a few seconds, the 7000 file delayed in their backup (thanks to returning from Costa Rica) started to be backed up again by CrashPlan.

Seriously, we would prefer to see Code42 (CrashPlan editor) adding a small code snippet detecting this kind of situation described in the web page and applying by itself the changes without requiring dancing to a tune that is utterly inaudible for most of the users…

There is still to understand the restoration issues on Marion’s PC (it’s much more important to be able to restore than to be able to backup).

My recent experience left me with a few dauting questions that will need to receive quick answer if I want to keep my faith in CrahsPlan. Since I have near me three different computers on three different accounts, backing up on a daily basis, I witnessed two annoying incidents nearly simultaneously.

First, my main desktop computer (which holds all my pictures/photos) no longer can backup anything because the CrashPlan engine keeps crashing (no pun intended) after only a few minutes. Seevral days, no idea why despite multiple reboots. Is the fact that I recently reached 1TB of data (a lot of photos, admittedly) a kind of explanation? Did I hit a limitation? a bug of CrashPlan?

Next, on Marion’s laptop computer, we experienced a dreadful moment (violent crash leading to full Windows re-install without any warning). So, the natural reaction was to say “Merde !” and go quietly to the CrashPlan restore option, to get it all back. But, where restoring a few files had always been pretty easy (you should always test the restore function of your backup solution), an intensive restore is proving very similar to a nightmare. I believe that we will have every back soon, but it’s slow not only because of connection or server speed but because the server keeps disconnecting us and we had to restore bit by bit, very small group of file, by very small directory. Anything big or long (videos?) is nearly impossible to get before the next disconnect every 30 minutes or so). Ugly and painful as hell.

I tried many things. I think that we will be able to restore everything (except Marion’s photo archive – that is already on a local back, thanks to our multi-tier backup strategy). But it is uselessly painful and slow (several hours of tiresome manual work during a full week for less than 10GB; This is too much for a subscription-based solution).

I’ll keep you posted about the way my opinion evolves here.