Archive for the 'Storage' category

When you look into the largest images accessible from the Internet, you MUST check Google maps and Google Earth (the application that allows you to navigate on the surface of the Earth globe from your desktop with the satellite images collected by Digital Globe and Google).

From its inception, Google Earth provided images of the Earth globe at a resolution of 15m/pixel (each pixel is a square of 15 meter by 15 meter). But from 23rd of March 2006, 20% of the emerged land mass (about 1/3 of the world population) is now covered by exclusive photos in the amazing résolution of 70cm/p (sometimes even down to 10cm/p, enough to recognize the plane model on an airport and to distinguish people in a house garden).

If we only keep the basic resolution, the Google Earth image amounts already to 2,111 giga-pixels. Impressive, but this is not all. If we include the high-resolution zones (let’s forget about the isolated 10cm/p zones), we must conclude that the Google Earth image is:

• 20% of land mass at 0.7m/p: 56,616 giga-pixels
• All the rest at 15m/p: 1,988 giga-pixels

A stagegring total of 58,604 giga-pixels! 58 T-pixel! We’re just switching to a different scale.

It will be difficult to make it bigger but Google is already working at it since the systematic movie to 0.7m/p for continental masses would lead to about 5 times this volume, and surfaces covered at 0.1m/p will simply blow the total up when it is no longer applied to isolated test examples (like the current example in part of Las Vegas city).

Let’s be clear and simple, the main limit will soon be (for Google): How to keep such a big amount of data accessible knowing that there is a constraint on access times (tens of thousands of simultaneous accesses from all over the world cannot be served by a mere big disk drive on the corner of a desk and keeping it up-to-date is not only a matter of cut’n paste anymore).

One of the most important and still one of the most poorly applied tasks of computer calysthenics in the world is data backup. Usually, it’s only when you discover that you lost your important data that you remember that you should have been backing things up. A little too late…

Requirement

What are the qualities of a good backup?

• It’s done regularly
• It contains useful data (and only them)
• You can easily perform a restore operation

If you look at these qualities, it is easy to see why so often we do not have the right backup sets. Problems abund:

• I forgot to run the backup
• I did not have enough time, so I backed up only part of what was needed
• I can no longer read the backup set or I can’t find the restoration tool

The Roumazeilles solution

Advantages

We can’t over-stress the importance of being attentive to these problems. But it is sometimes difficult to find a good solution. I could explain how to backup with expensive and complex software tools (or even hardware solutions), but I decided to give you a tip about how to do fairly good PC backups, as I do it right now. My method has plenty of advantages:

• Free
• Works with all Windows versions
• Uses no complicated software (you don’t need to click on the advertisment messages appearing around this page)
• Works automatically, even if I forgot or I’m hard pressed by some kind of other emergency
• Backups only the important data, but all of them
• Restore uses NO special tool and needs even less competence (the most sensitive operation is the safest and simplest)

What are you waiting for?

Your answer: “Please, tell us what is this miracle solution?”

(more…)

In addition to the recent articles posted here about testing a software RAID solution on OpenSuse, here are a few complementary elements and recommended useful links:

• The Software-RAID HOWTO
• Software RAID in the new Linux 2.4 kernel, Part 1
• Software RAID in the new Linux 2.4 kernel, Part 2

Worth reading!

In the midst of the posts I made about the reliability of optical media, I want to stop a minute to look at the reliability of magnetic hard disc drives. The recent publication of two interesting studies on this subject create a kind of immediate actuality that is useful to comment upon.

The first publication is a paper by Google research lab titled “Failure Trends in a Large Disk Drive Population” that is quite interesting because it relies upon the analysis of more than 100,000 drives installed all over the Google corporation. They used a lot of data collected mostly through SMART and draw several interesting or counter-intuitive conclusions like:

• High temperatures and heavy use patterns do not correlate positively with failure rates (on the contrary, if the drive is hotter, it seems to be slightly more reliable).
• Only four SMART attributes (scan errors, reallocation counts, offline reallocations and probational counts) relate to the reliability of the drive but they are not able to predict the individual failures (56% of the drives failed without such a warning).

Another report from Carnegie Mellon University titled “Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you?” -presented at the 5th Usenix Conference in San Jose, CA- points at the fact that the announced (or promised) reliability given as MTTF figures by the HD manufacturers are grossly overestimated. Bianca Schroeder and Garth A. Gibson insist on the fact that they -too- collected data from more than 100,000 hard disc drives and they remarked that the replacement rate predicted by the usual 1,000,000 to 1,500,000 hours MTTF of the manufacturers should translate into replacement rates of less than 0.88% per year. However, they observed in the field that it is usually more than 1%, that 2-4% is common and maximum runs at 13%.

All in all, it seems that magnetic media is still quite reliable but the figures given by the manufacturer are grossly overestimated. Would it be one more reason to think about installing RAID-based storage solutions?

In the context of the interest I have in evaluating the reliability of optical discs (more specifically CD-R and DVD-R whose use for archival purposes fully justify special attention), I found an interesting document published by the Electronic Library team of the French National Library (BNF).

The BNF is the recipient of the legal deposit in France and, in this context, archives all the Audio-CD (CD-DA) phonographic production since 1983. Alain Carou and Thi Phuong Nguyen summarized the observations made in this context, with two decennies of history. Overall, the resulting opinion is quite re-assuring since the CD-DA comply with their specification and promises (the most conservative estimations gave 20 years of shelf life for pressed optical CDs).

Furthermore, the published report (titled ” Contribution à l’étude du vieillissement des CD – Recherches sur les collections de la Bibliothèque nationale
de France ” – French for Contribution to the studies about CD ageing – Research about the collections at the French National Library) finally contains many interesting observations and informations inside its 21 pages. In order to entice you, here are some of the elements I noticed:

• A clear explanation about the measurable parameters of CD quality (BLER, BERL, C1/C2, jitter, etc.), in French.
• Detailled comments about the problems coming from silver+nitrocellulosic varnish discs (PDO UK discs known as “bronzed” or Optimes dics), where the varnish did not play its normal protecting role and let the silver layer corrode quickly (more quickly than in the case of aluminium layers).
• An analysis of “misted” discs where the polycarbonate substrate degraded much quicker than expected (bad chemical stabilization?).

A must read.

Source: http://bibnum.bnf.fr/conservation/supports.html

RAID storage is using several hard disc drives together to improve the reliability (sometimes the performance) over what can be done with one single disc drive. The recent posts here:

• Creating a RAID-1 on OpenSUSE
• Testing RAID-1 on OpenSUSE
• Software RAID: a question of light

For me, the CompactDrive PD70X is still a marvellous machine when you want to copy your flash memory cards onto portable hard disc storage. It still owns the speed record and it behaves like a camel when it comes to electric energy.

Today, I learned a few interesting informations that I want to share with the other owners:

• The HyperDrive HD80 (best distributed in the USA) seems to use the same electronics core and share the same firmwares
• Some users have had difficulties with 2+GB cards
• A new firmware version (2.2) is available from the German site (Resolves compatibility issues with certain Lexar CF cards; Fixes bug in usage of fragmented non-contiguous hard disk space (requires reformatting of hard drive); It is still unclear if this solves all the 2GB/4GB card problems)
• The new firmware increases the battery charge time (to better compatibility with most recent large capacity batteries)
• The PD70X is now officially considered by its manufacturer as a product having reached end of life and will no longer receive new firmware upgrades (but people will certainly keep checking the HyperDrive firmwares since up to now it was a successful bet)

A good product reaching end-of-life. Many advices from people wishing to replace their fastest Portable Hard Drive Storage. I remarked (but I cannot give my own opinion here):

• HyperDrive HD80 (internally and externally similar to PD70X; probably condemned in the short term, too)
• HyperDrive SPACE (well received by the press; very positive article in March 2007 Chasseur d’Images)
• NeXto CF Ultra

Thanks Jérôme.

Today, I am asking a simple question to solve a little problem I have :

Since I installed a RAID array in an OpenSUSE machine, I can observe that the little red LED light of disk access flashes once every seoncd -even when I do not try to use the array. Would somebody be able to explain me why the machine finds it necessary to regularly check at least one disk so often?

We, geeks, are always on the look for a designer box for our PC in the obvious objective to seaprate ourselves from the ugly standard grey box. Our desire has now also extended to the boxes protecting our external hard drives. After a generation of grey, metal or black boxes, we start seeing nice packages. But GeekStuff4U.com offers a GW2.5HD-U2 box that totally reverses the trend: aimed at containing a 2.5″ hard drive USB-connected, it look s exactly like a bare 3.5″ disk.

Moreover, at 19€ only, it’s a bargain.

Minor event here: With today’s purchase of a Seagate 400 GB SATA-2 hard drive (in an external USB-2/eSATA enclosure), the accumulated storage went over the 1 Tera-Byte limit (1024 GB). And I can still envision myself seriously thinking about buying my first hard disc – it was a 500 kB external disc for an Apple ][e…

As I described in another post, I wanted to test RAID technology on my GNU/Linux OpenSUSE configuration. In the previous post, I have shown how to create/install the RAID array.

Now my intent is to see what happens when a hard drive fails, as it is supposed to protect me against it. But I prefer to test rather than believe the hype. Just after that I will need to observe what happens when we plug a new drive to replace the failed one. And I will suppose that once again it’s a different drive (after a few years of operation and a failure, there is no reasonable chance of finding an identical drive).

(more…)

I have always be willing to test-run RAID technology for the storage of the files on my personal LAN. This technology has the advantage of being far more secure than the usual hard disc storage. By using more than one disc drive, RAID allows to gain additional security. In short, if a disc drive fails (it happens!), the RAID array (that’s its name) will keep working with no data loss [more about it on the wikipedia encyclopedia].

However, using such a technology was not for everybody up to now. You had to be a computer guy to know about it, you had to be a real computer guru to use it. It all changed with two things:

• Some RAID appliances start appearing on the market. You buy them and you get it.
• GNU/Linux now includes the needed tools to allow building of a RAID array if you are willing to connect enough drives to your GNU/Linux box.

I wanted to test the later. So here is a summary of my experiences with RAID technologies just using a couple of hard drives (two drives in RAID-1 redundant mode) and OpenSUSE 10 that I had recently installed on this machine.

(more…)

The questions

Many people are using optical media for their archival needs. They are very affordable and the manufacturers claim a storage life of more than ten years, often a century. However, when such claims are made people are always interested in ensuring that the figures are right and provable (archive management cannot accept to discover in a few years that investments made for a life long storage were not up to the task).

Today, a lot of users comment about the optical discs that cannot be read either just after writing or after a few months or years (much less than anticipated). So, it is a very legitimate question to ask what really is the reliability of the CD-R or DVD-R media and if there are issues when choosing these in the perspective of very long storage. Additionaly, you should be concerned with the requirements for a good storage.

(more…)

Ars Technica (one of my prefered web sites about PC computer technology) just published its perfect PC recommendation article. Obviously, perfection changes according to the price you are ready to pay for it. So, there are three different levels of PC perfection and each is described down to the finest details (what motherboard, what CPU, what hard disc drive, what graphic card, but also what mouse or what network card).

I am specially looking to the one titled “budget box” (whose cost must stay down to “very affordable”) and its possible equipment with either an AMD Athlon 64 x2 3800+ dual-core processor or an Athlon 64 3200+ single core CPU.

As I still advise to buy a CompactDrive PD70X to empty your Flash memory cards, I had to inform you about a weird little series of incidents I have had these recent days with my own device.

To my knowledge, the Compact Drive PD70x is only available from MacWay (in France) at a reasonable cost of 259€. Right now, there is an offer for an added free 12V car battery charger that will be welcome for all interested into photo safari or any other outdoor photo activity forcing you to stay far from a 2″0V plug because you are in an SUV or a car.