Securely erase hard drives

In a previous post I wrote about disk wiping tools. If you haven’t read that article, I suggest reading it before reading this one. Also, this article is written with the assumption that the reader has a fair amount of technical knowledge.

The reasons for wanting to wipe a hard disk drive are many:

  • You might expect a visit by NSA-SCS.
  • You might suspect a rootkit infestation.
  • You might wish to sell your old computer or throw it away and you don’t want your anyone else to get a hold of your private data 1).
  • Same as above, only for companies, hospitals, law firms or anyone else with legal obligation to prevent the spreading of sensitive data.
  • You’re simply paranoid.

Whatever your reasons are, you need to make sure that the data that used to be on your hard drive are gone, hence the need for a disk wiping tool.

But as mentioned in my previous post, wiping tools – that is; block erase wiping tools (BEWTs) – have certain limitations:

  • Hidden data areas (HPA/DCO) might not be wiped, possibly leaving rootkits in place (although probably non-functional).
  • Blocks marked as bad by the hard drive itself are not wiped (blocks marked as bad by the operating system only will be wiped). This information is possible to recover using exotic forensic techniques 2).

In addition to these two previously mentioned limitations, the following apply:

  • When data is overwritten (block erased), the old data on the hard drives might leave magnetic information on off-track areas of the hard drive. BEWTs have no way of erasing this off-track information. This information is theoretically possible to recover using exotic forensict techniques. 3)
  • On modern, high-capacity drives, multiple overwrites are no more effective than a single overwrite. 4)
  • BEWTs are suseptible to malware attacks. 5)
  • Using BEWTs takes time. Following the old DoD 5220.22-M directive of 3 consecutive wipes might take as much as 24 hours on a 250GB disk. BEWTs can exceed the DoD standard and wipe a drive as many as 35 times, leaving the computer used for wiping non-operational for weeks.
  • When you’ve run a BEWT, you have little posibility of verifying that a complete wipe has taken place; that all user accessible areas has been wiped. You’ll have to rely on the information that the BEWT gives you.

Enter Security Erase
Lately there’s been some hype about the relatively new ATA command addition called Secure Erase (part of the ATA Security Feature Set), from now on refered to as “ATA-SE”. ATA-SE is an ATA command (SECURITY ERASE UNIT) built into hard drive firmware that, if executed, orders the hard disk drive to wipe itself. Using software similar to BEWTs (boot disk with an ATA command program) you can trigger this built-in function, wiping your hard drive. Alternatively, you can use the same program to simply lock the hard drive rendering it useless, so that only a wipe (or providing the correct password) can unlock the drive to make it usable again 6). If you have a disk drive produced 2001 or later (with a capacity of 15GB or higher) there’s a 99% chance that your hard drive implements ATA-SE.

But what’s the hype about?

This article by ZDNet writer Robin Harris, for example, incorrectly portrays ATA-SE as some magical solution to disk security. Mr. Harris is not really at fault, though, since his article is based entirely on this paper [PDF Warning] called “Tutorial on Disk Drive Data Sanitization“. In the paper, Dr Gordon Hughes7) at Center for Magnetic Recording Research at University of California, San Diego and Tom Coughlin7) of Coughlin Associates lists the various faults of block erase methodology in a such way that poor mr Harris is led to believe that ATA-SE solves all these problems. This is unfortunately not the entire truth. Let’s review!

  • In the paper, the reader is led to believe that wiping the HPA/DCO is inherently difficult using BEWTs and inherently simple using ATA-SE. This is not the case. Wiping HPA/DCO is a tool implementation problem, not a wipe methodology problem. Some BEWTs (such as Blancco) implement HPA/DCO wiping by default, other tools could allow the user to chose whether or not to wipe HPA/DCO while other tools are not able to wipe HPA/DCO at all. The same principle would apply to ATA-SE tools. The ATA-SE tool “HDDErase” allows the user to chose whether or not to wipe HPA/DCO.ATA-SE is not superior to Block Erase on this point.
  • In the paper, the reader is led to believe that ATA-SE can wipe the off-track areas of the HDD platters while BEWTs cannot. The actual phrasing in the paper is:”It is difficult for external software to reliably sanitize user data stored on a hard disk drive. [...] Off-track overwrites could be effective in some drives, but there is no such drive external command for a software utility to move heads offtrack.“While it is certainly true that BEWTs cannot write to off-track areas, ATA-SE cannot overwrite off-track areas either. ATA-SE does a single on-track erasure of the data on the disk drive.ATA-SE is not superior to Block Erase on this point.
  • In the paper, the reader is led to believe that BEWTs are inherently more suseptive to malware attacks. This is, to the best of my knowledge, simply not true. If you want to read my view on this point, see footnote number 5.As far as I know, ATA-SE is not superior to Block Erase on this point.
  • In the paper, the reader is led to believe that ATA-SE is significantly faster than Block Erase. While speed is not a security issue per se, I agree that the more time-consuming a wipe is, the less likely will it be that users will wipe their disks, so I’ll adress this issue as well. On this one, the good doctor is comparing apples and oranges. While he claims that a single overwrite is not less secure than multiple overwrites he still compares a single ATA-SE overwrite to multiple Block Erase overwrites. Still, a single ATA-SE wipe will be faster than a single BEWT wipe.ATA-SE is only slightly superior to Block Erase on this point.
  • In the paper, the reader is led to believe that blocks marked by the hard drive itself as bad are not wiped wiped by BEWTs while they ARE wiped by ATA-SE. This claim is true. Blocks marked as bad by the hard drive itself (g-list blocks) are wiped with ATA-SE. This is impossible (or very difficult) to achieve using BEWTs.On this point, ATA-SE is superior to Block Erase.

Not emphasized in the paper (to my surprise) is the fact that ATA-SE provides much more reliable feedback regarding the wiping of the disk. If the hard drive successfully wiped the drive it will say so. If anything went wrong; not only will you know that it went wrong, but the drive cannot be used until a successful wipe is completed. This provides the user with a whole new level of feedback. The malware threat is not eliminated, though – read footnote 5.

Summary:
ATA-SE methodology is superior to Block Erase methodology due to ATA-SE’s ability to overwrite bad blocks, due to it’s reduced time demand and due to it’s improved completion feedback.

When should you use ATA-SE and when should you use block erase? If you’re in a hurry, is a “hard disk lock” sufficient?

Let’s start with the disk lock question. Disk lock as a means of security should only be used for strictly non-sensitive data! Locking the drive does not erase any data and the disk lock can easily be circumvented by any of the thousands of disk recovery companies around the world 6). While it is more secure than just “deleting the files” or formatting the drive, it is certainly not sufficiently secure for any degree of sensitive data.

How about ATA-SE wipes vs BEWTs? Well, for SCSI drives you don’t have much choice but to use BEWTs. Even though the SECURE ERASE UNIT can be implemented in SCSI disks as well as ATA disks, no SCSI disks have – to the best of my knowledge – implemented this function yet. When it comes to ATA disks, however, I believe BEWTs pretty much have played out their role. ATA-SE will take over the role of disk wiping (especially when the next version of MS Windows implements direct access to ATA-SE along with the format command, as my magic 8-ball said it will). ATA-SE is just as good as BEWTs – or better – for home users, hospitals, law firms and the like. Even government agencies should use ATA-SE, provided that the data that needs to be wiped is of a low-sensitive nature (yeah, now I’ve given the recommendation, not just some obscure government institute 8)).

However: remember that data CAN (probably) be recovered from a drive wiped by ATA-SE. Granted, the level of expertise and equipent is high and time committed is huge – but it CAN (probably) be done. With high probability the NSA have this capability, as well as intelligence agencies in other countries, such as China, Russia, Israel, France, UK, Germany, India and others. If you have information that you need any of these agencies NOT to see, then ATA-SE is not sufficiently secure. In addition to wiping the drive with ATA-SE you will have to use some physical destruction method 9).

Of course, if you’re anything like me you’ll use both ATA-SE and Block Erase for your sensitive data, then yanking the platters from the drive, grinding the surfaces and bending the platters. Then melting the platters in a burning furnace and taking a boat to somewhere in the middle of the pacific ocean and dropping the furnace overboard. Then having NASA hurling the Pacific Ocean into the sun. Then making God destroy the universe.

It all depends on how paranoid you are.

—————————————————–

Epilogue: In a related post I will be writing about “wiping” hard drives using built in HDD level encryption. But I’ll write about encryption basics first. And the general concept of HDD-based disk encryption. I think.

Also, note that ATA-SE in itself (paired with incorrectly written BIOS) poses a malware/security threat. But that’s a whole other story.

—————————————————–

1)  In 2003, Simson Garfinkel and Abhi Shelat at MIT bought 158 used hard drives at secondhand computer stores and on eBay. 129 of these drives were functional. 69 of these still had recoverable files on them. 49 contained “significant personal information” including medical correspondence, love letters, pornography and credit card numbers. One of the disks even had transactions with account numbers from a cash machine. 51 of the drives above had been formatted, yet 19 of those still contained “easily” recoverable data.

2, 3) There are literally thousands of data recovery companies around the world that can perform “normal” forensic techniques:  replacement of the firmware or the electronics board or cleanroom replacement of the spindle motor, base casting, head stack, etc. These techniques all rely on the hard drive’s own hardware (or replacement of equivalent hardware) for retrieving the user data from the drive. Any recovery from the disk platters when the platters are outside the drive’s normal environment are referred to as “exotic forensic techniques”. The number of commercial players are few, if any; the techniques require expensive equipment (such as scanning magnetic force microscopes), sophisticated decoding algorithms (since the data recovered from the platters are encoded – and each hard drive will have a unique encoding) and vast amounts of time (scanning each sector multiple times, calculating averages and assembling the scanned sectors into decodeable blocks) and are not commercially viable. The only organizations who are likely to be able to perform such operations are government signal analysis agencies, such as NSA-SIGINT, and even they will be limited in the number of cases per year.

4) A study made by Center for Magnetic Recording Research at University of California, San Diego indicates that multiple overwrites do not provide any significant improvement in data remover over a single overwrite. A single overwrite provides more than sufficient wiping of the information to prevent the information from being recovered by normal forensic methods.

A paper on the CMRR study can be found here: http://www.tomcoughlin.com/Techpapers/Secure%20Erase%20Article%20for%20IDEMA,%20042502.pdf

The study was led by Dr. Gordon Hughes and Tom Coughlin (see footnote 7).

5) There are really only a few ways that malware could attack a block erase tool such as DBAN, ExpertEraser, Killdisk or Blancco. They all boot from a CD, floppy or USB memorystick, loading their own operating system into RAM before starting the wiping process (presumeably all with the ATA command WRITE SECTOR(S) (EXT)).

Given these facts, there really aren’t that many ways malware can interfere with the process. The most likely way is for the boot CD/floppy/USB to be infected (another theoretical method is by a BIOS rootkit).  If you downoload DBAN and run the LiveCD, you really have no way of knowing whether it really does what it says it does. The CD will boot, the program will start and the message on the screen will inform you that a wipe is underway. Theoretically, if the DBAN LiveCD is infected by malware, the program could do NOTHING, only pretending to wipe the disk for two days, then tell you that the disk is wiped. If vital parts of the file system is destroyed (by simply formatting the drive) you have no way of telling whether the drive has been wiped or not – unless you turn the drive over to a disk recovery company for analyses. This infection is fairly easily implemented using rootkit-like system hooks.

Now, the same method would work for any tool implementing ATA-SE wipe. With ATA-SE you boot from your own DOS boot disks and run the HDDErase program. If the HDDErase tool was infected with malware the tool could behave like the above example. The program would tell the user that a SECURITY SET PASSWORD is being set and that a SECURITY ERASE UNIT is being performed, when in fact only a standard disk format is being performed.

My conclusion is that BEWTs and ATA-SE have the same level of risk of being twarted by malware (and a very small one at that).

6) According to ATA standardization organisation T13 and HDD manufacturers,  a locked drive cannot be unlocked again without the correct password – neither by the drive manufacturers themselves or by anyone else. The aforementioned data recovery company Ibas has proven this to be a big fat lie a) b).

7) Dr. Gordon Hughes is the Associate Director of the Center for Magnetic Recording Research at the University of California, San Diego. Tom Coughlin is President of the data storage consulting firm Coughlin Associates.

Dr. Hughes and Mr. Coughlin were involved in the invention of the ATA-SE command and were involved in the adoption of ATA-SE into the ATA standard by the ATA standardization organisation T13.

8) The single overwrite ATA-SE method has been recommended by the National Institute of Standards and Technology as the standard method for erasing ATA hard drives at a “purge” level (the three levels are “clear”, “purge” and “destroy”). The new NIST 800-88 [PDF warning] standard (Guidelines for Data Sanitation) has replaced the old DoD 5220.22-M standard (National Industrial Security Program Operating Manual).

9) Bending the platters is one of the most effective destruction methods available; preventing even exotic forensic recovery. As little as a millimeter of bending makes all forms of practical recoverability impossible – even though the data is still theoretically intact.

12 Comments

Filed under ATA, data forensics, data recovery, encryption, hard drive wiping, hard drives, Ibas, NSA, security

12 responses to “Securely erase hard drives

  1. farvish

    Drive wipe is a process performed on a hard drive to help ensure that all of the data and information on the drive are removed beyond possibility of recovery. Many hard drive wipe programs perform this overwriting process multiple times, which increases the security provided. Stellar Drive Wipe is powerful and robust drive wipe software that completely wipes hard drive data using advanced data wiping algorithms. It erases the data beyond recovery.

  2. ultraparanoid

    While farvish’s comment smells of self-promotion I prefer not to censor it.

  3. lou turturro

    Although I agree with your final conclusion (that one overwrite is not enough), I must tell you bad blocks are not erased by Secure Erase, but by Enhanced Secure Erase (not supported by any drive).
    I agree with what you say about off-track feature.

  4. moreparanoid

    The new ATA-8 revision now specifies the limit of the erase to include the HPA for normal ATA-SE and HPA+DCO for enhanced ATA-SE. HPA+DCO now no longer has to be removed before ATA-SE for newer drives, making erasure of those areas one less thing to worry about.
    ATA-SE is much faster than block overwriting because the drive does not have to wait for the host to transfer the write pattern on each write command to a different LBA. This transfer time gets more significant because as capacities increase the number of LBAs also increases. The overwrite pattern for ATA-SE (all 0’s) is already known to the drive so there is no data transfer across the bus once the command is issued.
    I have seen a SAS drive with the equivalent ATA-SE command, called “security initialize” in the SCSI standards. Its erasure limits are the same as in ATA, but there is no “locking” mechanism.
    To date, I know of no software that deletes the log data, which is another potential for user data to reside.

  5. jay armstrong

    Great post. My understanding is that ATA-SE is faster than a thorough BEWT wipe and cleans the g-list blocks. If HDDErase could detect SATA drives, I’d love to try it (FAQ doesn’t help).

    A question for you: how many random passes from a BEWT would be roughly equivalent to an SE? Or are there other programs that can trigger the SE besides HDDErase?

  6. ultraparanoid

    lou turturro: “bad blocks are not erased by Secure Erase, but by Enhanced Secure Erase”

    I did not know that. Thanks for the info.

    moreparanoid: Thanks for the insight.

    jay armstrong: “how many random passes from a BEWT would be roughly equivalent to an SE?”

    BEWT and ATA-SE are basically the same operations from a purely magnetic viewpoint, so I would say that a single BEWT overwrite is roughly equivalent to a (single) ATA-SE overwrite.

    Since ATA-SE overwrites the user data with all zeros and BEWTs overwrite the user data with a “random” pattern, one might think that BEWTs are more secure (since it is inherently simpler to recover user data overwritten with ANY known pattern, especially all zeros or all ones). This, however, is not the case. The pattern that is actually written to disk in an ATA-SE wipe is NOT all zeros but rather a pseudo-random sequence of ones and zeros (due to HDD scrambling, RLL, ECC, CRC and parity).

    “are there other programs that can trigger the SE besides HDDErase?”

    I do not know of any other program written specifically to perform an ATA-SE, but this program ( http://ata-atapi.com/atademo.html ) might do the trick? Also, creating your own would be fairly easy in a low-level programming language such as C/C++ or even assembler code if you have any programming experience. If you want to write your own program here’s a starting point for you: http://www.geocities.com/SiliconValley/2072/atapi.htm

  7. Giacomo Brussino

    The link for the paper “Tutorial on Disk Drive Data Sanitization” must have moved. It is now http://cmrr.ucsd.edu/people/Hughes/DataSanitizationTutorial.pdf.

    It looks like it may be possible to remove HPA and DCO as well as secure erase a drive using hdparm, although most / all the necessary command may still be experimental: http://linuxreviews.org/man/hdparm/. You may have alluded to this utility in your previous post on disk wiping tools when you wrote “… use a bootable Linux CD to remove the HPA and / or DCO…”. If not, I’d like to know what you had in mind.

    Also, in the previous article you had a link for ExpertEraser. The domain experteraser.com expired on 02/02/2009.

  8. Steve

    “With high probability the NSA have this capability, as well as intelligence agencies in other countries, such as China, Russia, Israel, France, UK, Germany, India and others.”

    You’re an idiot.

  9. Pingback: Secure Erase - Remote Exploit Forums

  10. Alan

    The hdparm(8) command in modern Linux distros has at least experimental support for some ATA security features. In particular, it looks like it supports setting passwords, secure erase, and enhanced secure erase. Caveat emptor.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s