Davor Josipovic Just another WordPress blog – rather tryout

02/06/2013

Ignoring device I/O errors during copy with PowerShell

Filed under: PowerShell,Programming — Tags: , , — Davor @ 19:14

I had a failing hard drive with lots of RAR files with recovery records. Problem was that I couldn’t get the files off the hard drive with tools like robocopy and xcopy because the drive had many bad sectors resulting in CRC and I/O device errors. I also couldn’t repair the drive with chkdsk /R because the bad sectors kept reappearing.

I also tried Unstoppable Copier, but on best settings it seems to use 0,1% of the file size per read operation, which results in large data corruption when that read operation fails even if there are only a few bad sectors.

So I wrote a little PowerShell script which will copy the source files and replace the unreadable data with zero’s, as accurately as the partition’s cluster size allows. It will also make a XML file per file where it will store bad sector data positions for further reference. I have placed the script code on GitHub.

Note on reading bad sector data

When a hard drives fails to read a sector due to a CRC error, it doesn’t give back any data. Instead it raises an error: I/O device error. With old hard drives it was possible to issue the READ LONG command to skip the error correction part, and simply give back the data. Some programs like Spinrite use this to recover data based on re-read statistics. (If on hundred consecutive re-read operations for sector X your starting bits are “1011…”, then it is reasonably safe to assume those bits are not corrupted.) But the READ LONG command doesn’t work with modern hard drives. Modern hard drives either succeed or fail a read operation. There is no third option. What I have noticed during my own recovery is that even though the hard drive fails reading a sector the first time, there is some chance it will succeed on a retry. I have seen it succeed even after 100 fails! That’s why the -MaxRetries is so important in this script. But setting it too high will greatly slow down the recovery process in case a sector is truly unreadable.

Notes on script usage

  1. First you need to download the script. Here is a link to the file. Save it as Force-Copy.ps1 to, for example, your desktop. (Make sure the extension is correct: ps1!) (There is also a more simple script available at stackoverflow.)
  2. This is a PowerShell script, so it has to run under the PowerShell environment. Run PowerShell and navigate it to your desktop directory. For example, run cmd, type cd desktop and then type powershell -ExecutionPolicy bypass. (Execution policy needs to be changed because PowerShell will not allow scripts to be run by default. With bypass, the script is not blocked and there are no warnings or prompts. This is because the script is unsigned. Read more about PowerShell execution policy and signing here.)
  3. Now, to see some examples of use, type Get-Help .\Force-Copy.ps1 -Examples. The most simple command to copy a file from one to an other location would be:
    .\Force-Copy.ps1 -SourceFilePath "C:\bad_file.txt" -DestinationFilePath "W:\recovered_file.txt"

25 Comments »

  1. Nice script, thank you! 🙂

    Comment by NLZ — 16/08/2013 @ 19:41

  2. how can i use this? sorry for being a noob. hehe

    Comment by minstrit — 25/08/2013 @ 07:01

  3. I have added a section about script usage.

    Comment by Davor — 25/08/2013 @ 09:20

  4. Hey I can copy files but I cant copy full folder and its contents. Please help

    P.S Thanks for the script its awesome

    Comment by max — 09/07/2014 @ 13:18

  5. This script made wonders. Tried two softs trying to copy from a bad block and both failed. Thank you so much.

    Comment by Adrian — 04/01/2016 @ 11:06

  6. Thank you for this script, I was able to rescue a vmware vmdk that fails at 99.1% when trying to copy. This successfully copied it with 6 I/O errors.

    Comment by Eric — 13/09/2016 @ 21:11

  7. Thank you so much. I have copied my virtual disk successfully and it is also running normal in Vmware. From below result you will find 4KB WAS UNREADABLE. Now, I am going to repair my bad sectors via HDD regenerator. Thanks again please keep posing good work. I have also visted other programming sections on this website and got good knowlege about c++(how keylogger works) and powershell.

    PS C:\> .\Force-Copy.ps1 -SourceFilePath “D:\VmwareMachines\Windows764bit\Windows7.vmdk” -DestinationFilePath “C:\recovered_file.vmdk”
    Starting copying of D:\VmwareMachines\Windows764bit\Windows7.vmdk…
    Started reading from source file at offset 0.
    Can not read 4096 bytes starting at 4316884992 bytes: Data error (cyclic redundancy check).

    4096 bytes are bad.
    Copied 22056468480 bytes in 00:10:41.0937828 (32.81 MB/s)
    Finished copying D:\VmwareMachines\Windows764bit\Windows7.vmdk!

    Comment by Ashok from india — 27/10/2016 @ 17:14

  8. Thank you for the script.
    But I would like to know how to copy a path instead of a single file?
    Is there a way to copy everything from C:/ to D:/ for example?

    Comment by Nicky — 05/05/2017 @ 12:12

  9. The powershell script is indeed meant to copy 1 file at a time. To copy a whole directory you will have to put it into a loop. See for example this post. Or just google recursive loop files directory.

    Normally you don’t want to copy unbroken files with this script. You want to move the files first (from C:), and for those that can not be moved (due to some error like the cyclic redundancy check failing) you want to use this script.

    Comment by Davor — 05/05/2017 @ 12:28

  10. Great tool! This saves me broken download files from my old notebook!

    I have a suggestion: [Bad sector isolation technique]
    Could you please try to add a file splitting feature where it will isolate bad sactors as files. Hence, system will not write onto them anymore by occupiting the sectors. Then we can keep using our old broken HDD! XD (Of course for downloading files only. I still strongly recommand to replace for a new drive if the old drive is from your working computer.)

    Comment by Eric — 21/07/2019 @ 09:33

  11. Great that it was helpful.

    Note that bad sector isolation is usually automatic or can be forced with a format. For example: https://superuser.com/a/655090/161109.

    In case of a failing disk and critical data, what you want is as quickly as possible, to get as much data from it as you can. That is what the tool is made for. Isolating bad sectors would be more like an other tool if you ask me.

    Comment by Davor — 21/07/2019 @ 12:53

  12. Thank you for the script. Most appreciated. Recovered a Linux Server VM VHDX file.

    Comment by Don Scott — 15/08/2019 @ 10:49

  13. Hey, I’m totally stuck in a very bad situation right now my hard disk is getting disconnected automatically after few minutes, i know this tool of your is working, there might be something suspicious with my HD, but does any one have any solution for this?

    Comment by Dishant Nankani — 31/10/2019 @ 06:03

  14. Thanks a lot, I could repair a 22GB MDF file. A Life saver tool!!! Thanks again.

    Comment by Harbey Alvarado — 18/04/2020 @ 02:02

  15. Notes on script usage doesn’t work for me
    I cannot make it work even though following the procedure
    I need a detailed explanation maybe with pics or video
    Help me
    Pls
    thanks

    Comment by Kishore — 13/05/2020 @ 11:26

  16. I have tried variations of your examples to no avail. I know that this script is meant to do one file at a time and that example 2 puts it into a loop but the wording is rather difficult for me. Every variation I try keeps erroring out. Perhaps you can just help me write the command. What I want to do seems simple. I want to copy everything I can (i know that some files will be bad) from drive H: and all sub directories to a directory onto C:\g7.
    I have been working on this for about 3 weeks now and am about to give up. Any help would be really appreciated. Thanks

    Comment by Scott Zechman — 10/11/2020 @ 10:40

  17. Hi Scott. Start step by step with only 1 file. At least something like this should work:

    .\Force-Copy.ps1 -SourceFilePath “H:\filename.extension” -DestinationFilePath “C:\g7\filename.extension”

    Make sure you run PowerShell as administrator so you can write to C:\. For programming help you can ask questions at stackoverflow.com where a whole community will try to help you. This page is really not a good place for such questions.

    Comment by Davor — 10/11/2020 @ 10:56

  18. Hi,

    First of all, thank you for this great script 🙂

    I am trying to rescue files from failing disk.

    I run this line from the directory which contains files & folders to be rescued (e:\sourcefoulder) and files are going to d:\sourcefoulder

    Get-ChildItem “*.*” -recurse | foreach {c:\scripts\Force-Copy.ps1 -SourceFilePath $_.FullName -DestinationFilePath (“D:\”+(Split-Path $_.FullName -NoQualifier)) -Maxretries 2}

    I fails & stops at one point with error below, I would like it to ignore possible errors and continue (goal is to rescue all readable files as soon as possible and ignore broken ones). This script seems to be working better than for example Roadkill’s unstoppable copier.

    Get-ChildItem : ASSERT FAILED: Specified position out of source file bounds.
    At line:1 char:1
    + Get-ChildItem “*.*” -recurse -ErrorAction Continue | foreach {c:\scripts\Force-C …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : OperationStopped: (ASSERT FAILED: …ce file bounds.:String) [Get-ChildItem], RuntimeExc
    eption
    + FullyQualifiedErrorId : ASSERT FAILED: Specified position out of source file bounds.,Microsoft.PowerShell.Comman
    ds.GetChildItemCommand

    Do you have any suggestions for this?

    Comment by Markku Mäenpää — 05/12/2020 @ 11:16

  19. Markku, the blocking line is this: Assert {$Position -lt $SourceFile.length} "Specified position out of source file bounds.";. I think that the file you are trying to copy is 0 bytes. Just remove the 0 byte files from the source before running the script. An other approach would be to ignore errors in the loop. See here.

    Comment by Davor — 05/12/2020 @ 22:40

  20. hvala ti. greatness is in simplicity.

    Comment by vlada — 23/10/2021 @ 17:20

  21. Why i get this error?
    PS C:\Users\ACER\Desktop> .\Force-Copy.ps1 -SourceFilePath “J:\All Pictures In This Computer\New folder\FileBrowser\Videos\VID_20170619_155259.mp4” DestinationFilePath “C:\Users\ACER\Desktop”
    C:\Users\ACER\Desktop\Force-Copy.ps1 : Cannot process argument transformation on parameter ‘BufferSize’. Cannot convert value “C:\Users\ACER\Desktop” to type “System.Int32”. Error: “Input string was not in a
    correct format.”
    At line:1 char:143
    + … \VID_20170619_155259.mp4″ DestinationFilePath “C:\Users\ACER\Desktop”
    + ~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : InvalidData: (:) [Force-Copy.ps1], ParameterBindingArgumentTransformationException
    + FullyQualifiedErrorId : ParameterArgumentTransformationError,Force-Copy.ps1

    Comment by Jagtar Singh — 02/12/2021 @ 16:34

  22. Seems straightforward to me: you’re copying a file-path to a dir-path. DestinationFilePath “C:\Users\ACER\Desktop\vid.mp4” should work.

    Comment by Davor — 02/12/2021 @ 19:04

  23. I keep getting this and it takes too long, what can I do? Thanks!
    Started reading from source file at offset 0.
    Can not read 4096 bytes starting at 9431482368 bytes: Data error (cyclic redundancy check).

    Can not read 4096 bytes starting at 9692839936 bytes: The semaphore timeout period has expired.

    Can not read 4096 bytes starting at 9692848128 bytes: The semaphore timeout period has expired.

    Can not read 4096 bytes starting at 9692852224 bytes: The semaphore timeout period has expired.

    Can not read 4096 bytes starting at 9692856320 bytes: The semaphore timeout period has expired.

    Comment by Timea — 24/10/2022 @ 05:40

  24. You cannot force a disk to read a bad block faster than it already does. There is a -MaxRetries option, but by default it is 0 (= no retry). So the only thing you can do is have patience ;).

    Comment by Davor — 24/10/2022 @ 18:34

  25. Dear Davor,
    Thank you for this awesome script. It allowed me to recover a personal family video from a failing hard drive that all the “supposedly modern” systems and software were unable to resolve. The fact that this script is working seamlessly in it’s 10th year is a testament to your skill and genius.

    Many thanks !!

    Comment by Ajit — 10/03/2023 @ 06:22

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress