File copying: Still stupidly slow
Mar. 10th, 2013 07:16 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
andrewduckerI'm currently grabbing a backup of all of Julie's important files from her PC before I reinstall Windows from scratch.
And the varying file copying speeds tell me that something brain dead is going on.
Copying a single large file brings transfer speeds up to 50MB/s (400Mbit/s). Copying lots of small files brings this down to 1MB/s, and occasionally down to 300kB/s when dealing with multiple tiny files.
So either there's a massive overhead in setup/teardown of individual network connections for each transfer (which would be dumb), there's a massive overhead of updating the file-system for each file (which would be dumb) or at the end of each file it's pausing and flushing all the buffers before going on to the next one (which would be dumb).
In any case, there's _something_ dumb going on.
And the varying file copying speeds tell me that something brain dead is going on.
Copying a single large file brings transfer speeds up to 50MB/s (400Mbit/s). Copying lots of small files brings this down to 1MB/s, and occasionally down to 300kB/s when dealing with multiple tiny files.
So either there's a massive overhead in setup/teardown of individual network connections for each transfer (which would be dumb), there's a massive overhead of updating the file-system for each file (which would be dumb) or at the end of each file it's pausing and flushing all the buffers before going on to the next one (which would be dumb).
In any case, there's _something_ dumb going on.
no subject
Date: 2013-03-10 10:53 pm (UTC)Time to DL: about two minutes.
Time to file transfer: about ten minutes.
I would have been better off logging into GOG on her machine.
no subject
Date: 2013-03-10 10:54 pm (UTC)I eventually solved it using http://sourceforge.net/projects/freefilesync/ - which copied at 30-60MB/s quite happily, taking an hour to do what had threatened to be a 10-hour job.