How can I move groups of files from a folder with nearly 50,000 files

[bizzybody]

With nearly 50,000 files, all the same type, in a single folder, Windows Explorer just chokes on it trying to open it - even on a Windows 7 x64 system with 4 gig RAM and a T5600 CPU.

Sorting by type won't work because they're all the same type. They're also all pretty close to the same size. 167 GB total.

What I'd like to do is have a way to "blindly" grab the first 1,000 files, doesn't matter what their names are, and move them to another folder. Repeat until the folder is empty. 1,000 at a time is a much easier and faster chunk to process. Processing will sort them into other folders in much smaller lots.

[bizzybody]

I've found this, was for 500, I changed the 500 to 1000 but it doesn't move 1,000 files. It first moved 997 files and each time it moves fewer. The 21st iteration only moved 923. The decrement in the number of files it moves is not always the same, it goes up and down but is always less than the previous number.

echo offmd folder1SETLOCAL EnableDelayedExpansionset movedFiles=0for /R d:\folder\ %%G in (*) do (    echo moving... "%%G"    move /Y "%%G" d:\folder1\    set /a movedFiles+="1"    if !movedFiles! EQU 1000 GOTO endOfCopy rem if you moved 1000 files  )  :endOfCopy  echo Done, %movedFiles% files Were copied successfully  pauseENDLOCAL

I added the line to make the directory. I manually increment the new folder's number then doubleclick this again to get another folder1 full of files. The files will be in batches small enough to process but it would be nice if it would move exactly 1,000 files each time.

Even better would be to have it automatically create sequentially numbered folders to move the batches of files into, then gracefully quit when there's less than 1,000 files left in the source directory or when it's empty.

[Ponch]

Windows Explorer just chokes on it trying to open it

Have you tried deactivating you anti-virus while doing this ?

[jaclaz]

@Ponch

Antivirus or not antivirus, Explorer normally "chokes" when a large number of files are selected, more specifically it "freezes" when you right click to select Copy or Move.

@bizzybody

What I would personally do (which not necessarily is a good idea, mind you), would be to rename all files in the folder adding to each of them a progressive number, and later use that info to copy/move them.

As an example, this is a suitable simple renaming tool:

http://www.softpedia.com/get/System/File-Management/CKRename.shtml

(but you can do the same through a batch)

jaclaz

[Guest]

Even better would be to have it automatically create sequentially numbered folders to move the batches of files into [...]

The following code was tested with a max file count of 5 instead of 1000.

@ECHO OFF &SETLOCAL EnableExtensions EnableDelayedExpansionSET "nFolderCount=0"SET "bNewFolder=DEFINED"FOR /R "d:\folder" %%# IN (*) DO (  IF DEFINED bNewFolder (    SET "nFileCount=0"    SET /A "nFolderCount+=1"    MD "d:\folder!nFolderCount!"    SET "bNewFolder="  )  ECHO moving... "%%~nx#" to "d:\folder!nFolderCount!\"  MOVE /Y "%%#" "d:\folder!nFolderCount!\" >NUL  SET /A "nFileCount+=1"  IF !nFileCount! EQU 1000 SET "bNewFolder=DEFINED")IF !nFolderCount! GTR 0 (  ECHO Done, !nFolderCount! new folders were created.  ECHO The last folder contains !nFileCount! files.  ECHO The rest should contain 1000 files each.) ELSE ECHO Done, no files found.PAUSEGOTO :eof

[...] then gracefully quit when there's less than 1,000 files left in the source directory or when it's empty.

As written above, the script should exit gracefully. It will always empty the source folder. Allowing the script to leave files in the source folder may require violating the "'blindly' grab" rule. I believe two passes would be necesary per thousand files. The first pass would only count the number of files up to one thousand in the source folder. The second pass would move the counted files only if the number counted is equal to one thousand. If more than that number remain, the cycle would repeat (pass 1, pass 2, pass 1, ...) until less than a thousand files remain.

EDIT: Removed ":FolderLoop" label and its associated GOTO statement. It's no longer necessary to jump out of the FOR loop by using the boolean bNewFolder, which replaced bNewLoop as a more descriptive variable name.

Edited December 23, 2013 by 5eraph

[jumper]

Or once the source folder is empty, the last block move can be undone--but that can also easily be done manually.

[Ponch]

@Ponch

Antivirus or not antivirus, Explorer normally "chokes" when a large number of files are selected, more specifically it "freezes" when you right click to select Copy or Move.

I've had that some times, but being patient was more efficient that waiting several hours for an answer on a forum. Eventually it comes with a reaction. Then again maybe there were less files.

[bizzybody]

No antivirus on the system being used for this. It's not connected to the internet.

5eraph, I'll try your script, thanks.

[Yzöwl]

I'm not expecting this batch to be particularly quick but it may suit your needs:

@ECHO OFFSETLOCAL ENABLEEXTENSIONS ENABLEDELAYEDEXPANSIONSET "SRC=X:\MY SOURCE"SET "DEST=X:\MY DESTINATION"SET/A "CHUNK=1000"SET/A "COUNT=0"FOR /F "TOKENS=*" %%# IN ('DIR/B/A-D "%SRC%"') DO (	ECHO=MOVE "%%#" "%DEST%"	SET/A "COUNT+=1"	SET/A "MOD=COUNT %% CHUNK"	IF !MOD! EQU 0 GOTO OUTPUT):OUTPUTECHO= %COUNT% FILES MOVED FROM %SRC% TO %DEST%PAUSEEXIT

Just in case you're interested something like this Powershell one liner may suit:

Get-Childitem "X:\MY SOURCE" | Select-Object -First 1000 | Move-Item "X:\MY DESTINATION"