Is there a way to compare folders in a batch file?

[glaurung]

Say I have two folders, \A and \B. Folder B has files also appearing in A, but also has some files that do not appear in A.

I can generate a list of files in B that are also in A, using:

cd B
for %%F in (*.*) do dir ..\A\%%F /b >> ..\duplicates.txt

I also want to generate a list of the files in B that are not in A. It seems to me that this should work, but all it does is put a single file name in unique.txt:

cd B
for %%F in (*.*) do if not exist ..\A\%%F echo %%F >> ..\unique.txt

I've also tried replacing the "echo %%F" with "dir %%F /B" Same result.

First, what am I doing wrong, and second, is there a simple way to do what I want? Am I not allowed to do a for-if combination, or is there some quirk about redirects that I don't know about?

[oscardog]

fc /L c:\filemon.txt c:\dos.txt > c:\unique.txt

if you want to read the text, if not look at the find command and use its error codes or xxcopy16 /clone if you want to copy files different (lfn needed)

HTH

[os2fan2]

It should not be that hard. I wrote a script that deleted duplicates out of a tree, but this was heavily based on 4NT / REXX / CRC / TSORT. Files with matching size and crc were deemed equal. I suppose i could have used fc or fcb to do this final test!

W

[tain]

Would you mind sharing that script with us?

[jaclaz]

second, is there a simple way to do what I want?

I don't think there is in Windows 9x batch file, it is relatively simple in 2K/XP:

@echo off
::Find duplicates files in two directories - DUPLI.CMD
::by Jacopo Lazzari
::thanks to Rob van Der Woude for the examples, tutorials and 
::info on his page [url="http://www.robvanderwoude.com/"]http://www.robvanderwoude.com/[/url]
::and to Simon Sheppard [url="http://www.ss64.com/"]http://www.ss64.com/[/url]
::-----------------------------------------------------------
::Usage: DUPLI.CMD <folderA> <folderB>
::-----------------------------------------------------------
setlocal ENABLEDELAYEDEXPANSION
Set dirA=%1
Set dirB=%2

dir /B /O %dirA% >dirA.txt
dir /B /O %dirB% >dirB.txt


Echo Files that are found in BOTH folders %dirA% AND %dirB% >both.txt
Echo. >>both.txt
findstr /G:dirA.txt dirB.txt >>both.txt
echo ------------------------------------- >>both.txt


Echo Files that are found in folder %dirA% but NOT in folder %dirB% >AnotinB.txt
Echo. >>AnotinB.txt
find /V /C "this_is_an_absurd_string" dirB.txt >numlines.txt
for /f "tokens=3 delims= " %%B in (numlines.txt) do set /A maxnum=%%B

for /F  "tokens=* delims= " %%A in (dirA.txt) do (
find /V /C "%%A" dirB.txt >numlines.txt
for /f "tokens=3 delims= " %%B in (numlines.txt) do set /A foundnum=%%B
if %maxnum%==!foundnum! echo %%A >>AnotinB.txt
)
echo ------------------------------------- >>AnotinB.txt


Echo Files that are found in folder %dirB% but NOT in folder %dirA% >BnotinA.txt
Echo. >>BnotinA.txt
find /V /C "this_is_an_absurd_string" dirA.txt >numlines.txt
for /f "tokens=3 delims= " %%B in (numlines.txt) do set /A maxnum=%%B

for /F "tokens=* delims= " %%A in (dirB.txt) do (
find /V /C "%%A" dirA.txt >numlines.txt
for /f "tokens=3 delims= " %%B in (numlines.txt) do set /A foundnum=%%B
if %maxnum%==!foundnum! echo %%A >>BnotinA.txt
)
echo. >>BnotinA.txt
echo ------------------------------------- >>BnotinA.txt

copy /B both.txt+AnotinB.txt+BnotinA.txt dupli.txt >nul
more dupli.txt

for %%A in (dirA.txt dirB.txt both.txt AnotinB.txt BnotinA.txt numlines.txt) do (
if exist %%A del %%A
)

You could see in Rob van der Woude's site if there is such a script in DOS/W9x version.....

jaclaz

Edited September 3, 2006 by jaclaz

[os2fan2]

This is the source tape for "deltree". It contains all the goings on for the batch files etc. Most of my batch foles are woven in this manner.

Extproc is a cmd/4os2/4nt command which tells the processor to let an external processor handle the batch. Here the external processor is weave.rex. You can fake the process by using regina.exe weave.rex zerase.cmd.

Note that we use weave.rex as a kind of preprocessor type script. The settings table tells you where everything hides. This form of weave.rex was written largely for direct readability, with limited functionality. None the same, it supports writing binary files as well: you could have a relatively short weave script that makes blank bootable floppy diskettes!

REGINA is a freeware REXX process. Google for Regina REXX.

TSE4 is buyware. You can use an alternate sort algorithm.

CRC is from the UTILSOS2 package, comes in DOS, Windows and OS/2. This makes the script work under OS/2 as well.

extproc weave.rex
goto :end

Rewrite of zdelsame.bat


!topic - The files to use
!src
!!  home / work			  !!inc bartpe
!inc part2
!lbl home
!! The command drive		 !!set ydrv d:\
!! The script drive		  !!set yscr #ydrv#
!! where the batch file goes !!set ycmd cdata\batch
!! where uerase.rex lives	!!set ydll cdata\batch\util
!! tsort.com:  oou		   !!set yapps d:\mswin\apps
!! where the exe files go	!!set ypath d:\mswin\exe
!lbl work
!! The command drive		 !!set ydrv h:\
!! The script drive		  !!set yscr #ydrv#
!! where the batch file goes !!set ycmd apps
!! where uerase.rex lives	!!set ydll apps\dll
!! tsort.com:  oou		   !!set yapps c:\winnt\data\apps
!! where the exe files go	!!set ypath c:\winnt\data\mswin\sh
!lbl bartpe
!! tsort.com:  OOU		   !!set ydrv i:\
!! where the exe files go	!!set yscr #%SystemDrive%\#
!! where the batch files go  !!set ycmd Programs\Exe
!! where unerase.rex lives   !!set ydll Programs\Ini
!! tsort.com:  oou		   !!set yapps #%SystemDrive%\Programs\Exe
!! where the exe files go	!!set ypath #%SystemDrive%\Programs\Exe
!say #ydrv#ycmd#
!end

This mob of settings is not layout specific.  Also this is where we do
the main batch files and rexx files from.

!src part2
!set ytemp #_delsame.ba##
!set ycrcx #ypath#\crc.exe -ls
!set ysort #ypath#\tsort.com
!set yrexx #ypath#\regina.exe
!set ydeltree del /sxeq
!set ydosdel del /sqexz
!set yvoidtxt lpt2
!set ydelete del /q /e /x
!set yutil udelsame
!set yname delsame
!set yldr #call lineout batfile,#
!exe erasecmd #ydrv#ycmd#\#yname#.bat
!exe erasecmd #ydrv#ycmd#\#yname#.cmd
!exe erasecmd #ydrv#ycmd#\#yname#.btm
!exe eraserex #ydrv#ydll#\#yutil#.rex
!end


!topic - The erase batch file.

!src erasecmd
!new #zvar1#
!put #@echo off ##:: #zcomment##::~ Delete files by size, crc##
!var #ydosdel# #ytemp#?
!var #yrexx# #yscr#ydll#\#yutil#.rex /1 %1$
!var #call %temp%\#ytemp#t
!var #yrexx# #yscr#ydll#\#yutil#.rex /2
!var #call %temp%\#ytemp#t
!end


What we do with the first command, is to write a batch file, based on the
command options.  This is then run, to generate the first round of output.
This output is then passed through the delsame utility, to give a list of
deletable files.

!topic The erase.rex utility

!src eraserex
!new #ydrv#ydll#\#yutil#.rex
!put #/* REXX #zcomment# */##/*~ ZDELETE utility */##
numeric digits 15; parse arg option cmdtail;
!put #batfile=value('TEMP',,'ENVIRONMENT') || '\#ytemp#'; #
unsort= batfile'u'; sorted=batfile's'; batfile = batfile || 't'
!var #sortexe='#ysort#'; crcexe = '#ycrcx#'; delete='#ydelete#';
!put #select; #
when option = '/1' then call writebat
when option = '/2' then call dosorted
otherwise; call optwrong; end; exit
!inc writebat
!inc dosorted
!inc optwrong
!end


!topic - writebat
Here we read the tail, and write the command accordingly.

!src writebat
!put #writebat:; #
!exe filewrite #batfile##
!put #if pos('/?', cmdtail) > 0
!put # then call ttyhelp;#
!put # else call databat;##
!exe fileclose #batfile##
return
ttyhelp:
!var #yldr# 'echo #yname# is used to delete files with duplicate size and CRC'.
!var #yldr# 'echo.'
!var #yldr# 'echo #   run #yname# in the directory base where the duplicate files are'
!var #yldr# 'echo #   eg  #yname# *.txt *r*.cmd'
!var #yldr# 'echo.'
!var #yldr# "echo #yname# uses TSE4's tsort, UTILOS2 crc and REGINA rexx"
return
databat:
if cmdtail = '' then cmdtail = '*'
!var #yldr# '@echo off'
!var #yldr# crcexe cmdtail '>' unsort
!var #yldr# sortexe unsort sorted
!var
return
!end

If the command option contains a /?, then a batch file is written to give
help indications.  This is what ttyhelp does.  Otherwise, the role is to
make a data batch.

In the command tail, we replace '' by '*' because crc expects a wildspec.

!topic - dosorted


!src dosorted
dosorted:; osize = -1; ocrc = 0; deleted = 0
!exe fileread #sorted#
!exe filewrite #batfile#
do while lines(sorted)
 incard = linein(sorted); parse var incard nsize ncrc nname; select
 when nsize > osize then call filejump
 when ncrc <> ocrc  then call filejump
 otherwise; call delfile; end
!! do while !!var end
!var #yldr# 'echo' format(deleted, 6) 'files deleted'
!var #yldr# '#ydeltree# #yvoidtxt#'
!exe fileclose #sorted#
!exe fileclose #batfile#
return
filejump:; osize = nsize; ocrc = ncrc; return
!inc delfile
!end

By the time we get here, we have files sorted by size, crc and name, in
the form as shown in the table below.  The idea is that we delete files
if the first two columns match.  This overcomes the obvious ability to
touch files and rename them, but not where the files are genuinely
different.

[D:\CDATA\batch\source]crc -ls *r*.* | sort
		 233  0x79C0585B  "zrpn.cmd"
		 248  0x3483E3DF  "descript.ion"
		1577  0x71253737  "zerase.cmd"
		5861  0xA31FB720  "nrxch.cmd"
		6702  0x554C58FE  "matrix4.rex"
		6775  0xEE945191  "matrix5.rex"
	   24781  0x2FC9582C  "ZRXC.CMD"

It still is possible for different files to have the same crc.  We can also
do a trick where we leave undeleted files under a certian size, eg 5 bytes.


!topic - - delfile

This is what happens when we want to delete the file.

   deleted = number of files deleted (counter)
   nname = name of duplicate file
   #yldr# is command to write batch to file
   delete = command required to delete file

!src delfile
!put #delfile:;  deleted = deleted + 1; #
!var #yldr# delete nname; #yldr# echo 'Deleting' nname; return
!end


!topic - optwrong

!src optwrong
optwrong:
!var #say '#yutil# is called as part of #yname#.  Please run #yname#.bat.'
return
!end


!topic Utilities  fopen, fclose

!src fileread
!var #call stream #zvar1#, 'c', 'open read'
!lbl filewrite
!var #call stream #zvar1#, 'c', 'open write replace'
!lbl fileclose
!var #call stream #zvar1#, 'c', 'close'
!end



:end

The output consists of two files, a batch file and a rexx script. This is the batch file. Note we use explicit paths to prevent files in the current directory being used. (These could be OS/2 or DOS programs!)

@echo off
:: Woven from zerase.cmd by Wendy Krieger on 20040604 at 21:34:58
::~ Delete files by size, crc
del /sqexz _delsame.ba?
d:\mswin\exe\regina.exe d:\cdata\batch\util\udelsame.rex /1 %1$
call %temp%\_delsame.bat
d:\mswin\exe\regina.exe d:\cdata\batch\util\udelsame.rex /2
call %temp%\_delsame.bat

This is the rexx script.

/* REXX Woven from zerase.cmd by Wendy Krieger on 20040604 at 21:34:58 */
/*~ ZDELETE utility */
numeric digits 15; parse arg option cmdtail;
batfile=value('TEMP',,'ENVIRONMENT') || '\_delsame.ba'; unsort= batfile'u'; sorted=batfile's'; batfile = batfile || 't'
sortexe='d:\mswin\exe\tsort.com'; crcexe = 'd:\mswin\exe\crc.exe -ls'; delete='del /q /e /x';
select; when option = '/1' then call writebat
when option = '/2' then call dosorted
otherwise; call optwrong; end; exit
writebat:; call stream batfile, 'c', 'open write replace'
if pos('/?', cmdtail) > 0 then call ttyhelp; else call databat;
call stream batfile, 'c', 'close'
return
ttyhelp:
call lineout batfile, 'echo delsame is used to delete files with duplicate size and CRC'.
call lineout batfile, 'echo.'
call lineout batfile, 'echo	run delsame in the directory base where the duplicate files are'
call lineout batfile, 'echo	eg  delsame *.txt *r*.cmd'
call lineout batfile, 'echo.'
call lineout batfile, "echo delsame uses TSE4's tsort, UTILOS2 crc and REGINA rexx"
return
databat:
if cmdtail = '' then cmdtail = '*'
call lineout batfile, '@echo off'
call lineout batfile, crcexe cmdtail '>' unsort
call lineout batfile, sortexe unsort sorted
return
dosorted:; osize = -1; ocrc = 0; deleted = 0
call stream sorted, 'c', 'open read'
call stream batfile, 'c', 'open write replace'
do while lines(sorted)
 incard = linein(sorted); parse var incard nsize ncrc nname; select
 when nsize > osize then call filejump
 when ncrc <> ocrc  then call filejump
 otherwise; call delfile; end
end
call lineout batfile, 'echo' format(deleted, 6) 'files deleted'
call lineout batfile, 'del /sxeq lpt2'
call stream sorted, 'c', 'close'
call stream batfile, 'c', 'close'
return
filejump:; osize = nsize; ocrc = ncrc; return
delfile:;  deleted = deleted + 1; call lineout batfile, delete nname; call lineout batfile, echo 'Deleting' nname; return
optwrong:
say 'udelsame is called as part of delsame.  Please run delsame.bat.'
return

[tain]

Thanks for the scripts! I like the CRC management.