Modify REG_BINARY with cmd/ps

[Finnish_Fellow]

Hello, I've currently hit a stop on my project was wondering, if here someone could help me.

 

-Going to quote out my previous message since it seems extremely long to read/understand even to me-

 

What I am trying to do:

1. Dump REG_BINARY data

2. Modify data at text file / variable or any other form.

3. Replace data.

 

NOTES: Data size: ~80000 characters !!

 

Steps:

Now, so far I've been able to modify any data from FOR loop through processed keys with help of tools like FART.EXE assigning Binary values as SET and contructing a loops to process through, but the size of data now grown óver 80K of values CMD / findstr cannot process these size of data any longer.

 

Current cmd/batch:

FOR /F "tokens=1-2* delims=\" %%a IN ('findstr "HKEY_LOCAL_MACHINE\TEMP_ REG_BINARY" tmp\bin.dat') DO (IF "%%a"=="HKEY_LOCAL_MACHINE" ( set "_MBS2014_keyname=%%a\%%b\%%c" )IF "%%b"=="REG_BINARY" echo REG ADD "!_MBS2014_keyname!" /v %%a /t REG_BINARY /d %%c /f>>binary_patch.cmd)

bin.dat is simply REG QUERY including REG_BINARY types.

 

Currently I am at conclusion that it is impossible to modify the data at cmd because of: Find has limitation of 1 string, findstr has issue of 576 size even echo fails to deliver secondary batch files and only solution is to use Powershell which I literally know nothing about (well basically I was able to parse out the value, 80000 character data and dump it to cmd or ps1 file, but after week fighting with 'select-string' I cannot seem to get working $matched_HKEY (line before) -> $matched_REGBIN joined line to out-file. I always end up with result 2 separate lines where HKEY_LOCAL_MACHINE is written to 1st line and value=data to second line.

 

For example:

 

$regkey = Get-Content bin.dat | Select-String 'HKEY_LOCAL_MACHINE'
$bindata = Get-Content bin.dat | select-String ' BinaryValue ' | %{$_ -replace ' BinaryValue','='} | %{$_ -replace ' REG_BINARY ',''}
write-output $regkey $bindata | out-file -filepath something_new.txt <- does not output as written

 

 

 

---Edit---

 

 

File to parse (value is fictional and note that data is 81800 characters long so it will not go through cmd echo or findstr):

HKEY_LOCAL_MACHINE\SOFTWARE\Mykey    datavalue1    REG_BINARY    000000000000000000000000900000000000000000000000000000000000000end of entriesHKEY_LOCAL_MACHINE\SOFTWARE\Mykey997012379    datavalue2    REG_BINARY    000000000000111111111111000000000000000111111111110000000000000end of entries

I need to form lines in cmd/ps1:

REG ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Mykey" /v datavalue1 /t REG_BINARY /d 000000000000000000000000900000000000000000000000000000000000000 /fREG ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Mykey997012379" /v datavalue2 /t REG_BINARY /d 000000000000111111111111000000000000000111111111110000000000000 /f

with cmd or powershell any help is welcome.

Edited December 23, 2014 by Finnish_Fellow

[jaclaz]

I am not sure to understand (actually I am sure I do not understand) the "nature" of the the data 80Kb seems like a lot to be needed parsing/changing nor in what consists the actual find/replace.

 

Can you post an actual example (using "dummy" data if needed)?

 

A tool often used by me is gsar:

http://home.online.no/~tjaberg/

but cannot say, without an actual example, if it might do in your case.

 

Or there is the need to only do it with "built-in" tools?

Also, which specific OS?

 

jaclaz

[Finnish_Fellow]

"the "nature" of the the data 80Kb seems like a lot to be needed parsing/changing nor in what consists the actual find/replace."

 

= Find and replace itself is not issue I can deal with that by multiple tools(example fart.exe) or even by looping it. Writing new command from such a long binary data however is an really big issue as it won't go through any cmd pipeline. Example find/replace itself is simple as replacing "000011111111111000" to "000022222222121000" and can be done by set variable from binary values from a-z,A-Z,0-9 and then combining new binary from user given entry. This can be done from powershell also of course:

get-content somefile.dat | %{$_ -replace '000011111111111000','000022222222121000'} | out-file -filepath somefile2.dat

So, data size at this point is not an issue issue becomes when trying to inject it back to registry from above example REG QUERY dump you have to construct new REG ADD command or use powershell Set-Content, but I still face same issue with selecting Keyname for below value, type, data. Issue there: 

* I simply cannot build for loop at powershell which would Select-String/-match "HKEY_LOCAL_MACHINE" -> Add this to $keyname -> while getting value,type,data -> build command against this info including idea that the $keyname variable would have to be set until next is found.

 

Can you post an actual example (using "dummy" data if needed)?

 

Yes, I can all I have to do is press 00, 80000 times to above example. I can of course do quick search and dump some reg_binary with REG.EXE for example some of microsoft own keys use same size data as binary.

 

--edit/ TEST FILE from your own system-

 

In fact now that I did fast search on this regular Windows 8.1 you can get test file with command:

REG QUERY "HKLM\SYSTEM\ControlSet001\Control\Session Manager\AppCompatCache" /f AppCompatCache /v /d /t REG_BINARY>mytestfile.dat

and try replacing for example \Program Files\ to something else (inside file this would be "5C00500072006F006700720061006D002000460069006C00650073005C"). Now you see with this key replacement of data is nothing compared to writing command to get it back to registry.

 

A tool often used by me is gsar

 

- not looking for replacement tool as explained now above I can do replacement from powershell or 3rd party tools like fart. Not saying it wouldn't be good tool.

 

OS:

 

I am looking to get this working with any NT6.x OS. So, we're talking Windows Vista->10 and server variants.

 

What I need:

 

IF/FOREACH/WHILE/whatever gets the job done loop from powershell like:

get-content regquerydump.dat | select-string '(HKEY_LOCAL_MACHINE)|(REG_BINARY)' | IF($_ -match "HKEY_LOCAL_MACHINE") { set-variable -name key - Value $_ } | IF($_ -match "REG_BINARY") { %{$_ -replace '    REG_BINARY    ','/t REG_BINARY /d '} | %{$_ -replace '    ',' /v '} } | write-host "REG ADD ";$key;$_ | out-file -filepath backtoregistry.ps1

Edited December 24, 2014 by Finnish_Fellow

[jaclaz]

I am starting maybe to understand, but I have not yet fully clear the situation/constraints.

 

You are not exporting a key, you are redirecting the output of REG.EXE to a file, and then you want to edit this output "raw"?

 

Why not export using a .reg file and the re-import/merge the .reg file?

 

Or, while still using for the search/replace the output of REG.EXE, why not re-importing by creating a .reg from the "raw" data?

 

jaclaz

[Finnish_Fellow]

I am starting maybe to understand, but I have not yet fully clear the situation/constraints.

 

You are not exporting a key, you are redirecting the output of REG.EXE to a file, and then you want to edit this output "raw"?

 

Why not export using a .reg file and the re-import/merge the .reg file?

 

Or, while still using for the search/replace the output of REG.EXE, why not re-importing by creating a .reg from the "raw" data?

 

jaclaz

 

You are not exporting a key, you are redirecting the output of REG.EXE to a file, and then you want to edit this output "raw"?

 

Correct, answer follow..

 

Why not export using a .reg file and the re-import/merge the .reg file?

 

Because of the format of data, the .reg data format is as follows:

Windows Registry Editor Version 5.00[HKEY_LOCAL_MACHINE]key=hex:00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00\

Now consider you have 80K bytes and line split happens at \, what you would have to process is a for loop join all 80K bytes to singular line to be sure replacement entry is not at the split then resplit the data and reconstruct the entire 25xbyte format to import it back to registry.

 

So, you now probably understand that I did consider this method, but it will fail at regular cmd for loop because of size of data being expanded even further by ',' separator and it's even harder to re-split the data. It would even be easier to replace at raw data then reconstruct the line with ',' separator.

 

Kinda need a xmas miracle powershell script here, heh.

 

BTW, Merry Christmas to everyone, if convenient.

[Yzöwl]

Your project isn't replacing data in random keys so please stop trying to confuse those willing to help by offering generic data examples.

Tell us the actual registry keys you're parsing, the exact data you are looking to replace and the replacement data you are hoping to replace it with.

[jaclaz]

Now consider you have 80K bytes and line split happens at \, what you would have to process is a for loop join all 80K bytes to singular line to be sure replacement entry is not at the split then resplit the data and reconstruct the entire 25xbyte format to import it back to registry.

 

So, you now probably understand that I did consider this method, but it will fail at regular cmd for loop because of size of data being expanded even further by ',' separator and it's even harder to re-split the data. It would even be easier to replace at raw data then reconstruct the line with ',' separator.

 

Of course I am gonna approach it from the side you did consider but then swiftly discarded.

 

The problem can be "generalized", as I see it, it amounts to two three questions:

• How can I get a .reg file containing the export of a SINGLE key with binary data?
• How can I convert such a .reg file representing a SINGLE key with binary data into a "bastardized" format that I find useful for search and replace?
• How can I convert back, once I have performed the needed search and replace, the data from this "no separator" format into a .reg file, ready to be re-imported in the Registry?

Answers:

#1 Use REG.EXE export  [ROOT\]RegKey FileName.reg <- and this is pretty much all you can do with "built-in tools"

For #2 and #3 external tools are needed, and some pretty much complex fiddling

 

I have jumped over the "once I have performed the needed search and replace" and tried putting together two small batches, one is intended to convert the .reg file to the specified "RAW" format, and the other one is intended to recreate the .reg file from it.

 

I have used what I had handy (and that I often use), i.e. gsar and SFK (SwissFileKnife):

http://stahlworks.com/dev/swiss-file-knife.html

 

These batches  have NOT the ambition of being well written or "good", they are as usually half-@§§ed, quickly put together, batches with very little (if not none) "error control", making use of a stupid amount of temporary files and not even cleaning the mess after having been run, but maybe they could be of inspiration to make something "better".

 

regtoraw.cmd:

 

@ECHO OFFSETLOCAL ENABLEEXTENSIONS ENABLEDELAYEDEXPANSIONSET SourceReg=testbinary.regSET /A Counter=0IF EXIST RegHeader.txt del RegHeader.txtFOR /F "tokens=1 delims=:" %%A IN ('TYPE %SourceReg%') DO (SET /A Counter+=1SET Line=%%AIF "!Line:~-3,3!"=="hex" SET Line=!Line!:&ECHO !Line!>FirstLine.txt&GOTO :Out_of_ForECHO !Line!>>RegHeader.txtIF "!Line:~-4,4!"=="5.00" ECHO.>>RegHeader.txt):Out_of_ForIF EXIST myraw.txt DEL myraw.txtFOR /F "skip=3 tokens=1,2 delims=: " %%A IN ('TYPE %SourceReg%') DO (SET Line=%%AIF "!Line:~-3,3!"=="hex" (ECHO %%B>>myraw.txt) ELSE (ECHO %%A>>myraw.txt))gsar -s:x5C:x0D:x0A -r -o myraw.txtgsar -s:x2C -r -o myraw.txt

rawtoreg.cmd

 

@ECHO OFFSETLOCAL ENABLEEXTENSIONS ENABLEDELAYEDEXPANSIONgsar -s:x0D:x0A -r -o FirstLine.txtCALL :get_size FirstLine.txtSET /A offlen=%FirstLine.txt%-2SET /A offset=%FirstLine.txt%-2sfk filter myraw.txt +hextobin myraw2.txtsfk hexdump -hexsrc -recsize 25 -nofile -offlen 0 %offlen% myraw2.txt>myraw3.txtsfk hexdump -hexsrc -recsize 25 -nofile -offset %offset% myraw2.txt>>myraw3.txtgsar -s0x -r -o myraw3.txtgsar -s:x0D:x0A -r:x5C:x0D:x0A:x20:x20 -o myraw3.txtsfk replace -case myraw3.txt "/A/a/" "/B/b/" "/C/c/" "/D/d/" "/E/e/" "/F/f/" -yesCOPY RegHeader.txt + FirstLine.txt + myraw3.txt mytempreg.txtsfk partcopy mytempreg.txt -fromto 0 -7 myreg.txt -yesECHO.>Unicode_myreg.txtCOPY myreg.txt + Unicode_myreg.txt + Unicode_myreg.txt myreg.txtgsar -s:x0D:x0A -r:xFF:xFE -o Unicode_myreg.txtCMD /U /C Type myreg.txt >> Unicode_myreg.txtGOTO :EOF:get_sizeSET %1=%~z1GOTO :EOF

SFK is a very comprehensive tool and I know only part of it's usage, so it is very likely that there are "better" ways using it or that "more suited" tools exist.

I used gsar only because I am more familiar with it for the simple replacement, most probably it is not *needed* at all and everything can be done within SFK.

Most of the complication is in re-creating a file identical to the original source, very likely the REG.EXE IMPORT will not be so "picky" on line length of the comma separated values, or with the CaSe of the letters corresponding to hex values.

 

Have a Merry Christmas.

jaclaz

[Finnish_Fellow]

Your project isn't replacing data in random keys so please stop trying to confuse those willing to help by offering generic data examples.

Tell us the actual registry keys you're parsing, the exact data you are looking to replace and the replacement data you are hoping to replace it with.

 

I am sorry, if my examples were not good enough.

 

I found it extremely hard to explain my intentions as there were few ways to approach the matter(cmd, powershell, vb + 3rd party) and hand full of tools(3rd party) that could of been used with few different formats to process not to mention I am actually processing through over 5000 registry keys even while only handful are actually binary and having this issue which I can isolate and process separately, but I believe generic example was also needed, so, it's not restricted to singular case.

 

 

@jaclaz

 

That is pretty amazing.

 

sfk, I did try sfk earlier I think I ended up not getting wanted result at hexdump, it's pretty amazing too just pain to understand correctly as it's indeed a "do it all, no matter what".

gsar, if I understand correctly you replaced end spaces, \, line split with gsar at "regtoraw.cmd" this is kinda function I didn't have anywhere.

 

Still need to understand all you wrote there, even those 'regtoraw.cmd' if statements are something I've never used before, heh, but it's pretty easy to read thanks for that.

 

Going to try it as Christmas ends and because that works the way it works got few ideas now on my head how to lose a few steps there.

Edited December 25, 2014 by Finnish_Fellow

[Yzöwl]

 

Your project isn't replacing data in random keys so please stop trying to confuse those willing to help by offering generic data examples.

Tell us the actual registry keys you're parsing, the exact data you are looking to replace and the replacement data you are hoping to replace it with.

 

I am sorry, if my examples were not good enough.

 

I found it extremely hard to explain my intentions as there were few ways to approach the matter(cmd, powershell, vb + 3rd party) and hand full of tools(3rd party) that could of been used with few different formats to process not to mention I am actually processing through over 5000 registry keys even while only handful are actually binary and having this issue which I can isolate and process separately, but I believe generic example was also needed, so, it's not restricted to singular case.

[Detailed Information]==[Appropriate Assistance]

 

I don't believe that you are doing a replace on 5000 registry keys; and since you have decided that your project detail is to be kept a secret, then even though I'm quite familiar with powershell and nt command scripting, I will be unable to provide you with the assistance you require.

[Finnish_Fellow]

@Yzöwl
[Detailed Information]==[Appropriate Assistance]:
 
I think I did this at first post by providing format to parse and even command to get key large enough to test and replicate the issue as said I found this issue quite hard to explain.
 
Offtopic:
My previous project, search and replace at Windows Enterprise N (x86 edition) is 5084 rough 8000-9000 entries at x64 architecture side entries as I was moving entire \Windows root (which is, btw, currently successful project working from almost any existing writable media) beside few thousand other modifications like programdata, users, program files x86/x64. Currently running any NT6.xx OS from any folder name. Which also considers Program Files, Users and other folders true movable by reboot.
 
Currently, I am working using registry near database a like and we are talking about corporate data, so, revealing the data itself from custom made keys is exactly the same as me giving 'generic data' as example. So, I provided example from Microsoft Windows own key as everyone has that key at any Windows NT6 system and it was way easier to use and dump as an example than me sending files to forum and as above we did see solution was found in whole another style of what I was using, so, this example gave way more ways to think about matter to come up with solution.
:Offtopic
 
So, you can believe me or not that still is the honest truth that, we are talking about several thousand keys not just single entry processed through a single batch.
 
I do appreciate that you're willing to help, specially with powershell side which is, well, quite a bit trickier, but also extremely useful and I believe most, if not all could of been processed with powershell script (even better idea to search, save, modify and directly modify back to registry through powershell which is what I was thinking originally, if I would of understand how to redirect found data as variable through powershell get-content or such which I was trying to explain few posts back).

[jaclaz]

Hmmm, little by little we are managing, through applying some (light) torture , to extract from you the actual goal, which now seems to me more *like* "How can I change each and every PATH reference in the Registry to a set of new values?". 

 

You should be aware of the generic risk of slipping on chocolate covered bananas, JFYI:

http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/put-down-the-chocolate-covered-banana.html

 

Now, the next question is:

Are you doing these modifications on an "online" Registry or are you doing it on a set of offline hives temporarily mounted in the online Registry?

 

I mean, this may be of use:

http://reboot.pro/topic/11212-offline-registry-library/

http://reboot.pro/topic/11312-offline-registry/

 

Or, depending on the actual use of the thingy, you may be able to afford paying a few bucks for a Shareware program, like (examples only, not necessarily a valid solution):

http://www.funduc.com/registry_toolkit.htm

 

I would also explore/experiment to see how much it is still functional the REGFIND from good ol' W2K Resource Kit:

http://www.petri.com/download_free_reskit_tools.htm

 

jaclaz

[Yzöwl]

I think I did this at first post by providing format to parse and even command to get key large enough to test and replicate the issue as said I found this issue quite hard to explain.

 

<SNIP />

 

Currently, I am working using registry near database a like and we are talking about corporate data

 

<SNIP />

If this is Corporate data, and you are looking for remuneration for someone else's work then provide the information or go elsewhere

 

If you are searching the entire registry or a selection of hives then instead of making things difficult just say so.

 

I asked for the exact search and replacement data, and you still appear to have failed to provide it!

 

Finally, if you are, as we are guessing, intending to inject replacement paths into entire registries, (offline or otherwise), then batch scripting and/or powershell would be far from my recommendation.