Interesting ISO/NTFS issue

[awkduck]

This is just a kinda note, in case someone else runs into it.

I have NTFS support (Paragon), on Win98.

This system has no internal drives, with NTFS. USB drives with NTFS and EXT2/3 are connected and used.

When a USB drive "having NTFS/EXT*" is plugged in and active, certain ISO files cannot be used. This is tested with D-Tools and VirtualCloneDrive. I cannot even copy these ISO files. For example, from a network FTP to my FAT32 drive. These ISO files do not copy correctly. Once the NTFS filesystem "or other foreign type" is inactive, everything is back to normal.

I took me awhile to figure out the culprit, because the files did not have to be on the foreign filesystem. It only mattered that the file system was active.

Some ISO files did work. It did not matter if it was a CD or DVD image; nor the Rock Ridge, Joliet, or UDF type. It had something to do with how empty space was written. In hex, if the empty space was written with "20" or "20,00", the image would not work. If the image used only "00" for empty spaces, it would work; even if stored on the foreign filesystem.

I don't know much about ISO structure. There is probably some specification difference. But I don't know what it is.

Anyway, it was weird and perplexing.

 

[jaclaz]

It is strange, but I am not sure to understand.

20 means "space" in ASCII and "20,00" means "unicode space", neither are used as "empty" (which remains 00 or 00,00), this is not related to ISO (ISO 9660) file format, AFAIK.

I have never seen a .iso where 20 is used for empty sectors (if this is what you mean)[1].

What would happen with a .txt file filled with spaces (either ASCII or Unicode)?

Or, if you prefer, what happens if you change the file extension from .iso to <something else>?

jaclaz

[1] the standard has the first 16 sectors (2048 byte each, i.e. 32,768 bytes) "unused" or - maybe better - "reserved for other uses", they are (for normal CD/DVD images) all 00's or (as an example) have a MBR or other data for "hybrid" images, though it could probably be fine according to the standard to fill this area with 20 hex, it never happened to me to find one such .iso image and it would anyway make very little sense.

 

Edited April 12, 2023 by jaclaz

[awkduck]

1 hour ago, jaclaz said:

20 means "space" in ASCII and "20,00" means "unicode space", neither are used as "empty" (which remains 00 or 00,00), this is not related to ISO (ISO 9660) file format, AFAIK.

"20" as a space in a string, is okay. That did not prevent loading. It was empty space using "20" or "20,00". Data gap, if you will. Normally found right after offset 8000. Large sections of "20", and sometimes additionally "20,00", around and after the Volume Label.

The use of "20" and "20,00" may be irrelevant. It may be related to some format/specification, that just so happened to allow for that. Perhaps just certain ISO/Burner software. It may be possible that the "20" and "20,00" could be fine; but as a coincidence, the only images that worked, didn't have them.

I'm guessing I could modify a working ISO, filling some empty chunk with "20"s, and it would still work.

Some Vendor strings, of non-woking ISOs,  "GEAR CD DVD RECORDIG" and " ULTRAISO". A working one is "Smart Storage, Inc".

The "Smart Storage, Inc" ISOs have a much cleaner layout.

[jaclaz]

It seems more like *something else*, offset 8000 is within the 32,768 leading (normally) empty bytes/16 sectors, there would be no reason to have part of them filled with spaces, around the volume label (I presume you mean the volume descriptor, i.e. "CD001") could well be something like a fixed length field filled with trailing spaces.

I have no idea what "Smart Storage, Inc." could be related to, Ultraiso is a known tool that may well be not fully compliant to specs (and actually I seem to remember a few issues with it when using it to  edit a .iso) I have no experience with Gear software.

An easy test that you can try is zeroing the first 32,768 bytes of a copy of a (non-working) .iso and see if it still misbehaves.

Also, checking the actual .iso files with isoinfo (from the cdrtools package) might (or might not) find some particular inconsistency in those not working files.

Cdrtools port to windows should be available here:

https://sourceforge.net/projects/cdrtfe/files/tools/binaries/cdrtools/

jaclaz

 

[awkduck]

2 hours ago, jaclaz said:

I presume you mean the volume descriptor, i.e. "CD001"

The Volume Label is at "8028" just below the Volume descriptor.

-this is not char count accurate, due to font spacing-
...........CD001..Win3
2                           
      SUMVOLLAB      
                    ......
...V!..!V.............
......................
......................
............".(......(
........{.............
..<- from here (80BE) to (823D) is "20" -> Vendor Name <- "20" till (832C)

Just as an example.

[SweetLow]

>I have NTFS support (Paragon)

There are different Paragon NTFS drivers. If you are using UFSD.VXD then try to use PNTFS.VXD

[jaclaz]

Yep, you are using hex offsets, 0x8000=32768=16x2048.

After the CD001 and up to relative offset 0xBE or 190 there is normally a huge (until the end of the sector) data field and many iso tools use space or 0x20 to fill it.

This should be "normal", the last 512+653 bytes should be zero, but before those the presence of 0x20's is OK,. see Volume Descriptors here:

https://pierrelib.pagesperso-orange.fr/filesystems/iso9660_simplified.html

http://www.dubeyko.com/development/FileSystems/ISO9960/ISO9960.html

It must be *something else*, not those values in the Volume Descriptor.

jaclaz

Edited April 13, 2023 by jaclaz

[awkduck]

20 hours ago, SweetLow said:

There are different Paragon NTFS drivers. If you are using UFSD.VXD then try to use PNTFS.VXD

Eventually, I'll probably try them both. I am using BIO95DRV.VXD.

It isn't important that I have access to ISOs, while the foreign formats are active. But, different circumstances are foreseeable.

I mainly wanted to point it out, in case someone has a similar issue. Then a probably culprit would be known.

9 hours ago, jaclaz said:

It must be *something else*, not those values in the Volume Descriptor.

I think we are both in agreement about this. I only differ, from your view, a little. I can not state is as fact, yet.

Keep in mind, all of these ISO work "normally", on this machine/OS. So there is nothing wrong with them. While the foreign partitions are active, the issue arises; at no "actual" fault due to any ISO specification/format.

At a glance, the difference has been mentioned. It may be that, at one time, there was once a specification difference. Perhaps not even an official one. But, specification difference may have nothing to do with.

When the foreign partitions are active, I can also not copy these files; they do not copy correctly. This is what, in my opinion, dismisses the "20" issue; since copying the files has nothing to do with ISO formats. The stunning part, is that the issue persists when the foreign partitions are not involve. Just being active on the system is enough. I should examine what does copy over, and compare that to the original.

It is also interesting, with the foreign partitions active, that the other ISOs copy and function fine. This would mean there is something about those files. ISO format may be, inadvertently, playing a part. I initially thought is was partition alignment, and that may still be involved. All of my considerations may be too narrow. Something else could be causing the issue; and BIO95DRV.VXD is only partially involved. I also considered hardware disk issues, but all ISO files otherwise work fine.

It is also strange, that the issue is only with ISO files. I thought it might be a large file issue. But, one of the working ISOs is over 4Gb.

All of this is really back burner for me. What I wanted the files for, is more important to me. There is an innate desire to flesh it all out. But for now, under my own time demands, I just copied the ISO content to folders.

[SweetLow]

1 hour ago, awkduck said:

I am using BIO95DRV.VXD

It is NOT the real file system driver. The real Paragon NTFS driver (UFSD.VXD or PNTFS.VXD) is located in %windir%\SYSTEM\IOSUBSYS

[awkduck]

36 minutes ago, SweetLow said:

The real Paragon NTFS driver (UFSD.VXD or PNTFS.VXD) is located in %windir%\SYSTEM\IOSUBSYS

You are right. It is PNTFS.VXD.

[jaclaz]

It could be well something connected with the various standards (original ISO, joliet, rockridge, but also iso-level and el-torito emulation or no emulation, multi-volume disks and what not).

There are many possibilities, and the actual references/documentations are far from being "crystal clear", as a completely unrelated example, it took us (actually rloew and cdob) years to find out that the part specifying size of floppy images for el-torito emulation had been read "wrong" by everyone and we could actually have 36 MB superfloppies, JFYI:

https://msfn.org/board/topic/152399-on-bootable-cds-floppy-emulation/

jaclaz

 

[SweetLow]

11 hours ago, awkduck said:

It is PNTFS.VXD.

Ok, could you make minimal .ISO with reproducible problem and put zipped image somewhere to download?

Edited April 14, 2023 by SweetLow

[jaclaz]

Still essentially OT, here:

https://msfn.org/board/topic/182116-winsetupfromusb-problem-installing-xp-on-legacy-system/?do=findComment&comment=1191406

is a small batch that allows for testing iso creation with different parameters and with "tricky" file names (but this won't touch other possible issues such as el-torito emulation or no emulation, multivolumes, etc.)

Though the batch could be useful (maybe) as a base to develop on, the issue remains that we don't actually (yet) know <what exactly> is triggering the misbehaviour, so it is not reproducible.

jaclaz

[awkduck]

I have gotten a CD ISO to work, that didn't before. I'm guessing that the reason it did not work, previously, was because of errors already tampering with the system. EDIT:{Or, I made an assumption.} But it still did not work, from the foreign partition. I had to copy it, which worked without corruption.

The still existing copying issue, with larger files, may be related to Rloew's File64. I can't really test it, when his API is disables. Maybe foreign partition to/from FTP. But File64 is not causing the issue with some ISOs working/not working , from foreign partitions. This still remains, with File64 disabled; both CD and DVD images. EDIT:{I guess it has been tested, since the still working DVD ISOs copied fine.

That "potentially" clears it up a bit. Now it is just an issue with why some do/don't work, from the foreign partition.

If File64 really was the cause, of file copy corruption, then if points back to some ISO specification difference.

I'll try to test ISOs, from opensource projects. Likely most will not work, and serve as examples. If I can't find a certain type, then I'll create one, and upload it somewhere. What will be harder, is making/finding one that works. I can't share the ISOs that work; especially here. I don't have the tools that made the original CD/DVD. I do have some of the tools that cloned them.

Knowing how to make working ones, might end up being useful.

If anyone has D-tool/VirtualCloneDrive, Paragon's NTFS PNTFS.VXD (Thank's SweetLow), and a USB NTFS partition, this could be tested. If others do not have the same problem, then it might even have to do with my NTFS partition's attributes.

Edited April 15, 2023 by awkduck

[awkduck]

Here are four ISOs. All failed, but work when copied. All are CD images.

FreeDos Live.

FreeDos Bonus.

ReactOS Live.

Aros Live.