blog: long filenames ahoy

[nikos]

here's the comment area for today's blog post found at
http://zabkat.com/blog/long-8.3-path-names.htm

[Kilmatead]

A lot of people miss the fine-print regarding the \\?\ prefix - while the overall length is indeed kicked up to 32767, the limit for any given component within that path still remains MAX_PATH. Who's going to parse that gotcha every time in the name of total-security?

Quoting performance gains for dropping 8.3 names sounds reasonable...

A similar argument (with questionable yields in discernible performance benefits) arises with one of the more common tasks: recursive folder searching. Instead of using the plain old FindFirstFile (etc), the most popular is to use the Ex version along with FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH. Essentially FindExInfoBasic stops the function from querying the 8.3 name when populating the WIN32_FIND_DATA struct. You can spend a lot of time reading different people's attempts at quantifying such a difference... usually the end result is to just shrug and say "Well, it can't hurt, at least".

While MAX_PATH is slightly claustrophobic these days, 32767 is complete overkill. Regardless of bad-practice, at least you can conveniently tuck a quick MAX_PATH buffer allocation on the stack without anyone noticing the extravagance too much... imagine all the L-drivers who'll just up the buffer size without realising the difference. Worse, memory management allows these kind of errors to go undetected. Just because you manually allocate a million bytes doesn't mean you actually sequester a million bytes right then... you only get it "when you need/use it", thus allowing it to pass basic testing unnoticed. Embrace the heap.

I vaguely recall reading somewhere that when MS ported parts of the traditional API (and .NET malarkey) into their atrocious UWP nonsense, they just cheated and obfuscated the \\?\ away, claiming "oh yeah, we natively support long paths now". Don't quote me on that, but it's exactly the kind of thing they'd do... which is Win10 in a nutshell.

And, like you say, since many of these things don't apply to PIDL-related API's, how is MS going to get around that one? Messy.

[nikos]

in an unrelated point, I'm pissed off that all my win7 virtual machines and even my old laptop are basically screwed with automatic updates. It just consumes 100% CPU and updates nothing. All the remedies with windows update troubleshooter and what have you had no result. The only solution is to turn off auto updates

you could easily imagine that's just malice from MS that intentionally screwed up win7 to get people to update to win10, fek!

[Kilmatead]

Yeah, I noticed that too... one day a month (that notorious second-Tuesday) an MS process sits there sucking 14% CPU (of a quad-core) for about an hour before deciding that there's a couple of updates available. Once you let it do its thing it goes away, but that never used to happen on new installs.

I'm fairly convinced that it's just parsing the currently installed .NET CAB's looking for differences; unpacking those things sequentially eats up cycles like crazy.



Considering that Win10 is actually losing market share now, they're revealed as the tossers we knew them to be anyway.

[nikos]

no, in my case it just eats up all CPU and does nothing, unbelievable idiots

[Tuxman]

they just cheated and obfuscated the \\?\ away, claiming "oh yeah, we natively support long paths now".

\\?\ paths are native. They're usually hidden from most file managers though.

[Kilmatead]

I never said they weren't native (if anything they're more native-than-native in the UNC sense of the term) - except that MS is now claiming them as a new addition to the LFS, which is just the same old obfuscation as it's always been.

They're not hidden from file-managers, they're hidden by file-managers, amongst other programmes which employ the necessary prefix. I'm using "native" in the sense of "one less thing the end user sees as a work-around". Probably why Win10 treats them as a disabled-by-default option... much like claiming mould on a tennis-shoe as an environmentally-friendly "feature". Users will lap up any old soured-milk if it's sold right. (In fact, if you're Swedish, they already do, those Filmjölk-loving weirdos.)

[nikos]

apparently what's happening now is that an application manifest just informs the system that the app is "long path aware" so \\?\ prefix isn't required at all

[Kilmatead]

Yeah, but in the same way as setting DPI-awareness in the manifest doesn't actually make the application itself DPI-aware (it's just a flag that says "don't add our own rubbish solution"), it (as a declaration) probably doesn't actually do anything other than spool up the UWP API's as substitute. The developer still has to do the real work if they want it done correctly.

[BezantSoft CTO]

Uuuughhh!
I always hated the way that 8dot3 names were mangled. I once had an app that generated long names that were IMHO reasonable, but a old tool I was using wanted shortnames. The disambiguation for the shortened name was no fun at all to build scripts for.

What I want to know is does the non-appearance of shortnames on non-system volumes affect in any manner symlinks, hard links, and junctions? More and more SSD's are out there, now-a-days, and if you use the symbolic links and the like to move the "real" file contents to another volume that is non-system and non-solid-state, are there any ramifications, or have I not thought this through enough?

Bz

[nikos]

I don't know how symbolic links handle long paths. You'd have to experiment with a really long path and create a link to it and see how it is stored in the file (a hex viewer will help)

[Kilmatead]

Ok, let's take a deep-breath here... ready? Let us sally forth unto the breach once more (I didn't learn this stuff the hard way for nothin')...

Symbolic link semantics are a little different from Junctions or Mount points in that they can contain relative paths (within a single volume), or absolute paths (cross-volume), stored via the REPARSE_DATA_BUFFER (or more specifically REPARSE_GUID_DATA_BUFFER structure) at FSCTL_SET_REPARSE_POINT creation time in the SubstituteName-data subreference portions.

SubstituteName paths are limited to 32,760 characters (which tells you something), subject to the 255-character path-component limit and are often just UNC-prefixed GUID's. Since a relative path is variable text, that's what you set/get (following the dot-convention), whereas GUID's and absolute paths are SMB and streamname friendly.

If you pursue the forensic method of gleaning reparse targets (a hex editor, Nikos? Really? ), you'll find that DeviceIoControl always returns SubtituteNames (ignore the PrintName-data, that's just for prettification purposes and can be blank) in (absolute-path) \\?\UNC\server\share (or plain old \??\ -> \\?\) prefix format (whether you want it or not). Relative paths are returned... well... just relative (dot-double-dot), as you'd expect.

As an interesting aside, reparse-point target paths may themselves contain up to a maximum of 31 reparse point (subcomponents) allowed in any one particular path. Exactly why anyone would actually do that is beyond me, but hey, it's nice that the designers of NTFS had crazy people like me in mind when designing this mousetrap maze.

So, long story short, as long as the Symbolic Links were not originally created with 8.3 paths (which is highly unlikely), the actual reparsing done by NTFS (at browsing time) follows UNC and long-form LFS. Or, better yet, just embrace a GUID proper to be sure.