Drag'n'drop on user command icon doesn't allow >50 files

[Fr0sT]

Looks like a bug, and I haven't found such issue mentioned already.
I created user command button on toolbar to quickly gzip all dropped files. If I drop quite many files (about 50 and more) x2 returns an error "access denied - you don't have permission". Lesser number of files works fine.
x2 pro 1.8.0.12 [Unicode]

[Kilmatead]

What precisely did you use as the User Command text?

I just did a few tests (under x86 and x64), dropping 250+ files onto different toolbar user-commands (including one for gzip, which I don't normally use) and had no problems, all files were processed accordingly.

Have you tried this on different sets of multiple files?  (As gzip seems to, by default, remove the original file and replace it with *.gz, this may be a R/W issue when it processes a particular file, causing the queue to terminate prematurely.)

[Fr0sT]

Well, I tryed as well with simple windows utility called by this command:
>C:\Soft\Utils\File\Hasher\Hasher.exe "$F"
and it fails on more than 22 files from path D:\Coding\Components\Common\ICSv7\Delphi\Vc32. However, the same files from folder D:\tmp\test were processed OK. Seems that maximum length of total paths is exceeded or something like that.
For test: create folder
"D:\this is very very very very very very very very very very very very very very very very very very very very very very long folder name" and place there about 15 files (F7-Enter-F7-Enter). And drop them altogether on any icon. That worked for me (I got an error described above).

[fgagnon]

A usable workaround should be to reference the path/filenames in compact form with $f  (vs "$F").   That will reduce cases where the argument string length overflows.

[Kilmatead]

Ok, I can more or less reproduce this, but it has nothing to do with x2 or dropping on icons, it appears there's an "overall argument string" length buffer which is to blame, which different applications handle differently.

For simplicities' sake, I created a test.bat file of:

Code: Select all

@echo off
echo %*
pause

...which simply outputs all the arguments passed to it (albeit slightly unformatted).  If all your test files are numbered, it's easy enough to tell what was passed.

I tried this from your "very very very..." (~140 character length) folder and for some reason 54 arguments get passed just fine, but 55 or above will not.

I then shortened to the folder name to 77 characters, and it seems 90 arguments are passed but 91 will not.

At 41 characters, it allowed 145 arguments, but not 146.

And so on.

There would appear to be some overall limit for a "passed arguments" string length, made up of all the filenames.  That said, this is wholly dependent upon the application being called: for instance, from the original folder (140 length) Notepad++ would accept 215 arguments, but not 216.  From a shorter foldername it had no problem with upwards of 251 (I got tired of hitting F7 ) - but test.bat (above) would not accept those same 251 files from the same foldername.

The same results also came from dragging/dropping the files directly from one Windows Explorer window directly onto the drop targets in another Win Explorer window, so it's independent of x2's milieu.

So, it's all very unscientific, but definitely "real world" results.

(One has to wonder why I punish myself like this.  An odd way to spend an evening, to be sure; I really must "get a life", as they say.)

And no, fgagnon, I'm not repeating those same tests with $f just to see what numerical effect it may have.  :evil:  Considering that it appears to be application "dependant" for what it can handle, there's no simple dictum of correlation.

[Mr.Pleasant]

I remember an entertaining discussion with you, Kilmatead, whether to use $S or $F in user commands. The trick was that one can use ">>" or "$$" to run the same command on each selected file (I had grown used to this), or on the other hand, ">" or "$" to run it on a bunch of files (I learned that from you). In the former case $F would be the token to use: ">> [path to application.exe] $F". In the latter, $S should be used: "> [path to application.exe] $S".
$S is usually better in queueing stuff, but is limited in number of files, as it is filled in like:

Code: Select all

> [path to application.exe] [path to selected file 1] [path to selected file 2] [path to selected file 3] [etc.]

... and this command cannot exceed a certain length (and now I'm still used to applying $F).
Now if you like you could abide your time summing up the characters involved , but I got a hunch that this and the limitation of the drag and drop commands are connected.

[Kilmatead]

I believe you're referring to the enjoyably lovely War Against Reason thread, which, as arguments go, I believe I lost to you.  Never let it be said I'm not gracious in defeat, you tulip-head, you.

...but I got a hunch that this and the limitation of the drag and drop commands are connected.

Actually, just playing around with this I think there is a bug in how x2 handles dropping on user-commands.  Your hunch is correct, but not in the way you think it is.

Using the test.bat code above I was playing about with $F, $S, $f, etc., and discovered that if the user command is invoked directly from the Customize -> User Commands menu the argument is passed as expected (according to the converted rules of the $-token).

However, if the same user command (in this case 'test.bat') is invoked from a filename being dropped on a toolbar button, the argument will always be the full path+filename, regardless if the user-command is $f, $F, or $S.

In other words, $-tokens are not parsed properly (or at all) in a drag/drop operation.

There really shouldn't be a difference between invoking from the menu, and invoking from drag/drop to a toolbar button.

This does smell like a bug.  I seem to recall something about this being mentioned once, but my brain has gone fuzzy - the changelog says toolbar user-command buttons first became drop-targets in [1.8.0.0] 25 November 09.

If nothing else, it means that any drag/drop tests I did above would show absolutely no difference depending on the token used.

[Fr0sT]

A usable workaround should be to reference the path/filenames in compact form with $f  (vs "$F").   That will reduce cases where the argument string length overflows.

You're right, but unfortunately this won't work with for example search results.

Kilmatead, wow, great investigation!

The reasons are clear now. I guess it's time to decide how to handle this issue .

First, I still don't understand why I get the error when I have this gzip command:
>>%X2DIR%\utils\gzip.exe --force --name --best --suffix .z "$F"
- >> is intended to run app file by file, isn't it? Maybe it's the same bug you described (different command handling from icon and from menu)? Moreover, menu execution seems buggy too: when I launched gzip command from menu on a 247 files set, it left 8 files unprocessed.

Second, when I launch command via >> on a plenty of files I get a huge heap of process instances launched. Is there a way to limit a number of simultaneously launched processes or to wait for their finish like DOS' start /wait command? If not, it would be nice to add it.

[Kilmatead]

Nothing overly helpful here, but I'll chime in anyway (rather hoping Nikos will pipe up with an explanation).

I can see why you weren't mad about $f with gzip... the filenames gzip outputs are mush!  Not a lot to be done about that, unfortunately.  Try using $S instead of "$F", and see if it helps, which it will do if called from a single folder, but search results are largely dependant upon filepath, so no help there.

I should mention I did have a successful test with $S and gzip where it successfully passed 814 arguments from the "very very... long folder".

I'm not sure why you're getting the error with '>>', except that the gzip site reports there is a patch for 1.2.4 which fixes

gzip 1.2.4 may crash when an input file name is too long (over 1020 characters)

I used the beta (1.3.3) when I did the tests, so that might influence it.

As for spawning multiple jobs... well, that's what it's supposed to do, x2 has no built in facility to be a surrogate batch handler - it wouldn't be that difficult to create your own batch file which would write x2's output into a queue which could then repass the arguments to gzip in a more linear fashion using whatever delimiters you choose.

Like I said, not overly helpful, but it's difficult to force-ably recreate this without the same folder structure you have (all the tests I try outside of the drag/drop $-token bug above work fine for me).  You might try temporarily testing with 7zip or WinRAR just to see if they handle the buffer-size differently - as I discovered above, different applications seem to interpret en-masse arguments in different ways.  I always use WinRAR and have never encountered a problem with it, though I use it's own shell menu command rather than piping via x2.

[Mr.Pleasant]

Ah yes, that War against Reason... Your memory serves you well. It ought to, when it comes to wars.

You write:

it wouldn't be that difficult to create your own batch file which would write x2's output into a queue

Not sure about that (or my imagination is lacking), especially since the '>>' commands don't give any tangible output, as does for instance the '$$' commands (which goes into '%temp%\x2tmpScript.cmd'). Not one that I know of, anyway.

Fr0sT, I don't know about gzip, but 7zip has an option in the command line syntax to refer to a text file which sums up all items that needs to be zipped. Something like this would be the command line:

Code: Select all

"C:\Program Files\7-Zip\7z.exe" a -tzip "[path and name of the destination zip file" @"[path and name of a text file containing paths and names of all the files to be zipped.txt"

If gzip has something similar, I'd imagine a two-step procedure for big zipping jobs: the first step being writing all selected files and their paths to this text file, and the second running the zip command referring to this text file.

[nikos]

Are you sure you want to use $F and not $S?