So, HERE's a way for Nikos to spend his copious free time

[Gary M. Mugford]

Hi there Nikos (and lurkers of the Scottish kind, plus others of fine heritage),

In the renamer thread of earlier this month, I opined Nikos would be engaging in a bit of a Sisyphean task. We ALL have our favourite renamers, including some who are happy with the existing one in x2. So, I suggested Nikos find other ways of engaging his time. Most of them were in his best interests. Now, I change my tune. I have a completely and utterly self-serving way to bide his time. And by self-serving, I mean ME.

Monthly, I find myself with a motley group of new files of limited number of file extensions. For the PURPOSE of this example, let's call them audio files with Wav's, MP3's, Flac's and Ogg's. They could just as easily be BMPs, JPGs and PNGs. SOME of the files are listed with more than one of the extensions. i.e. Anthem.MP3 and Anthem.Ogg.

Bringing up the Incoming folder, I would like VERY MUCH, to have a single button called Uniques. Clicking that button (or Mark menu choice) would select all unique base names. Where there are multiple choices of an instance of a unique file name, I'd like a preference to rule WHICH unique file is selected. Say I prefer audio file selection priority to be Ogg, Flac, MP3, Wav (leaving Anthem.Ogg selected and Anthem.MP3 not) and the picture format to be PNG, JPG, BMP. Once the Unique button has selected all of the files I would want to keep (more or less), I could use the Invert Selection menu choice and ditch the excess. I figure this would save me about an hour of cross-eyed inspection per month.

I WOULD BE WILLING TO PAY FOR THIS. Not a helluva lot. But pay, I would.

Now honestly, there's probably only three people in the world who need this. Me and two others. Unnamed. BUT there might be others out there. As many as six, maybe seven.

So, what's your favourite charity m'lord Kilmatead? Nikos?

GM

[nikos]

probably there's already a solution for this, see here
www.zabkat.com/x2h_5.htm#xa_fuzzy
you have to experiment a little with the fuzzy factor, try 80 or more

[Kilmatead]

Just because Nikos doesn't understand the "Amusement of the Niche" as an existential state-of-mind, I decided to whip up a little something that can do this for you properly, not relying on fuzzy-dice or rednecks or experimenting.

While it's easy enough to find common basenames, the complex part of your request is providing a reliable means for you to define a "Priority-Order" for extension-based matching.

To this end, I decided to add it as an ugly growth on the x2Filter utility since that already provided a framework of what I needed to conveniently do this.

Download this beta version (outdated link removed, see x2Filter) and make a toolbar button for it if you don't already have one. The user-command should consist only of "> x2Filter.exe", to keep things simple. I recommend adding the /Amalgam command-line switch at the moment as that will provide a quick reference to File-Groups (you'll see why this is important in a moment). You must use the appropriate bit-type for your version of x2, x64 ~ x64, x86 ~ x86. If you use the Ultimate version with an INI instead of the registry, be sure to add /INI to the line. If the INI file is not where the x2 binary is, then define a path in /INI="<path>". You also cannot have "hide extensions of known filetypes" checked within explorer options. Hey, live with it - no real user would ever have that ticked anyway.

For simplicity, you just select the "<Unique Basenames>" option from the popup menu and all the contents of the currently active pane in x2 will be sorted/highlighted accordingly where multiple independently matching basenames are found.

But how do you define the priority of selection to distinguish between MP3 and FLAC? This is the clever bit: you use the File-Groups feature in x2 itself to do it.

File-Groups are the defined filters in x2 which are found in Tools -> Advanced Options -> Global (Tab) -> File Types. For example, in x2Filter if you select {audio files} from the menu, only those files matching the extension-masks therein will be highlighted.

Normally x2 treats these definitions as dummy blind-lists, where the actual order of their definition is irrelevant.

I've changed this - x2Filter will prioritise multiple basenames by analysing the user-defined precedence in the order of extension masks:



Using the above order, given:

Nikos_The_Lazy_Parvenu.MP3 <-- highlighted automatically
Nikos_The_Lazy_Parvenu.FLAC
Nikos_The_Lazy_Parvenu.OGG

...but what if we want OGG files to have priority? Simple, just edit the File-Group filter so *.OGG comes before *.MP3 or *.FLAC:



Nikos_The_Lazy_Parvenu.MP3
Nikos_The_Lazy_Parvenu.FLAC
Nikos_The_Lazy_Parvenu.OGG <-- highlighted automatically

Simple.

I have added floating tooltips to the x2Filter File-Groups entries (in the popup menu) which will show their defined text masks, as a quick reference helper so you don't have to go into x2's advanced options each time you want to check the priority order of any given text. (You do still need to edit them the hard way in x2's editor, though, if you want to change them.)



Naturally, you can define as many groups as you want of whatever extensions you please - as long as matching basenames have extensions which are in the same group, their priority is ensured. But what if we add a different filetype that isn't in {audio files} to the mess? Well, groups are themselves based on priority, so if a file with the .FLATULANCE extension was in the list with the others, and you had a group called {Aardvarks} (which comes before {Audio...}) which defined '*.flatulance', then given:

Nikos_The_Lazy_Parvenu.MP3
Nikos_The_Lazy_Parvenu.FLAC
Nikos_The_Lazy_Parvenu.FLATULANCE <-- highlighted automatically
Nikos_The_Lazy_Parvenu.OGG

If an extension is not found in any group then it is given the lowest priority, and will not be auto-selected, unless you have a whole list of undefined extensions which all have the same basename - in that case the first-found will be given priority. Also, any objects without extensions will just be ignored, as will any objects which have wholly unique basenames (i.e., there is only one of each).

Now, this doesn't just work for one basename type - all of them in the active pane will be processed at the same time . Say you have a list like:

Nikos_Is_An_Octothorp.DOC
Nikos_Is_An_Octothorp.TXT <-- highlighted automatically
Nikos_The_Lazy_Parvenu.MP3
Nikos_The_Lazy_Parvenu.FLAC
Nikos_The_Lazy_Parvenu.OGG <-- highlighted automatically
Nikos_Wishes_He_Was_In_ZZTop.BMP
Nikos_Wishes_He_Was_In_ZZTop.GIF <-- highlighted automatically
Nikos_Wishes_He_Was_In_ZZTop.JPG

The number of highlighted entries will be displayed in the x2 status bar for convenience - all other entries will be specifically de-selected. As the <Unique Basenames> option only processes by "highlighting" matching set members, it's not a real filter per se, but it works well enough in this niche context.

I think that about covers it. As stated, this menu option in x2Filter is beta, and may or may not stay in it in this current state - this was more of a quick "proof-of-concept" for you specifically, just so you can see it working and offer recommendations or requests if I missed something. And just for fun, this version of x2Filter also clears any text leftover in the filterbar when <Unfiltered> is selected. You were complaining about that last time, as I recall.

(The results are not necessarily guaranteed, be sure to always check them yourself - it has been tested, but as you can see it's a little complex so weird things could still occur.)

Suggestions welcome. I was up kinda late last night working on this, so I might not have thought of everything.

[Gary M. Mugford]

I knew if I threw a little Irish love into the original self-serving request, I might find a solution if it didn't trigger Nikos' obsession with making x2 the all-knowing, all-seeing master of the file-management universe. Two shots, one direct hit.

Okay, the beta is downloaded and I have a freshly munged-together group of files I have already processed to experiment on. This will be something approaching a fun exercise. Will report the result.

Thanks. And do you have a charity that might like a (small) donation in your name?

GM

[Gary M. Mugford]

probably there's already a solution for this, see here
http://www.zabkat.com/x2h_5.htm#xa_fuzzy
you have to experiment a little with the fuzzy factor, try 80 or more

Nikos,

I know the love (there's that word again) you have for your scrap containers. Me .... not so much. I never fully understood them and abstained, beyond the flattening feature which works despite my ignorance. X2's been amazingly efficient even without using scrap containers (and tabs too). Pretty impressive when I think about it.

I am going to try out the Kilmatead solution because I actually understand it pretty well from the post and think I can see it through to success. AFTER that, I plan to go through the fuzzy match blog and see about your solution path too. One, I owe it to you to respect the time you spent thinking of and replying. Two, it's time. Even old codgers can learn new tricks. And sometimes, they SHOULD learn new tricks.

So, thanks. I WILL get back to you later. It'll probably be for another self-serving feature not usable for anybody else, but I will at least document my successes with scrap containers and fuzzy logic.

[Kilmatead]

I owe it to you to respect the time you spent thinking of and replying.

Ain't that always the way - I do all the work, and he gets all the credit! He didn't spend any time thinking about this, he just fobbed you off to a leftover blog from 1976 and went off to make coffee. Feck this for a game of soldiers - what's a guy gotta do around here to get a little respect?





(And Nikos doesn't like scrap-containers either - if you press him on it, even he'll admit multi-windows is a fiddly design.)

My charity is the "Save Kilmatead from Aardvark Flatulance Fund" (SKAFF). A worthy cause, if one best frequented at a fair distance.

[nikos]

you can trust kilmatead for an overengineered solution any day of the week... beer money to him then :cheers:

I WOULD BE WILLING TO PAY FOR THIS. Not a helluva lot. But pay, I would.

[Kilmatead]

you can trust kilmatead for an overengineered solution any day of the week

There are those still sloshing about in the primordial ooze claiming that bipedism was just a passing trend. Me and Isambard Kingdom Brunel (great name, that) will toast your plankton-fest from the nice warm shoreline, thank you mighty.

Over-engeneered indeed! Harumph! Clever little monkeys get the best apples!

[Gary M. Mugford]

(The results are not necessarily guaranteed, be sure to always check them yourself - it has been tested, but as you can see it's a little complex so weird things could still occur.)

I live in interesting times. I set up a user command and put it on a small toolbar with other filtering conditions. I even used an Irish flag as the icon:lol: I pretty well left the file types as is, having added an eBook filetype eons ago. Well, if not eons, ages at least. So, clicking the pretty little flag (with an obscuring U, forgive me), I ended up with this a full menu of previously saved filters, along with the expanded list of file types, then Unfiltered, Edit Definition, Unique Basenames and Highlight/Select.

I tried it on a set of music files. In the sample, I had three MP3's, and three OGG versions of the same basenames. ONE Ogg name was altered to be different than the MP3 it was created from. By my count, it made four unique basenames, two pairs of a prefered MP3 version with an excess OGG version. So, I clicked on Highlight and then Unique Basenames. The result: It selected the two MP3 files that were in pairs. It was smart enough to make the call, but failed to continue on and get the rest of the unique file names.

I switched to a folder of books where I knew there to be a lot of pairs of books. Comes from having both a Kindle and a Kobo (we are talking v3 of the former, v1 for the latter). Out of maybe 200 files, it found EACH mobi file (I love my Kindle, I put up with my Kobo) in pairs. And again, none of the none-paired unique file names.

I experimented without the /Amalgam switch ... which leads to the FileTypes being in a sub-menu. No difference. I AM running the x64 version. I have the 0303 version of the filter (no x32/x64 version was available at the time). Clicking on it brought up the usual menu with a file group sub-menu. Clicking Highlight and then one of the file groups does, what it always does. The same behaviour observed with 0306 when going highlight and then a File type group. The Uniques didn't work.

What was semi-interesting was switching to the x2 folder itself. Clicking highlight and uniques selected EVERY pair's unique name, but I couldn't discern a pattern of choosing. MenuHack.exe was selected over the au3 and ico file, ed2skin.png over the ed2skin.pngx, etc. Not clicking highlight led to a filtered view in all cases, but without the uniques even being partially selected.

So, where am I? An interesting tool with possibilities. Getting the pair right counts, but so would have getting the uniquely uniques. I turn the conversation over to you now.

Thanks for the effort, GM (now headed to bed, the meds are about to take hold)

[Gary M. Mugford]

you can trust kilmatead for an overengineered solution any day of the week... beer money to him then :cheers:

I WOULD BE WILLING TO PAY FOR THIS. Not a helluva lot. But pay, I would.

As a non-drinker of the Gods' mead, I'd rather fund water in Africa or even a bit of a school. But I'm dealing with an Irishman. I suspect a pint of SOMETHING will be involved. NOT that there's anything wrong with that.

GM

Now off to bed. The local baseball team is playing and I'd rather be asleep then put up with yet one more desultory effort.

[Kilmatead]

It "sounds" like it's doing what it was supposed to do: right now (Rome was not built in a day, nor Carthage destroyed the same ) it only concentrates on finding unique objects where there exist potential basename duplicates. It intentionally does not worry about single "uniques" (in the code I refer to them as "orphans") because that would just clutter up the display at the moment and they don't need any special handling.

Your original post described a circumstance where the multiple basenames were causing you trouble, and you wanted to select from between them - you didn't actually mention that you wanted all uniques (including orphans) selected too.

The /Amalgam switch adds (as you discovered) the File-Groups directly to the main menu - this lets you hover the mouse over them directly allowing you to check the contents (since those are what decide the selection-rules). It's merely for quick reference during debugging (deciphering why PNG comes before JUJUBEE, for example) , it wouldn't change anything else, and is optional. (Believe it or not, adding floating tooltips to what are glorified context-menus is not an exact science - I'm still working the bugs out. )

The bottom line: Ignoring the question about orphans for the moment, does the multiple-basename auto-selection work as you expected? Remember that the rules are derived on where the extensions are defined within the groups (both horizontally and vertically), so it may take some careful thinking to find the culprit of unexpected results.

Assuming the answer to the above to be more or less "yes", I'll activate the detection of orphan-highlighting as well, and we'll see where we stand. Like I said, orphans are ignored at the moment so as to keep that clutter out of the way (since 95% of folders are usually full of unique orphans to begin with.

I shall tweak for a bit then post the fix. And for those with dirty minds, I said tweak, not twerk, so don't get your knickers in a twist. Trust me, nobody wants to watch a 46 year old guy twerk. Yuck.

Addendum: As to the curious results in the x2 folder, consider that some of the files there have "double" extensions, such as xplorer2.exe.manifest, etc. My routine only checks the first "." and considers anything after that to be a full extension, thus ".exe.manifest" not just ".manifest". Maybe I should parse from the other direction. It's also a loose comparison at the moment for simplicity.

(It's funny how even a thousand miles away I can still hear Nikos sniggering quietly to himself with a self-satisfied gleam in his eye. Since this is my week for mainstream references, as Han Solo said: "Laugh it up, fuzzball". )

[Kilmatead]

Ok, x2Filter_Beta2 is up (outdated link removed), so try this one when you're ready. Ignore, for the moment, the results reported in the x2 status bar, the counters are all wrong since orphans screw everything up, as is the way with orphans - Fagin deserves our pity, not our scorn. Anyway, harmless for now.

I should note that objects without extensions are still intentionally ignored and should never be auto-selected (such as folders). I'm undecided on how these should be handled, since I can't actually distinguish between folders and non-folders in the display at the moment. Long story.

Added: Selection/highlighting of all unique nomenclature: orphans and basename conflicts alike
Fixed: Menu item name changed from Unique Basenames to the more grammatically incorrect "Select Uniques"
Fixed: Extension-parsing was reversed leading to unexpected compound extensions being processed unpredictably
Fixed: Tooltips could display incorrectly depending on non-amalgamated menus (still mouse-based, not rectangle-based [for keyboards], but at least the display content order is accurate now)

[Gary M. Mugford]

Ok, x2Filter_Beta2 is up, so try this one when you're ready. Ignore, for the moment, the results reported in the x2 status bar, the counters are all wrong since orphans screw everything up, as is the way with orphans - Fagin deserves our pity, not our scorn. Anyway, harmless for now.

I should note that objects without extensions are still intentionally ignored and should never be auto-selected (such as folders). I'm undecided on how these should be handled, since I can't actually distinguish between folders and non-folders in the display at the moment. Long story.

Added: Selection/highlighting of all unique nomenclature: orphans and basename conflicts alike
Fixed: Menu item name changed from Unique Basenames to the more grammatically incorrect "Select Uniques"
Fixed: Extension-parsing was reversed leading to unexpected compound extensions being processed unpredictably
Fixed: Tooltips could display incorrectly depending on non-amalgamated menus (still mouse-based, not rectangle-based [for keyboards], but at least the display content order is accurate now)

M'Lord,

Everything works perfectly on the music folder and on the books and x2 folder based on my preferential rules, although I'm not sure WHAT rules were applied to the x2 folder. If I HAD my druthers, I think you SHOULD only take the first extension. So yes, the exe should be selected and the exe.manifest not. Just my two cents.

Backtracking to your prior message but one, I will point out a relevant quote from my original missive: "Bringing up the Incoming folder, I would like VERY MUCH, to have a single button called Uniques. Clicking that button (or Mark menu choice) would select all unique base names. Where there are multiple choices of an instance of a unique file name, I'd like a preference to rule WHICH unique file is selected." The second, emboldened sentence said I wanted all unique basenames. And the third sentence made mention of a decision tree in cases where a choice would be made as to which would be in the highlighted/selected list. Thus, I BELIEVE I WAS explicit in my dreams and desires and wanton lust for a solution.

But I was obviously wrong.

And given the solution I just saw for a job that takes me about 15 minutes a week (obviously, with a special set of work files, not music or books), I will cheerfully admit I WAS WRONG. I WAS WRONG. I WAS WRONG. And you were right [G]. The Uniques button will save me about an hour a month and a CONSIDERABLE amount of eyestrain. I am a very happy person. And a very GRATEFUL person. If you want to talk via direct email (I believe you have access to my details), then we can get down to the serious task of karmic recompense.

Thank you very much. In a weekend of not much else going right (although the local heroes of the ball diamond did, in fact win. That win is under what appears to be a successful protest in the making, Naturally), that this works brings is a very good thing. A ray of sunshine, if you will.

Best regards, GM

[Gary M. Mugford]

(It's funny how even a thousand miles away I can still hear Nikos sniggering quietly to himself with a self-satisfied gleam in his eye. Since this is my week for mainstream references, as Han Solo said: "Laugh it up, fuzzball". )

Never give away extra information when wanting somebody to WORK for the reference. The good captain said, "Laugh it up, fuzzball!" Then, either Nikos gets it or he doesn't. And it's always more fun when one can lord over another with a famous quote, since those are SOOOOOO much more difficult to look up on Google.

GM

[Kilmatead]

Everybody's a critic of my pop-culture quotes this week - ah well. Nikos wouldn't pay attention to such things anyway, he only perks up when I use words like tyrannodidaskalos to describe him. He gets a kick out of that.

I believe you have access to my details

I am neither moderator nor employee nor clairvoyant nor NSA flunky - minions like us are stuck with the PM system. Anyway, recompense for this sort of thing is not really necessary (men should be paid for real work, not this white-collar nonsense). I'm also not %100 happy with the results myself - the algorithm can still be spoofed by accident and is somewhat held together with sticky-tape: put a "." in a folder-name and you'll see what I mean - I haven't found a reliable workaround yet. It's going to stay as a beta-only option for a bit longer to work (at least) some of the kinks out. Just because something works under test conditions doesn't mean that it won't blow up a nuclear plant when you're not looking.

In the case of multi-extension objects, I decided to follow x2 (or the shell?) as it decides that the last one is pertinent (the Extension [S] column shows .manifest for .exe.manifest), so that's that. It could also do with an option to disable all such rules and just dependably select the "first" of any given set, but I was hoping not to have to add a configuration menu to x2Filter (the one in SetFolderColour is insane), and that sort of nonsense even I admit always gets over-engineered in the end. Also, it doesn't properly handle extensions defined with embedded wildcards. Not the end of the world, but I'm a stickler for esoteric details, and if you're going to over-engineer something... by God, do it right!

Clicking that button (or Mark menu choice) would select all unique base names

Yeah, I sort of got fixated on the basename part, as that was always going to be the sticking point. Forest for the trees. (You also don't need to click "highlight" first as you keep intimating - the Select Uniques entry can only ever work that way (you don't have a choice) - it can't operate as a "real" filter. Blame nikos for not providing us with an API we can actually use.)

[As we watch the Google-Search hits for tyrannodidaskalos go through the roof... Rave On, John Donne...]