SizeES: A Plugin for Fast, Persistent FolderSizes in x2 via Everything Search

A collection of especially useful xplorer² topics and ideas. New users may find it helpful to look here before searching the other forums for information. >>>>>> Please post new material in the relevant forum. (New stuff posted here will be removed.) Thanks. -fg-

Moderators: fgagnon, nikos

User avatar
johngalt
Gold Member
Gold Member
Posts: 607
Joined: 2008 Feb 10, 19:41
Location: 3rd Rock
Contact:

Re: SizeES: A Plugin for Fast, Persistent FolderSizes in x2 via Everything Search

Post by johngalt »

First of all, my apologies for such a late, late reply. My years played catch up with me in a not-nice manner.
Kilmatead wrote: 2024 Jul 10, 09:12
johngalt wrote: 2024 Jul 09, 18:58 Does this make this version the bastard halfling that is still better at everything than the pure stock version?
When I was young we used to steal those old fashioned Mercedes-Benz bonnet-emblems off the streets and use them as a poor-man's brass-knuckles, or at least we pretended as much. This version of the plugin is what the kids from your neighbourhood will use to adorn their keychains and mantelpieces for years to come. And if, by chance, one of those scallywag hoodlums should find a way to use it to become the street's newest discount dentist, then, yeah, this version is indeed the bastard halfling I would want guarding the integer representations of my folders. :D
Noted. I'll be sure to keep an eye on it then!
Kilmatead wrote: 2024 Jul 10, 09:12
johngalt wrote: Or is that a perennially 56 years old birthday that you've settled on celebrating for the rest of your life?
Around 1.77 trillion milliseconds ago back in nineteen sixty-and-eight during the Ides of March, little Kilmatead came into the world with a dangling Gauloises off his lips and a bedraggled (yet still stylish) Fedora perched atop his head.
Wait, what? You preceded me 3 years, exactly, to the day?

I have friends who are Ides babies from '69, '67, and '73, but never have met a '68 Ides baby. Until now.

You'll find this part hilarious, in that my sister was born some years later on the anniversary of the Kennedy assassination. My only sibling, and we were both born on dates of infamy, both being assassinations, no less.
Kilmatead wrote: 2024 Jul 10, 09:12 At least in my imagination he did. In reality I should think he was spanked, he spit, he cried, and promptly sought out the nearest teat. That in later years he misremembered that teat as belonging to La Lupa Capitolina is, of course, purely the product of whimsy, conjecture, and youthful exuberance. Don't all men reinvent the past to fit the form they are most pleased to inhabit?

That I raised Eric Hoffer to a hero's pedestal was not only to honour a stevedore philosopher's contrition to the hard labour of hard men, but also to draw attention to the last line of his wikipedia entry, which happily spits in the face of the Ivory Tower biographies of lesser-intellectuals:
To this day, no one ever has claimed to have known Hoffer in his youth, and no records apparently exist of his parents, nor indeed of Hoffer himself until he was about forty, when his name appeared in a census.
My continuous surprise at the many people who desperately cling to the identities with which they were assigned and burdened at the whim of officialdom, is always balanced by the multitudes whose imaginary claims of "ages not their own" parade quietly through their own biographies. I've never been bothered by getting older - I neither embrace it nor fear it - but I do like to study it. From the daily morning pains engendered by my workaday 8-tonne excursions, to the entropy which inhibits a willingness of the eyes to see unaided, the process is probably not one to be challenged or retarded by chemicals from, and ablutions to, the "miracles of modern medicine". Rather, I'd like to strive to a mindful stasis, remaining acutely aware of the diminishing sunsets and the lessening grip of mental disciplines pervading my days... not with regret, or sad refrain, but instead with the child's surprise and curiosity as to why it is he will never see "that" blade of grass again, or coax the same cicada up the same tree as he remembers he may have done in his yesterdays.

So yeah, in 3 years time I'll have 16-tonne of gravel delivered to your door, and I'll pit my Hoffer to your Rand, and we'll see which of our dead geriatric idealogues can fling the dark stones of their unrepatriated youths the furthest in mind and space. :wink:

If nothing else, you'll get a new driveway out of it, and I'll just have another twinge in my back to complain about the next day. Such is life. :)
Brilliant exposition on both the burden of identity that some, if you'll pardon the pun, burden themselves with, but also on the the part about chemicals, in that, other than my on-going love of the 'eins' - caffeine and nicotine (yes, we spell things boorishly sometimes), I cannot agree with you more about the massive need I have to avoid overt ingestion of all things branded as 'medications'. It's getting harder to do so, but I'm happy to say that I still am not on any 'prescribed' medications, and as my last physical investigator mentioned, I'm as healthy as a horse. It was touch and go for a bit last year, but I suspect it was more latent effects of COVID paired with a violent resurgence of my allergies, than anything else, and yet I still received a nice bill of health, my last few weeks notwithstanding.

As for the gravel - if you could see my so-called front yard, you'd note I have no need for it, sirrah, as I have carefully crafted my very own gravel-laced planting - taking over my entire front 'yard', as it were (with 147 plantings, last I counted).

But the visit itself will be cherished, for sure. We'll see if Rand is all she is cracked up to be, for sure. I've less doubt about Hoffer than her, to be frank. Or, perhaps, more appropriately, how her ideas are being (mis)used in modern society by fools that don't actually understand what she stood for in the first place.
Image
Kilmatead
Platinum Member
Platinum Member
Posts: 4666
Joined: 2008 Sep 30, 06:52
Location: Dublin

Re: SizeES: A Plugin for Fast, Persistent FolderSizes in x2 via Everything Search

Post by Kilmatead »

  • Are you tired of living your life a quarter-mile at a time?
  • Are you overwhelmed with dread, doom, and fatigue every time you hear someone use using old-fashioned phrases like "in the blink of an eye" or "faster than snapping your fingers"?
  • Do you have a nagging feeling that you should toss out your old George Winston records and embrace your inner speed-metal demons?
  • Do milliseconds just feel too slow - and indeed, passé even - the older you get as you uncomfortably reflect upon the fewer you have left?
  • Does your significant other constantly inspire you to wonder, really, just what is an acceptable amount of time to waste furiously waiting for foldersizes?
  • Do you conclude, as surely you must, when you regularly glance longingly at that yellowed portrait of Madame Curie you secretly keep in your night-table, that perhaps the half-life of polonium-214 would be a more appropriate amount of time to wait?
If you answered yes (or heck, even no?) to any of the above questions, you might want to rethink your life's ambitions. :wink:

In the meantime, we have v3 of the SizeES plugin! :D

Code: Select all

Changelog for 3.0.0.0

Added: Everything SDK3 (aka 1.5 SDK) integration: IPC uses Pipes (if available) instead of Messages
	- Reduces average folder query/response time from under 3ms to ~164μs
	- To reduce bloat, the SDK source included is a highly edited subset of the original (only code necessary for this plugin)
	- Suggested use of Everything Search version 1.5.0.1387a or later: https://www.voidtools.com/forum/viewtopic.php?t=9787
	- If the previous install set /alpha_instance=0 (for this plugin), this must now be reversed [=1] to take advantage of pipes
	- If Pipes are not available (ES 1.4 and older 1.5 builds) IPC will fallback to using messages

Added: INI option 'ForceCopyData[=0]' to force the use of Messages and ignore Pipes (mainly for debug testing)
Added: The Status.ES [X] column shows current IPC-type: e.g. '<Ok> (Pipe)' or '<Ok> (Msg)'

Fixed: Zero-size folders could not be distinguished between "not indexed" and "legitimately empty" (now they are)
Fixed: By default, errors on Root-folders were not deferred while non-Root were (long boring explanation would normally ensue...)
Fixed: Regression where disabling the plugin via the INI setting didn't always defer folders

Moved to GCC 14.2 LLVM 19.1.5 MinGW-w64 12.0.0 MSVCRT toolchain: https://github.com/mstorsjo/llvm-mingw
Refactored for -std=c23
TLDR: Reduces average response time to ~164μs.

Those aren't your grandfather's milliseconds, those are micro-bleedin'-seconds, or millionths of a second. In other words, yes, the half-life of polonium-214! :roll:

I can't take much credit for this - Everything Search now uses pipes under the bonnet for IPC instead of (the slower) WM_COPYDATA, so it's just a wee bit faster than your average explosive diarrhoea. Anyway, I added this to the plugin (the piping, not the plumbing), so... well, bon appétit!

As with all ironies of computer-type-stuff, if you had installed a previous version of SizeES and went through the steps to adapt it to the newer alpha-version of Everything Search, that must now be reversed (to benefit from pipes), so:

Open Everything Search (be sure to update it), and in the main search-bar, type:

/alpha_instance=1 <Press Enter>
/restart <Press Enter>

...and you're good to go. The recommended version of Everything has been bumped to 1.5.0.1387a or later. The plugin will still work with older versions (including the much older 1.4), but it will "fall back" to using the slower IPC instead of the nicer newer betterer stuff if it can't detect a pipe-server. Use the Status.ES [X] column to inform you which one it's using. If you don't see "<Ok> (Pipe)" writ large in the mirror of your skull as you brush your teeth in the morning, you're just not one of the cool kidz and should hang your head in shame and go back to bed.

I don't anticipate any further major updates to this plugin, barring pertinent changes to the (still-evolving) ES SDK source-code, so this can be considered a final version, more or less. Point-updates might ensue as needs or inspiration strikes. The download link in the original post has been updated, or just use this one.

Enjoy.

Incidentally, in the interests of stability I implemented the SDK in such a way that the client is created and destroyed for each individual folder-query, which sounds slow and impractical... while it is possible to leave a connected pipe "open" between calls to halve the times to around 60μs (yes, I tested it), but that leads to instability in reclaiming a link if either x2 or ES is closed and/or restarted during use. Given that possibility, I went for the more robust option, since most humans wouldn't be able to sense the delta unless they were on cocaine, which would also most likely be rather detrimental, hindering their ability to use x2 productively. The disappointment of living with sobriety asks, "Which is the lesser evil?" :wink:

(If there actually are a multitude of cocaine-obsessed productivity-based x2 users out there, we would love to hear from you. This is a safe space. We don't judge. We celebrate. They say sharing is the first step.)

Incidentally (Part II), The folder-queries are adapted to WSL (Windows Subsystem for Linux), where case-sensitivity matters. I don't know if x2 itself adapts to that or not, but at least we're future-proofed for when the end-times commence with all due heralding and pomp as we anticipate the crushing fall and inevitable overthrow of the current order. 8)

Hang the rich.
Tuxman
Platinum Member
Platinum Member
Posts: 1633
Joined: 2009 Aug 19, 07:49

Re: SizeES: A Plugin for Fast, Persistent FolderSizes in x2 via Everything Search

Post by Tuxman »

Kilmatead wrote: 2024 Dec 05, 14:37 this can be considered a final version, more or less.
I nominate you for the Nobel Prize. Finished software has been a myth for so long!
Kilmatead wrote: 2024 Dec 05, 14:37(If there actually are a multitude of cocaine-obsessed
I'm all ears!
Kilmatead wrote: 2024 Dec 05, 14:37productivity-based
Oh, ok. :(
Kilmatead wrote: 2024 Dec 05, 14:37The folder-queries are adapted to WSL (Windows Subsystem for Linux), where case-sensitivity matters.
Does it work with NTFS case sensitivity?
Tux. ; tuxproject.de
registered xplorer² pro user since Oct 2009, ultimated in Mar 2012
Kilmatead
Platinum Member
Platinum Member
Posts: 4666
Joined: 2008 Sep 30, 06:52
Location: Dublin

Re: SizeES: A Plugin for Fast, Persistent FolderSizes in x2 via Everything Search

Post by Kilmatead »

Tuxman wrote: 2024 Dec 09, 17:18 Does it work with NTFS case sensitivity?
Interestingly enough, according to the lord high executioner himself...
The new Everything3_GetFolderSizeW API will lookup folders by case first.
A case insensitive search is performed if a folder is not found.
Diacritics are always matched.
In a largely case-insensitive environment like Windows, it would seem an odd choice to explicitly state that the initial search is actually conducted case-first... which suggests he knows something about the secretive high-cabal to which you worship that I do not. Therefore I can only assume it applies to all disc environments where such distinctions are possible.

Even more curiously, from my culling of the API source code, it appears that Everything handles all internal data explicitly and exclusively as UTF-8 (going out of its way to convert wide-chars into it), which would also be an odd choice if you didn't anticipate dealing often with the non-UTF-16 (wchar_t) world.

Again, I can't speak for whether x2 takes these things seriously or not ("nikos is as nikos does"), so your successes and failures may well be his epitaph etched in some of that leftover Notre-Dame stone. :wink:

Perhaps these drugs have a larger market-availability than previously thought. One can only hope.
Last edited by Kilmatead on 2024 Dec 09, 19:34, edited 1 time in total.
Kilmatead
Platinum Member
Platinum Member
Posts: 4666
Joined: 2008 Sep 30, 06:52
Location: Dublin

Re: SizeES: A Plugin for Fast, Persistent FolderSizes in x2 via Everything Search

Post by Kilmatead »

<Post Edited out of Existence>
Tuxman
Platinum Member
Platinum Member
Posts: 1633
Joined: 2009 Aug 19, 07:49

Re: SizeES: A Plugin for Fast, Persistent FolderSizes in x2 via Everything Search

Post by Tuxman »

Kilmatead wrote: 2024 Dec 09, 18:59 Everything handles all internal data explicitly and exclusively as UTF-8 (going out of its way to convert wide-chars into it), which would also be an odd choice if you didn't anticipate dealing often with the non-UTF-16 (wchar_t) world.
One of the countless joys of developing software that is orientated towards compiler specifics via low-level development is the manifold differences that sometimes even occur between different versions of the same compiler. A wchar_t has at least 8 bits, but perhaps more. I would now have to reverse-engineer everything in order to obtain reasonably reliable information about which compiler is used there. So in short: If you squeeze EVERYTHING (hehe, you understand?) into UTF-8, then it should be possible to work compiler-agnostically with a handful of #ifdefs.

Thankfully, the days of 7 and 18 bit architectures (outside of banks, universities and the military) are over. :)
Tux. ; tuxproject.de
registered xplorer² pro user since Oct 2009, ultimated in Mar 2012
Kilmatead
Platinum Member
Platinum Member
Posts: 4666
Joined: 2008 Sep 30, 06:52
Location: Dublin

Re: SizeES: A Plugin for Fast, Persistent FolderSizes in x2 via Everything Search

Post by Kilmatead »

Tuxman wrote: 2024 Dec 09, 20:19 A wchar_t has at least 8 bits, but perhaps more.
Well, except in WinAPI where it's guaranteed to be 2-bytes - mayhap only 8-bits of the 16 available are actually used, but Windows will kick your caboose to the next track over if you confuse variable-byte unicode for UTF-16. I don't think he's targeting compilers (based on his API providing named functions for ANSI/WCHAR/UTF8 explicitly), I think he's targeting a database format standard, probably in the interests of size-compression. What makes it interesting (to me) is that he actually wrote his own bidirectional wchar_t <-> UTF-8 <-> ANSI conversion functions (surrogates, et al) rather than trusting library functions to do it, so he must have had some elaborate grand scheme in mind.

I know he's using the VS compiler as he used some VS/MS-only extension-notations in the API code which I had to work-around to be able to compile it via GCC/LLVM in the first place. In the end I needn't have bothered as I just decimated 95% of the code anyway (including the stuff I "fixed"), so it was an afternoon of two steps forward one step back. If he was considering compilers other than MS's, he wouldn't have done that.

That being said, I have on more than one occasion used non-standard extensions in code just to frustrate and alienate VS users - but I'm a cruel goose that way. He actually wants people to easily use and adapt his code. :D
Tuxman
Platinum Member
Platinum Member
Posts: 1633
Joined: 2009 Aug 19, 07:49

Re: SizeES: A Plugin for Fast, Persistent FolderSizes in x2 via Everything Search

Post by Tuxman »

Kilmatead wrote: 2024 Dec 09, 21:33 I know he's using the VS compiler as he used some VS/MS-only extension-notations in the API code which I had to work-around to be able to compile it via GCC/LLVM in the first place.
As far as my (albeit aging) memory does not deceive me, the VS compiler has been (a version of) LLVM for a few years now and is therefore reasonably standard-compliant. Of course, the same cannot be said of GCC, which, to put it with the necessary restraint, has been conspicuous for about two decades, above all with overzealous ‘optimisation’ (i.e. error generation). Are there any subtle nuances in the depths of the obvious nastiness with which Microsoft usually does its business that I should be aware of?
Tux. ; tuxproject.de
registered xplorer² pro user since Oct 2009, ultimated in Mar 2012
Post Reply