Page 1 of 2

declone: now finds duplicates

Posted: 2021 Jun 30, 15:54
by nikos
build 000.7 alpha available for your brutal testing!

DOWNLOAD: www.zabkat.com/test/dc_alpha.zip
duplicates scan engine is fully functional (except searching file contents and approximate mode)

thanks for your feedback

Re: declone: now finds duplicates

Posted: 2021 Jul 01, 10:39
by nikos
do you see the little clock icon on the progress dialog?
Image
if you click on it it's meant to turn into an arrow (resume scan)

Re: declone: now finds duplicates

Posted: 2021 Jul 04, 17:26
by Gandolf
Not a great deal of use to me yet since I always want to search contents!

I also want to set the extension. I could want to search ONLY jpg (and jpeg) but NOT png, gif or any other Photo extensions. Hence, Photos is not suitable.

Re: declone: now finds duplicates

Posted: 2021 Jul 05, 04:44
by nikos
if you set it to EXPERT mode (use the options or press CTRL+SHIFT+E keys) then you can edit the file types, so set the pics to be just JPG

Re: declone: now finds duplicates

Posted: 2021 Jul 05, 06:17
by Gandolf
Yes, but that is not a solution.

The next search I might want to do is for png or gif, so I have to again edit the Photo file types. It is much easier to have a drop down list of file types as TC has or to be able to filter the files types before checking for duplicates, as I do in x².

Re: declone: now finds duplicates

Posted: 2021 Jul 05, 07:50
by Gandolf
I've also realised that htm and html are not in any list.

Re: declone: now finds duplicates

Posted: 2021 Jul 05, 10:00
by nikos
declone is targeted to the "simple" people so too much flexibility isn't good for them :)
there are ways to change things but the main interface is super-plain

Re: declone: now finds duplicates

Posted: 2021 Jul 05, 10:20
by Gandolf
I'll take it as a compliment that you don't regard me as a "simple" person!

I'll stick to x² and TC for my duplicate finding. They do the job very well and there is not much to choose between them regarding speed.

Re: declone: now finds duplicates

Posted: 2021 Jul 05, 11:10
by nikos
declone is going to do approximate duplicate matching (similar not identical files), which xplorer2 doesn't do (not sure about TC)

Re: declone: now finds duplicates

Posted: 2021 Jul 05, 21:32
by Gandolf
I only want to check for identical contents.

The odd occasions I've tried similar file checking utilities I've not been impressed. One regarded as similar a photo of my black cat and one of a Caribbean friend, who dresses in black, as similar!

I'll probably see what DeClone does.

Re: declone: now finds duplicates

Posted: 2021 Jul 12, 17:37
by lnbrantley
Gandolf wrote: 2021 Jul 05, 21:32 The odd occasions I've tried similar file checking utilities I've not been impressed. One regarded as similar a photo of my black cat and one of a Caribbean friend, who dresses in black, as similar!
Gandalf,

Two recommendations for finding similar photos:

1) DupDetector from Prismatic Software
I used this for years to find similar photos (same photo at different resolution or different quality/compression)
Using the 99% comparison option, false positives are extremely rare, false negatives not too common. I use automatic deletion when using this comparison level.
When using 98% or less, I would recommend using manual deletion.
Caveats:
Old - no longer maintained, not updated since 2011, 32 bit only.
Slow
Can't handle a huge number of files, or huge images
Ui is clunky.
Can't handle unicode.
Have to minimize its window to do anything else while it runs.

2) Find.Same.Images.OK - by the QDir guy.
I've started using this recently instead of DupDetector. Detection is almost as good, and is MUCH faster
It's 64 bit, so doesn't have the same limitations as DupDetector
Has options to find rotated or flipped images, which is nice.
Seems to have more false negatives than DupDetector.



I will experiment with DeClone to compare.

Re: declone: now finds duplicates

Posted: 2021 Jul 12, 17:56
by lnbrantley
Oh, Declone doesn't do similarity check yet, so can't compare.

Re: declone: now finds duplicates

Posted: 2021 Jul 12, 18:22
by Gandolf
@lnbrantley

DupDetector:
Used it for many years. Worked well but, as you say, it's no longer supported and was very, very slow.

Find.Same.Images.OK:
That was the program that found my cat and my friend similar. Regular crashes and lockouts with large numbers of files. Disappointing, since other O.K. software is good. I haven't tried it since version 2.01, I see it's up to 4.49 now.

As I wrote, similar is something I would use very rarely.

DeClone looks pretty good for identical files, and is fast. I did some speed comparison tests and sent the results to Nikos by e-mail. There are many factors that determine speed, disk fragmentation looks like my main speed restriction, and you have no idea what background processing is going on. If you want speed, set Settings > Scan options > Low scan priority to NO. The latest beta has debug code, so is slower.

Re: declone: now finds duplicates

Posted: 2021 Jul 12, 19:37
by lnbrantley
Gandolf,

Yes, give the latest version of Find.Same.Images.OK another try. I'm using 4.49 and see false positives at the 90% comparison level only when there is a very tiny difference between images. More often I have to crank it down to 80% or 75% to match images that DupDetector catches at 99% .

I may not be giving it as many files to process as you, so that could be a factor in the crashes and lockout s you experienced.

Re: declone: now finds duplicates

Posted: 2021 Jul 13, 04:53
by nikos
I started work on approximate comparisons, more to come later!