Never Completed
Moderator: nikos
-
- Member
- Posts: 12
- Joined: 2014 Oct 22, 23:25
Never Completed
Tried it twice to find duplicates in ~22,500 photos (Mostly jpg & cr2 & XMP, some tif and psd). Both times it started progressing through the folders using 7% CPU time at the beginning to 16% later and eventually stabilized at ~1.2 GB of memory. Did that for maybe 1/2 hr., then stopped advancing through folders. Stalled at a different place the second time.
Was this too many files to throw at it?
Was this too many files to throw at it?
Re: Never Completed
declone shouldn't have a problem with lots of files (although it does consume a lot of memory for the time being).
what kind of search parameters did you use? did you leave on the default options or have you tweaked them?
it's hard to say what went wrong.... did the progressbar appear "working"?
what kind of search parameters did you use? did you leave on the default options or have you tweaked them?
it's hard to say what went wrong.... did the progressbar appear "working"?
-
- Member
- Posts: 12
- Joined: 2014 Oct 22, 23:25
Re: Never Completed
Search for duplicate: Photos
Extensions: bmp,ico,jpg,gif,img,pcx,png,tif,xcf,raw,emf,tga,heic,jpeg
Scan options
Similarity level: 90 %
Ignore system files: True
Compare same types: True
Ignore small files: True
Folders to search
[+] My Pictures (that's "E:\Users\Mike\Pictures\Personal", an internal HD)
Compare these properties
Photo.DateTaken
Check file content: True
Also I checked and it should have scanned 43,800 files.
I just ran it again. At "phase 1: gathering files" progress bar advanced and the "searching in" folder at the bottom advanced.
About 9 min. later the progress bar stopped, the folder searching stopped advancing (in a folder containing 756 files), but CPU use continued at ~18%, Memory use ~1.1 GB, and disk ~ 0.1 MBs.
I waited another 30 min. after it stalled but nothing changed so I cancelled it.
Extensions: bmp,ico,jpg,gif,img,pcx,png,tif,xcf,raw,emf,tga,heic,jpeg
Scan options
Similarity level: 90 %
Ignore system files: True
Compare same types: True
Ignore small files: True
Folders to search
[+] My Pictures (that's "E:\Users\Mike\Pictures\Personal", an internal HD)
Compare these properties
Photo.DateTaken
Check file content: True
Also I checked and it should have scanned 43,800 files.
I just ran it again. At "phase 1: gathering files" progress bar advanced and the "searching in" folder at the bottom advanced.
About 9 min. later the progress bar stopped, the folder searching stopped advancing (in a folder containing 756 files), but CPU use continued at ~18%, Memory use ~1.1 GB, and disk ~ 0.1 MBs.
I waited another 30 min. after it stalled but nothing changed so I cancelled it.
Re: Never Completed
ok so you are using similarity photo search and I suspect that declone had a problem getting the pic preview somewhere and it got stuck
do you think you can use this procdump tool and send me the report when declone is stuck?
www.zabkat.com/blog/procdump-debug-hung-processes.htm
thanks
ps. also try to see if you can narrow down the problem, instead of checking your entire pics collection, do them one folder at a time? Your hint is the folder name where declone got stuck in the first place
do you think you can use this procdump tool and send me the report when declone is stuck?
www.zabkat.com/blog/procdump-debug-hung-processes.htm
thanks
ps. also try to see if you can narrow down the problem, instead of checking your entire pics collection, do them one folder at a time? Your hint is the folder name where declone got stuck in the first place
-
- Member
- Posts: 12
- Joined: 2014 Oct 22, 23:25
Re: Never Completed
Started declone this morning, 6 min. later it had stalled at a different place than before. Let it run for 31 min. after.
I then ran procdump64 declone. The dump is here: https://www.dropbox.com/sh/re6yful6y5w7 ... giySa?dl=0. When I was looking for the dump file I ran Everything file search and got a messed up display. Also once before I got a messed up display on the desk top instead of a window. I included an image of the Everything file search: it looked to me like the window didn't refresh as I dragged the lower right corner out. That problem stopped as soon as I cancelled declone.
After clicked cancel for the first time I got output! 862 groups, up to year/folder 2009. The photos start in 2002 and go up to now. At a glance (don't have more time to look now) the output looked correct. Maybe it was working all the time but just the folder and progress display was stuck?
Don't really have time to try to narrow it down to a folder, and since it stalls at a different place each time not sure that's useful.
I then ran procdump64 declone. The dump is here: https://www.dropbox.com/sh/re6yful6y5w7 ... giySa?dl=0. When I was looking for the dump file I ran Everything file search and got a messed up display. Also once before I got a messed up display on the desk top instead of a window. I included an image of the Everything file search: it looked to me like the window didn't refresh as I dragged the lower right corner out. That problem stopped as soon as I cancelled declone.
After clicked cancel for the first time I got output! 862 groups, up to year/folder 2009. The photos start in 2002 and go up to now. At a glance (don't have more time to look now) the output looked correct. Maybe it was working all the time but just the folder and progress display was stuck?
Don't really have time to try to narrow it down to a folder, and since it stalls at a different place each time not sure that's useful.
Re: Never Completed
thanks for the DMP file, i will look into it
however if declone responded to cancel button, then it wasn't stuck to begin with
however if declone responded to cancel button, then it wasn't stuck to begin with
Re: Never Completed
i see from the DMP file that the problem could be related to 32/64 bit (the beta is 32 bit) so it may go away with the final version which is going to be x64
-
- Member
- Posts: 12
- Joined: 2014 Oct 22, 23:25
Re: Never Completed
If you do make a 64 bit version I'd love to try it. I'm actually trying to solve a problem now (organize years of photos from different cameras and sources with lots of file name collisions) and declone could be a big help.
Re: Never Completed
if you open declone installer with something like 7zip, you will find inside an 64 bit version, but it is in trial mode, so it won't let you delete more than 100 files. But please extract and try it, I want to see if it will crash on detection or not?
-
- Member
- Posts: 12
- Joined: 2014 Oct 22, 23:25
Re: Never Completed
I tried the 64 bit version. After 4 min. it stalled, i.e. folder searching didn't change and progress bar froze. An hour later I looked at it and the progress bar was gone, declone was using 0% CPU and 120 MB of memory, the search folder at the bottom was partially clipped off and otherwise unchanged, and the list of files on the screen was empty. I closed it. I never saw any results.
Re: Never Completed
just to be certain, are you sure you were running the 64 bit version? Did it say x64 in HELP > ABOUT?
-
- Member
- Posts: 12
- Joined: 2014 Oct 22, 23:25
Re: Never Completed
In 7z I tried to unpack everything in declone_setup_beta.exe into a new folder but it didn't work because somehow in the 7z plugins folder you have multiple files with the same name. What's with that? I thought that wasn't possible and didn't know what to do with it so I just copied declone64.exe only and tried it. Perhaps not my best moment: did that mess things up? Anyway Help doesn't work without some of the other files but I know I'm executing declone64.exe. If you want me to install the other files tell me what to do about the duplicate names.
Re: Never Completed
ok if you were running declone64 then you did the right thing. You can always double check using HELP > ABOUT menu command (next to the toolbar button), which says if the program is 64 bit or not
Re: Never Completed
there's definitely a problem with the thumbnail cache once you read a few thousands of photos. I will try to avoid the cache and see how it goes
Re: Never Completed
I believe I fixed the problem, see the other post for the download link
I tested it on 30K photos, and although it takes some time to generate the thumbnails, it works without crashing
I tested it on 30K photos, and although it takes some time to generate the thumbnails, it works without crashing