Page 1 of 1

Serialization of link/file state

Posted: 2013 Mar 16, 02:44
by Euclid
Problem:
For every search, links that have been previously determined to be invalid/not available are checked again. When dealing with thousands of links this has a big performance impact.

Solution:
Serialize the link/file state (availability) into a database (e.g. sqlite). If development resources are constrained the existing "blacklist" infrastructure can be (ab)used: Store every link for which the status is determined to be "not available" into the blacklist.

Thank you.

Posted: 2013 Mar 16, 05:55
by nikos
this is indeed an issue however crack tracker doesn't know if a link was only temporarily unavailable because of a variety of reasons, or if it was truly removed. So it checks again to be safe.

what I do for xplorer2 is add a version number, so instead of searching for xplorer2 I search for xplorer2 2.3, which will reduce this problem