Serialization of link/file state

Forum for Discussion of Crack Tracker Issues

Moderator: nikos

Post Reply
Euclid
Member
Member
Posts: 63
Joined: 2012 Mar 21, 11:59

Serialization of link/file state

Post by Euclid » 2013 Mar 16, 02:44

Problem:
For every search, links that have been previously determined to be invalid/not available are checked again. When dealing with thousands of links this has a big performance impact.

Solution:
Serialize the link/file state (availability) into a database (e.g. sqlite). If development resources are constrained the existing "blacklist" infrastructure can be (ab)used: Store every link for which the status is determined to be "not available" into the blacklist.

Thank you.

User avatar
nikos
Site Admin
Site Admin
Posts: 14650
Joined: 2002 Feb 07, 15:57
Location: UK
Contact:

Post by nikos » 2013 Mar 16, 05:55

this is indeed an issue however crack tracker doesn't know if a link was only temporarily unavailable because of a variety of reasons, or if it was truly removed. So it checks again to be safe.

what I do for xplorer2 is add a version number, so instead of searching for xplorer2 I search for xplorer2 2.3, which will reduce this problem

Post Reply