Thunderstone Search Appliance Manual

Multiple Fetches

Syntax: select Y or N

Multiple Fetches allows a page to be fetched multiple times, and can potentially slow down a walk. It should only be used in specific situations in conjunction with Off-Site Pages.

For example, Consider the situation of walking two sites, a.com and b.com with Off-Site Pages enabled. A link from a page on a.com to b.com/page.htm is considered off-site, so it will be walked but its links won't. Then, when b.com starts its walk, b.com/page.htm won't be processed because it has already been done, causing b.com/page.htm's links to not be included.

Multiple Fetches allows the 2nd encounter of b.com/page.htm to be processed again, which will allow its links to be properly processed.