Yahoo Spider Problems– Too many IPs!
So a suggestion I made over at the Yahoo suggestions board seems to be growing legs, so I figured I would post it here.
The long and short of it is that Yahoo has changed the way they crawl websites. Instead of a nice, slow meandering pace from limited originating IPs, Yahoo is now using a different IP for nearly every page request when crawling websites.
This is leading to increased and overly-inflated guest counts on nearly all scripts that utilize guest-visitor totals, or guest visitor tracking. It is also causing undue load upon many servers.
Here is my Yahoo suggestion: http://suggestions.yahoo.com/detail/?prop=SiteExplorer&fid=31431
Here is the Digg: http://www.digg.com/software/Yahoo_Spider_wrecking_havoc_on_website_guest_tracking

