常见搜索引擎蜘蛛名称大全

选择字号： 超大 标准 发布时间：2014-7-18 16:41:10 | 作者：Searcheo | 0个评论 | 人浏览

本文地址：http://www.searcheo.cn/post/131.html 转载请注明出处！

‍‍‍‍‍

百度 Baiduspider

http://www.baidu.com/search/spider.htm

据说是百度MP3搜索的蜘蛛 Baidu-Transcoder

据说是百度图片搜索的蜘蛛 Baidu-Thumbnail

百度手机搜索 baiduspider-mobile-gate

Google 谷歌

1） Googlebot：从Google的网站索引和新闻索引中抓取网页 

Googlebot/2.1

2） Googlebot-Mobile：针对Google的移动索引抓取网页 

3） Googlebot-Image：针对Google的图片索引抓取网页 

4） Mediapartners-Google：抓取网页确定 AdSense 的内容。只有在你的网站上展示 AdSense 广告的情况下，Google才会使用此漫游器来抓取您的网站。 

5） Adsbot-Google：抓取网页来衡量 AdWords 目标网页的质量。只有在你使用 Google AdWords 为你的网站做广告的情况下，Google才会使用此漫游器。

http://www.google.com/bot.html 中文版请打开以下页面

http://www.google.cn/support/webmasters/bin/topic.py?topic=8843

雅虎中国

Yahoo! Slurp China

http://misc.yahoo.com.cn/help.html

Yahoo 英文

http://www.yuzhiguo.com/articleview.asp?id=437

Yahoo! Slurp

http://help.yahoo.com/help/us/ysearch/slurp

Yahoo 搜索竞价蜘蛛

Yahoo ContentMatch Crawler/2.1

Yahoo-MMCrawler 图片

有道 YodaoBot

http://www.yodao.com/help/webmaster/spider/

搜搜 Sosospider 网页蜘蛛 / Sosoimagespider 图片蜘蛛

http://help.soso.com/webspider.htm

http://help.soso.com/soso-image-spider.htm

搜狗 sogou spider

http://www.sogou.com/docs/help/webmasters.htm#07

新浪爱问 iaskspider

微软 msnbot

http://search.msn.com/msnbot.htm 打开后跳转到以下页面

http://help.live.com/help.aspx?mkt=zh-cn&project=wl_webmasters

*MSNBot：主网页爬虫(www.live.com)

*MSNBot-Media：图片及其它媒体爬虫(images.live.com)

*MSNBot-NewsBlogs：新闻及blog爬虫(search.live.com/news)

*MSNBot-Products：产品及购物爬虫(products.live.com)

*MSNBot-Academic：学术搜索爬虫(academic.live.com)

gigablast.com----Gigabot

http://www.gigablast.com/spider.html

webalta.net----WebAlta

http://www.webalta.net/ru/about_webmaster.html

国内的一个实验室的

apex.sjtu.edu.cn

Alexa ia_archiver iaarchiver

AOL sqworm

Altavista的蜘蛛：Scooter

Lycos的蜘蛛： Lycos_Spider_(T-Rex) 

Alltheweb的蜘蛛： FAST-WebCrawler/ 

INKTOMI的蜘蛛： Slurp

ASPSeek  ASPSeek 

东方网景 lanshanbot

日本的一个蜘蛛 BSpider

ask.com Ask

projecthoneypot.org Java (Often spam bot)

yandex.ru Yandex bot

Exabot.com Exabot

Netcraft

SurveyBot

larbin

http://www.majestic12.co.uk MJ12bot

http://www.cuil.com twiceler

Googlebot (Search Bot) 

Google Translate (Someone Behind Proxy) 

Unverified Googlebot (Search Bot) 

SiteUptime (Site Helper) 

Baidu Spider (Search Bot) 

Yahoo! Slurp (Search Bot) 

NimbuBot (Site Helper) 

mon.itor.us (Site Helper) 

Are My Sites Up? (Site Helper) 

Host Tracker (Site Helper) 

SurveyBot (Search Bot) 

Facebook External Hit (Crawler) 

Twingly Recon (Search Bot) 

PostRank Bot (Search Bot) 

Kosmix Voyager (Search Bot) 

TweetMeme Bot (Crawler) 

OneRiot Bot (Search Bot) 

Knowmore Bot (Search Bot) 

Comodo Certificates Spider (Crawler) 

Jayde Crawler (Search Bot) 

Netcraft Survey Bot (Search Bot) 

MSN/Bing Bot (Search Bot) 

XML Sitemaps Generator (Site Helper) 

Dotcom Bot (Site Helper) 

Pingdom Bot (Site Helper) 

CatchBot (Search Bot) 

Stack Rambler (Search Bot) 

Yandex Bot (Search Bot) 

Google Feedfetcher (Feed Fetcher) 

Montastic Bot (Site Helper) 

Google AdsBot (Search Bot) 

Speedy Spider (Search Bot) 

ExaBot (Search Bot) 

Panscient Bot (Search Bot) 

FindLinks (Search Bot) 

Gigabot (Search Bot) 

Flight Deck Bot (Crawler) 

MJ12 Bot (Search Bot) 

YaCy-Bot (Search Bot) 

obot (Crawler) 

DotBot (Crawler) 

Alexa Crawler (Search Bot) 

Kisiwa VSE (Crawler) 

ICC-Crawler (Crawler) 

Website Optimization Analyzer (Site Helper) 

Daum Bot (Search Bot) 

Lightspeed Crawler (Crawler) 

Websense Crawler (Crawler) 

Google Toolbar (Browser) 

DNS Digger Explorer (Crawler) 

CCBot (Crawler) 

L.webis (Crawler) 

Ask Crawler (Search Bot) 

Trend Micro Crawler (Crawler) 

Yeti (Naver Bot) (Search Bot) 

Netnir Rank Check (Crawler) 

spbot (Crawler) 

MLBot (Crawler) 

Xenu's Link Sleuth (Site Helper) 

Web Link Validator (Site Helper) 

Mail.Ru Bot (Search Bot) 

AgHaven Bot (Search Bot) 

Seexie Bot (Search Bot) 

Dom2Dom (Crawler) 

Setoozbot (Search Bot) 

ia_archiver (Search Bot) 

Corbina Bot (Crawler) 

RuleSpace Bot (Crawler) 

Blekko ScoutJet (Search Bot) 

IBM SAI Crawler (Crawler) 

OmgiliBot (Search Bot) 

Bixo Labs Crawler (Crawler) 

PickUpJob Bot (Search Bot) 

Labhoo Bot (Search Bot) 

Linkspimp (Crawler) 

Google Wireless Transcoder (Someone Behind Proxy) 

Envolk Bot (Search Bot) 

Soso Spider (Search Bot) 

DuckDuckBot (Search Bot) 

PHP OpenID (Site Helper) 

TurnitinBot (Crawler) 

Sogou Spider (Search Bot) 

008 (80legs Crawler) (Crawler) 

Cyveillance Bot (Crawler) 

psbot (Search Bot) 

KAIST Bot (Crawler) 

FlaxCrawler (Search Bot) 

Snapbot (Search Bot) 

LinkedIn Bot (Crawler) 

Careerjet Bot (Search Bot) 

Camont Spider (Crawler) 

Change Detection Bot (Site Helper) 

TextDigger Bot (Crawler) 

Statools Bot (Crawler) 

my6sense Bot (Crawler) 

TwengaBot-Discover (Search Bot) 

CyberPatrol Webbot (Crawler) 

Incapsula Service (Site Helper) 

SpokeSpider (Crawler) 

Sygol Bot (Search Bot) 

Seznam Bot (Search Bot) 

Jaxified RSS Feeder (Feed Fetcher) 

Solomono Bot (Search Bot) 

Youdao Bot (Search Bot) 

Hailoo Bot (Search Bot) 

Google Custom Search (Search Bot) 

MSR-ISRCCrawler (Crawler) 

SimplePie RSS Feeder (Feed Fetcher) 

Keywen Bot (Search Bot) 

UrlCheck (Site Helper) 

Periscope It (Site Helper) 

UrlCheck (Site Helper) 

Discobot (Crawler) 

Webmetrics Monitoring Service (Site Helper) 

FindFiles.net Bot (Search Bot) 

Yahoo External Cache (Crawler) 

Conduit Toolbar (Browser) 

FeedBurner (Feed Fetcher) 

Microsoft Web Services Client (Site Helper) 

ClickTale bot (Site Helper) 

Hyperspin (Site Helper) 

Pinterest Bot (Crawler) 

Proximic Bot (Crawler) 

Google Plus Share (Crawler) 

Amazon aranhabot (Site Helper) 

Microsoft Lync (Site Helper) 

Acoon Bot (Search Bot) 

elkMonitor (Site Helper) 

Covario IDS (Crawler) 

Yottaa (Site Helper) 

JikeSpider (Crawler) 

Magpie Crawler (Crawler) 

Panopta (Site Helper) 

GSA Crawler (Crawler) 

Disguised Google Bot (Search Bot) 

Spinn3r (Search Bot) 

Imperva MX Server (Crawler) 

CareerBot (Crawler) 

Sucuri Bot (Site Helper) 

OpenIndex Bot (Search Bot) 

Bountii Bot (Search Bot) 

Google Partner Monitor (Site Helper) 

Project Wonderful Ad/Compliancy Bot (Crawler) 

Yahoo Pipes (Feed Fetcher) 

Microsoft Adidxbot (Crawler) 

eToro Flash Client (Feed Fetcher) 

tra.cx.pider (Crawler) 

eToro IOS Client (Feed Fetcher) 

Radian6 Bot (Crawler) 

Topsy Butterfly (Search Bot) 

Grepnetstat (Site Helper) 

Gnip Bot (Crawler) 

Updowner bot (Crawler) 

NetSeer Crawler (Crawler) 

EvriNid bot (Crawler) 

InternetVista Site Monitor (Site Helper) 

Aihit bot (Crawler) 

eToro Mobile Client (Feed Fetcher) 

Yunyun Spider (Search Bot) 

CodeGuard Bot (Site Helper) 

VaultPress Bot (Site Helper) 

YourCronTask (Site Helper) 

Outbrain (Crawler) 

ShopWiki Bot (Search Bot) 

Server Density (Site Helper) 

GlobalSign Domain Verification Agent (Site Helper) 

Amazon ProductAdsBot (Crawler) 

Amazon CloudFront Client (Site Helper) 

Rackspace Monitoring Agent (Site Helper) 

SiteTruth (Site Helper) 

W3C - Markup Validator (Site Helper) 

W3C - Unicorn Unified Validator (Site Helper) 

W3C - Internationalization Checker (Site Helper) 

W3C - Nu Markup Validation Service (Site Helper) 

SpotFN iPhone App (Browser) 

SpotOption iPhone App (Browser) 

Google Webmasters Tools Agent (Site Helper) 

W3C - CSS Validator (Site Helper) 

W3C - Feed Validation (Site Helper) 

W3C - Mobile OK Checker (Site Helper) 

W3C - Link checker (Site Helper) 

http://www.botopedia.org 蜘蛛目录

‍‍‍‍‍

标签：搜索引擎  

猜你喜欢