### TEMPORARILY SUSPEND ALL [Disallow All] ### # TEMP-BLOCK "ALL=Disallow" #User-agent: * #Disallow: / ### END-OF TEMPORARILY SUSPEND ALL [Disallow All] ### # robots.txt file for http://www.nativeamericanembassy.net # robots.txt Renewed on Sunday October 21 21:50:35 EDT 2007 # REVISED: COMPLETELY: Saturday, February 24, 2008 09:50:35 EDT 2008 # REVISED: UPDATED: Tuesday, September 30, 2008 12:08:35 EDT 2008 # field-name:field-value # # field-name: # field-value: #PLACED AT BOTTOMUser-agent: * #PLACED AT BOTTOMDisallow: / #BEGIN ITEMIZED ROBOTS User-agent: A Disallow: / User-agent: AbachoBOT Disallow: / User-agent: abcdatos_botlink Disallow: / User-agent: AboutUsBot Disallow: / User-agent: Acoo Disallow: / User-agent: Acoon Disallow: / User-agent: Acoon Robot Disallow: / User-agent: Acorn Disallow: / User-agent: Acorn Browser Disallow: / User-agent: Adsbot-Google Disallow: / User-agent: AESOP_com_SpiderMan Disallow: / User-agent: Agent-admin Disallow: / User-agent: Agent-admin/ Disallow: / User-agent: ah-ha.com Disallow: / User-agent: ah-ha.com crawler Disallow: / #User-agent: Alexa #Disallow: / # TEST & WATCH CLOSELY: MAY BE "ALEXA" #User-agent: Alexibot #Disallow: / #User-agent: Altavista #Disallow: / User-agent: AMD64 Disallow: / User-agent: America Online Disallow: / User-agent: America Online Browser 1.1 Disallow: / # AMAZON User-agent: AMZNKAssocBot Disallow: / User-agent: AMZNKAssocBot/4.0 Disallow: / User-agent: antibot Disallow: / #User-agent: AOL #Disallow: / User-agent: AOL/1.1 Disallow: / User-agent: AOL 1.1 Disallow: / User-agent: ApacheBench Disallow: / User-agent: Apple Disallow: / User-agent: AppleWebKit Disallow: / User-agent: AppleWebKit/523.12.2 Disallow: / User-agent: Aqua_Products Disallow: / User-agent: Arachmo Disallow: / User-agent: Arachnoidea Disallow: / User-agent: Aranha Disallow: / User-agent: ArchitectSpider Disallow: / User-agent: Arcor Disallow: / User-agent: Arcor 5.004 Disallow: / User-agent: Arcor/5.004 Disallow: / #User-agent: Ask #Disallow: / #User-agent: AskJeeves #Disallow: / #User-agent: Ask Jeeves #Disallow: / #User-agent: Ask Jeeves/Teoma #Disallow: / User-agent: asterias Disallow: / User-agent: Atomz Disallow: / User-agent: Atomz/1.0 Disallow: / User-agent: Avant Disallow: / User-agent: AvantGo Disallow: / User-agent: AxmoRobot Disallow: / User-agent: AXP Disallow: / User-agent: B Disallow: / User-agent: Babya Disallow: / User-agent: Babya Discoverer Disallow: / User-agent: Babya Discoverer 8.0 Disallow: / User-agent: Babya Discoverer/8.0 Disallow: / User-agent: Babya Discoverer 8.0 Disallow: / User-agent: BackDoorBot Disallow: / User-agent: BackDoorBot 1.0 Disallow: / User-agent: BackDoorBot/1.0 Disallow: / User-agent: baidu Disallow: / User-agent: baiduspider Disallow: / User-agent: baidu.com Disallow: / User-agent: Become Disallow: / User-agent: BeOS Disallow: / User-agent: BePC Disallow: / User-agent: bgft Disallow: / User-agent: bigfoot Disallow: / User-agent: Black Hole Disallow: / User-agent: blackspider Disallow: / User-agent: Blazer Disallow: / User-agent: Blekko #UpComing [ Purpose: To Take-On / Challenge GOOGLE! ] Disallow: / #BlogPeople Java/1.4.2_04 User-agent: BlogPeople Disallow: / #BlogsNowBot, V 3.0 (+http://www.blogsnow.com/) User-agent: BlogsNowBot Disallow: / User-agent: BlowFish Disallow: / User-agent: BlowFish 1.0 Disallow: / User-agent: BlowFish/1.0 Disallow: / User-agent: Boitho Disallow: / User-agent: boitho.com Disallow: / User-agent: boitho.com-dc Disallow: / User-agent: boitho.com-dc/0.85 Disallow: / User-agent: boitho.com-dc/0.85 ( http://www.boitho.com/dcbot.html ) Disallow: / #User-agent: bonWeb #Disallow: / #User-agent: BonWeb #Disallow: / User-agent: Bookmark search tool Disallow: / User-agent: bot Disallow: / User-agent: bot/ Disallow: / User-agent: bot- Disallow: / User-agent: bot 1.0 Disallow: / User-agent: bot/1.0 Disallow: / User-agent: BotALot Disallow: / User-agent: BotRightHere Disallow: / # Added (Questionable) by: JVN # User-agent: BOTW Disallow: / User-agent: BOTW Spider Disallow: / User-agent: BrightCrawler Disallow: / User-agent: BrowseX Disallow: / User-agent: build 01218 Disallow: / User-agent: BuiltBotTough Disallow: / User-agent: Bullseye Disallow: / User-agent: Bullseye/1.0 Disallow: / User-agent: BunnySlippers Disallow: / User-agent: Buscaplus Robi Disallow: / User-agent: b2w Disallow: / User-agent: b2w 0.1 Disallow: / User-agent: b2w/0.1 Disallow: / User-agent: C Disallow: / User-agent: Cabot Disallow: / User-agent: Cabot/ Disallow: / User-agent: Cabot/Nutch-0.9 Disallow: / User-agent: Cabot/Nutch-0.9 (Amfibi/'s web-crawling robot; http://www.amfibi.com/cabot/; agent@amfibi.com) Disallow: / # Proxy Cache User-agent: Cache Disallow: / # Novell iChain Cool Solutions caching User-agent: caching Disallow: / User-agent: Camino Disallow: / User-agent: Camino 1.5.5 Disallow: / User-agent: Camino/1.5.5 Disallow: / User-agent: CanSeek Disallow: / User-agent: CanSeek/ Disallow: / User-agent: CatchBot Disallow: / User-agent: CatchBot/1.0 Disallow: / User-agent: Cazoodle Disallow: / User-agent: Cegbfeieh Disallow: / User-agent: Centrum Disallow: / User-agent: cfetch/1.0 Disallow: / User-agent: Charlotte Disallow: / User-agent: Charlotte/1.0 Disallow: / User-agent: Charlotte/1.0b Disallow: / # Added (Questionable) by: JVN # User-agent: Check&Get Disallow: / User-agent: Check&Get 1.14 Disallow: / User-agent: CheeseBot Disallow: / User-agent: CherryPicker Disallow: / User-agent: CherryPicker/1.0 Disallow: / User-agent: CherryPicker /1.0 Disallow: / User-agent: CherryPickerElite Disallow: / User-agent: CherryPickerElite 1.0 Disallow: / User-agent: CherryPickerElite/1.0 Disallow: / User-agent: CherryPickerSE Disallow: / User-agent: CherryPickerSE 1.0 Disallow: / User-agent: CherryPickerSE/1.0 Disallow: / User-agent: Chimera Disallow: / User-agent: ChristCRAWLER Disallow: / User-agent: cityreview Disallow: / User-agent: Cityreview Robot Disallow: / User-agent: Claria Disallow: / User-agent: Clushbot Disallow: / User-agent: Cobion Disallow: / User-agent: Commerce Disallow: / User-agent: Commerce Client Disallow: / User-agent: Commerce Client v1.0 Disallow: / User-agent: compatible Disallow: / User-agent: CompuServe Disallow: / # Secure Computing User-agent: Computing Disallow: / User-agent: Configuration Disallow: / User-agent: Configuration/CLDC-1.1 Disallow: / User-agent: Convera Disallow: / # Novell iChain Cool Solutions caching User-agent: Cool Disallow: / User-agent: CopyRightCheck Disallow: / User-agent: Cosmix Disallow: / User-agent: cosmos Disallow: / User-agent: Crawl #Seen &/or Shown as 'Unknown' Disallow: / User-agent: Crawler Disallow: / #User-agent: crawler100.ask.com #Ask Jeeves #Disallow: / User-agent: Crazy Browser Disallow: / User-agent: Crescent Disallow: / User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 Disallow: / User-agent: CSSCheck Disallow: / User-agent: cuill Disallow: / User-agent: Curl Disallow: / User-agent: custom Disallow: / User-agent: cybermapper Disallow: / User-agent: D Disallow: / User-agent: DaAdLe Disallow: / User-agent: DaAdLe.com Disallow: / User-agent: DaAdLe.com ROBOT Disallow: / User-agent: DaAdLe.com ROBOT/ Disallow: / User-agent: DataCha0s Disallow: / User-agent: DataCha0s 2.0 Disallow: / User-agent: DataCha0s/2.0 Disallow: / User-agent: DAUM Disallow: / User-agent: DBLBot Disallow: / User-agent: DBLBot/1.0 Disallow: / User-agent: Debian Disallow: / User-agent: DeepIndex Disallow: / User-agent: Deepnet Disallow: / User-agent: Deepnet Explorer Disallow: / User-agent: dial Disallow: / User-agent: discobot Disallow: / User-agent: DittoSpyder Disallow: / User-agent: DLM Disallow: / User-agent: DLM/3 Disallow: / User-agent: DoCoMo/ Disallow: / User-agent: DoCoMo/1.0/ Disallow: / User-agent: DoCoMo/1.0/N505i/ Disallow: / User-agent: DoCoMo/1.0/N505i/c20/ Disallow: / User-agent: DoCoMo/1.0/N505i/c20/TB/ Disallow: / User-agent: DoCoMo/1.0/N505i/c20/TB/W20H10 Disallow: / User-agent: DocZilla Disallow: / # NEW INTRUDER: August 8, 2008 [crawl6.dotnetdotcom.org] [DotBot/1.0.1 (http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org)] User-agent: dotbot Disallow: / User-agent: E Disallow: / User-agent: EARTHCOM Disallow: / User-agent: EARTHCOM.info Disallow: / User-agent: ELinks Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSearch Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: EmailWolf 1.00 Disallow: / User-agent: EmailWolf/1.00 Disallow: / User-agent: EmbeddedWB Disallow: / User-agent: EmbeddedWB 14.57 Disallow: / User-agent: EmbeddedWB/14.57 Disallow: / User-agent: EmeraldShield Disallow: / User-agent: eMusic Disallow: / User-agent: eMusic DLM Disallow: / User-agent: eMusic DLM/3 Disallow: / User-agent: en Disallow: / User-agent: en-US Disallow: / User-agent: EnaBot Disallow: / User-agent: EnaBot/1.2 Disallow: / User-agent: EnergyPlugIn Disallow: / User-agent: Entireweb Disallow: / # Envolk Links to Us #*User-agent: envolk #*Disallow: / #*User-agent: envolk/1.7 #*Disallow: / User-agent: Epiphany Disallow: / User-agent: Epiphany 1.4.4 Disallow: / User-agent: EroCrawler Disallow: / User-agent: EuripBot Disallow: / #User-agent: ExaBot #O.K. LinksUs #Disallow: / #User-agent: exabot.com #O.K. LinksUs #Disallow: / User-agent: Exabot-Thumbnails Disallow: / User-agent: ExpertRank Disallow: / User-agent: Explorer Disallow: / User-agent: Explorer 7 Disallow: / User-agent: Explorer/7 Disallow: / User-agent: Explorer 7.0b Disallow: / User-agent: Explorer/7.0b Disallow: / User-agent: extrabot Disallow User-agent: extrabot.com Disallow: / User-agent: ExtractorPro Disallow: / User-agent: EZResult Disallow: / User-agent: F Disallow: / User-agent: FairAd Client Disallow: / # Review/Check: User-agent: fast Disallow: / User-agent: fastbot Disallow: / User-agent: fastbot.de Disallow: / User-agent: fastbot.de crawler Disallow: / User-agent: fastbot.de crawler/2.0 Disallow: / User-agent: fastbot.de crawler 2.0 Disallow: / User-agent: fastbot.de crawler 2.0 beta Disallow: / User-agent: fastbot.de crawler 2.0 beta (http://www.fastbot.de) Disallow: / User-agent: FAST Data Search Crawler Disallow: / User-agent: FAST Data Search Document Retriever Disallow: / User-agent: Fast PartnerSite Crawler Disallow: / User-agent: FAST-WebCrawler Disallow: / User-agent: favorstar Disallow: / User-agent: Faxo Disallow: / # FDSE Robot User-agent: FDSE Disallow: / User-agent: FDSE Robot User-agent: fedora Disallow: / User-agent: Fedora/1.0.7-1.1.fc4 Disallow: / User-agent: FeedDemon Disallow: / User-agent: FeedDemon/2.5 Disallow: / User-agent: Fetch # URI::Fetch 0.08 # Disallow: / User-agent: Fetch 0.08 # URI::Fetch 0.08 # Disallow: / User-agent: Fetch/0.08 # URI::Fetch 0.08 # Disallow: / # Added (Questionable) by: JVN # User-agent: FileDownloader Disallow: / User-agent: findlinks Disallow: / User-agent: findlinks 1.1.3-beta8 Disallow: / User-agent: findlinks/1.1.3-beta8 Disallow: / User-agent: Firebat Disallow: / User-agent: Firebat 2.7.13 Disallow: / User-agent: Firebird Disallow: / User-agent: Firebird/0.5+ Disallow: / User-agent: Firefox Disallow: / User-agent: Firefox/1.0RC2 Disallow: / User-agent: Firefox/1.0.1 Disallow: / User-agent: Firefox/1.5.0.3 Disallow: / User-agent: Firefox/1.6a1 Disallow: / User-agent: Firefox/2.0.0.9 Disallow: / User-agent: Firefox/2.0.0.11 Disallow: / User-agent: Firefox/2.0.0.12 Disallow: / User-agent: Firefox/2.0.0.12,gzip Disallow: / User-agent: Firefox/2.0.0.12,gzip(gfe) Disallow: / User-agent: Flaming AttackBot Disallow: / User-agent: flatlandbot Disallow: / User-agent: Flock Disallow: / # FollowSite Bot [http://www.followsite.com/bot.html] User-agent: FollowSite Bot Disallow: / User-agent: FollowSite Disallow: / User-agent: FollowSiteBot Disallow: / User-agent: Foobot Disallow: / User-agent: FOTOCHECKER Disallow: / User-agent: france.misesajour.com Disallow: / ### TEMP ALLOW For FREEFIND SEARCH ENGINE ### User-agent: freefind allow: / User-agent: FriendFeedBot Disallow: / User-agent: FriendFeedBot/0.1 Disallow: / User-agent: FunWebProducts Disallow: / User-agent: FyberSearch Disallow: / User-agent: G Disallow: / User-agent: Gaisbot Disallow: / User-agent: GalaxyBot Disallow: / User-agent: Galeon Disallow: / User-agent: Galeon/1.2.0 Disallow: / User-agent: Gecko Disallow: / User-agent: Gecko/ Disallow: / User-agent: Gecko/20020326 Disallow: / User-agent: Gecko/20020910 Disallow: / User-agent: Gecko/20021207 Disallow: / User-agent: Gecko/20030504 Disallow: / User-agent: Gecko/20040707 Disallow: / User-agent: Gecko/20040815 Disallow: / User-agent: Gecko/20041103 Disallow: / User-agent: Gecko/20050512 Disallow: / User-agent: Gecko/20050716 Disallow: / User-agent: Gecko/20051002 Disallow: / User-agent: Gecko/20051102 Disallow: / User-agent: Gecko/20060206 Disallow: / User-agent: Gecko/20060426 Disallow: / User-agent: Gecko/20060918 Disallow: / User-agent: Gecko/20071025 Disallow: / User-agent: Gecko/20071127 Disallow: / User-agent: Gecko/20080201 Disallow: / User-agent: Gecko/70061023 Disallow: / User-agent: geckobot Disallow: / User-agent: GenCrawler Disallow: / User-agent: genieBot Disallow: / User-agent: genieBot wgao Disallow: / User-agent: genieBot wgao@genieknows.com Disallow: / User-agent: Geona Disallow: / User-agent: Geona 1.2 Disallow: / User-agent: GeonaBot Disallow: / User-agent: getRAX Disallow: / User-agent: GetRight Disallow: / User-agent: GetRight/4.2 Disallow: / User-agent: GetRight/5.2d Disallow: / User-agent: GetRight/6.2a Disallow: / # SECURITY ALERT: GigaBlast (GigaBot) IGNORS Robots.txt & ENTERS FORBIDDEN DIRECTORY/FILES: BAN Associated IPs From Entire Web Domain in .HTACCESS! # User-agent: Gigablast # UNFRIENLY ROBOT/SPIDER: DISOBEYS &/Or IGNORS ROBOTS.TXT [BAN IPs] Disallow: / User-agent: Gigablast 3.0 # UNFRIENLY ROBOT/SPIDER: DISOBEYS &/Or IGNORS ROBOTS.TXT [BAN IPs] Disallow: / User-agent: Gigabot # UNFRIENLY ROBOT/SPIDER: DISOBEYS &/Or IGNORS ROBOTS.TXT [BAN IPs] Disallow: / User-agent: Gigabot 2.0 # UNFRIENLY ROBOT/SPIDER: DISOBEYS &/Or IGNORS ROBOTS.TXT [BAN IPs] Disallow: / User-agent: Gigabot/2.0 # UNFRIENLY ROBOT/SPIDER: DISOBEYS &/Or IGNORS ROBOTS.TXT [BAN IPs] Disallow: / User-agent: Gigabot/3.0 # UNFRIENLY ROBOT/SPIDER: DISOBEYS &/Or IGNORS ROBOTS.TXT [BAN IPs] Disallow: / User-agent: Gigabot/3.0 (http://www.gigablast.com/spider.html) # UNFRIENLY ROBOT/SPIDER: DISOBEYS &/Or IGNORS ROBOTS.TXT [BAN IPs] Disallow: / # User-agent: Pingdom GIGRIB User-agent: GIGRIB Disallow: / User-agent: girafa Disallow: / User-agent: girafa.com Disallow: / User-agent: Goo Disallow: / User-agent: Google Disallow: /? Disallow: /add-guest.php Disallow: /addguest.php Disallow: /guest-book.php Disallow: /guestbook.php Disallow: /bbclone/ Disallow: /bbclone/? Disallow: /bbclone/show_detailed.php Disallow: /bbclone/show_detailed.php? Disallow: /bbclone/show_detailed.php?lng=en Disallow: /bbclone/show_views.php Disallow: /bbclone/show_views.php? Disallow: /bbclone/show_views.php?id=18716&lng=en Disallow: /botanical/audio/ Disallow: /botanical/mgmh/a/images/ Disallow: /botanical/mgmh/b/images/ Disallow: /botanical/mgmh/c/images/ Disallow: /botanical/mgmh/d/images/ Disallow: /botanical/mgmh/e/images/ Disallow: /botanical/mgmh/f/images/ Disallow: /botanical/mgmh/g/images/ Disallow: /botanical/mgmh/h/images/ Disallow: /botanical/mgmh/i/images/ Disallow: /botanical/mgmh/j/images/ Disallow: /botanical/mgmh/k/images/ Disallow: /botanical/mgmh/l/images/ Disallow: /botanical/mgmh/m/images/ Disallow: /botanical/mgmh/n/images/ Disallow: /botanical/mgmh/o/images/ Disallow: /botanical/mgmh/p/images/ Disallow: /botanical/mgmh/q/images/ Disallow: /botanical/mgmh/r/images/ Disallow: /botanical/mgmh/s/images/ Disallow: /botanical/mgmh/t/images/ Disallow: /botanical/mgmh/u/images/ Disallow: /botanical/mgmh/v/images/ Disallow: /botanical/mgmh/w/images/ Disallow: /botanical/mgmh/x/images/ Disallow: /botanical/mgmh/y/images/ Disallow: /botanical/mgmh/z/images/ Disallow: /cgi-bin/ Disallow: /cgi-bin/guestbook.php Disallow: /cgi-bin/guestbook/ Disallow: /cgi-bin/wwwboard/ Disallow: /.htdocs/ Disallow: /.htdocs/jsActive/ Disallow: /.htdocs/jsDisabled/ Disallow: /images Disallow: /images/ Disallow: /images/? Disallow: /Images Disallow: /Images/ Disallow: /Images/? Disallow: NAINDIANER/? Disallow: NAINDIANER/mitglied.lycos.de/? Disallow: NAINDIANER/mitglied.lycos.de/frlakota/? Disallow: NAINDIANER/mitglied.lycos.de/frlakota/Lakota1.jpg Disallow: /NAINDIANER/? Disallow: /NAINDIANER/mitglied.lycos.de/? Disallow: /NAINDIANER/mitglied.lycos.de/frlakota/? Disallow: /NAINDIANER/mitglied.lycos.de/frlakota/Lakota1.jpg Disallow: /redirection/ User-agent: GoogleBot Disallow: /? Disallow: /add-guest.php Disallow: /addguest.php Disallow: /guest-book.php Disallow: /guestbook.php Disallow: /bbclone/ Disallow: /bbclone/? Disallow: /bbclone/show_detailed.php Disallow: /bbclone/show_detailed.php? Disallow: /bbclone/show_detailed.php?lng=en Disallow: /bbclone/show_views.php Disallow: /bbclone/show_views.php? Disallow: /bbclone/show_views.php?id=18716&lng=en Disallow: /botanical/audio/ Disallow: /botanical/mgmh/a/images/ Disallow: /botanical/mgmh/b/images/ Disallow: /botanical/mgmh/c/images/ Disallow: /botanical/mgmh/d/images/ Disallow: /botanical/mgmh/e/images/ Disallow: /botanical/mgmh/f/images/ Disallow: /botanical/mgmh/g/images/ Disallow: /botanical/mgmh/h/images/ Disallow: /botanical/mgmh/i/images/ Disallow: /botanical/mgmh/j/images/ Disallow: /botanical/mgmh/k/images/ Disallow: /botanical/mgmh/l/images/ Disallow: /botanical/mgmh/m/images/ Disallow: /botanical/mgmh/n/images/ Disallow: /botanical/mgmh/o/images/ Disallow: /botanical/mgmh/p/images/ Disallow: /botanical/mgmh/q/images/ Disallow: /botanical/mgmh/r/images/ Disallow: /botanical/mgmh/s/images/ Disallow: /botanical/mgmh/t/images/ Disallow: /botanical/mgmh/u/images/ Disallow: /botanical/mgmh/v/images/ Disallow: /botanical/mgmh/w/images/ Disallow: /botanical/mgmh/x/images/ Disallow: /botanical/mgmh/y/images/ Disallow: /botanical/mgmh/z/images/ Disallow: /cgi-bin/ Disallow: /cgi-bin/guestbook.php Disallow: /cgi-bin/guestbook/ Disallow: /cgi-bin/wwwboard/ Disallow: /.htdocs/ Disallow: /.htdocs/jsActive/ Disallow: /.htdocs/jsDisabled/ Disallow: /Images Disallow: /Images/ Disallow: /Images/? Disallow: NAINDIANER/? Disallow: NAINDIANER/mitglied.lycos.de/? Disallow: NAINDIANER/mitglied.lycos.de/frlakota/? Disallow: NAINDIANER/mitglied.lycos.de/frlakota/Lakota1.jpg Disallow: /NAINDIANER/? Disallow: /NAINDIANER/mitglied.lycos.de/? Disallow: /NAINDIANER/mitglied.lycos.de/frlakota/? Disallow: /NAINDIANER/mitglied.lycos.de/frlakota/Lakota1.jpg Disallow: /redirection/ User-agent: Googlebot/2.1 Disallow: /? Disallow: /add-guest.php Disallow: /addguest.php Disallow: /guest-book.php Disallow: /guestbook.php Disallow: /bbclone/ Disallow: /bbclone/? Disallow: /bbclone/show_detailed.php Disallow: /bbclone/show_detailed.php? Disallow: /bbclone/show_detailed.php?lng=en Disallow: /bbclone/show_views.php Disallow: /bbclone/show_views.php? Disallow: /bbclone/show_views.php?id=18716&lng=en Disallow: /botanical/audio/ Disallow: /botanical/mgmh/a/images/ Disallow: /botanical/mgmh/b/images/ Disallow: /botanical/mgmh/c/images/ Disallow: /botanical/mgmh/d/images/ Disallow: /botanical/mgmh/e/images/ Disallow: /botanical/mgmh/f/images/ Disallow: /botanical/mgmh/g/images/ Disallow: /botanical/mgmh/h/images/ Disallow: /botanical/mgmh/i/images/ Disallow: /botanical/mgmh/j/images/ Disallow: /botanical/mgmh/k/images/ Disallow: /botanical/mgmh/l/images/ Disallow: /botanical/mgmh/m/images/ Disallow: /botanical/mgmh/n/images/ Disallow: /botanical/mgmh/o/images/ Disallow: /botanical/mgmh/p/images/ Disallow: /botanical/mgmh/q/images/ Disallow: /botanical/mgmh/r/images/ Disallow: /botanical/mgmh/s/images/ Disallow: /botanical/mgmh/t/images/ Disallow: /botanical/mgmh/u/images/ Disallow: /botanical/mgmh/v/images/ Disallow: /botanical/mgmh/w/images/ Disallow: /botanical/mgmh/x/images/ Disallow: /botanical/mgmh/y/images/ Disallow: /botanical/mgmh/z/images/ Disallow: /cgi-bin/ Disallow: /cgi-bin/guestbook.php Disallow: /cgi-bin/guestbook/ Disallow: /cgi-bin/wwwboard/ Disallow: /.htdocs/ Disallow: /.htdocs/jsActive/ Disallow: /.htdocs/jsDisabled/ Disallow: /Images Disallow: /Images/ Disallow: /Images/? Disallow: NAINDIANER/? Disallow: NAINDIANER/mitglied.lycos.de/? Disallow: NAINDIANER/mitglied.lycos.de/frlakota/? Disallow: NAINDIANER/mitglied.lycos.de/frlakota/Lakota1.jpg Disallow: /NAINDIANER/? Disallow: /NAINDIANER/mitglied.lycos.de/? Disallow: /NAINDIANER/mitglied.lycos.de/frlakota/? Disallow: /NAINDIANER/mitglied.lycos.de/frlakota/Lakota1.jpg Disallow: /redirection/ User-agent: Googlebot-Image Disallow: / Disallow: /? # SAMPLE FOR SPECIFIC DISALLOW: In GOOGLS.s & OTHER's Case, ALL IMAGES Are FORBIDDEN! # #User-agent: Googlebot-Image #Disallow: /images/dogs.jpg User-agent: Googlebot-Mobile Disallow: / User-agent: Googlebot-Mobile/2.1 Disallow: / #TEMP-WATCH+STUDY~User-agent: Google Browser # 72.14.192.33 #TEMP-WATCH+Disallow: / User-agent: Grabbitbot Disallow: / User-agent: Grabbitbot/1.0 Disallow: / User-agent: Grabbitbot/1.0(feedback@grabb.it) Disallow: / User-agent: Grub Disallow: / # http://www.grub.org (User-agent) User-agent: grub-client Disallow: / User-agent: grub-client-2.6.0 Disallow: / User-agent: GrubNG Disallow: / User-agent: GrubNG 20080128 Disallow: / # User-agent: gulliver # allow Northern Light crawler # Disallow: / # GurujiBot #TEST ALLOW (2008-03-03) #User-agent: Guruji #Disallow: / #TEST ALLOW (2008-03-03) #User-agent: Guruji 1.0 #Disallow: / User-agent: GurujiBot Disallow: / User-agent: gzip Disallow: / User-agent: gzip(gfe) Disallow: / User-agent: H Disallow: / User-agent: Harbot Disallow: / User-agent: Harvest Disallow: / User-agent: Harvest/1.5 Disallow: / User-agent: HbTools 4.8.4 Disallow: / User-agent: HbTools/4.8.4 Disallow: / User-agent: HD Disallow: / User-agent: HD nutch Disallow: / User-agent: HD nutch agent Disallow: / User-agent: HD nutch agent/1.0 Disallow: / User-agent: Heritrix Disallow: / User-agent: heritrix/1.12.1 Disallow: / User-agent: hiptop Disallow: / User-agent: hiptop 4.7 Disallow: / User-agent: hit #Idientifird as, on robots.txt Disallow: / User-agent: hloader Disallow: / User-agent: Hotzonu Disallow: / User-agent: HouxouCrawler Disallow: / User-agent: HouxouCrawler/Nutch-0.8.2 Disallow: / User-agent: HouxouCrawler/Nutch-0.8.2-dev Disallow: / User-agent: HouxouCrawler/Nutch-0.9 Disallow: / User-agent: HP Disallow: / User-agent: HP Web Disallow: / User-agent: HP Web PrintSmart Disallow: / User-agent: HTTPClient Disallow: / User-agent: HTTPClient 0.3 Disallow: / User-agent: httplib Disallow: / User-agent: HttpUnit Disallow: / User-agent: http://bestsexyes.info Disallow: / User-agent: http://www.gigablast.com Disallow: / User-agent: http://www.webalta.net Disallow: / User-agent: HTTrack Disallow: / User-agent: HTTrack 3.0 Disallow: / User-agent: humanlinks Disallow: / User-agent: I Disallow: / # TEST & WATCH CLOSELY: MAY BE "ALEXA" #User-agent: ia_archiver # Internet Archive crawler #Disallow: / # TEST & WATCH CLOSELY: MAY BE "ALEXA" #User-agent: ia_archiver/1.6 #Disallow: / # Internet Archive {archive.org} {ia331424.us.archive.org} [Associated with "Alexa"] #User-agent: ia_archiver-web.archive.org #Disallow: / User-agent: Iceweasel Disallow: / User-agent: Iceweasel/2.0.0.10 Disallow: / # Novell iChain Cool Solutions caching User-agent: iChain Disallow: / # Mozilla/4.0 (compatible; ICS 1.2.105) # User-agent: ICS 1.2.105 Disallow: / User-agent: IEMB3 Disallow: / User-agent: IIgs Disallow: / # May be O.K. - Indian Language (University) Web Search Engine WATCH/REVIEW # #User-agent: Indian Language (University) Web Search Engine #Disallow: / User-agent: Indy Library Disallow: / User-agent: InfoNaviRobot Disallow: / User-agent: InfoPath Disallow: / User-agent: InfoPath.1 Disallow: / User-agent: InfoPath.2 Disallow: / User-agent: Inquent Disallow: / User-agent: Inquent-Metamend Disallow: / User-agent: Inquent-Metamend 1.0 Disallow: / User-agent: Inquent-Metamend/1.0 Disallow: / User-agent: InternetSeer Disallow: / User-agent: Intraformant Disallow: / User-agent: IntraVnews Disallow: / User-agent: ips-agent Disallow: / User-agent: Iron33 Disallow: / User-agent: Iron33/1.0.2 Disallow: / User-agent: IRLbot Disallow: / User-agent: IRLbot 3.0 Disallow: / User-agent: IRLbot/3.0 Disallow: / # Unsure if Legitimate # User-agent: ISC Disallow: / User-agent: ISC Systems Disallow: / User-agent: ISC Systems iRc Disallow: / User-agent: ISC Systems iRc Search Disallow: / User-agent: ISC Systems iRc Search 2.1 Disallow: / User-agent: iSense Disallow: / User-agent: iSense bot Disallow: / User-agent: iSense bot v1.21 Disallow: / User-agent: i686 Disallow: / User-agent: i686 Linux Disallow: / User-agent: J Disallow: / User-agent: Jack Disallow: / # Entered (Questionable) by: JVN # User-agent: Jakarta Commons-HttpClient Disallow: / User-agent: Jakarta Commons-HttpClient/3.0 Disallow: / User-agent: Java Disallow: / User-agent: Java/1.4.1_04 Disallow: / User-agent: Java/1.4.2 Disallow: / User-agent: Java/1.4.2_03 Disallow: / User-agent: Java/1.4.2_04 Disallow: / User-agent: Java/1.4.2_05 Disallow: / User-agent: Java/1.4.2_08 Disallow: / User-agent: Java/1.4.2_13 Disallow: / User-agent: Java/1.5.0 Disallow: / User-agent: Java/1.5.0_01 Disallow: / User-agent: Java/1.5.0_02 Disallow: / User-agent: Java/1.5.0_03 Disallow: / User-agent: Java/1.5.0_04 Disallow: / User-agent: Java/1.5.0_05 Disallow: / User-agent: Java/1.5.0_06 Disallow: / User-agent: Java/1.5.0_07 Disallow: / User-agent: Java/1.5.0_08 Disallow: / User-agent: Java/1.5.0_09 Disallow: / User-agent: Java/1.5.0_10 Disallow: / User-agent: Java/1.5.0_11 Disallow: / User-agent: Java/1.6.0 Disallow: / User-agent: Java/1.6.0-oem Disallow: / User-agent: Java/1.6.0-rc Disallow: / User-agent: Java/1.6.0_01 Disallow: / User-agent: Java/Java/1.0 Disallow: / #User-agent: Jeeves #Disallow: / #User-agent: Jeeves/Teoma #Disallow: / User-agent: JennyBot Disallow: / User-agent: Jetbot/1.0 Disallow: / # JP - Japanese # User-agent: J-PHONE Disallow: / User-agent: J-PHONE/2.0 Disallow: / User-agent: J-PHONE/2.0/J-SH03 Disallow: / User-agent: J-PHONE/2.0/J-SH03 (compatible; Y!J-SRD/1.0; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-27.html) Disallow: / User-agent: Jyxo Disallow: / # Entered (Questionable) by: JVN # User-agent: Jyxobot Disallow: / User-agent: Jyxobot/1 Disallow: / User-agent: K Disallow: / User-agent: Kazehakase Disallow: / User-agent: Kenjin Spider Disallow: / User-agent: Keyword Disallow: / User-agent: Keyword Density Disallow: / User-agent: Keyword Density/0.9 Disallow: / User-agent: KHTML Disallow: / User-agent: KHTML 3.5.5 Disallow: / User-agent: KHTML/3.5.5 Disallow: / User-agent: KIT Disallow: / User-agent: KIT-Fireball Disallow: / User-agent: KKman # WebCapture (Robot) # Disallow: / User-agent: K-Meleon Disallow: / User-agent: Knight Disallow: / User-agent: Knight/0.2 Disallow: / User-agent: Konqueror Disallow: / User-agent: Konqueror 3.0 Disallow: / User-agent: Konqueror/3.0 Disallow: / User-agent: Konqueror/3.0-rc1 Disallow: / User-agent: Konqueror 3.0-rc1 Disallow: / User-agent: Konqueror 3.0-rc5 Disallow: / User-agent: Konqueror 3.0-rc6 Disallow: / User-agent: Konqueror 3.1 Disallow: / User-agent: Konqueror/3.1 Disallow: / User-agent: Konqueror/3.1-rc6 Disallow: / User-agent: Konqueror 3.5 Disallow: / User-agent: Konqueror/3.5 Disallow: / User-agent: Krugle Disallow: / User-agent: Krugle/ Disallow: / User-agent: Krugle/Krugle Disallow: / User-agent: Krugle/Krugle,Nutch Disallow: / User-agent: Krugle/Krugle,Nutch/0.8+ Disallow: / User-agent: Krugle,Nutch Disallow: / User-agent: Krugle web crawler Disallow: / User-agent: L Disallow: / User-agent: Larbin Disallow: / User-agent: larbin_2.6.3 Disallow: / User-agent: larbin2.6.3@unspecified.mail Disallow: / User-agent: LexiBot Disallow: / User-agent: libWeb Disallow: / User-agent: libWeb/ Disallow: / User-agent: libWeb/clsHTTP Disallow: / User-agent: libWWW Disallow: / User-agent: libwww-perl Disallow: / User-agent: libwww-perl/5.65 Disallow: / User-agent: libwww-perl/5.69 Disallow: / User-agent: libwww-perl/5.803 Disallow: / User-agent: libwww-perl/5.805 Disallow: / User-agent: libwww-perl/5.808 Disallow: / # Pete-Spider Light/1.46 # User-agent: Light Disallow: / # Pete-Spider Light/1.46 # User-agent: Light/1.46 Disallow: / User-agent: like Gecko Disallow: / User-agent: Link Disallow: / User-agent: Linkcheck Disallow: / User-agent: LinkextractorPro Disallow: / User-agent: LinkScan Disallow: / User-agent: LinkScan/8.1a Disallow: / User-agent: LinkScan/8.1a Unix Disallow: / # Xenu Link Sleuth User-agent: LinkSleuth Disallow: / # Link Walker User-agent: LinkWalker Disallow: / User-agent: Linux Disallow: / User-agent: Linux i986 Disallow: / User-agent: Livedoor Disallow: / User-agent: LLWC Disallow: / User-agent: LNSpiderguy Disallow: / # OK! TRUST, BUT WATCH! SEARCHES "LOCAL" By DEFAULT "Knows User's City" OR USER ENTERS CITY, ZIP, etc.: DOES LINK US, & SHOWS ADDRESS, PHONE, etc. DETAILS! #User-agent: LocalcomBot #Disallow: / User-agent: Localization Disallow: / User-agent: lwp-trivial Disallow: / User-agent: lwp-trivial/1.34 Disallow: / User-agent: Lycos_Spider_(T-Rex) Disallow: / User-agent: Lynx Disallow: / User-agent: M Disallow: / User-agent: Mac Disallow: / User-agent: Mac_PowerPC Disallow: / User-agent: Mail.Ru Disallow: / User-agent: Mail.Ru/1.0 Disallow: / User-agent: Majestic Disallow: / User-agent: Majestic-12 Disallow: / User-agent: Majestic12 Disallow: / # Unsure if Legitimate # User-agent: MarkWatch/1.0 Disallow: / User-agent: Mata Hari Disallow: / User-agent: Maxthon Disallow: / User-agent: Media Disallow: / User-agent: Media Center Disallow: / User-agent: Media Center PC Disallow: / User-agent: Media Center PC 5.0 Disallow: / User-agent: Mediapartners-Google Disallow: / User-agent: Mediapartners-Google/2.1 Disallow: / User-agent: Media Player Disallow: / # Added (Questionable) by: JVN # User-agent: MEGAUPLOAD Disallow: / User-agent: MEGAUPLOAD 1.0 Disallow: / User-agent: MEGAUPLOAD 2.0 Disallow: / User-agent: MEGAUPLOAD TOOLBAR 1.0 Disallow: / User-agent: Mercator Disallow: / #Inquent-Metamend/1.0 User-agent: Metamend Disallow: / User-agent: Microsoft URL Control Disallow: / User-agent: Microsoft URL Control - 5.01.4511 Disallow: / User-agent: Microsoft URL Control - 6.00.8169 Disallow: / User-agent: Microsoft URL Control - 6.00.8862 Disallow: / User-agent: MIIxpc Disallow: / User-agent: MIIxpc/4.2 Disallow: / User-agent: Missigua Disallow: / User-agent: Missigua Locator Disallow: / User-agent: Missigua Locator 1.9 Disallow: / User-agent: Mister PiX Disallow: / User-agent: MJ12bot Disallow: / User-agent: MJ12bot/v1.2.1 Disallow: / User-agent: MLBot Disallow: / User-agent: Mnenhy Disallow: / User-agent: moget Disallow: / User-agent: moget/2.0 Disallow: / User-agent: moget/2.1 Disallow: / User-agent: MOOX M3 Disallow: / #Moreoverbot/5.00 (+http://www.moreover.com) User-agent: Moreoverbot Disallow: / User-agent: Moreoverbot/5.00 Disallow: / User-agent: Mozilla Disallow: / User-agent: Mozilla/2.0 Disallow: / # Added (Questionable) by: JVN # User-agent: Mozilla/2.0 compatible; Check&Get 1.14 (Windows NT) Disallow: / User-agent: Mozilla/3.0 Disallow: / User-agent: Mozilla/3.0 (compatible; TALWinHttpClient) Disallow: / User-agent: Mozilla/3.0 (compatible; WebCapture 2.0; Auto; Windows) Disallow: / User-agent: Mozilla/3.0 (compatible; WebMon 1.0.10; Windows XP) Disallow: / User-agent: Mozilla/3.01 Disallow: / User-agent: Mozilla/4 Disallow: / User-agent: Mozilla/4.0 Disallow: / User-agent: Mozilla/4.0 (compatible;) Disallow: / User-agent: Mozilla/4.0 (compatible; BOTW Spider; +http://botw.org) Disallow: / User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95) Disallow: / User-agent: Mozilla/4.0 (compatible; grub-client-2.6.0) Disallow: / User-agent: Mozilla/4.0 (compatible; ICS 1.2.105) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows ME) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows 95; MSIECrawler) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; America Online Browser 1.1; rev1.2; Windows NT 5.1; SV1; .NET CLR 1.1.4322) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Netcraft SSL Server Survey - contact info@netcraft.com) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; DGD (AutoProxy4) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FDM) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; .NET CLR 1.1.4322; PeoplePal 6.2) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; iSense bot v1.21) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; MEGAUPLOAD 1.0) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; MEGAUPLOAD TOOLBAR 1.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; MRA 4.6 (build 01425); MRSPUTNIK 1, 5, 0, 19 SW) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322) Babya Discoverer 8.0 Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; bgft; IEMB3) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; {FBCBFCB5-A146-427D-BD3D-3C7E995F10CB}; .NET CLR 1.0.3705; .NET CLR 1.1.4322) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; FileDownloader; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.1.4322; .NET CLR 2.0.50727; MEGAUPLOAD 1.0; .NET CLR 1.0.3705; InfoPath.1; Media Center PC 4.0; FileDownloader; MEGAUPLO... Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; HbTools 4.8.4; .NET CLR 2.0.50727) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SU 3.005; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506; .NET CLR 1.1.4322) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 6.0) Disallow: / User-agent: Mozilla/5 Disallow: / User-agent: Mozilla/5.0 Disallow: / User-agent: Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020424) Disallow: / User-agent: Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020526) Disallow: / User-agent: Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020910) Disallow: / User-agent: Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020802) Disallow: / User-agent: Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Exabot-Thumbnails) Disallow: / User-agent: Mozilla/5.0 (compatible; MJ12bot/v1.2.1; http://www.majestic12.co.uk/bot.php?+) Disallow: / User-agent: Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html) Disallow: / User-agent: Mozilla/5.0 (compatible; Yoono; http://www.yoono.com/) Disallow: / User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 Disallow: / User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 Waterunicorn/0.10 StumbleUpon/1.998 Disallow: / User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.3) Gecko/20040924 Epiphany/1.4.4 (Ubuntu) Disallow: / User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 Disallow: / User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20061201 Firefox/2.0.0.6 (Ubuntu-feisty) Disallow: / User-agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-CA; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 Disallow: / User-agent: Mp3Bot Disallow: / # Added (Questionable) by: JVN # User-agent: MPlayer Disallow: / User-agent: MPlayer/dev-CVS-050721-06:00-3.2.3 Disallow: / User-agent: MRA Disallow: / User-agent: MrCarlito Disallow: / User-agent: MrCarlito-0.1 Disallow: / User-agent: MRSPUTNIK Disallow: / User-agent: MRSPUTNIK 1, 5, 0, 19 SW Disallow: / User-agent: MSIE Disallow: / # Mozilla/4.0 (compatible; MSIE 5.01; Windows 95; MSIECrawler) # User-agent: MSIECrawler Disallow: / User-agent: msn Disallow: /? Disallow: /botanical/mgmh/a/images/ Disallow: /botanical/mgmh/b/images/ Disallow: /botanical/mgmh/c/images/ Disallow: /botanical/mgmh/d/images/ Disallow: /botanical/mgmh/e/images/ Disallow: /botanical/mgmh/f/images/ Disallow: /botanical/mgmh/g/images/ Disallow: /botanical/mgmh/h/images/ Disallow: /botanical/mgmh/i/images/ Disallow: /botanical/mgmh/j/images/ Disallow: /botanical/mgmh/k/images/ Disallow: /botanical/mgmh/l/images/ Disallow: /botanical/mgmh/m/images/ Disallow: /botanical/mgmh/n/images/ Disallow: /botanical/mgmh/o/images/ Disallow: /botanical/mgmh/p/images/ Disallow: /botanical/mgmh/q/images/ Disallow: /botanical/mgmh/r/images/ Disallow: /botanical/mgmh/s/images/ Disallow: /botanical/mgmh/t/images/ Disallow: /botanical/mgmh/u/images/ Disallow: /botanical/mgmh/v/images/ Disallow: /botanical/mgmh/w/images/ Disallow: /botanical/mgmh/x/images/ Disallow: /botanical/mgmh/y/images/ Disallow: /botanical/mgmh/z/images/ Disallow: /Images Disallow: /Images/ Disallow: /Images/? User-agent: msnbot Disallow: /? Disallow: /botanical/mgmh/a/images/ Disallow: /botanical/mgmh/b/images/ Disallow: /botanical/mgmh/c/images/ Disallow: /botanical/mgmh/d/images/ Disallow: /botanical/mgmh/e/images/ Disallow: /botanical/mgmh/f/images/ Disallow: /botanical/mgmh/g/images/ Disallow: /botanical/mgmh/h/images/ Disallow: /botanical/mgmh/i/images/ Disallow: /botanical/mgmh/j/images/ Disallow: /botanical/mgmh/k/images/ Disallow: /botanical/mgmh/l/images/ Disallow: /botanical/mgmh/m/images/ Disallow: /botanical/mgmh/n/images/ Disallow: /botanical/mgmh/o/images/ Disallow: /botanical/mgmh/p/images/ Disallow: /botanical/mgmh/q/images/ Disallow: /botanical/mgmh/r/images/ Disallow: /botanical/mgmh/s/images/ Disallow: /botanical/mgmh/t/images/ Disallow: /botanical/mgmh/u/images/ Disallow: /botanical/mgmh/v/images/ Disallow: /botanical/mgmh/w/images/ Disallow: /botanical/mgmh/x/images/ Disallow: /botanical/mgmh/y/images/ Disallow: /botanical/mgmh/z/images/ Disallow: /Images Disallow: /Images/ Disallow: /Images/? User-agent: msnbot/1.1 Disallow: /? Disallow: /botanical/mgmh/a/images/ Disallow: /botanical/mgmh/b/images/ Disallow: /botanical/mgmh/c/images/ Disallow: /botanical/mgmh/d/images/ Disallow: /botanical/mgmh/e/images/ Disallow: /botanical/mgmh/f/images/ Disallow: /botanical/mgmh/g/images/ Disallow: /botanical/mgmh/h/images/ Disallow: /botanical/mgmh/i/images/ Disallow: /botanical/mgmh/j/images/ Disallow: /botanical/mgmh/k/images/ Disallow: /botanical/mgmh/l/images/ Disallow: /botanical/mgmh/m/images/ Disallow: /botanical/mgmh/n/images/ Disallow: /botanical/mgmh/o/images/ Disallow: /botanical/mgmh/p/images/ Disallow: /botanical/mgmh/q/images/ Disallow: /botanical/mgmh/r/images/ Disallow: /botanical/mgmh/s/images/ Disallow: /botanical/mgmh/t/images/ Disallow: /botanical/mgmh/u/images/ Disallow: /botanical/mgmh/v/images/ Disallow: /botanical/mgmh/w/images/ Disallow: /botanical/mgmh/x/images/ Disallow: /botanical/mgmh/y/images/ Disallow: /botanical/mgmh/z/images/ Disallow: /Images Disallow: /Images/ Disallow: /Images/? User-agent: msnbot-media Disallow: / User-agent: msnbot-media 1.0 Disallow: / User-agent: msnbot-media/1.0 Disallow: / User-agent: MSN Explorer Disallow: / User-agent: msrbot Disallow: / User-agent: MSR-ISRCCrawler Disallow: / # Added (Questionable) by: JVN # User-agent: My agent Disallow: / User-agent: MyIE2 Disallow: / User-agent: N Disallow: / User-agent: Natsu Mican Disallow: / User-agent: Naver Disallow: / User-agent: Naver 1.0 Disallow: / User-agent: Naver/1.0 Disallow: / User-agent: NaverBot Disallow: / User-agent: NaverBot/1.0 Disallow: / User-agent: NetAnts Disallow: / # Added (Questionable) by: JVN # User-agent: NetCaptor Disallow: / User-agent: NetCaptor 7.5.0 Disallow: / User-agent: NetCaptor 7.5.0 Gold Disallow: / User-agent: NET CLR Disallow: / User-agent: .NET CLR Disallow: / User-agent: .NET CLR 1.1.4322 Disallow: / User-agent: .NET CLR 2.0.50727 Disallow: / User-agent: Netcraft Disallow: / User-agent: Netcraft SSL Server Survey Disallow: / User-agent: NetMechanic Disallow: / User-agent: NetNewsWire Disallow: / User-agent: Netscape Disallow: / User-agent: Netscape 4.5 Disallow: / User-agent: Netscape/4.5 Disallow: / User-agent: Netscape 6.2.1 Disallow: / User-agent: Netscape6/6.2.1 Disallow: / User-agent: Netscape/8.0 Disallow: / User-agent: Netscape/8.0.4 Disallow: / User-agent: NetSear Disallow: / User-agent: netsear.com Disallow: / User-agent: NewsGator Disallow: / User-agent: NextGenSearchBot Disallow: / User-agent: NICErsPRO Disallow: / User-agent: Nimble Disallow: / User-agent: NimbleCrawler Disallow: / # Bad Bot, well worth banning! User (healthline.com) [IP block: 72.5.115.0 - 72.5.115.127]: Claims to "obeys...[robots.txt?]" User-agent: NimbleCrawler 2.0.2 Disallow: / User-agent: NimbleCrawler/2.0.2 Disallow: / User-agent: Novell Disallow: / User-agent: Novell iChain Cool Solutions caching Disallow: / User-agent: Novell iChain Cool Solutions caching 4.0 Disallow: / User-agent: noXtrum Disallow: / # Added (Questionable) by: jvn # User-agent: NSPlayer Disallow: / User-agent: NSPlayer/9.0.0.3265 Disallow: / User-agent: NSPlayer/9.0.0.3265 WMFSDK Disallow: / User-agent: NSPlayer/9.0.0.3265 WMFSDK/ Disallow: / User-agent: NSPlayer/9.0.0.3265 WMFSDK/9.0 Disallow: / User-agent: NT Disallow: / User-agent: Numeric Disallow: / User-agent: Nutch Disallow: / User-agent: Nutch-0.8.2 Disallow: / User-agent: Nutch-0.9 Disallow: / User-agent: Nutch/Nutch Disallow: / User-agent: Nutch/Nutch-0.9 Disallow: / User-agent: Nutch/Nutch-0.9 (Nutch; http://nutch; nutch) Disallow: / # Added (Questionable) by: JVN # User-agent: NutchCVS Disallow: / User-agent: NutchCVS/Nutch Disallow: / User-agent: NutchCVS/Nutch-0.9 Disallow: / User-agent: NutchCVS/Nutch-0.9 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) Disallow: / User-agent: O Disallow: / User-agent: OffByOne Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Omni Disallow: / #* May Be O.K. User-agent: OmniExplorer Disallow: / User-agent: OmniExplorer_Bot Disallow: / User-agent: OmniExplorer_Bot/6.10.5 Disallow: / User-agent: OmniWeb Disallow: / User-agent: OOZBOT Disallow: / User-agent: Openbot Disallow: / User-agent: Openfind Disallow: / User-agent: Openfind data Disallow: / User-agent: Openfind data gathere Disallow: / User-agent: Openfind data gatherer Disallow: / User-agent: Opera Disallow: / User-agent: Opera/7.11 Disallow: / User-agent: Opera/8.00 Disallow: / User-agent: Opera/9.00 Disallow: / User-agent: Opera Mini Disallow: / User-agent: Oracle Ultra Search Disallow: / User-agent: Oregano Disallow: / User-agent: P Disallow: / # Theft/Misuse of All Page's Inner-Link(s) causing (Copyable) Page-PreView-Images # User-agent: Pagebull Disallow: / User-agent: pagebullbot Disallow: / User-agent: Pagebull http://www.pagebull.com/ Disallow: / User-agent: panscient.com Disallow: / User-agent: PC Disallow: / User-agent: PC 2.0 Disallow: / User-agent: PC 5.0 Disallow: / User-agent: PeoplePal Disallow: / User-agent: PeoplePal 6.2 Disallow: / User-agent: PeoplePal/6.2 Disallow: / User-agent: PerMan Disallow: / User-agent: Pete-Spider Disallow: / # 62.118.12.190 - Pete-Spider is a Persistent & Repeat Offender # User-agent: Pete-Spider Light/1.46 Disallow: / User-agent: Phoenix Disallow: / User-agent: Phoenix/0.5 Disallow: / # PHP version tracker User-agent: PHP Disallow: / User-agent: PHP version tracker Disallow: / # UNFRIENDLY ROBOT: #ALLOWED FOR ONLY 7 To 14 DAYS! User-agent: PicSearch #* Actual robot: * Psbot# #ALLOWED FOR ONLY 7 To 14 DAYS! Disallow: / User-agent: PicSearch Disallow: / User-agent: Pingdom Disallow: / User-agent: Pingdom GIGRIB Disallow: / User-agent: Plagger Disallow: / User-agent: PlantyNet 1.9 Disallow: / User-agent: PlantyNet_WebRobot_V1.9 # dhkang@plantynet.com Disallow: / User-agent: PluckIt Disallow: / User-agent: PluckItCrawler Disallow: / User-agent: PluckItCrawler/1.0 Disallow: / User-agent: Pluggd Disallow: / User-agent: Pluggd/Nutch Disallow: / User-agent: Pluggd/Nutch-0.9 Disallow: / User-agent: Pluggd/Nutch-0.9 (automated crawler; http://www.pluggd.com; support at pluggd dot com) Disallow: / User-agent: Powermarks Disallow: / User-agent: Powermarks/3.5 Disallow: / User-agent: Powermarks 3.5 Disallow: / User-agent: PrintSmart Disallow: / User-agent: Profile Disallow: / User-agent: Profile/MIDP-2.0 Disallow: / User-agent: ProPowerBot Disallow: / User-agent: ProPowerBot/2.14 Disallow: / User-agent: ProWebWalker Disallow: / # Proxy Cache User-agent: Proxy Disallow: / User-agent: Proxy Cache Disallow: / # UNFRIENDLY ROBOT: #ALLOWED FOR ONLY 7 To 14 DAYS! User-agent: psbot #* Shown as * PicSearch# #ALLOWED FOR ONLY 7 To 14 DAYS! Disallow: / # UNFRIENDLY ROBOT: #ALLOWED FOR ONLY 7 To 14 DAYS! User-agent: Psbot #* Shown as * PicSearch# #ALLOWED FOR ONLY 7 To 14 DAYS! Disallow: / User-agent: psbot Disallow: / User-agent: PycURL Disallow: / User-agent: PycURL/7.15.5 Disallow: / User-agent: Python-url Disallow: / User-agent: Python-urllib Disallow: / User-agent: Python-urllib/1.16 Disallow: / User-agent: Python-urllib/1.17 Disallow: / User-agent: Q Disallow: / User-agent: Qihoo Disallow: / User-agent: QueryN Metasearch Disallow: / User-agent: Q312461 Disallow: / User-agent: R Disallow: / User-agent: RaBot Disallow: / User-agent: Radiation Retriever Disallow: / User-agent: Radiation Retriever 1.1 Disallow: / User-agent: Radiation Retriever/.1 Disallow: / User-agent: Rambler Disallow: / User-agent: Real Player Disallow: / User-agent: RepoMonkey Disallow: / User-agent: RepoMonkey Bait Disallow: / User-agent: RepoMonkey Bait & Tackle Disallow: / User-agent: RepoMonkey Bait & Tackle/v1.01 Disallow: / User-agent: RGcrawler Disallow: / User-agent: RMA Disallow: / User-agent: roach.smo.av.com-1.0 Disallow: / User-agent: Robot Disallow: / User-agent: Robot Genius Disallow: / User-agent: RobotGenius Disallow: / User-agent: Robotgenius crawler Disallow: / User-agent: Robotgenius crawler/Nutch Disallow: / User-agent: Robotgenius crawler/Nutch-0.9 Disallow: / User-agent: robtexbot Disallow: / User-agent: RSSMicro Disallow: / User-agent: ru Disallow: / User-agent: ru-RU Disallow: / User-agent: rv Disallow: / User-agent: rv:1.8.1.9 Disallow: / User-agent: S Disallow: / User-agent: Safari/523.12.2 Disallow: / #http://boston.lti.cs.cmu.edu" [Language Technologies Institute] Within: "Carnegie Mellon University's School of Computer Science" User-agent: Sapphire Disallow: / #IBID: ALSO USING "NUTCH" User-agent: SapphireWebCrawler Disallow: / User-agent: Sauce Reader Disallow: / User-agent: SBIder Disallow: / User-agent: scooter # AltaVista crawler? Disallow: / User-agent: Scooter2 Disallow: / User-agent: Scooter2 Mercator Disallow: / User-agent: Scooter2_Mercator Disallow: / User-agent: Scooter2_Mercator_3-1.0 Disallow: / User-agent: Scooter2 Mercator/-1.0 Disallow: / User-agent: Scooter2_Mercator/-1.0 Disallow: / User-agent: ScoutJet Disallow: / User-agent: ScSpider Disallow: / User-agent: ScSpider 0.2 Disallow: / User-agent: ScSpider/0.2 Disallow: / User-agent: Semager Disallow: / User-agent: Semager/1.2 Disallow: / User-agent: SeaMonkey Disallow: / User-agent: Search Disallow: / User-agent: Search/ Disallow: / User-agent: search.ch Disallow: / User-agent: search.ch V1.4.2 Disallow: / User-agent: search.ch V1.4.2 (spiderman@search.ch, http://www.search.ch) Disallow: / User-agent: SearchHub Disallow: / User-agent: SearchHub Bot Disallow: / User-agent: searchpreview Disallow: / User-agent: Search 2.1 Disallow: / User-agent: Search/2.1 Disallow: / # Added (Questionable) by: JVN # User-agent: Search 4.0 Disallow: / User-agent: Search/4.0 Disallow: / User-agent: Search 4.0 Robot Disallow: / # Secure Computing User-agent: Secure Disallow: / User-agent: ShopWiki Disallow: / User-agent: SIS Disallow: / User-agent: SiteSnagger Disallow: / #WATCH User-agent: SitiDiBot #WATCH Disallow: / User-agent: SL Disallow: / User-agent: SLCC1 Disallow: / User-agent: SL Commerce Disallow: / User-agent: SL Commerce Client v1.0 Disallow: / User-agent: Sleipnir Disallow: / # Xenu Link Sleuth User-agent: Sleuth Disallow: / User-agent: sloleht-gw.sloleht.ee Disallow: / #User-agent: Slurp #Disallow: / User-agent: Slurp.so Disallow: / User-agent: Slurp.so/1.0 Disallow: / User-agent: Slurp/2.0j Disallow: / User-agent: Slurp/2.0-KiteHourly Disallow: / User-agent: Slurp/2.0-OwlWeekly Disallow: / User-agent: Slurp/3.0-AU Disallow: / User-agent: SlySearch # NOW "TurnItInBot" Disallow: / User-agent: Snapbot Disallow: / User-agent: Snapbot/1.0 Disallow: / User-agent: Snoopy Disallow: / User-agent: Snoopy v1.2 Disallow: / User-agent: Snoopy/v1.2 Disallow: / User-agent: SoftBank Disallow: / User-agent: SoftBank/1.0/920T/TJ001 Disallow: / User-agent: Sogou Disallow: / User-agent: Sogou 4.0 Disallow: / User-agent: sogou spider Disallow: / User-agent: Sogou web spider Disallow: / User-agent: Sogou web spider 3.0 Disallow: / User-agent: Sogou web spider/3.0 Disallow: / # Novell iChain Cool Solutions caching User-agent: Solutions Disallow: / User-agent: songbird Disallow: / User-agent: songbird/0.1 Disallow: / User-agent: Sosospider Disallow: / User-agent: SPAM Disallow: / User-agent: SpamBlockerUtility Disallow: / User-agent: SpamBlockerUtility/4.8.4 Disallow: / User-agent: SpamBlockerUtility 4.8.4 Disallow: / User-agent: SpankBot Disallow: / User-agent: spanner Disallow: / User-agent: Spectrum Disallow: / User-agent: Spectrum Internet Disallow: / User-agent: Spectrum Internet Suite Disallow: / # EntireWeb User-agent: Speedy Disallow: / User-agent: Speedy Spider Disallow: / User-agent: Spider Disallow: / User-agent: spider1.picsearch.com Disallow: / User-agent: spider2.picsearch.com Disallow: / User-agent: spider3.picsearch.com Disallow: / User-agent: spider4.picsearch.com Disallow: / User-agent: spider5.picsearch.com Disallow: / User-agent: spider6.picsearch.com Disallow: / User-agent: spider7.picsearch.com Disallow: / User-agent: spider8.picsearch.com Disallow: / User-agent: spider9.picsearch.com Disallow: / User-agent: spider10.picsearch.com Disallow: / User-agent: spider11.picsearch.com Disallow: / User-agent: spider12.picsearch.com Disallow: / User-agent: spider13.picsearch.com Disallow: / User-agent: spider14.picsearch.com Disallow: / User-agent: spider15.picsearch.com Disallow: / User-agent: spider16.picsearch.com Disallow: / User-agent: spider17.picsearch.com Disallow: / User-agent: spider18.picsearch.com Disallow: / User-agent: spider19.picsearch.com Disallow: / User-agent: spider20.picsearch.com Disallow: / User-agent: spider25.yandex.ru # Yandex 1.01.001 Disallow: / User-agent: SpokeSpider Disallow: / User-agent: SpokeSpider/1.0 Disallow: / User-agent: Sputnik Disallow: / User-agent: statedept-crawler Disallow: / User-agent: StumbleUpon Disallow: / User-agent: StumbleUpon 1.998 Disallow: / User-agent: StumbleUpon/1.998 Disallow: / User-agent: Sunrise Disallow: / User-agent: SurfControl Disallow: / User-agent: SUSE Disallow: / User-agent: SUSE/7.0-70 Disallow: / User-agent: suzuran Disallow: / User-agent: SV1 Disallow: / User-agent: Swift Disallow: / User-agent: SWSE Disallow: / User-agent: Sylera Disallow: / User-agent: Synoobot Disallow: / User-agent: Synoobot/0.9 Disallow: / User-agent: Szukacz Disallow: / User-agent: Szukacz/1.4 Disallow: / User-agent: T Disallow: / User-agent: Tablet Disallow: / User-agent: Tablet PC Disallow: / User-agent: Tablet PC 2.0 Disallow: / #User-agent: tags2dir.net #Disallow: / #User-agent: tags2dir.net/0.8 #Disallow: / User-agent: TALWinHttpClient Disallow: / User-agent: Tamu Disallow: / User-agent: Tamu Crawler Disallow: / User-agent: Tamu Crawler 3.0 Disallow: / User-agent: Tamu Crawler/3.0 Disallow: / #User-agent: tc-gw.exabot.com #O.K. LinksUs #Disallow: / User-agent: Technoratibot Disallow: / User-agent: Technoratibot/0.7 Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: Telesoft Disallow: / #User-agent: Teoma #Ask Jeeves #Disallow: / User-agent: The Intraformant Disallow: / User-agent: TheNomad Disallow: / User-agent: TheRarestParser Disallow: / User-agent: TheRarestParser/0.2a Disallow: / User-agent: Thunderbird Disallow: / User-agent: Thunderbird 1.0.6 Disallow: / User-agent: TightTwatBot Disallow: / User-agent: Titan Disallow: / User-agent: toCrawl Disallow: / User-agent: toCrawl/UrlDispatcher Disallow: / User-agent: T-Online Disallow: / # May Be OK? User-agent: T-Online Browser Disallow: / User-agent: Toutatis Disallow: / User-agent: Toutatis 2.5-2 Disallow: / User-agent: True_Robot Disallow: / User-agent: True_Robot/1.0 Disallow: / User-agent: turingos Disallow: / User-agent: Turnitin Disallow: / User-agent: Turnitinbot/1.4 Disallow: / User-agent: Turnitinbot/1.5 Disallow: / User-agent: TurnitinBot Disallow: / User-agent: TurnitinBot/1.5 Disallow: / User-agent: TutorGig Disallow: / User-agent: TÜzilla Disallow: / User-agent: Tv_Merc_resh_26_1_D-1.0 Disallow: / User-agent: twiceler Disallow: / User-agent: Twiceler-0.9 Disallow: / User-agent: T312461 Disallow: / User-agent: U Disallow: / User-agent: Ubuntu Disallow: / User-agent: Ubuntu-feisty Disallow: / User-agent: UP Disallow: / User-agent: UP.Browser Disallow: / User-agent: URI Disallow: / User-agent: URI:Fetch 0.08 Disallow: / User-agent: URI::Fetch 0.08 Disallow: / User-agent: URL Control Disallow: / User-agent: UrlDispatcher Disallow: / User-agent: URL_Spider_Pro Disallow: / User-agent: URLy Disallow: / User-agent: URLy Warning Disallow: / User-agent: V Disallow: / User-agent: VCI Disallow: / User-agent: VCI WebViewer Disallow: / User-agent: VCI WebViewer Win32 Disallow: / User-agent: VCI WebViewer VCI WebViewer Win32 Disallow: / User-agent: Version/3.0.4 Disallow: / User-agent: Visvo Disallow: / User-agent: Voila Disallow: / User-agent: VoilaBot Disallow: / User-agent: VoilaBot BETA Disallow: / User-agent: VoilaBot BETA 1.2 Disallow: / User-agent: v1.21 Disallow: / User-agent: W Disallow: / # Link Walker User-agent: Walker Disallow: / User-agent: WAP Disallow: / User-agent: Waterunicorn Disallow: / User-agent: Waterunicorn 0.10 Disallow: / User-agent: Waterunicorn/0.10 Disallow: / User-agent: WDL Disallow: / User-agent: WDL6.1.1 Disallow: / User-agent: WebAlta Disallow: / User-agent: WebAlta Crawler Disallow: / # BOGUS User-Agent's URL/PATH (All Possibles): http://www.webalta.net/ru/about_webmaster.html USED BY IP 77.91.224.12: Does Not Read "robots.txt" User-agent: WebAlta Crawler/2.0 Disallow: / User-agent: WebAlta 2.0 Disallow: / User-agent: WebAuto Disallow: / User-agent: WebBandit Disallow: / User-agent: WebBandit/3.50 Disallow: / User-agent: WebCapture # KKman # Disallow: / User-agent: WebCapture 2.0 Disallow: / User-agent: WebCapture/2.0 Disallow: / User-agent: WebCopier Disallow: / User-agent: WebCopier v.2.2 Disallow: / User-agent: WebCopier v3.2a Disallow: / User-agent: WebCrawler Disallow: / # (IP 72.249.126.142) & (IP 66.90.118.101) BOTH [USING: WebDataCentreBot/1.0] IGNORES RULES! User-agent: WebData Disallow: / User-agent: WebDataCentre Disallow: / User-agent: WebDataCentreBot Disallow: / # IGNORES RULES! BAN ANY IPs USING IT! User-agent: WebDataCentreBot/1.0 Disallow: / User-agent: WebEnhancer Disallow: / User-agent: Web Image Collector Disallow: / User-agent: WebKit Disallow: / User-agent: WebmasterWorldForumBot Disallow: / # Added (Questionable) by: JVN # User-agent: WebMon Disallow: / User-agent: WebMon 1.0.10 Disallow: / User-agent: Web PrintSmart Disallow: / User-agent: WebReaper Disallow: / User-agent: WebSauger Disallow: / User-agent: Websense Disallow: / User-agent: Website Quester Disallow: / User-agent: Webster Disallow: / User-agent: Webster Pro Disallow: / User-agent: WebStripper Disallow: / User-agent: Webtv Disallow: / User-agent: WebZip Disallow: / User-agent: WebZip/4.0 Disallow: / User-agent: WebZIP/4.21 Disallow: / User-agent: WebZIP/5.0 Disallow: / User-agent: Wget Disallow: / User-agent: Wget 1.10.2 Disallow: / User-agent: Wget/1.4.0 Disallow: / User-agent: Wget/1.5.3 Disallow: / User-agent: Wget/1.6 Disallow: / User-agent: Win Disallow: / User-agent: Windows Disallow: / User-agent: Windows Media Player Disallow: / User-agent: Windows-Media-Player Disallow: / #May not be needed the way shown here# User-agent: Windows Media Player/10.00.00.3990 Disallow: / User-agent: Windows-Media-Player/10.00.00.3990 Disallow: / User-agent: Windows NT Disallow: / User-agent: Windows NT 5.1 Disallow: / User-agent: WinNT Disallow: / User-agent: WinNT4.0 Disallow: / User-agent: Win16 Disallow: / User-agent: Win64 Disallow: / User-agent: WIRE/0.13 Disallow: / # Added (Questionable) By: jvn # User-agent: WMFSDK Disallow: / User-agent: WMFSDK/9.0 Disallow: / User-agent: www.baidu.com Disallow: / User-agent: WWWC Disallow: / User-agent: WWW-Collector Disallow: / User-agent: WWW-Collector-E Disallow: / User-agent: www.gigablast.com Disallow: / User-agent: www.girafa.com Disallow: / User-agent: www.netsear.com Disallow: / User-agent: www.webalta.net Disallow: / User-agent: WWW-Mechanize Disallow: / User-agent: Wysigot Disallow: / User-agent: W3C Validator Disallow: / User-agent: W3C-mobile Disallow: / User-agent: W3C-mobileOK Disallow: / User-agent: W3C-mobileOK DDC-1.0 Disallow: / User-agent: W3C-mobileOK/DDC-1.0 Disallow: / User-agent: W3C-mobileOK/DDC-1.0 (see http://www.w3.org/2006/07/mobileok-ddc) Disallow: / User-agent: w3m Disallow: / User-agent: X Disallow: / # Xenu Link Sleuth User-agent: Xenu Disallow: / User-agent: Xenu's Disallow: / User-agent: Xenu's Link Sleuth Disallow: / User-agent: Xenu's Link Sleuth 1.1c Disallow: / User-agent: Xenu's Link Sleuth/1.1c Disallow: / User-agent: X11 Disallow: / User-agent: Y Disallow: / User-agent: Yacy Disallow: / User-agent: yacybot Disallow: / #User-agent: Yahoo #Disallow: / User-agent: YahooFeedSeeker Disallow: / # WATCH THIS CLOSELY! User-agent: Yahoo! Slurp China Disallow: / # WATCH YANDEX CLOSELY! Has a Decent Search Engine, altho it Ignors Robot.txt Rules #User-agent: Yandex 1.01.001 # spider25.yandex.ru #Disallow: / #User-agent: Yandex # spider25.yandex.ru ? #Disallow: / #User-agent: Yandex/1.01.001 # spider25.yandex.ru #Disallow: / #User-agent: Yandex/1.01.001 (compatible; Win16; I) # spider25.yandex.ru #Disallow: / User-agent: Yodao #* Disallow: / User-agent: Yodao1.0 Disallow: / User-agent: Yodao/1.0 Disallow: / User-agent: Yodao_1.0 Disallow: / User-agent: YodaoBot Disallow: / User-agent: YodaoBot/ Disallow: / User-agent: YodaoBot/1.0 Disallow: / User-agent: yoofind Disallow: / User-agent: yoofind-0.1 Disallow: / User-agent: yoofind/yoofind-0.1 Disallow: / User-agent: Yoono Disallow: / User-agent: yoono webcrawler Disallow: / User-agent: ZoomInfo Disallow: / User-agent: Z Disallow: / # IGNORS ROBOTS.TXT & RULES User-agent: Zermelo Disallow: / User-agent: zermelo Mozilla/5.0 compatible; heritrix/1.12.1 (+http://www.powerset.com) Disallow: / User-agent: Zeus Disallow: / User-agent: Zeus Link Scout Disallow: / User-agent: Zeus 32297 Disallow: / User-agent: Zeus 32297 Webster Disallow: / User-agent: Zeus 32297 Webster Pro Disallow: / User-agent: Zeus 32297 Webster Pro V2.9 Disallow: / User-agent: Zeus 32297 Webster Pro V2.9 Win32 Disallow: / User-agent: Zune Disallow: / User-agent: Zune/2.0 Disallow: / User-agent: Zune 2.0 Disallow: / # NUMERIC # User-agent: 1.1.4322 Disallow: / User-agent: 2.0.50727 Disallow: / User-agent: 20020526 Disallow: / User-agent: 72.249.126.241 # Extension:"Numeric" CRAWLING / NOT VIEWING Disallow: / User-agent: 77.91.224.13 # WebAlta Crawler Robot.txt Violator Disallow: / # spider25.yandex.ru [Yandex 1.01.001] User-agent: 77.88.26.25 Disallow: / User-agent: 91.121.23.84 #ENTERED: HIDDEN KEYWORD PAGE (Acts Like Robot/Crawler/Harvester) Disallow: / # NONSENSE USER-AGENTS # User-agent: sgulnofpfelsskodfxgn #[65.67.83.241] Disallow: / User-agent: ti jeybrwxggcwrceeorhplwiod johlrs3gx Disallow: / User-agent: 65.67.83.241 Disallow: / # TEMP-BLOCK "ALL=Disallow" #User-agent: * #Removed From Top #Disallow: / #Removed From Top # To Cover Bots, and Directories, Files (Images) Not Listed! User-agent: * # BLOCK ALL USER-AGENTS FROM ROOT "/", TEMPORARILY DEACTIVATED FOR TESTING! Disallow: / #TEST BLOCK ROOT: Disallow: / Disallow: /? Disallow: /& Disallow: /add-guest.php Disallow: /addguest.php Disallow: /ADMINISTRATIVE/ Disallow: /ADS/ Disallow: /AmerInd-Music/ Disallow: /badbot.html Disallow: /badspammer.html Disallow: /bbclone/ Disallow: /bbclone/? Disallow: /bbclone/show_detailed.php Disallow: /bbclone/show_detailed.php? Disallow: /bbclone/show_detailed.php?lng=en Disallow: /bbclone/show_views.php Disallow: /bbclone/show_views.php? Disallow: /bbclone/show_views.php?id=18716&lng=en Disallow: /botanical/mgmh/a/images/ Disallow: /botanical/mgmh/b/images/ Disallow: /botanical/mgmh/c/images/ Disallow: /botanical/mgmh/d/images/ Disallow: /botanical/mgmh/e/images/ Disallow: /botanical/mgmh/f/images/ Disallow: /botanical/mgmh/g/images/ Disallow: /botanical/mgmh/h/images/ Disallow: /botanical/mgmh/i/images/ Disallow: /botanical/mgmh/j/images/ Disallow: /botanical/mgmh/k/images/ Disallow: /botanical/mgmh/l/images/ Disallow: /botanical/mgmh/m/images/ Disallow: /botanical/mgmh/n/images/ Disallow: /botanical/mgmh/o/images/ Disallow: /botanical/mgmh/p/images/ Disallow: /botanical/mgmh/q/images/ Disallow: /botanical/mgmh/r/images/ Disallow: /botanical/mgmh/s/images/ Disallow: /botanical/mgmh/t/images/ Disallow: /botanical/mgmh/u/images/ Disallow: /botanical/mgmh/v/images/ Disallow: /botanical/mgmh/w/images/ Disallow: /botanical/mgmh/x/images/ Disallow: /botanical/mgmh/y/images/ Disallow: /botanical/mgmh/z/images/ Disallow: /cgi-bin/ Disallow: /cgi-bin/guestbook.php Disallow: /cgi-bin/guestbook/ Disallow: /cgi-bin/wwwboard/ Disallow: /genealogy-board/ Disallow: /genealogy-board/msg/ Disallow: /guest-book.php Disallow: /guestbook.php Disallow: /.htdocs/ Disallow: /.htdocs/jsActive/ Disallow: /.htdocs/jsDisabled/ Disallow: /Images Disallow: /Images/ Disallow: /Images/? Disallow: /images/ Disallow: /images/? Disallow: /mboard-gen-pub/ Disallow: /mboard-gen-pub/msg/ Disallow: /NAINDIANER/mitglied.lycos.de/frlakota/Images/ Disallow: /php/ Disallow: /PHP_Master/ Disallow: /PHP_TESTS/ Disallow: /PRIVATE/ Disallow: /SpamPot/ Sitemap: http:/www.nativeamericanembassy.net/siteindex/ # LEGEND: * = Violator of robots.txt rules #COPYRIGHT 2008 By Priscilla Jessie Renee Noaker