.htaccess - htaccess regex Keep spiderbots outside -


i can't part of htaccess working :

rewriteengine on rewritebase /  rewritecond %{http_user_agent} ^(360spider|acoon|ahrefs|aihit|altushost|amazon|archive|airjordanin|applecreek|baidu|bezeqint|bilbo|bork|botje|butterfly|buy) [nc,or] rewritecond %{http_user_agent} ^(catchbot|career|cloud-ips|coach|contabo|corbina) [nc,or] rewritecond %{http_user_agent} ^(dataprovider|digext|discobot|dotbot|ec2|edisterbot|energis|ephorus|exabot|exigen|ezooms|fastlink|genieo|huaweisymantecspider) [nc,or] rewritecond %{http_user_agent} ^(iguana|ileadz|ipxserver|jaunty|jadynave|jikespider|keywordspy|kimsufi|kobala|komodiabot|koula|longchamp) [nc,or] rewritecond %{http_user_agent} ^(magpie|majestic12|malas|media|michael|mj12bot|modx|mohitseo|mor306|mulberryoculos) [nc,or] rewritecond %{http_user_agent} ^(nashirnet|ncsa|netwiz|obot|openindex|osso|panscient|pathcom|peoplepal|piki|presto|purity) [nc,or] rewritecond %{http_user_agent} ^(santrex|scoutjet|shabnet|sistrix|siteexplorer|softlayer|sogou|sosospider|spinn3r|splims|statsbot|survey) [nc,or] rewritecond %{http_user_agent} ^(tangoo|targetdevelopmentdirect|thunderstone|trabajoo|track|trellian|trendictionbot|turnitinbot|tweetmemebot|ukraine) [nc,or] rewritecond %{http_user_agent} ^(vagabondo|vanekeris|vectranet|webcijfers|wise|yabrowser|yandex|yodao|zh-cn) [nc]  # issue 403 / serve errordocument rewriterule . - [f,l]

tried without ^ @ beginning , $ @ end bots still coming, regarding statistics in php tried instead of http_user_agent referer , host , forwarded, , in numbers remote_address.

any ?

the rule looks far, though change 2 things.

  • exclude robots.txt rule, because many of these adhere robots exclusion standard. add 

    rewritecond %{request_uri} !^/robots.txt$

    and fill robots.txt meaningful set of rules. if want exclude robots 

    user-agent: * disallow: /

  • change . ^, because home page not yet included in rule

    rewriterule ^ - [f]

Comments

Post a Comment

Popular posts from this blog

c# - Send Image in Json : 400 Bad request -

from android trying send image class of data, iis webservice. (c#) the problem 400 bad request . the image being encoded base64 . , placed json rest of class elements. my guess base64 not valid inside json. server not understand it. if set string "" , post accepted fine. so question is, how make base64 valid within json array? ( tried url.encode no success). or how should send image android webservice? gson gson = new gson(); string json = gson.tojson(record); // record has param { string base64photo } how big image? i'm pretty sure surpassing iis json size limit (default 4 mb). check http://geekswithblogs.net/frankw/archive/2008/08/05/how-to-configure-maxjsonlength-in-asp.net-ajax-applications.aspx or http://www.webtrenches.com/post.cfm/iis7-file-upload-size-limits good luck!
Read more

jquery - Fancybox - apply a function to several elements -

this source code below posted cristiano g. carvalho in answer question: calling fancybox gallery other link made user. <div class="details_gallery"> <a href="#" class="manualfancybox">manual call fancybox</a> <div class="details_gallery_min"> <a rel="details" href="max/1.jpg" class="fancybox"><img src="min/1.jpg" alt="" /></a> <a rel="details" href="max/2.jpg" class="fancybox"><img src="min/2.jpg" alt="" /></a> <a rel="details" href="max/3.jpg" class="fancybox"><img src="min/3.jpg" alt="" /></a> <a rel="details" href="max/4.jpg" class="fancybox"><img src="min/4.jpg" alt="" /></a> </div> </div> <script> $(document).read
Read more

An easy way to program an Android keyboard layout app -

i program android app replaces standard keyboard 1 one-hand optimized coffee++ keyboard layout. what special problems have face in progress? guess such project go deep android core, cause keyboard such essential thing. is wise start android developer tools (adt)—eclipse plugin described in this tutorial ? or there better way achieve goal? i new android programming, firm in php, mysql , javascript , use eclipse php i guess such project go deep android core, cause keyboard such essential thing. no. can create inputmethodservice implementation of keyboard. there sample soft keyboard in android sdk installation (if chose download sample code sdk manager), , there open source input methods floating around well, such hacker's keyboard. inputmethodservice distributed part of ordinary android application, , user can elect activate input method if user chooses. is wise start android developer tools (adt)—eclipse plugin described in tutorial? that fine s
Read more