diff options
| author | jomo <github@jomo.tv> | 2014-06-25 02:18:03 +0200 |
|---|---|---|
| committer | jomo <github@jomo.tv> | 2014-06-25 02:18:03 +0200 |
| commit | bfe68ce1613a6235e69945a926c028aa9c5c920d (patch) | |
| tree | 033f637b3eefa50b1b13cdc3ace21f3641dac95c /public | |
| parent | 194cd0d658a272a2498e3c3edf8314762621be57 (diff) | |
robots.txt
Diffstat (limited to 'public')
| -rw-r--r-- | public/robots.txt | 60 |
1 files changed, 59 insertions, 1 deletions
diff --git a/public/robots.txt b/public/robots.txt index 27b5b1d..8c507d8 100644 --- a/public/robots.txt +++ b/public/robots.txt @@ -1 +1,59 @@ -# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file +# +# .-. +# ( ) +# '-' +# J L +# | | +# J L +# | | +# J L +# .-'.___.'-. +# /___________\ +# _.-""' `bmw._ +# .' `. +# J `. +# F L +# J J +# J ` +# | L +# | | +# | | +# | J +# | L +# | | +# | ,.___ ___....--._ +# | ,' `""""""""' `-._ +# | J _____________________`-. +# | F .-' `-88888-' `Y8888b.`. +# | | .' `P' `88888b \ +# | | J # L # q8888b L +# | | | | )8888D ) +# | J \ J d8888P P +# | L `. .b. ,88888P / +# | `. `-.___,o88888o.___,o88888P'.' +# | `-.__________________________..-' +# | | +# | .-----.........____________J +# | .' | | | | +# | J---|-----..|...___|_______| +# | | | | | | +# | Y---|-----..|...___|_______| +# | `. | | | | +# | `'-------:....__|______.J +# | | +# L___ | +# """----...______________....--' +# +# +User-agent: * + +# Laws of Robotics +Disallow: /harming/humans +Disallow: /ignoring/human/orders +Disallow: /harm/to/self + +# 10 seconds until next request +Crawl-delay: 10 + +# People don't like being stalked ;) +Disallow: /users
\ No newline at end of file |