projects
/
irspy-moved-to-github.git
/ blobdiff
commit
grep
author
committer
pickaxe
?
search:
re
summary
|
shortlog
|
log
|
commit
|
commitdiff
|
tree
raw
|
inline
| side by side
avoid double utf8 encoding, IR-355
[irspy-moved-to-github.git]
/
web
/
htdocs
/
robots.txt
diff --git
a/web/htdocs/robots.txt
b/web/htdocs/robots.txt
index
beef7ee
..
a0fdc6c
100644
(file)
--- a/
web/htdocs/robots.txt
+++ b/
web/htdocs/robots.txt
@@
-1,8
+1,15
@@
User-agent: *
User-agent: *
-Disallow: /full.html
-Disallow: /all.html
+Disallow: /admin/
Disallow: /add.html
Disallow: /add.html
-Disallow: /upload.html
+Disallow: /add_target.html
+Disallow: /all.html
+Disallow: /doc.html
+Disallow: /find.html
+Disallow: /full.html
Disallow: /raw.html
Disallow: /raw.html
-Disallow: /admin/
+Disallow: /stats.html
+Disallow: /upload.html
+
+User-agent: rogerbot
+Disallow: /