Robots.txt для LiveStreet
Готовый robots.txt выглядит так:
User-agent: *
Disallow: /people
Disallow: /blogs
Disallow: /tags
Disallow: /tag
Disallow: /top
Disallow: /comments
Disallow: /talk
Disallow: /search
Disallow: /classes Disallow: /config Disallow: /include Disallow: /logs Disallow: /templates Disallow: /login Disallow: /settings Disallow: /robostat Disallow: /error Disallow: /profile Disallow: /registration Disallow: /blog Disallow: /slovar Disallow: /bad Disallow: /personal_blog Disallow: /new Disallow: /my Disallow: /talk Disallow: /error Disallow: /link Disallow: /index
Данный вариант обеспечивает максимально чистые результаты поиска
comments powered by HyperComments