Skip to:
Content
Pages
Categories
Search
Top
Bottom

robots.txt help


  • memnunfor
    Member

    @memnunfor

    i want to disallow the ”search” and the ”user name” urls,what should i write in robots.txt?

    Thanks.

Viewing 8 replies - 1 through 8 (of 8 total)

  • Greitas kreditas
    Member

    @greitas-kreditas

    Hey,

    it should look something like this:

    User-agent: *

    Disallow: /user-name/

    Disallow: /search/


    Greitas kreditas
    Member

    @greitas-kreditas

    Hey,

    it should look something like this:

    User-agent: *

    Disallow: /user-name/

    Disallow: /search/

    There is one new method, which is mostly used for SEO:

    <meta name=”robots” content=“noindex, follow”/>

    You should put this to all the pages you don’t want to get to SE’s index.

    Your pages won’t be indexed, but still will pass link juice.

    There is one new method, which is mostly used for SEO:

    <meta name=”robots” content=“noindex, follow”/>

    You should put this to all the pages you don’t want to get to SE’s index.

    Your pages won’t be indexed, but still will pass link juice.

    Do I need to remove “Disallow” from robots file then? How should this file look after I updated my pages with this code <meta name=”robots” content=“noindex, follow”/>?

    Do I need to remove “Disallow” from robots file then? How should this file look after I updated my pages with this code <meta name=”robots” content=“noindex, follow”/>?


    atlieku surinkimas
    Member

    @atlieku-surinkimas

    Yes, you can leave only:

    User-agent: *

    Disallow:

    Then it should work just fine.


    atlieku surinkimas
    Member

    @atlieku-surinkimas

    Yes, you can leave only:

    User-agent: *

    Disallow:

    Then it should work just fine.

Viewing 8 replies - 1 through 8 (of 8 total)

You must be logged in to reply to this topic.