Skip to:
Content
Pages
Categories
Search
Top
Bottom

robots.txt help


  • memnunfor
    Member

    @memnunfor

    i want to disallow the ”search” and the ”user name” urls,what should i write in robots.txt?

    Thanks.

Viewing 8 replies - 1 through 8 (of 8 total)

  • Greitas kreditas
    Member

    @greitas-kreditas

    Hey,

    it should look something like this:

    User-agent: *

    Disallow: /user-name/

    Disallow: /search/


    Greitas kreditas
    Member

    @greitas-kreditas

    Hey,

    it should look something like this:

    User-agent: *

    Disallow: /user-name/

    Disallow: /search/


    Amber 11
    Member

    @amber-11

    There is one new method, which is mostly used for SEO:

    <meta name=”robots” content=“noindex, follow”/>

    You should put this to all the pages you don’t want to get to SE’s index.

    Your pages won’t be indexed, but still will pass link juice.


    Amber 11
    Member

    @amber-11

    There is one new method, which is mostly used for SEO:

    <meta name=”robots” content=“noindex, follow”/>

    You should put this to all the pages you don’t want to get to SE’s index.

    Your pages won’t be indexed, but still will pass link juice.


    Voayge
    Member

    @voayge

    Do I need to remove “Disallow” from robots file then? How should this file look after I updated my pages with this code <meta name=”robots” content=“noindex, follow”/>?


    Voayge
    Member

    @voayge

    Do I need to remove “Disallow” from robots file then? How should this file look after I updated my pages with this code <meta name=”robots” content=“noindex, follow”/>?


    atlieku surinkimas
    Member

    @atlieku-surinkimas

    Yes, you can leave only:

    User-agent: *

    Disallow:

    Then it should work just fine.


    atlieku surinkimas
    Member

    @atlieku-surinkimas

    Yes, you can leave only:

    User-agent: *

    Disallow:

    Then it should work just fine.

Viewing 8 replies - 1 through 8 (of 8 total)

You must be logged in to reply to this topic.