Keyword

Should the following be in robots.txt

  • boodoo
  • boodoo's Avatar Topic Author
  • Offline
  • Junior Member
More
11 years 5 months ago #112478 by boodoo
Hello again,

I notice all my images related to my K2 module are stored here:

www.mysite.com/media/k2/items/cache/

Therefore, if I want these crawled by googlebot etc, should I have the following line in my robots.txt:

Allow: /media/k2/items/cache/


Thanks!!

Please Log in or Create an account to join the conversation.

  • Krikor Boghossian
  • Krikor Boghossian's Avatar
  • Offline
  • Platinum Member
More
11 years 5 months ago #112479 by Krikor Boghossian
Replied by Krikor Boghossian on topic Re: Should the following be in robots.txt
Hello boodoo,

By default Joomla! (not K2) blocks the media and images folders from robots.
You can comment out these two lines in your robots.txt with a hashtag.

JoomlaWorks Support Team
---
Please search the forum before posting a new topic :)

Please Log in or Create an account to join the conversation.

  • boodoo
  • boodoo's Avatar Topic Author
  • Offline
  • Junior Member
More
11 years 5 months ago #112480 by boodoo
Replied by boodoo on topic Re: Should the following be in robots.txt
Thanks for the reply.

If I do not allow bots to crawl the /media/ folder (via robots.txt), will the K2 comments on my website still be crawled if I have links to that page?

Please Log in or Create an account to join the conversation.

  • Krikor Boghossian
  • Krikor Boghossian's Avatar
  • Offline
  • Platinum Member
More
11 years 5 months ago #112481 by Krikor Boghossian
Replied by Krikor Boghossian on topic Re: Should the following be in robots.txt
This is a template specific issue. Some templates have rel="nofollow" attributes in the comments sections.
If they don't have something like that then it's up to the search engine to honour the no-index request.

JoomlaWorks Support Team
---
Please search the forum before posting a new topic :)

Please Log in or Create an account to join the conversation.

  • boodoo
  • boodoo's Avatar Topic Author
  • Offline
  • Junior Member
More
11 years 5 months ago #112482 by boodoo
Replied by boodoo on topic Re: Should the following be in robots.txt
Thanks Krikor,

Last question. If I want my K2 images to be crawled by google image bot for example, which line should I add to my robots.txt (one with trailing backslash and one without).

Allow: /media/k2/items/cache/

or

Allow: /media/k2/items/cache

Please Log in or Create an account to join the conversation.

  • Krikor Boghossian
  • Krikor Boghossian's Avatar
  • Offline
  • Platinum Member
More
11 years 5 months ago #112483 by Krikor Boghossian
Replied by Krikor Boghossian on topic Re: Should the following be in robots.txt
As I said I would comment out the entire Disallow: media and Disallow: images rules in your position

JoomlaWorks Support Team
---
Please search the forum before posting a new topic :)

Please Log in or Create an account to join the conversation.


Powered by Kunena Forum