Optimal robots.txt for Opencart Store
What is robots.txt file. The Robots.txt - a text file that contains the site indexing parameters for the search engine robots.
What is the robots.txt
1) Excluding pages from the search index of a search engine;
2) Optimization of the site load by web crawlers.
On the Internet, a lot of examples of a robots.txt file for OpenCart. Here is the best example:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
User-Agent: *
Disallow: /*.js
Disallow: /*.css
Disallow: /*.jpg
Disallow: /*.jpeg
Disallow: /*.gif
Disallow: /*.png
Disallow: /*?
Allow: /index.php?route=feed/sitemap_pro
Allow: /index.php?route=product/category&path=
Allow: /*?path=
Allow: /index.php?route=product/product&product_id=
Allow: /index.php?route=product/product&path=
Allow: /*?product_id=
Allow: /index.php?route=information/information&information_id=
Allow: /index.php?route=product/special
User-agent: YandexImages
Disallow: /
Allow: /image/cache
User-agent: Googlebot-Image
Disallow: /
Allow: /image/cache
User-agent: msnbot-media
Disallow: /
Allow: /image/cache
Sitemap: http://site.com/robot.txt
|
