Generation of useless Urls

-- url of the page with the problem -- : leza-shaving.com
-- HikaShop version -- : 2.6.1
-- Joomla version -- : 3


Hello,
Problem is that our site continues to generate URLs like:
Example 1:
leza-shaving.com/en/store/vm/checkout/notice/cart_type-cart/tmpl-component
leza-shaving.com/en/aftershave/checkout/notice/cart_type-wishlist/tmpl-component
leza-shaving.com/en/store/thiers-issard/checkout/notice/cart_type-cart/tmpl-component

Example 2:
leza-shaving.com/en/home/index.php?option=com_hikashop&view=product&layout=listing&Itemid=143&lang=ru
leza-shaving.com/en/component/hikashop/category/mugs
leza-shaving.com/en/component/hikashop/category/podstavki
leza-shaving.com/en/component/hikashop/category/krem
leza-shaving.com/en/component/hikashop/

Example 3
leza-shaving.com/ru/hikashop-menu-for-brands-listing/category/chashi
leza-shaving.com/ru/hikashop-menu-for-brands-listing/category/gel
leza-shaving.com/ru/hikashop-menu-for-brands-listing/category/chekhly

If we can`t stop this madness through hikashop menu, ss there a way to hide those URLs from search engines, using robots.txt?
We already have Disallow: /checkout/ in our rules, does it stop engines to analyse pages from example 1?

Hi,

For Example 1, the URLs are already set to not being indexed thanks to a meta data tag in the header: take.ms/3MEo7
So that shouldn't be a problem.

For Example 2 and 3, you need to enter the URL you want to be used for each product and category in their "canonical URL" field (the URL you set needs to be among the ones generated for the product/category). That way, it will also add a meta data tag to tell the search engines to use that URL instead of the others and thus it won't be content as duplicate.

nicolas wrote: For Example 2 and 3, you need to enter the URL you want to be used for each product and category in their "canonical URL" field (the URL you set needs to be among the ones generated for the product/category). That way, it will also add a meta data tag to tell the search engines to use that URL instead of the others and thus it won't be content as duplicate.

Thx for fast reply Nicolas.
It may sound funny, but how do I know the "canonical URL" for categories? With products it`s easy, their url can be accessed via frontend.
But where do I find urls for categories, if they are hidden on my site?

I also encountered some 404 pages like:
ru/hikashop-menu-for-categories-listing/proraso/75-preshave-sensitive
ru/hikashop-menu-for-brands-listing/aktsiya/view-article/id-5
ru/hikashop-menu-for-brands-listing/omega/view-article/id-18
ru/store/eshave/checkout/notice/cart_type-wishlist/tmpl-component

Is there a way to hide them in robots.txt? Its code shouldn`t be too hard, but I`m just too afraid to make a mistake.
Disallow: */hikashop-menu-for-brands-listing/
Disallow: */checkout/
Am I correct?

Hi,

They are not hidden at all.
For example, that's the link you want to use I guess for the canonical URL of "gift sets":
leza-shaving.com/en/store/gift-sets
It's the link you have on the left "store" module.
You can remove the language tag in them so that they are language agnostic.

Please take a closer look once again. Left menu contains only brands and as an exception - mentioned by you "gift sets".
Gels, soaps, mugs and etc Urls are not seen.

Moreover, I also encountered some 404 pages like:
ru/hikashop-menu-for-categories-listing/proraso/75-preshave-sensitive
ru/hikashop-menu-for-brands-listing/aktsiya/view-article/id-5
ru/hikashop-menu-for-brands-listing/omega/view-article/id-18
ru/store/eshave/checkout/notice/cart_type-wishlist/tmpl-component

Is there a way to hide them in robots.txt? Its code shouldn`t be too hard, but I`m just too afraid to make a mistake.
Disallow: */hikashop-menu-for-brands-listing/
Disallow: */checkout/
Am I correct?

Hi,

You sure can hide them with the robots.txt that's not a problem.

I`m just too afraid to make a mistake, but it must be easy once you know how to do it Can you please confirm my approach:
Disallow: */hikashop-menu-for-brands-listing/
Disallow: */checkout/
Am I correct?

Hi,

The best would be to have a look at this documentation and use the tester:
support.google.com/webmasters/answer/6062596?hl=en