Problem with search on bilingual site

-- url of the page with the problem -- : www.collectcamera.com/nl/producten
-- HikaShop version -- : 4.2.1
-- Joomla version -- : 3.9.11
-- PHP version -- : 7.2.20

Hi Guys,

When searching (standard search) on this website we get different results in different languages.
In the default language everything is fine.
In the alternate language Dutch using Falang, we have several issues:
1. unpublished products and products with zero stock still show in the results.
2. we get funny extra dummy results showing with just a number, no title
3. first part of the product description shows in stead of the title
Any idea where I may be doing something wrong?

BTW: the site is still on Hikashop 4.1 because the new ajax display of the filters and product listing disagrees with my overrides.
I am still working to fix those. Will be done before the weekend

On the test site we have 4.2.1 installed and are testing with the SmartSearch because above issues were still there after the upgrade.
Here we find that Falang does not translate the search results, but that may be functionality that's not yet in the plug-ins.
Can you comment on that?

Thanks very much for your help and opinion.

Cheers,
André

Hi Nicolas,

Sorry for late reply.
In the mean time we updated the site to all latest versions.
We had to adapt our overrides for the filter and product listing to allow for the different dynamic display introduced in this version of Hikashop.

Part 1: Standard search
I will give more details about the site in question, hopefully something triggers your mind to put me straight.

There is an older copy of the site available to play with, I created a super user for you to check and try things.
I will send the details through the contact form.

A little background on the site:
This site is for a webshop selling collectors items of photographic equipment - the (sometimes very) old stuff.
Virtual all items are unique, so stock is generally set at 1.
The owner wanted the site to use 2 languages: EN_US and NL_NL.
So there are 3 languages in the site of which the Joomla default (EN_GB) is not used.

The site has been set up as a standard multilingual Joomla site, with the duplicated menu's, etc. and all that.
The Hikashop part has been made multilingual using Falang as that's essentially the only way to go.
Basically everything works correctly, site and product display, multiple level filters, checkout, etc.
Only when searching is when things start to go wrong and I do not understand what goes wrong where.
There are 3 things in this process: Joomla search, Hikashop search plug-in, Falang.

Please humor me and check out the following from the front-end:
Go to: www.collectcamera.com/colcam-test/en-us/
This enters the site in English.
You can browse through the products and filters and switch between languages, there is no problem there.

Now enter the search string: "summicron" and click "Search".
In EN you will get 20 items in the search results, in NL you'll get 30.
The last 2 shown items are dummies, showing just a sequence number, no content.
The extra real items are all "unpublished" products or products with no stock.
You can change language in the search results and see this effect.

Check the top item when the list is in NL and "newest first" => "Leica RE met Summicron...".
This item is unpublished and when clicking it you get an error 404 and get redirected to the home page.

Next try search phrase "leica m8".
This will yield 52 hits in EN and 53 in NL, now the dummy result in NL is not so obvious.
Now select the "Exact phrase" and the results go down to 3 (EN) resp. 4 (NL) and here the dummy is very obvious.
In this selection there are no unpublished products.

I have no explanation for this behaviour and have no idea where to look.
Being a programmer (but not aquainted with PHP and Joomla internals) I would generally expect a procedure like:
1. Search starts of extracting the relevant sections of the database into a list
2. Hika search plug-in adds the product contents and other selected field ( plug-in options) to this list.
3. Falang plug-in does its magic translating the content items in this list.
4. Search actually goes through the list searching for the keywords, obtaining results and displaying these.

So Falang should not add items to the list, only translate what's already there. I'm puzzled.

Earlier we had an override for the mod_search-default.php
We added a line to the search form to have the exact phrase as default setting at the start.
This worked ok as far as searching wnet, but with this setting we quite often had the titles of search results in NL replaced by the content part of the results, which looked really funny.
So we removed this override.

---

Part 2: SmartSearch
I understand your situation, this plug-in is brand new and will be improved in the future no doubt.
As the Joomla team will be phasing out the standard search eventually, the SmartSearch is the way to go in the future.
Essentially the Hikashop plug-in already works quit well.
Feeding the results through Falang may or may not be hard, I cannot be a judge on that - insufficient info.

One thing that did catch my eye is that search results (so also in the indexes) duplicate products for each category they are in.
In our case that results to the same item showing up 2 to 4 times in the results.
This is due to the structure we created to organize our products. This closely mimicks how our customers search for items on the site.
An item is "Leica equipment" from the "Leica M series".
At the same time it is in the product group "Lenses" - "Telefocus"
We implemented this through the Products (top menu) and Product Selection (side menu) then drill down using the filters.
Our override suppresses filters when there is no content/selection to show.

With the current implementation these items show up multiple times in the search results of SmartSearch because they are in multiple categories. Even when categories are not enabled for the search.

---

Thank you for your patience reading through all the above.
I hope you can help pinpoint the area or plug-in that is causing our troubles.

Cheers,
André

Attached are our settings for the standard search plug-in.
The selected fields are:
product_name,product_description,product_code,product_keywords,serial,year,catnumber
All 3 included custom fields are of type "text".

Thanks Nicolas for the quick fix.
I installed it and it is indeed much better - not there yet 100%, but I'm not complaining.

As to the SmartSearch plug-in:
Obviously being able to use the category in a search is extremely valuable in general.
Having the option to either use or not use it will allow different ways of using it like in our website.
So that's definitely a very good thing.

Thank you very much for the effort.
Cheers,
André

Hi Guys,

I reopen this topic from over a year ago, because today I have been digging into this problem and found the cause.
The fixed file is attached for your review.

Recapping the problem:
On a bilingual site (Dutch + USA) Falang is used to preform all product translations. The standard Joomla language system is used for everything else.
When a search is performed certain keywords would return search results where the meta-description is in the spot of the title in the secondary language only. For a few search keywords everything is fine.
The behaviour is not stable and 'right' and 'wrong' keywords may change over time.

My debugging and the fix are based on the latest version of Hikashop Search plugin.

While digging through the code I added lots of 'echo' statements to help me trace what is going on, I left them in (commented out) for the moment.
From these traces I noticed what was happening.
In essence the search plugin applies the search criteria to the products in the site, then Falang does its thing and all data captureed by the search plugin is already in the translated form.
The reults is an array of products that should show up in the results.
This array is then matched and changed over to the format required by Joomla Search.
At this point things may go wrong - ref the attached file lines 235-250.
Here Hika fields ProductName and ProductDescription are matched against required fields for Joomla search results.
As a default the ProductName is filled with the value of any other Hika field.
Depending on the order in which the fields appear in the preceding query the name may be inserted before or after that 'any' field.
By removing the setting of the default value only the actual ProductName will be inserted here, solving our problem.
Perhaps a more intelligent way could be thought of for setting a default value in case the ProductName turns out to be blank/missing.


I had a bit of a problem understanding what was happening in the section at lines 285-345.
An previously array variable is reused and seemingly added to. To me it seems that actually 2 different arrays should be used as use of the one might interfere woth the other.
For that reason I changed the name of one array from $row to $rowx.
This may or may not have anything to do with my problem. I did not want to revert just to find this out.

I leave it to you guys to review and clean up as needed

For now I'm happy and my customer is happy. So all is well.

Thanks a lot for a great product and great support.

Cheers,
André

PS: Attached file is: /plugins/search/hikashop_products/hikashop_products.php

Hi Nicolas,

Thanks for the reply.
I reverted back all the $rowx => $row.

I made a quick check and found that the canonical field is usually triggering this particular else-clause.
The canonical is properly filled in for all our products otherwise google/bing-search might possibly return bad results.
I also noted that the product_name is already present in its translated form before the translation loop is started.

The change you suggested therefore results in the product_name having the canonical appended to it (which is truncated again at display time).
So a cleaner and more defensive solution would be to check first if the product_name is empty and fill it only if not.
So the line might read something like:
if (!$row->product_name) $row->product_name=$item->value;

It's your code, so I'm just suggesting...

Hi Nicolas,

Sorry for late reply, but I have been away from office for a few days and could not respond.

I implemented your latest solution and can confirm that it works correctly.
Your addition to the SQL speeds thing up a bit as less items now need to be checked - all items not relevant to the search and results are now filtered out.

The extending of the ProductDescription only has effect in combination with relatively short descriptions as the description is truncated at display time to just a few lines.
So the person performing the search in most cases will just have to trust the correctness of the results anyway.

Thanks a lot for your help.
André