Shop is very slow everywhere i click

  • Posts: 24
  • Thank you received: 0
10 years 4 weeks ago #211750

-- HikaShop version -- : 2.5
-- Joomla version -- : 3.4
-- PHP version -- : 5.3
-- Browser(s) name and version -- : Chrome,Firefox,IE,Edge

Need helps
21 second page load - 2800 Brands ? On mysql takes less than 1 sec
18 sec search results - on mysql takes less than 1 sec to query same item

Seems like everything shop related is slowed to crawl.
- Search(filter)
- Brands
- Category
- Clicking Links from results even..
- Click Brand link from Prod details
Anything not shop related open sub 1 sec.

Hikashop Updated to current version - This is dedicated server with 6core 16gb ram SSD..
Please Advise and thanks in advance

Please Log in or Create an account to join the conversation.

  • Posts: 84305
  • Thank you received: 13700
  • MODERATOR
10 years 4 weeks ago #211757

Hi,

How many products and categories do you have ?
Displaying a 100 products a already quite a lot. When you do that it needs to load the product data, the images, the prices, the badges, the variants, the options, the custom fields etc for each product. So it can amount to a lot of queries, but also a lot of data to process in PHP. It's not the same as displaying the product entries from only one table via PHPMyAdmin.
But there are ways to improve it I suppose. First, correctly configuring the MySQL server to have the tables in RAM is a good idea if not already done. Also, adding indexes to the database might be necessary based on how the listings and filters are configured in order to speed up the queries.
Analyzing the queries done on your pages would be necessary to understand what's slowing down the process. It can be done by turning on the "debug mode" setting of the Joomla configuration.

Also, caching can be activated in HikaShop to further speed up the display of the pages.

The following user(s) said Thank You: tampasounds

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
10 years 4 weeks ago #211804

edited

Last edit: 9 years 11 months ago by tampasounds.

Please Log in or Create an account to join the conversation.

  • Posts: 84305
  • Thank you received: 13700
  • MODERATOR
10 years 4 weeks ago #211874

Hi,

I've checked and here is what I said for reference:

You have a really simple filter so it should be able to handle a few 100 000 products with maybe a 1 second processing time on a normal server (it's basically the limit of the MySQL query processing by your MySQL server here and since the query isn't complex, it should be quite fast).


Now, when I said "Displaying a 100 products a already quite a lot." in my last message, I meant that displaying these 100 products on the same page is a lot. By default, products listings display only 20 products per page. So that's 5 times the default amount.
And that has nothing to do with the total number of products stored in the database claim I made.
Also, I said "should" not "would" in my email. While "would" imply that you basically guarantee that it will happen, "should" only tells the other that you're confident that it will be the case.

But we're not here to discuss wording. We're both here to fix your issue with your load time. So could you answer my questions and try the different things I talked about so that we can help you solve the problem ? How many products do you have in total ?
Did you turn on the debug mode of the Joomla configuration and looked at the debug data on one of the slow pages ? Which queries were slow ?

The following user(s) said Thank You: tampasounds

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
10 years 4 weeks ago #211884

There are only approx 400k Product atm. We are not near our goal.
This is not a normal server. It is a private dedicated server with dual 6 core Xeons with 16gb ram and SSD...(Dell T620)

I agree, I am not here to split hairs with you and apologize for my reaction when misunderstanding your statement
I really did try not to over react

Debug mode is currently turned on. I can leave it like this for next 5 hours before regular business opens

Places where it is exceptionally slow are.
Brands Listing Page (linecard) (30 sec)
Click Brand link to Product Listing Table (no images) I test with brand that only had 20 items (27 sec)
Clicking Product link from Listing table is slow but almost bearable (11 sec)
Clicking Aero category at bottom is slow but better (12 sec)
Search Filter for exact match (36 seconds)

I noticed that it complains a little about duplicates. I did not think this was possible since i imported using the shops import feature.
Seems to be a lot of queries but my chrome crashes before i can see them all
Memory seems low only 20mb but references to hikashop are flagged red with 4k-7k ms latency

And i should say again , thank you in advance

Please Log in or Create an account to join the conversation.

  • Posts: 84305
  • Thank you received: 13700
  • MODERATOR
10 years 3 weeks ago #212000

Hi,

Great, we're moving forward !
So, here is for example a page:
take.ms/cHC3x
You can see that the MySQL queries takes 13 seconds, while the total processing of the page is 14 seconds.
So almost everything is eaten up by queries to the database.
Then, if you browse through the queries there, you can see this query:
take.ms/nNhkR
Only this query eats by itself 10 seconds of the processing.
Would it be possible to get an access to your phpmyadmin in order to run some tests with that query on the database in order to understand what is the exact drag on that query?
Maybe adding an index somewhere will solve the issue but without your data in the tables, it's hard to give a definitive answer.

If that's ok, you can provide it with a link to this thread via our contact form:
www.hikashop.com/support/contact-us.html

You can also turn off the debug mode for now.

The following user(s) said Thank You: tampasounds

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
10 years 2 weeks ago #212924

I sent you that info last week but never heard back ?

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
10 years 1 week ago #213565

Another week has gone by...
Hello support are you there ? :(

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
10 years 1 day ago #214163

edited

Last edit: 9 years 11 months ago by tampasounds.

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
10 years 14 hours ago #214296

edited

Last edit: 9 years 11 months ago by tampasounds.

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
9 years 11 months ago #214356

edited

Last edit: 9 years 11 months ago by tampasounds.

Please Log in or Create an account to join the conversation.

  • Posts: 84305
  • Thank you received: 13700
  • MODERATOR
9 years 11 months ago #214375

Sorry for the delay. I've had a lot on my plate in the last few weeks and such kind of debugging takes a lot of time.
I'm going to look at the issue this afternoon and get back to you.

The following user(s) said Thank You: tampasounds

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
9 years 11 months ago #214376

edited

Last edit: 9 years 11 months ago by tampasounds.

Please Log in or Create an account to join the conversation.

  • Posts: 84305
  • Thank you received: 13700
  • MODERATOR
9 years 11 months ago #214379

So I've run some tests on your SQL server.

The query

SELECT DISTINCT b.*
FROM jos_hikashop_product_category AS a
LEFT JOIN jos_hikashop_product AS b 
ON a.product_id=b.product_id 
WHERE b.product_published=1
AND b.product_type='main'
AND a.category_id IN (42)
AND (b.product_access = 'all' OR b.product_access LIKE '%,1,%')
ORDER BY ordering ASC
LIMIT 0,100
takes 3 seconds when you run it directly on phpmyadmin.
The issue comes from the ordering.
There are 81936 products in the category with the id 42.
So when your MySQL server wants to order the entries, it has to order all of these elements and then send you the 100 first elements. But in its memory, it has to generate a temporary table of all these 81K elements.

HikaShop has been made to work with a lot of products and a lot of categories, but we never expected that someone would have so many products in each idividual category.
Removing the JOIN is not possible as otherwise it would display all the products, not just the products of that category. And removing the query is not possible either as it is the query loading the products data, so it is necessary. And that's the query taking most of the time on the page, so that's where the problem is.
The simplest here will be to remove the ordering. As the products have apparently already been input in the correct order, it should produce the same result.
To do that, remove the line:
$order = ' ORDER BY '.$pageInfo->filter->order->value.' '.$pageInfo->filter->order->dir;
in the file components/com_hikashop/views/product/view.html.php

That's however a change we can't add in HikaShop. Having no ordering on the products listings would be a problem for a lot of our customers. The problem here is not really the way HikaShop is done, but the product/category structure.
I would recommend to not have more than a few thousands of products in the same category.

There might be some other issues elsewhere but that modification should already help a lot.

It is not false advertising that HikaShop can work with hundred of thousands of products. You can see that the pages display, albeit slow, even though the product/category strcuture is far from optimal.
You have to understand that here, the problem is not in HikaShop but in the huge amount of products you have, the category structure, and the optmization of the MySQL server (caching the queries, putting the tables in the RAM, etc). The query itself in HikaShop cannot be simplified as it does what it should do.

The following user(s) said Thank You: tampasounds

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
9 years 11 months ago #214381

When i profiled that one query i saw the same thing. When you remove the order by in the in-line editor it really made no difference.
Maybe it was a bad pull but it actually took 5.67 sec when i tried...

But my first and main concern is the search/filter
You told me we could do 1 million products and expect search result times to be 100k per sec
With 400k products i expected 4 sec not 27-32sec

I am not asking you to change the product for everyone
But if you can help me modify our unique (site) situation it would be forever and greatly appreciated
I'd even be willing to never update it again if we can achieve some acceptable results

The way we work (with this one specific site) we don't even need the category with the 82k ...
They can all be uncategorized really.
We do like the ability to filter by brand though from the one page (brands) only...

Last edit: 9 years 11 months ago by tampasounds.

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
9 years 11 months ago #214383

edited

Last edit: 9 years 11 months ago by tampasounds.

Please Log in or Create an account to join the conversation.

  • Posts: 84305
  • Thank you received: 13700
  • MODERATOR
9 years 11 months ago #214389

Hi,

If you try the query on your phpmyadmin with the ordering, you can see that it takes more than 3secs. And if you remove the ordering it takes less than 1ms. So the ordering is the problem for that specific query.

Removing the categories might help, but that's not the main issue.

The filters performances will depend on how the filters are configured and what the data is.
For maximum performances, instead of a brand filter, I would recommend a custom product field filter.
That way, no JOIN is required as the fitlering will be done directly with the product column on the hikashop_product table.
For the search on the name, the issue is the same as what I explained previously with the ordering. Removing the ordering should help.
If that's not enough, removing the categories JOIN will help. It can be done by modifying the code:

if(empty($select)){
			$parentCategories = implode(',',$pageInfo->filter->cid);
			$catName = 'a.category_id';
			$type = 'product';

			if(!empty($element->category_type) && $element->category_type=='manufacturer'){
				if($pageInfo->filter->order->value=='a.ordering' || $pageInfo->filter->order->value=='b.ordering'){
					$pageInfo->filter->order->value='b.product_name';
				}
				$type = 'manufacturer';
				$catName = 'b.product_manufacturer_id';
				$b = '';
				$a = hikashop_table('product').' AS b';
				$on = '';
				$select='SELECT DISTINCT b.*';
			}else{
				if($pageInfo->filter->order->value=='b.ordering'){
					$pageInfo->filter->order->value='a.ordering';
				}
				$b = hikashop_table('product_category').' AS a LEFT JOIN ';
				$a = hikashop_table('product').' AS b';
				$on = ' ON a.product_id=b.product_id';
				$select='SELECT DISTINCT b.*';
			}
			if($this->params->get('filter_type',2)==2){
				$defaultParams = $config->get('default_params');
				$this->params->set('filter_type',$defaultParams['filter_type']);
			}
			if(!$this->params->get('filter_type')){
				if(!empty($parentCategories)&& $parentCategories!='0') $filters[]=$catName.' IN ('.$parentCategories.')';
			}else{
				$categoryClass->parentObject =& $this;
				$categoryClass->type = $type;

				$children = $categoryClass->getChildren($pageInfo->filter->cid,true,array(),'',0,0);
				//
				// Depending the number of $children, we can determine the number of categories (without filter), check if we have the same number and not put the filter in this case.
				// We can also use an "NOT IN" if there are a lot of $children and the diff between $children and $allcategories gave a few number of categories.
				//
				$filter = $catName.' IN (';
				foreach($children as $child){
					$filter .= $child->category_id.',';
				}
				$filters[]=$filter.$parentCategories.')';
			}
		}
in the same file, just before the ordering.

The following user(s) said Thank You: tampasounds

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
9 years 11 months ago #214390

edited

Last edit: 9 years 11 months ago by tampasounds.

Please Log in or Create an account to join the conversation.

  • Posts: 24
  • Thank you received: 0
9 years 11 months ago #214391

edited

Last edit: 9 years 11 months ago by tampasounds.

Please Log in or Create an account to join the conversation.

  • Posts: 84305
  • Thank you received: 13700
  • MODERATOR
9 years 11 months ago #214540

Hi,

When you search by name, the problem of the query is the left join to the product_category table. That's why I proposed removing the that code which handles the join.
So yes, we're talking about the same thing.

As I said in a previous message, I'm talking about the file components/com_hikashop/views/product/view.html.php in the listing function. That's where the query loading the products is done and that's the one causing you the slow page processing. So that's what you want to modify.

You already provided a backend access to the website and I was able to check your settings already, so it is not necessary to provide it again.

The following user(s) said Thank You: tampasounds

Please Log in or Create an account to join the conversation.

Time to create page: 0.074 seconds
Powered by Kunena Forum