CSV Importing is too slow. Solution?

  • Posts: 156
  • Thank you received: 0
6 years 4 weeks ago #290455

-- HikaShop version -- : 3.3.2
-- Joomla version -- : 3.8.5
-- PHP version -- : 5.6

I am using a shared server but during the importing of only 6 products (with variants = 40 item) Hikashop takes at least 10-15 second.
It means with 18.000 there will be no chances to import all items?

Solution? Maybe a VPS could be the right choice?

Please Log in or Create an account to join the conversation.

  • Posts: 81539
  • Thank you received: 13069
  • MODERATOR
6 years 3 weeks ago #290471

Hi,

A more powerful server might help, but it might not help much.
For example, if the issue is that you have URLs for the images of the variants/products, then for each one, HikaShop has to download the image and store it on the server, before it can save the product in the database.
A more powerful server won't help here and in that case, it would be best to harvest the images yourself, upload them in the upload folder of HikaShop and only have the name of the image files in your CSV. That would probably give a 95% speed increase.

Now I don't know what you have in your CSV, so I can't say anything decisive.

Please Log in or Create an account to join the conversation.

  • Posts: 156
  • Thank you received: 0
6 years 3 weeks ago #290759

nicolas wrote: Hi,

A more powerful server might help, but it might not help much.
For example, if the issue is that you have URLs for the images of the variants/products, then for each one, HikaShop has to download the image and store it on the server, before it can save the product in the database.
A more powerful server won't help here and in that case, it would be best to harvest the images yourself, upload them in the upload folder of HikaShop and only have the name of the image files in your CSV. That would probably give a 95% speed increase.

Now I don't know what you have in your CSV, so I can't say anything decisive.


Is there a way to start importing only the images and then the full csv (without images)?

It could be useful to get more performace, using the server cronjobs and not hikashop cron plugin?

Last edit: 6 years 3 weeks ago by SuX.

Please Log in or Create an account to join the conversation.

  • Posts: 81539
  • Thank you received: 13069
  • MODERATOR
6 years 3 weeks ago #290782

Hi,

Using the server cronjob won't change anything.
The cron only triggers the process and the process is already done by your server.

If you want to import the product data without the images data, just change the column with the name "images" to something else like for example "images_not_imported" and the column will be ignored.
That way, you can see if the issue comes from the images or not.

Please Log in or Create an account to join the conversation.

  • Posts: 156
  • Thank you received: 0
6 years 3 weeks ago #290821

nicolas wrote: Hi,

Using the server cronjob won't change anything.
The cron only triggers the process and the process is already done by your server.

If you want to import the product data without the images data, just change the column with the name "images" to something else like for example "images_not_imported" and the column will be ignored.
That way, you can see if the issue comes from the images or not.


I guess I could improve the upload speed setting only the image reading and not a copy. Right?
Do you now a simple way to quick upload the images? Consider for each csv row, I have 3 external image path (in three single column) and about ten thousand row.
If I will be to do that, then I could import just csv text.

Please Log in or Create an account to join the conversation.

  • Posts: 81539
  • Thank you received: 13069
  • MODERATOR
6 years 2 weeks ago #290825

Hi,

I don't know of any tool to automatically download the URLs of a CSV.
A custom script could do that if you or a developer you would hire would develop it.

If you already have the images on your computer, then you could easily upload them all with FTP.

Please Log in or Create an account to join the conversation.

Time to create page: 0.068 seconds
Powered by Kunena Forum