Messages

Yes. The problem seems not caused by abantecart. The temp. files are in /system/cache folder. But It doesn't have this problem by using other shopping cart. I need to investigate it.
Thank you for your information.

I have 1079 categories and 31500 products. My website was crashed 3 times after I have set up the website 3 weeks ago.
I find that there are too many temp. files in the /tmp folder (sess_*). The first 2 times made the website stopped more than 30 minutes.
After I had cleared all the temp. files in /tmp, it worked again.
But this time, my debian server is crashed. After I reboot the server, the apache, mysql, email, ftp can't be started, the server hard disk becomes read only.
I have remount the hard disk with read/write manually and remove the disk quota in the /etc/fstab. Then the server can boot up properly.
I don't know what is the really problem causes this happened. But I think that if abantecart can create some sub-folders to store the temp. files, then it may solve some problems. (For exmaple, create sub-folder /tmp/sess_00/, /tmp/sess_01/, /tmp/sess_a0/, /tmp/sess_aa/, /tmp/sess_zz/, .....  and put the temp. files sess_01xxxxxxxxxx to /tmp/sess/01, files sess_aaxxxxxxx to /tmp/sess_aa/ , etc).
Hope that abantecart can improve it.
Thank you.

Thank you!

I find that signup up with 2 characters login name which can pass the error check when press F5 to refresh the screen. Abantecart will create the customer account during this process for login name which is shorter than 5 characters.
For the 1st time verification, the error message occurrs (Login name must be alphanumeric only and between 5 and 64 characters!). But if I ignore it and press F5 to refresh the screen, then it will create an account successfully.
Please fix this problem. (abantecart v1.2.10)
Thank you!

I disable "Display Stock" in backend->setting->general.
Then add an option in product with a few options with quantity and some with 0 quantity. The options with 0 quantity will show "In stock" in frontstore product page.
Please fix it.  (abantecart v1.2.10)
Thank you!

Abantecart 1.2.10
Backend
Edit product (Edit Order Details) in order edit doesn't work. The edit product popup windows is nothing inside.
Please fix it.

[However, open the product listing page in a category where it has 10-20pcs of products is the same time as the category where it has 450K products....]

I have tested it with 450K products.
The backend product browsing is improved. It takes 13-15 seconds to jump to next page previously but now only 4-5 seconds

The frontstore home page: It takes 1-2 minutes to load the home page previously but now 30 seconds (after clear all the caches and reload it).
The categories / products listing page : It takes 15-20 seconds to load the page previously but now 12-14 seconds.
browsing page by page in a category (jump to next page): It takes 15-20 seconds to load the page previously but now also 12-14 seconds.
However, open the product listing page in a category where it has 10-20pcs of products is the same time as the category where it has 450K products....
open product detail page : no change, it takes 4-6 seconds to open the page.

In fact, for the front store, it was failed to load the home page after add the new indexes to the tables. (I have uploaded abantecart v.2 /admin, /core to overwrites my abantecart v.1.2.9 but forget to overwrite the storefront.) The error is http 500 error and find memory exhausted in error log. I have increased memory_limit to 512M in php.ini, then it works.
It seems that the new indexes will also use some memories. But after I uploaded the abantecart v.2 /storefront to overwrites the v.1.2.9, it only needs 384M.

Besides, you have mistake in public_html/install/abantecart_database_upgrade.sql (abantecart v.2.0)
CREATE INDEX `ac_products_status_idx` ON `abc_products` (`product_id`, `status`, `date_available`);
should correct to :
CREATE INDEX `ac_products_status_idx` ON `ac_products` (`product_id`, `status`, `date_available`);

Improvements are there, but I think this is the max that we can squeeze out of Mysql and MyISAM on the code side for this case.

I like us to work on Postgres support and some other tools to make AbanteCart EE (Enterprise ECommerce) ready, but this will require some funding.

I fixed typos. Thank you.

This is for your reference : https://www.digitalocean.com/community/tutorials/sqlite-vs-mysql-vs-postgresql-a-comparison-of-relational-database-management-systems

Looks like your mysql server is a root of your problems.

FYI. I am working on some improvements for large product database.  I will post update in here soon.   

May be I have 6 live websites which are requesting data from mysql at the same time and abantecart has problem to access mysql at the moment.
I think that mysql should be tuned for performance setting. But I am not Familiar with it....
It's a good news to hear that you are improving abantecart to handle large product database.
Thank you!

There are a number of articles about Mysql performance tuning.
http://dba.stackexchange.com/questions/136349/mysql-performance-tuning-for-myisam

We already have plans to add InnoDB engine to boots performance for Mysql and also support PostgreSQL.
Timeframe is not yet clear as this requires a lot of effort and needs funding.

This is for your reference : https://www.selikoff.net/2008/11/19/why-too-much-database-normalization-can-be-a-bad-thing/

[However, open the product listing page in a category where it has 10-20pcs of products is the same time as the category where it has 450K products....]

I have tested it with 450K products.
The backend product browsing is improved. It takes 13-15 seconds to jump to next page previously but now only 4-5 seconds

The frontstore home page: It takes 1-2 minutes to load the home page previously but now 30 seconds (after clear all the caches and reload it).
The categories / products listing page : It takes 15-20 seconds to load the page previously but now 12-14 seconds.
browsing page by page in a category (jump to next page): It takes 15-20 seconds to load the page previously but now also 12-14 seconds.
However, open the product listing page in a category where it has 10-20pcs of products is the same time as the category where it has 450K products....
open product detail page : no change, it takes 4-6 seconds to open the page.

In fact, for the front store, it was failed to load the home page after add the new indexes to the tables. (I have uploaded abantecart v.2 /admin, /core to overwrites my abantecart v.1.2.9 but forget to overwrite the storefront.) The error is http 500 error and find memory exhausted in error log. I have increased memory_limit to 512M in php.ini, then it works.
It seems that the new indexes will also use some memories. But after I uploaded the abantecart v.2 /storefront to overwrites the v.1.2.9, it only needs 384M.

Besides, you have mistake in public_html/install/abantecart_database_upgrade.sql (abantecart v.2.0)
CREATE INDEX `ac_products_status_idx` ON `abc_products` (`product_id`, `status`, `date_available`);
should correct to :
CREATE INDEX `ac_products_status_idx` ON `ac_products` (`product_id`, `status`, `date_available`);

[It's database doesn't has too much normalization, so generate less quiries.(means run faster and use less resource)]

This is great. I wonder if we should consider eCommerce site with 380 thousand products as a large scale site. What is next? Entire database of Amazon?
Usually, sites with large catalogs and volumes, require special hardware and administration in addition to software tuning. This is a costly exercise.

Our prior focus on performance testing was towards fast growing customers and orders. 1/2 million of products was not ever tested and great that AbanteCart can handle it.

From my testing, abantecart v.1.2.9 is better than v.1.2.7 or before which can handle large products database but it is not ideal. However, it is better than OC, ZC or even some paid shopping cart. Agree with you that handling large catalogs and volumes required more resources. But I have used a free Chinese shopping cart which can already handle 1000K+ products smoothly and it doesn't need powerful hardware or no special request for the software. The cart was developed at least 10 years ago. (it uses smarty template and also apply caching methodology at the beginning. So handling large product database is efficiently and open the category / product listing page, home page is very fast. It's database doesn't has too much normalization, so generate less quiries.(means run faster and use less resource)) I am still using it but I want to use a shopping cart which can provide multi-shop, multi-language and multi-currencies functionality. Now, abantecart may be the choice as I have already checking on abantecart a few years.

Looks like your mysql server is a root of your problems.

FYI. I am working on some improvements for large product database.  I will post update in here soon.   

May be I have 6 live websites which are requesting data from mysql at the same time and abantecart has problem to access mysql at the moment.
I think that mysql should be tuned for performance setting. But I am not Familiar with it....
It's a good news to hear that you are improving abantecart to handle large product database.
Thank you!

I have cleared all the caches again and open the store front home page immediately.
This time, the following error occurs:

aban2/static_pages/index.php
==========================
There has been a critical error processing your request

SQL Error: MySQL server has gone away
Error No: 2006
SQL: DELETE FROM `abc_online_customers` WHERE `date_added`< (NOW() - INTERVAL 1 HOUR) in /aban2/core/database/amysqli.php on line 108
==========================

Waiting for 30 seconds later, go to /aban2 (home page) and open the storefront home page again, there is no problem and the page is cached.

This morning, I have tested it again and there has 450K products. For the 1st, 2nd time, there is no problem. But after I have cleared all the caches a few times and set sorting by Date New -> Old. Then the http 500 error (memory exhausted) occurs again. Then I have increased the memory_limit  to 512M in php.ini, no error occurs later.
I am running the test from production server. The resource may affect by other live websites.
Last night, I also created 1150K products and there is no error even the memory_limit set to 256M....
I am confusing right now and don't know where is the problem.
Anyway, the loading speed of home page and category / product listing page is really slow for 1st visit. If cache is enable, it is fast when the page is cached (2nd visit and no refresh of data and no expiration of caches). Hope that abantecart team can improve the loading speed of the home page and category / product listing page (data size inside one page is small, should load fast. just like the phpmyadmin, browsing a large/big table is very fast without delay).
I am also worrying about the Max number of concurrent processes. My server has 16GB Ram, then it should limit the no. of visitors / users and may caused memory exhausted easily from a few visitors open the web pages simultaneously.

I have reset the memory_limit to 512M in php.ini (previous setting is 256M) again. This time it works without error. I think that I may modify the incorrect php.ini file previously.
I will try to test it again. But the time for loading of category / product listing page is too long. Hope abantecart team can improve it.
I put all the dummy products in category "Skincare" (380K+ dummy products). When I click "Skincare", it takes very long time to open the products listing page. However, even the category "Makeup" (only 6 demo products), it still takes very long time to open the products listing page. (6 products should be assumed to be opened within 1 second in good design)
Thank you!

I installed abantecart v.1.2.9 on Debian 8 server with 32GB Ram, Xeon CPU L5630 x 2, 256GB SSD x 4 (Dell H700 Raid card, set raid 10), 4TB Hardisk x 6 (set raid 10). Mysql was installed on the SSD.
Running on PHP Version 5.6.29-0+deb8u1 (FPM).
Besides, I also install it on another Debian server with 16GB Ram, Intel i3, 3TB Hardisk x 4 (Raid 10). But the results are the same after insert 380K products. The page display HTTP 500 Error and find  memory exhausted error.
HTML cache can improve it. But if I clear the cache (because some new products are added, need to clear the cache) and visit the front page again, the loading speed is very slow.
For the category / product listing page, the loading speed is more slowly. If the page is not cached, the first person who open the page will take a long time.
Besides, do you enable the "Use SEO URL's" in "Setting/System" abantecart backend admin?