Make sure temporary tables always get cleaned up, even on faillure

[quisse]

Description (*)

Make sure temporary tables always get cleaned up, even on faillure.

Related Pull Requests

Fixed Issues (if relevant)

1. Fixes M2 catalogrule_product__temp Tables #22273

Manual testing scenarios (*)

1. ...
2. ...

Questions or comments

Contribution checklist (*)

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• All automated tests passed successfully (all builds are green)

[m2-assistant]

Hi @quisse. Thank you for your contribution
Here is some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

• @magento give me test instance - deploy test instance based on PR changes
• @magento give me 2.4-develop instance - deploy vanilla Magento instance

For more details, please, review the Magento Contributor Guide documentation.

[kandy]

I believe there is a drawback to your fix.
After your changes, the index table will be cleaned up if any exception happens

Better to modify IndexerTableSwapper class to:

• Create a temporary table instead of real in https://github.com/magento/magento2/blob/2.4-develop/app/code/Magento/CatalogRule/Model/Indexer/IndexerTableSwapper.php#L50
• or/and cleanup temporary table in class destructor

[quisse]

The "Database Compare" (for example) build task failed. All reports available show success. Am I missing something? See:

• https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/27012/c6949b214931a7c9a68034e3765e4038/DatabaseCompare/b2b-skip-db-compare-report.html
• https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/27012/c6949b214931a7c9a68034e3765e4038/DatabaseCompare/ce-db-compare-report.html
• https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/27012/c6949b214931a7c9a68034e3765e4038/DatabaseCompare/ce-skip-db-compare-report.html
• https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/27012/c6949b214931a7c9a68034e3765e4038/DatabaseCompare/ee-db-compare-report.html
• https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/27012/c6949b214931a7c9a68034e3765e4038/DatabaseCompare/ee-skip-db-compare-report.html

[hostep]

What's the status here? Just saw an error:

Table 'catalogrule_product__temp8f30230c' doesn't exist, query was: SHOW CREATE TABLE `catalogrule_product__temp8f30230c

happening on a shop during bin/magento setup:upgrade (shop was upgraded to Magento 2.3.5 yesterday).

Will this PR fix this problem?

Update: is probably something else, because on a second attempt to deploy to production, we now get:

Table 'catalogsearch_fulltext_scope4_tmp' doesn't exist, query was: SHOW CREATE TABLE `catalogsearch_fulltext_scope4_tmp`

Great, on to debugging 2.3.5 (again!)

Update 2: is probably a cron running during a deploy which causes it somehow, when we disable crons, then deploy and re-enable crons afterwards, it works.

Sorry for the noise in this PR, has probably nothing to do with it.

[quisse]

@kandy please, any response would be appreciated.

[santibm]

hi, what's the status of this!? it's happening now on two customer sites heavily using catalog rules, @quisse does your fix work?!

[engcom-Charlie]

Hi @quisse, could you please look through failed tests?

[quisse]

@engcom-Charlie Please read previous comment.
#27012 (comment)

[engcom-Alfa]

Hi @quisse .
Due to Magento Definition of Done all code must be covered by tests. Please cover your fix by automated test.
Thanks!

Pull Request state was updated. Re-review required.

[ihor-sviziev]

@magento run Performance Acceptance Tests

[ihor-sviziev]

Seems like test failures not related to changes from this PR. Let’s wait a bit and restart tests again

[ihor-sviziev]

@magento run Functional Tests B2B, Integration Tests

[ihor-sviziev]

@magento run Functional Tests B2B

[quisse]

Feels like a lottery, isn't it? 🙈😅
Appreciate the effort by the way ☺️👌

[ihor-sviziev]

There were some fixes in related repositories, so tests should be fixed already

[ihor-sviziev]

@magento run Functional Tests B2B

[quisse]

@magento run Functional Tests B2B

[ihor-sviziev]

@quisse could you merge recent changes from 2.4-develop? Maybe that's causing test failures?

[ihor-sviziev]

@magento run all tests

[magento-engcom-team]

Hi @ihor-sviziev, thank you for the review.
ENGCOM-8367 has been created to process this Pull Request

[ihor-sviziev]

@engcom-Alfa any updates on testing this PR?

[stefandoorn]

This change will still leave temporary tables in case Magento hit a PHP memory limit, which is not that hard in Magento landscape. Maybe the indexer should start by cleaning up old tables, instead of doing it in the destructor.

[ihor-sviziev]

@stefandoorn, that's actually another use case. From my perspective - this solution is good enough to be tested and merged. If you have a better solution - please send a Pull Request, we'll review it

[stefandoorn]

Yeah, sorry, wasn't meant to discard this PR. It's indeed another use case to keep in mind :-)

[m2-assistant]

Hi @quisse, thank you for your contribution!
Please, complete Contribution Survey, it will take less than a minute.
Your feedback will help us to improve contribution process.

[vzabaznov]

Hey @quisse, the Performance team took care of your PR. Thank you for your contribution.