GH-26255: Refactor CacheInvalidate sendPurgeRequest to fix incorrect tag splitting #26256

moloughlin · 2020-01-03T23:14:36Z

Description (*)

Issue: #26255

Refactored sendPurgeRequest to take in an array of tags instead of requiring the caller to implode(). This allows us to do smarter chunking, and avoids the incorrect splitting noted in the linked issue.
Backwards compatibility maintained for callers passing in a string, calls from core code have been updated to use the new array form
Allowed header size to be passed in via DI

Fixed Issues (if relevant)

Fixes Magento_CacheInvalidate mis-handles very large tag patterns when doing a PURGE #26255
Fixes Varnish "Connection reset by peer" error when large catalog is reindexed on schedule #8815 : Varnish "Connection reset by peer" error when large catalog is reindexed on schedule

Manual testing scenarios (*)

See Magento_CacheInvalidate mis-handles very large tag patterns when doing a PURGE #26255

Contribution checklist (*)

Pull request has a meaningful description of its purpose
All commits are accompanied by meaningful commit messages
All new or changed code is covered with unit/integration tests (if applicable)
All automated tests passed successfully (all builds are green)

…orrect purge batching, regardless of the size and content of provided tag pattern.

m2-assistant · 2020-01-03T23:14:40Z

Hi @moloughlin. Thank you for your contribution
Here is some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

@magento give me test instance - deploy test instance based on PR changes
@magento give me 2.4-develop instance - deploy vanilla Magento instance

For more details, please, review the Magento Contributor Guide documentation.

…rams. Fix failing test case.

lbajsarowicz

Could you provide some Unit Tests for that changes?

app/code/Magento/CacheInvalidate/Model/PurgeCache.php

…ing.

moloughlin · 2020-01-08T10:03:19Z

@lbajsarowicz, I've added a test for the splitting of large tag arrays and fixed the other outstanding suggestions. Ready for another look when you are available - thanks!

Added test coverage as requested.

moloughlin · 2020-02-03T06:50:03Z

@lbajsarowicz can you please confirm next steps on this? Thanks.

lbajsarowicz · 2020-02-08T19:12:13Z

@magento run all tests

lbajsarowicz · 2020-02-08T19:16:30Z

@moloughlin Let's wait for tests to pass.
Code looks fine, so I'll approve the PR after passing the tests.

Could you eventually merge latest 2.4-develop to your working branch?

…lidate_incorrect_tag_splitting

moloughlin · 2020-02-09T07:20:06Z

@lbajsarowicz awesome, thanks. 2.4-develop merged back in and only the typical false negatives on the test runs 👍

…lidate_incorrect_tag_splitting

moloughlin · 2020-03-02T22:30:11Z

Bump (merged 2.4-develop again since it's been a few weeks...)

lbajsarowicz · 2020-03-02T23:12:49Z

@moloughlin Thanks for your contribution (again).
We faced false-negative with your PR Static Tests.

After tests pass - I'm going to approve the PR.

magento-engcom-team · 2020-03-09T21:31:52Z

Hi @lbajsarowicz, thank you for the review.
ENGCOM-7073 has been created to process this Pull Request
✳️ @lbajsarowicz, could you please add one of the following labels to the Pull Request?

Label	Description
`Auto-Tests: Covered`	All changes in Pull Request is covered by auto-tests
`Auto-Tests: Not Covered`	Changes in Pull Request requires coverage by auto-tests
`Auto-Tests: Not Required`	Changes in Pull Request does not require coverage by auto-tests

engcom-Kilo · 2020-07-07T08:55:57Z

@engcom-Kilo could you test this PR?

Checking this.

…litting

engcom-Kilo · 2020-07-08T17:51:48Z

✔️ QA Passed

Changes implemented in Pull Request works correctly.

engcom-Kilo · 2020-07-08T17:54:27Z

@magento run all tests

engcom-Kilo · 2020-07-09T09:06:24Z

Failed Functional Tests B2B is not related to changes in current Pull Request.

VladimirZaets

Hi, @engcom-Kilo are tested this fix with varnish? If not please retest it.

engcom-Kilo · 2020-08-07T10:26:12Z

@VladimirZaets
Yes, this was tested with Varnish. Works correctly.

ihor-sviziev

Approved again in order to have an ability to move back to accept

magento-engcom-team · 2020-08-07T11:14:46Z

Hi @ihor-sviziev, thank you for the review.
ENGCOM-7073 has been created to process this Pull Request

…ix incorrect tag splitting #26256

m2-assistant · 2020-08-13T16:39:02Z

Hi @moloughlin, thank you for your contribution!
Please, complete Contribution Survey, it will take less than a minute.
Your feedback will help us to improve contribution process.

magentoGH-26255: Refactor CacheInvalidate sendPurgeRequest to allow c…

e212301

…orrect purge batching, regardless of the size and content of provided tag pattern.

moloughlin requested a review from kokoc as a code owner January 3, 2020 23:14

magento-engcom-team added Partner: Aligent Consulting partners-contribution Pull Request is created by Magento Partner labels Jan 3, 2020

moloughlin mentioned this pull request Jan 3, 2020

Magento_CacheInvalidate mis-handles very large tag patterns when doing a PURGE #26255

Closed

Matthew O'Loughlin added 2 commits January 6, 2020 11:58

magentoGH-26255: Fix logic error around handling both string/array pa…

4c22303

…rams. Fix failing test case.

magentoGH-26255: Fix codesniff errors/warnings

fc2ce73

lbajsarowicz previously requested changes Jan 6, 2020

View reviewed changes

app/code/Magento/CacheInvalidate/Model/PurgeCache.php Outdated Show resolved Hide resolved

app/code/Magento/CacheInvalidate/Model/PurgeCache.php Outdated Show resolved Hide resolved

ghost assigned lbajsarowicz Jan 6, 2020

ghost added the Progress: needs update label Jan 6, 2020

Matthew O'Loughlin added 2 commits January 8, 2020 09:33

magentoGH-26255: Add test coverage for multi-request purge tag splitt…

a9d376d

…ing.

magentoGH-26255: Add class descriptions to satisfy codesniff rules

5e69e82

moloughlin requested a review from lbajsarowicz January 8, 2020 09:59

Merge remote-tracking branch 'origin/2.4-develop' into fix/cache_inva…

479c0cc

…lidate_incorrect_tag_splitting

Merge remote-tracking branch 'origin/2.4-develop' into fix/cache_inva…

d0ef6ae

…lidate_incorrect_tag_splitting

lbajsarowicz approved these changes Mar 9, 2020

View reviewed changes

ghost added Progress: ready for testing and removed Progress: needs update labels Mar 9, 2020

magento-engcom-team added Release Line: 2.4 Component: CacheInvalidate labels Mar 9, 2020

Merge branch '2.4-develop' into fix/cache_invalidate_incorrect_tag_sp…

a19274e

…litting

ghost added Progress: testing in progress and removed Progress: ready for testing labels Jul 8, 2020

ghost added Progress: extended testing and removed Progress: testing in progress labels Jul 9, 2020

engcom-Kilo added the QA: Ready to add to Regression Scope Should be analyzed and added to Regression Testing Scope(if applicable) label Jul 9, 2020

VladimirZaets requested changes Aug 4, 2020

View reviewed changes

ghost assigned VladimirZaets Aug 4, 2020

ghost added Progress: needs update Priority: P2 A defect with this priority could have functionality issues which are not to expectations. and removed Progress: extended testing labels Aug 4, 2020

ihor-sviziev approved these changes Aug 7, 2020

View reviewed changes

ghost added Progress: ready for testing and removed Progress: needs update labels Aug 7, 2020

ghost added Progress: extended testing and removed Progress: ready for testing labels Aug 10, 2020

engcom-Alfa added QA: Added to Regression Scope Scenario was analysed and added to Regression Testing Scope and removed QA: Ready to add to Regression Scope Should be analyzed and added to Regression Testing Scope(if applicable) labels Aug 11, 2020

ghost added Progress: accept and removed Progress: extended testing labels Aug 11, 2020

magento-engcom-team mentioned this pull request Aug 13, 2020

Varnish "Connection reset by peer" error when large catalog is reindexed on schedule #8815

Closed

magento-engcom-team pushed a commit that referenced this pull request Aug 13, 2020

ENGCOM-7073: GH-26255: Refactor CacheInvalidate sendPurgeRequest to f…

2d548ac

…ix incorrect tag splitting #26256

magento-engcom-team merged commit 22c0827 into magento:2.4-develop Aug 13, 2020

GH-26255: Refactor CacheInvalidate sendPurgeRequest to fix incorrect tag splitting #26256

GH-26255: Refactor CacheInvalidate sendPurgeRequest to fix incorrect tag splitting #26256

Uh oh!

Conversation

moloughlin commented Jan 3, 2020 • edited by ihor-sviziev Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description (*)

Fixed Issues (if relevant)

Manual testing scenarios (*)

Contribution checklist (*)

Uh oh!

m2-assistant bot commented Jan 3, 2020

Uh oh!

lbajsarowicz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

moloughlin commented Jan 8, 2020

Uh oh!

moloughlin commented Feb 3, 2020

Uh oh!

lbajsarowicz commented Feb 8, 2020

Uh oh!

lbajsarowicz commented Feb 8, 2020

Uh oh!

moloughlin commented Feb 9, 2020

Uh oh!

moloughlin commented Mar 2, 2020

Uh oh!

lbajsarowicz commented Mar 2, 2020

Uh oh!

magento-engcom-team commented Mar 9, 2020

Uh oh!

engcom-Kilo commented Jul 7, 2020

Uh oh!

engcom-Kilo commented Jul 8, 2020

Uh oh!

engcom-Kilo commented Jul 8, 2020

Uh oh!

engcom-Kilo commented Jul 9, 2020

Uh oh!

VladimirZaets left a comment

Choose a reason for hiding this comment

Uh oh!

engcom-Kilo commented Aug 7, 2020

Uh oh!

ihor-sviziev left a comment

Choose a reason for hiding this comment

Uh oh!

magento-engcom-team commented Aug 7, 2020

Uh oh!

m2-assistant bot commented Aug 13, 2020

Uh oh!

Uh oh!

moloughlin commented Jan 3, 2020 •

edited by ihor-sviziev

Loading