MC-38593: Complex products children ids are not cached in Type class leading to multiple unnecessary db requests #30746

engcom-Echo · 2020-11-02T09:57:25Z

Description (*)

Related Pull Requests

Fixed Issues (if relevant)

Fixes Complex products children ids are not cached in Type class leading to multiple unnecessary db requests #30585

Manual testing scenarios (*)

See: Complex products children ids are not cached in Type class leading to multiple unnecessary db requests #30585

Questions or comments

Contribution checklist (*)

Pull request has a meaningful description of its purpose
All commits are accompanied by meaningful commit messages
All new or changed code is covered with unit/integration tests (if applicable)
All automated tests passed successfully (all builds are green)

…leading to multiple unnecessary db requests

m2-assistant · 2020-11-02T09:57:27Z

Hi @engcom-Echo. Thank you for your contribution
Here is some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

@magento give me test instance - deploy test instance based on PR changes
@magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

@magento run all tests - run or re-run all required tests against the PR changes
@magento run <test-build(s)> - run or re-run specific test build(s)
For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

Database Compare
Functional Tests CE
Functional Tests EE,
Functional Tests B2B
Integration Tests
Magento Health Index
Sample Data Tests CE
Sample Data Tests EE
Sample Data Tests B2B
Static Tests
Unit Tests
WebAPI Tests

You can find more information about the builds here

ℹ️ Please run only needed test builds instead of all when developing. Please run all test builds before sending your PR for review.

For more details, please, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

viktorpetryk · 2020-11-02T10:38:44Z

app/code/Magento/Bundle/etc/frontend/di.xml

@@ -14,6 +14,6 @@
        </arguments>
    </type>
    <type name="Magento\Catalog\Model\Product">
-        <plugin name="bundle" type="Magento\Bundle\Model\Plugin\Frontend\Product" sortOrder="100" />
+        <plugin name="add_child_identities_bundle" type="Magento\Bundle\Model\Plugin\Frontend\ProductIdentitiesExtender"/>


Please do not remove sortOrder as this can change the order of the plugins execution for this class.

viktorpetryk

As you rename the plugin, please rename the tests as well

app/code/Magento/Bundle/Test/Unit/Model/Plugin/Frontend/ProductTest.php
dev/tests/integration/testsuite/Magento/Bundle/Model/Plugin/Frontend/ProductTest.php.

Could you take a look at \Magento\Bundle\Model\Plugin\Product::afterGetIdentities. It seems like we have the same performance issue with executing DB requests when getParentIdsByChild is executed several times.

Also, could you check behavior with API requests and described methods call, as we have two identical plugins for different areas - frontend and global?

Please check other plugins and models like \Magento\ConfigurableProduct\Model\Plugin\ProductIdentitiesExtender::afterGetIdentities for configurable and grouped products as they have the same ...type->getParentIdsByChild and ...type->getChildrenIds calls and it may need to implement caching as well.

viktorpetryk · 2020-11-02T12:37:38Z

app/code/Magento/Bundle/Model/Plugin/Frontend/ProductIdentitiesExtender.php

@@ -37,12 +37,15 @@ public function __construct(Type $type)
     */
    public function afterGetIdentities(CatalogProduct $product, array $identities): array
    {
+        if ($product->getTypeId() !== BundleType::TYPE_CODE) {
+


Please remove an empty line here.

viktorpetryk · 2020-11-02T12:37:53Z

app/code/Magento/Bundle/Model/Plugin/Frontend/ProductIdentitiesExtender.php

        foreach ($this->type->getChildrenIds($product->getEntityId()) as $childIds) {
            foreach ($childIds as $childId) {
                $identities[] = CatalogProduct::CACHE_TAG . '_' . $childId;
            }
        }
-
        return array_unique($identities);


Please add an empty line here.

viktorpetryk · 2020-11-02T12:43:56Z

app/code/Magento/Bundle/etc/frontend/di.xml

@@ -14,6 +14,6 @@
        </arguments>
    </type>
    <type name="Magento\Catalog\Model\Product">
-        <plugin name="bundle" type="Magento\Bundle\Model\Plugin\Frontend\Product" sortOrder="100" />
+        <plugin name="add_child_identities_bundle" type="Magento\Bundle\Model\Plugin\Frontend\ProductIdentitiesExtender"/>


Maybe it would be better to rename it to add_bundle_child_identities?

viktorpetryk · 2020-11-02T12:50:43Z

app/code/Magento/Bundle/Test/Unit/Model/Product/TypeTest.php

@@ -173,7 +180,7 @@ protected function setUp(): void
            ->disableOriginalConstructor()
            ->getMockForAbstractClass();
        $this->bundleModelSelection = $this->getMockBuilder(SelectionFactory::class)
-            ->setMethods(['create'])
+            ->setMethods(['create', 'getChildrenIds'])


It seems like 'getChildrenIds' is redundant in this case.

engcom-Echo · 2020-11-03T07:46:35Z

@victorpetryk

Also, could you check behavior with API requests and described methods call, as we have two identical plugins for different areas - frontend and global?

These are plugin is not identical.
Magento\Bundle\Model\Plugin\Product - used getParentIdsByChild
Magento\Bundle\Model\Plugin\Frontend\ProductIdentitiesExtender - used getChildrenIds

zakdma · 2020-11-03T07:42:05Z

app/code/Magento/Bundle/Model/Product/Type.php

@@ -285,7 +290,12 @@ public function getRelationInfo()
     */
    public function getChildrenIds($parentId, $required = true)
    {
-        return $this->_bundleSelection->getChildrenIds($parentId, $required);
+        $cacheKey = $parentId . '-' . ($required ? 1 : 0);


AFAIK $parentId here can be array here.

No, it isn't. In the annotation to the function, the specified parameter takes only "int".

* @param int $parentId * @param bool $required * @return array */ public function getChildrenIds($parentId, $required = true) { $cacheKey = $parentId . '-' . ($required ? 1 : 0);

…leading to multiple unnecessary db requests

zakdma · 2020-11-04T14:06:17Z

app/code/Magento/Bundle/Model/Product/Type.php

@@ -296,7 +311,12 @@ public function getChildrenIds($parentId, $required = true)
     */
    public function getParentIdsByChild($childId)
    {
-        return $this->_bundleSelection->getParentIdsByChild($childId);
+        $cacheKey = is_array($childId) ? implode('-', $childId) : $childId;


In case of array is too big it can break because of lack of memory
Please test this solution for big amount of products
https://devdocs.magento.com/guides/v2.4/config-guide/cli/config-cli-subcommands-perf-data.html

zakdma · 2020-11-04T14:25:09Z

app/code/Magento/ConfigurableProduct/Model/Product/Type/Configurable.php

@@ -364,7 +369,12 @@ public function getChildrenIds($parentId, $required = true)
     */
    public function getParentIdsByChild($childId)
    {
-        return $this->_catalogProductTypeConfigurable->getParentIdsByChild($childId);
+        $cacheKey = is_array($childId) ? implode('-', $childId) : $childId;


In case of array is too big it can break because of lack of memory
Please test this solution for big amount of products
https://devdocs.magento.com/guides/v2.4/config-guide/cli/config-cli-subcommands-perf-data.html

…leading to multiple unnecessary db requests

engcom-Echo · 2020-11-09T07:35:03Z

@magento run all tests

viktorpetryk · 2020-11-09T16:46:13Z

app/code/Magento/Bundle/Model/Plugin/Frontend/ProductIdentitiesExtender.php

+     * @param mixed $entityId
+     * @return array
+     */
+    private function getChildrenIds($entityId)


Please add a return type hint.

viktorpetryk · 2020-11-09T16:49:03Z

app/code/Magento/Bundle/Model/Plugin/ProductIdentitiesExtender.php

+use Magento\Bundle\Model\Product\Type as BundleType;
+use Magento\Catalog\Model\Product as CatalogProduct;
+
+class ProductIdentitiesExtender


Please add DocBlock for class.

viktorpetryk · 2020-11-09T16:50:34Z

app/code/Magento/Bundle/Model/Plugin/ProductIdentitiesExtender.php

+        foreach ($this->getParentIdsByChild($product->getEntityId()) as $parentId) {
+            $identities[] = CatalogProduct::CACHE_TAG . '_' . $parentId;
+        }
+        return $identities;


Please add an empty line before.

viktorpetryk · 2020-11-09T16:50:44Z

app/code/Magento/Bundle/Model/Plugin/ProductIdentitiesExtender.php

+     * @param mixed $entityId
+     * @return array
+     */
+    private function getParentIdsByChild($entityId)


Please add a return type hint.

viktorpetryk · 2020-11-09T16:55:54Z

app/code/Magento/Bundle/Test/Unit/Model/Plugin/Frontend/ProductIdentitiesExtenderTest.php

 use Magento\Bundle\Model\Product\Type;
 use Magento\Catalog\Model\Product;
 use PHPUnit\Framework\MockObject\MockObject;
 use PHPUnit\Framework\TestCase;

-class ProductTest extends TestCase
+class ProductIdentitiesExtenderTest extends TestCase
 {
    /** @var  \Magento\Bundle\Model\Plugin\Product */


Please correct @var tag

viktorpetryk · 2020-11-09T17:08:25Z

app/code/Magento/Bundle/Test/Unit/Model/Plugin/Frontend/ProductIdentitiesExtenderTest.php

 use Magento\Bundle\Model\Product\Type;
 use Magento\Catalog\Model\Product;
 use PHPUnit\Framework\MockObject\MockObject;
 use PHPUnit\Framework\TestCase;

-class ProductTest extends TestCase
+class ProductIdentitiesExtenderTest extends TestCase


Please add DocBlock for class.

viktorpetryk · 2020-11-09T17:08:58Z

app/code/Magento/Bundle/Test/Unit/Model/Plugin/ProductIdentitiesExtenderTest.php

 use Magento\Catalog\Model\Product;
 use Magento\Framework\TestFramework\Unit\Helper\ObjectManager;
 use PHPUnit\Framework\MockObject\MockObject;
 use PHPUnit\Framework\TestCase;

-class ProductTest extends TestCase
+class ProductIdentitiesExtenderTest extends TestCase


Please add DocBlock for class.

viktorpetryk · 2020-11-09T17:21:41Z

app/code/Magento/ConfigurableProduct/Model/Plugin/ProductIdentitiesExtender.php

            $parentProduct = $this->productRepository->getById($parentId);
-            $identities = array_merge($identities, $parentProduct->getIdentities());
+            $parentProductsIdentities[] = $parentProduct->getIdentities();


Maybe it would be better to rename it to $parentProductIndentities

It is an array of Products where everyone has Identities

…leading to multiple unnecessary db requests

engcom-Echo · 2020-11-10T11:04:35Z

@magento run all tests

engcom-Kilo · 2020-11-11T11:22:09Z

@magento import pull request to https://github.com/magento-tsg/magento2ce

magento-engcom-team · 2020-11-11T11:24:47Z

@engcom-Kilo the pull request successfully imported.

m2-assistant · 2020-11-14T16:18:54Z

Hi @engcom-Echo, thank you for your contribution!
Please, complete Contribution Survey, it will take less than a minute.
Your feedback will help us to improve contribution process.

MC-38593: Complex products children ids are not cached in Type class …

cd8fe10

…leading to multiple unnecessary db requests

ghost added the Progress: pending review label Nov 2, 2020

magento-engcom-team added Component: Bundle Release Line: 2.4 labels Nov 2, 2020

engcom-Echo requested a review from viktorpetryk November 2, 2020 10:01

viktorpetryk suggested changes Nov 2, 2020

View reviewed changes

ghost assigned viktorpetryk Nov 2, 2020

ghost added Progress: needs update Progress: review and removed Progress: pending review Progress: needs update labels Nov 2, 2020

engcom-Echo added the Priority: P1 Once P0 defects have been fixed, a defect having this priority is the next candidate for fixing. label Nov 2, 2020

ghost added Progress: needs update Progress: review Severity: S1 Affects critical data or functionality and forces users to employ a workaround. and removed Progress: review Progress: needs update labels Nov 2, 2020

viktorpetryk suggested changes Nov 2, 2020

View reviewed changes

ghost added Progress: needs update and removed Progress: review labels Nov 2, 2020

zakdma suggested changes Nov 3, 2020

View reviewed changes

ghost assigned zakdma Nov 3, 2020

ghost added Progress: review Progress: needs update and removed Progress: needs update Progress: review labels Nov 3, 2020

engcom-Echo and others added 2 commits November 3, 2020 13:00

MC-38593: Complex products children ids are not cached in Type class …

e9b2930

…leading to multiple unnecessary db requests

Merge branch '2.4-develop' into MC-38593

4e0f75d

m2-community-project bot added the Progress: review label Nov 4, 2020

zakdma suggested changes Nov 4, 2020

View reviewed changes

m2-community-project bot added Progress: needs update and removed Progress: review labels Nov 4, 2020

zakdma suggested changes Nov 4, 2020

View reviewed changes

engcom-Echo and others added 4 commits November 5, 2020 13:42

MC-38593: Complex products children ids are not cached in Type class …

2d2c77d

…leading to multiple unnecessary db requests

Merge branch '2.4-develop' into MC-38593

4ef5701

Merge branch '2.4-develop' into MC-38593

6f38e2d

MC-38593: Complex products children ids are not cached in Type class …

bd51f4d

…leading to multiple unnecessary db requests

engcom-Echo force-pushed the MC-38593 branch from b886404 to c0a99e0 Compare November 6, 2020 17:19

MC-38593: Complex products children ids are not cached in Type class …

a01dc43

…leading to multiple unnecessary db requests

engcom-Echo force-pushed the MC-38593 branch from c0a99e0 to a01dc43 Compare November 9, 2020 07:34

viktorpetryk suggested changes Nov 9, 2020

View reviewed changes

m2-community-project bot added Progress: review and removed Progress: needs update labels Nov 9, 2020

engcom-Echo and others added 2 commits November 10, 2020 10:21

Merge branch '2.4-develop' into MC-38593

918e1d3

MC-38593: Complex products children ids are not cached in Type class …

c4c4535

…leading to multiple unnecessary db requests

viktorpetryk approved these changes Nov 11, 2020

View reviewed changes

m2-community-project bot added Progress: ready for testing and removed Progress: review labels Nov 11, 2020

magento-engcom-team merged commit b18a387 into magento:2.4-develop Nov 14, 2020

viniciusbordinhao-blueacorn mentioned this pull request Feb 15, 2021

Perfomance issue on product page with all product type #25111

Closed

hostep mentioned this pull request Jun 14, 2022

Why do we need parent identities on frontend? #35546

Open

m2-community-project bot assigned adarshkhatri Sep 5, 2023

m2-community-project bot removed the Progress: ready for testing label Sep 5, 2023

MC-38593: Complex products children ids are not cached in Type class leading to multiple unnecessary db requests #30746

MC-38593: Complex products children ids are not cached in Type class leading to multiple unnecessary db requests #30746

Uh oh!

Conversation

engcom-Echo commented Nov 2, 2020

Description (*)

Related Pull Requests

Fixed Issues (if relevant)

Manual testing scenarios (*)

Questions or comments

Contribution checklist (*)

Uh oh!

m2-assistant bot commented Nov 2, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

viktorpetryk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

engcom-Echo commented Nov 3, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

engcom-Echo commented Nov 9, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

engcom-Echo commented Nov 10, 2020

Uh oh!

engcom-Kilo commented Nov 11, 2020

Uh oh!

magento-engcom-team commented Nov 11, 2020

Uh oh!

m2-assistant bot commented Nov 14, 2020

Uh oh!

Uh oh!