Adding masked quote id to QuoteRepository::getList causing infinite loop

[ihor-sviziev]

In the Klaviyo extension, we have a plugin that adds a masked quote id to quote search API requests. That looks like quite a small plugin that shouldn't cause any performance or functional damage.

In most cases, it works well, but sometimes we see that such request causing eating 100% CPU by the php-fpm, and Gateway Timeout issues after quite a long time 60 seconds.

Here is a link to the plugin in the Klaviyo extension (it uses API class, which is ok):
https://github.com/klaviyo/magento2-klaviyo/blob/3.0.5/Plugin/Api/CartSearchRepository.php

Detailed investigation showed that issue is coming from the following line:

magento2/app/code/Magento/Quote/Model/QuoteIdToMaskedQuoteId.php

Lines 50 to 53 in afa2169

	public function execute(int $quoteId): string
	{
	/* Check the quote exists to avoid database constraint issues */
	$this->cartRepository->get($quoteId);

-> it goes to

magento2/app/code/Magento/Quote/Model/QuoteRepository.php

Line 121 in afa2169

$quote = $this->loadQuote('loadByIdWithoutStore', 'cartId', $cartId, $sharedStoreIds);

-> and then to

magento2/app/code/Magento/Quote/Model/Quote.php

Lines 912 to 917 in 7c6b636

	public function loadByIdWithoutStore($quoteId)
	{
	$this->_getResource()->loadByIdWithoutStore($this, $quoteId);
	$this->_afterLoad();
	return $this;
	}

Looks pretty small and simple, right? Yes... but... here we have the recollect totals and re-saving quote

magento2/app/code/Magento/Quote/Model/Quote.php

Lines 2461 to 2468 in 7c6b636

	protected function _afterLoad()
	{
	// collect totals and save me, if required
	if (1 == $this->getTriggerRecollect()) {
	$this->collectTotals()
	->setTriggerRecollect(0)
	->save();
	}

In case if the quote has trigger_recollect = 1:
we running the following code:

magento2/app/code/Magento/Quote/Model/Quote/TotalsCollector.php

Lines 146 to 148 in 7c6b636

	/** @var \Magento\Quote\Model\Quote\Address $address */
	foreach ($quote->getAllAddresses() as $address) {
	$addressTotal = $this->collectAddressTotals($quote, $address);

This code triggers an event:

magento2/app/code/Magento/Quote/Model/Quote/TotalsCollector.php

Lines 263 to 264 in 7c6b636

	$this->eventManager->dispatch(
	'sales_quote_address_collect_totals_before',

That execites an event observer:

magento2/app/code/Magento/SalesRule/etc/events.xml

Lines 30 to 32 in 7c6b636

	<event name="sales_quote_address_collect_totals_before">
	<observer name="coupon_code_validation" instance="Magento\SalesRule\Observer\CouponCodeValidation" />
	</event>

magento2/app/code/Magento/SalesRule/Observer/CouponCodeValidation.php

Lines 58 to 79 in 7c6b636

	public function execute(EventObserver $observer)
	{
	/** @var Quote $quote */
	$quote = $observer->getData('quote');
	$code = $quote->getCouponCode();
	if ($code) {
	//Only validating the code if it's a new code.
	/** @var Quote[] $found */
	$found = $this->cartRepository->getList(
	$this->criteriaBuilder->addFilter('main_table.' . CartInterface::KEY_ENTITY_ID, $quote->getId())
	->create()
	)->getItems();
	if (!$found || ((string)array_shift($found)->getCouponCode()) !== (string)$code) {
	try {
	$this->codeLimitManager->checkRequest($code);
	} catch (CodeRequestLimitException $exception) {
	$quote->setCouponCode('');
	throw $exception;
	}
	}
	}
	}

That... runs $this->cartRepository->getList method... for which we just added a plugin for adding masked quote ids!

As a result - we have an infinite loop.

Preconditions (*)

1. Magento 2.3.7 or 2.4-develop

Steps to reproduce (*)

1. Install Klaviyo extension (tested on version 3.0.5)

   OR Add a plugin, that adds masked quote id to response for the cart search API, for example:

   • https://github.com/klaviyo/magento2-klaviyo/blob/3.0.5/etc/webapi_rest/di.xml#L3-L5
   • https://github.com/klaviyo/magento2-klaviyo/blob/3.0.5/Plugin/Api/CartSearchRepository.php

2. Create Cart Price Rule with coupon (let's give some 10% discount)

3. Add some product to the shopping cart, apply the coupon from step 2

4. Run the following code with and w/o Klaviyo extension:

curl --location --request GET 'https://www.mymagento.com/index.php/rest/default/V1/carts/search?searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=updated_at&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=2021-06-11+11%3A38%3A16&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bcondition_type%5D=gt&searchCriteria%5BsortOrders%5D%5B0%5D%5Bfield%5D=updated_at&searchCriteria%5BsortOrders%5D%5B0%5D%5Bdirection%5D=DESC&searchCriteria%5BcurrentPage%5D=1&searchCriteria%5BpageSize%5D=50' \
--header 'Authorization: Bearer dk6zp483qmck8omd7sov1r3b9is7kxh'

Note: here is documentation on how to get access token https://devdocs.magento.com/guides/v2.4/get-started/authentication/gs-authentication-token.html#request-token
✔ we successfully retrieved the list of carts

5. Go to admin, edit the Cart Price Rule that you created in step 2, do the following steps:
   • change the coupon code, save the rule
   • disable the cart price rule, save it
6. Run the code from step 4

Expected result (*)

1. You should get the JSON with a list of carts

Actual result (*)

1. ❌We have 504 Gateway Time-out error
   

---

Please provide Severity assessment for the Issue as Reporter. This information will help during Confirmation and Issue triage processes.

• Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

[m2-assistant]

Hi @ihor-sviziev. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[github-jira-sync-bot]

✅ Jira issue https://jira.corp.magento.com/browse/AC-1190 is successfully created for this GitHub issue.

[m2-assistant]

✅ Confirmed by @engcom-Alfa. Thank you for verifying the issue.
Issue Available: @engcom-Alfa, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

[jsamhall]

Chiming in as this issue has impacted our production environment, and becomes somewhat severe when the payment processor relies on REST API to update orders. In our case, we use Bolt for payment processing and it uses webhooks / Magento REST API to manage order status according to payment status at the processor.

In our case, we have become inundated with "Request Timeout" from Bolt, but it is of course due to the issue described here.

I have resolved the issue by patching the Klaviyo CartSearchRepository plugin and exiting before it queries the QuoteRepository a second time. I have achieved this by caching imploded array keys of the passed CartSearchResults into a static property on the Plugin and checking that before querying the QuoteRepository.

My relevant changes here:

/**
     * Identification of CartSearchResults being processed; part of fix to avoid infinite loops
     * @var array 
     */
    private static $processingSearchResults = [];

    public function __construct(
        CartExtensionFactory $extensionFactory,
        QuoteIdToMaskedQuoteId $quoteIdToMaskedQuoteId
    ){...}
    /**
     * Add "kl_masked_id" extension attribute to order data object to make it accessible in API data
     *
     * @param CartRepositoryInterface $subject
     * @param CartSearchResults $searchResult
     *
     * @return CartSearchResults
     */
    public function afterGetList(CartRepositoryInterface $subject, CartSearchResults $searchResult)
    {
        $quotes = $searchResult->getItems();

        // Prevent infinite loop as Magento CouponValidator may invoke CartRepository::getList if Quote has a Coupon applied
        // @see https://github.com/magento/magento2/issues/33675
        if($this->isProcessingCartSearchResults($searchResult)){
            return $searchResult;
        }

        foreach ($quotes as $quote) {
            $maskedId = $this->getMaskedIdFromQuoteId($quote->getId());
            $extensionAttributes = $quote->getExtensionAttributes();
            $extensionAttributes = $extensionAttributes ? $extensionAttributes : $this->extensionFactory->create();
            $extensionAttributes->setData(self::KL_MASKED_ID, $maskedId);
            $quote->setExtensionAttributes($extensionAttributes);
        }

        return $searchResult;
    }

    /**
     * Evaluate if this CartSearchResults object has already been handled
     * @param CartSearchResults $searchResults
     * @return bool
     */
    private function isProcessingCartSearchResults(CartSearchResults $searchResults){
        $processingKey = implode('_', array_keys($searchResults->getItems()));
        if(in_array($processingKey, self::$processingSearchResults)){
            return true;
        }

        self::$processingSearchResults[] = $processingKey;
        return false;
    }

@ihor-sviziev It appears you are the author of the Klaviyo Magento 2 integration? May you provide any input to my patch?