Unquarantine CanSendAndReceiveBytes

[TanayParikh]

This test has been passing consistently for the past 30 days now:

(the 4 failures in the past 24 hours are due to an unrelated CI issue which has been separately investigated).

This is just an extension of: #31156

Underlying test stability issues should've been resolved in the October/November 2020 timeframe via #26519.

Edit: Also added independent Browser instances between CanSendAndReceiveBytes and BenchmarksRunWithoutError. The flakiness seems to have been rooted in shared Browser instances between tests.

Fixes: #23366

[TanayParikh]

Looks like a legitimate CI failure. BenchmarksRunWithoutError is another test which was quarantined due to #23366 but was later unquarantined via #31156. So it seems the tests fail when they're both "active". @HaoK wondering if you have any context on the relation here?

OpenQA.Selenium.BrowserAssertFailedException : Xunit.Sdk.EmptyException: Assert.Empty() Failure
Collection: [Element (id = 998de1bd-fcd8-4799-8dad-14a452284a07), Element (id = 55fd5e9a-f85d-4d58-9b66-bfb75ed57e14), Element (id = 78b326ec-137d-4fee-ae58-12912cd24578), Element (id = b126de36-52b2-46c1-bdd7-fa9fad7381a5), Element (id = a3d7b7a9-4a29-42ae-9219-4656f8626ca6), ...]
   at Xunit.Assert.Empty(IEnumerable collection) in C:\Dev\xunit\xunit\src\xunit.assert\Asserts\CollectionAsserts.cs:line 284
   at Microsoft.AspNetCore.E2ETesting.WaitAssert.<>c__DisplayClass17_0.<WaitAssertCore>b__0() in /_/src/Shared/E2ETesting/WaitAssert.cs:line 83
   at Microsoft.AspNetCore.E2ETesting.WaitAssert.<>c__DisplayClass18_0`1.<WaitAssertCore>b__0(IWebDriver ) in //src/Shared/E2ETesting/WaitAssert.cs:line 109
Screen shot captured at 'D:\workspace\_work\1\s\artifacts\TestResults\Release\Microsoft.AspNetCore.Components.E2ETests\92efa74db8e842dca7b5c80a40d6154b.png'
Encountered browser errors
[2021-04-29T21:49:31Z] [Debug] http://127.0.0.1:53201/_framework/dotnet.6.0.0-preview.4.21221.4.js 0:142749 "mono_wasm_runtime_ready" "fe00e07a-5519-4dfe-b35a-f867dbaf2e28"
[2021-04-29T21:49:32Z] [Debug] http://127.0.0.1:53201/_framework/dotnet.6.0.0-preview.4.21221.4.js 0:142749 "mono_wasm_runtime_ready" "fe00e07a-5519-4dfe-b35a-f867dbaf2e28"
[2021-04-29T21:49:33Z] [Severe] http://127.0.0.1:53201/benchmarks/lib/minibench/minibench.js 218:28 Error: Wrong number of items rendered
    at measureRenderList (http://127.0.0.1:53201/benchmarks/renderListBenchmark.js:22:11)
    at async ExecutionTimer._runBlock (http://127.0.0.1:53201/benchmarks/lib/minibench/minibench.js:149:17)
    at async ExecutionTimer.run (http://127.0.0.1:53201/benchmarks/lib/minibench/minibench.js:121:56)
    at async Benchmark._measureTimings (http://127.0.0.1:53201/benchmarks/lib/minibench/minibench.js:248:9)
    at async http://127.0.0.1:53201/benchmarks/lib/minibench/minibench.js:201:21
[2021-04-29T21:49:33Z] [Severe] http://127.0.0.1:53201/benchmarks/lib/minibench/minibench.js 218:28 Error: Wrong number of items rendered
    at measureRenderList (http://127.0.0.1:53201/benchmarks/renderListBenchmark.js:22:11)
    at async ExecutionTimer._runBlock (http://127.0.0.1:53201/benchmarks/lib/minibench/minibench.js:149:17)
    at async ExecutionTimer.run (http://127.0.0.1:53201/benchmarks/lib/minibench/minibench.js:121:56)
    at async Benchmark._measureTimings (http://127.0.0.1:53201/benchmarks/lib/minibench/minibench.js:248:9)
    at async http://127.0.0.1:53201/benchmarks/lib/minibench/minibench.js:201:21
[2021-04-29T21:49:33Z] [Debug] http://127.0.0.1:53201/_framework/dotnet.6.0.0-preview.4.21221.4.js 0:142749 "mono_wasm_runtime_ready" "fe00e07a-5519-4dfe-b35a-f867dbaf2e28"
[2021-04-29T21:49:35Z] [Debug] http://127.0.0.1:53201/_framework/dotnet.6.0.0-preview.4.21221.4.js 0:142749 "mono_wasm_runtime_ready" "fe00e07a-5519-4dfe-b35a-f867dbaf2e28"
[2021-04-29T21:49:36Z] [Debug] http://127.0.0.1:53201/_framework/dotnet.6.0.0-preview.4.21221.4.js 0:142749 "mono_wasm_runtime_ready" "fe00e07a-5519-4dfe-b35a-f867dbaf2e28"
[2021-04-29T21:49:37Z] [Severe] http://127.0.0.1:53201/_framework/blazor.webassembly.js 0:27256 "crit: Microsoft.AspNetCore.Components.WebAssembly.Rendering.WebAssemblyRenderer[100]
      Unhandled exception rendering component: Failed to execute 'setAttribute' on 'Element': '@bind' is not a valid attribute name.
      Error: Failed to execute 'setAttribute' on 'Element': '@bind' is not a valid attribute name.
          at V.applyAttribute (http://127.0.0.1:53201/_framework/blazor.webassembly.js:1:20408)
          at V.insertElement (http://127.0.0.1:53201/_framework/blazor.webassembly.js:1:19455)
          at V.insertFrame (http://127.0.0.1:53201/_framework/blazor.webassembly.js:1:18439)
          at V.insertFrameRange (http://127.0.0.1:53201/_framework/blazor.webassembly.js:1

[HaoK]

Right I don't have any context, but I noticed this too, our tests seem to interact with each other, my feeling is that there's too much parallelization/sharing of servers between our tests. It seems better to run our tests in a sandbox or at the least where its not possible to have this kind of interaction. I filed #31195 which was another set of tests that I encountered that cause each other to fail. You can probably add these tests to that list, and maybe you can try to identify what's causing the test interactions in our E2E tests

[HaoK]

Actually that bind error, is that our old caching friend @captainsafia ?

[HaoK]

I've made this suggestion before but got opposition from DOI, I think the quarantine runs should be running the full set of tests, not just the quarantined tests, that would dramatically increase the confidence that unquarantining tests actually is safe.

[captainsafia]

Actually that bind error, is that our old caching friend @captainsafia ?

The other issue occurs as a build error. This one is happening during runtime so I believe it is different.