-
Notifications
You must be signed in to change notification settings - Fork 10.4k
Reenable individual auth tests #26823
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@dotnet/aspnet-blazor-eng @dougbu how do we feel about maybe moving project template tests back to azdo for now (off helix), that would let us unquarantine a lot of tests that are broken due to issues around installing our custom bits correctly on the helix machines. Once the work involving moving to a model of testing against a coherent SDK, we can switch back to helix. It doesn't seem worth trying to hack something short term if we are moving away from our current model of testing. Thoughts? |
I don't think this is the right short-term solution because
Besides all that, we are basically in MQ and should spend some of that period to really fix whatever Helix infrastructure problems we've got. |
@HaoK could you elaborate what it is that's problematic here and how that would be resolved in the future:
|
Sure, so the main issue with testing templates is that we have to basically install the correct bits onto the helix machines. This actually fits perfectly into wider long term future goal where we test against a coherent product build as opposed to whatever it is that we generally cobble together (which is especially true for our template tests). To summarize where we are at: If the end goal is to start just pointing our tests against a version of the SDK, that removes the need to have any kind of install our private bits step on the helix machines. The flow would just be, send the correct SDK version as part of our helix jobs that contains the coherent SDK with templates, and then run the tests. Since things already work well enough on azdo, I'm suggesting we consider abandoning the logic which tries to install template bits on the helix machines and just run the template tests on azdo (We still run the blazor templates on azdo anyways). We don't lose much of the dump/hang work since most of that is part of dotnet [vs]test anyways. Also we will have to do work to update the template tests to work against official sdk bits anyways, irrespective of whatever short term fixes we do here. |
@HaoK I agree it would be nice if we were building against entire SDKs but we're not there yet and we don't know when in the 6.0 timeframe that will come. We need to be testing thoroughly long before that. @pranavkm I also would like to call attention to #26776 and my comment here:
We have support for things like the AzDO / local template tests' TemplateTests.props for many other tests that run on Helix but nothing for the template tests |
Right this actually helps my case. So right now we have degraded testing as more tests are quarantined or skipped on helix due to it just being less functional then letting them continue to run on azdo. So if our goal its just to have better coverage today, we should just run template tests on azdo. It just seems like a poor investment to enhance/build new test install logic that we know is most likely going to be obsolete and nuked in 6.0. |
My point was we don't know when testing against full ADKs will even start. And, the issue is not about moving back to AzDO. If that was enough, we'd be satisfied with the template job in PR builds. |
Fixes #26776