RefreshingAWSCredentials using global lock + slow AssumeRoleWithWebIdentityCredentials

[denis-nikolayev-warby]

Describe the bug

Encountered interesting issue with GetCredentialsAsync() + "slow" AssumeRoleWithWebIdentityCredentials + tight timeout on calls to our app.

https://github.com/aws/aws-sdk-net/blob/master/sdk/src/Core/Amazon.Runtime/Credentials/RefreshingAWSCredentials.cs#L146

1. it acquires global lock
2. check if token is expired
3. makes a slow network call to CredentialsFromAssumeRoleWithWebIdentityAuthenticationAsync
4. release lock

Unfortunately don't have statistics from aws .net sdk, but have from aws nodejs sdk. Assume this slow call is about the same

we tried to implement simple wrapper

that works pretty well, because number of timeout errors goes down after deploying this fix (24th)

Expected Behavior

background refreshing token

Current Behavior

global lock

Reproduction Steps

do not apply

Possible Solution

No response

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWSSDK.Core 3.7.8.2

Targeted .NET Platform

.NET 6.0

Operating System and version

aws eks, docker, Debian 11, mcr.microsoft.com/dotnet/aspnet:6.0

[wmundev]

can confirm we are having the same issue, using AWS EKS with AssumeRoleWithWebIdentity as recommended in this docs - https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html

i observe latency every 5 - 15 minutes or so when for example trying to make a dynamodb call and the token is expired and AssumeRoleWithWebIdentity is called to the regional aws sts endpoint with a average latency of around 100ms - 200ms

[dscpinheiro]

We just released preview 5 of V4 and it includes a PR with a fix for this issue: #3541

Please let us know if you still see any problems.

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.