Equalize marklists

[PeterSolMS]

Equalize mark lists across GC heaps in Server GC.

With real world workloads, we observe that the size of the mark lists is often very uneven across the heaps. This will also lead to uneven sort times across the heaps. As the GC threads need to synchronize at the end of sort_mark_list, the GC threads with less work will need to wait for those with more work to finish.

Therefore, it should be advantageous to even out the workload before we start sorting.

This PR does this adding a new method equalize_mark_lists that is called at the beginning of sort_mark_list. This method is executed in parallel on all GC threads and computes what pieces of the mark list need to be moved from one heap to another to achieve an even distribution. To avoid additional synchronization, each GC thread only writes its own mark list, and treats the other mark lists as read-only.

[ghost]

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

[Maoni0]

LGTM; there's some trivial styling things that I'll point out when we talk.

[Maoni0]

@PeterSolMS is on vacation and he didn't want to merge this till he gets back so please do not merge.

[PeterSolMS]

I analyzed perf traces from a first party customer (Bing). To estimate the impact of this change on GC pause time, I computed the difference of the last and first CPU samples spent in relevant methods for each ephemeral GC.

Here are charts and averages:

The summary information for an hour of runtime with and without this change shows a reduction of mean pause time of 2.0 milliseconds or 3.2%.