Changeset 91039 in webkit for trunk/Source/JavaScriptCore/heap/NewSpace.h

Timestamp:

Jul 14, 2011, 6:40:25 PM (14 years ago)

Author:

[email protected]

Message:

GC allocation fast path has too many operations.
​https://bugs.webkit.org/show_bug.cgi?id=64493

Patch by Filip Pizlo <​[email protected]> on 2011-07-14
Reviewed by Darin Adler.

Changed the timing of the lazy sweep so that it occurs when we land on
a previously-unsweeped block, rather than whenever we land on an unsweeped
cell. After the per-block lazy sweep occurs, the block is turned into a
singly linked list of free cells. The allocation fast path is now just a
load-branch-store to remove a cell from the head of the list.

Additionally, this changes the way new blocks are allocated. Previously,
they would be populated with dummy cells. With this patch, they are
turned into a free list, which means that there will never be destructor
calls for allocations in fresh blocks.

These changes result in a 1.9% speed-up on V8, and a 0.6% speed-up on
SunSpider. There are no observed statistically significant slow-downs
on any individual benchmark.

• JavaScriptCore.exp:
• heap/Heap.cpp:

(JSC::Heap::allocateSlowCase):
(JSC::Heap::collect):
(JSC::Heap::canonicalizeBlocks):
(JSC::Heap::resetAllocator):

• heap/Heap.h:

(JSC::Heap::forEachProtectedCell):
(JSC::Heap::forEachCell):
(JSC::Heap::forEachBlock):
(JSC::Heap::allocate):

• heap/MarkedBlock.cpp:

(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::lazySweep):
(JSC::MarkedBlock::blessNewBlockForFastPath):
(JSC::MarkedBlock::blessNewBlockForSlowPath):
(JSC::MarkedBlock::canonicalizeBlock):

• heap/MarkedBlock.h:
• heap/NewSpace.cpp:

(JSC::NewSpace::addBlock):
(JSC::NewSpace::canonicalizeBlocks):

• heap/NewSpace.h:

(JSC::NewSpace::allocate):
(JSC::NewSpace::SizeClass::SizeClass):
(JSC::NewSpace::SizeClass::canonicalizeBlock):

• heap/OldSpace.cpp:

(JSC::OldSpace::addBlock):

File:

• trunk/Source/JavaScriptCore/heap/NewSpace.h (modified) (5 diffs)

trunk/Source/JavaScriptCore/heap/NewSpace.h

@@ -51 +51 @@
             SizeClass();
             void resetAllocator();
-
+            void canonicalizeBlock();
+
+            MarkedBlock::FreeCell* firstFreeCell;
+            MarkedBlock* currentBlock;
             MarkedBlock* nextBlock;
             DoublyLinkedList<MarkedBlock> blockList;
@@ -65 +68 @@
         void addBlock(SizeClass&, MarkedBlock*);
         void removeBlock(MarkedBlock*);
+        
+        void canonicalizeBlocks();
 
         size_t waterMark();
@@ -116 +121 @@
     inline void* NewSpace::allocate(SizeClass& sizeClass)
     {
-        for (MarkedBlock*& block = sizeClass.nextBlock ; block; block = block->next()) {
-            if (void* result = block->allocate())
-                return result;
-
-            m_waterMark += block->capacity();
-        }
-
-        return 0;
+        MarkedBlock::FreeCell* firstFreeCell = sizeClass.firstFreeCell;
+        if (!firstFreeCell) {
+            // There are two possibilities for why we got here:
+            // 1) We've exhausted the allocation cache for currentBlock, in which case
+            //    currentBlock == nextBlock, and we know that there is no reason to
+            //    repeat a lazy sweep of nextBlock because we won't find anything.
+            // 2) Allocation caches have been cleared, in which case nextBlock may
+            //    have (and most likely does have) free cells, so we almost certainly
+            //    should do a lazySweep for nextBlock. This also implies that
+            //    currentBlock == 0.
+            
+            if (sizeClass.currentBlock) {
+                ASSERT(sizeClass.currentBlock == sizeClass.nextBlock);
+                m_waterMark += sizeClass.nextBlock->capacity();
+                sizeClass.nextBlock = sizeClass.nextBlock->next();
+                sizeClass.currentBlock = 0;
+            }
+            
+            for (MarkedBlock*& block = sizeClass.nextBlock ; block; block = block->next()) {
+                firstFreeCell = block->lazySweep();
+                if (firstFreeCell) {
+                    sizeClass.firstFreeCell = firstFreeCell;
+                    sizeClass.currentBlock = block;
+                    break;
+                }
+                
+                m_waterMark += block->capacity();
+            }
+            
+            if (!firstFreeCell)
+                return 0;
+        }
+        
+        ASSERT(firstFreeCell);
+        
+        sizeClass.firstFreeCell = firstFreeCell->next;
+        return firstFreeCell;
     }
 
@@ -156 +190 @@
 
     inline NewSpace::SizeClass::SizeClass()
-        : nextBlock(0)
+        : firstFreeCell(0)
+        , currentBlock(0)
+        , nextBlock(0)
         , cellSize(0)
     {
@@ -165 +201 @@
         nextBlock = blockList.head();
     }
+    
+    inline void NewSpace::SizeClass::canonicalizeBlock()
+    {
+        if (currentBlock) {
+            currentBlock->canonicalizeBlock(firstFreeCell);
+            firstFreeCell = 0;
+        }
+        
+        ASSERT(!firstFreeCell);
+        
+        currentBlock = 0;
+        firstFreeCell = 0;
+    }
 
 } // namespace JSC