Changeset 62031 in webkit for trunk/JavaScriptCore/parser/Lexer.h

Timestamp:

Jun 28, 2010, 1:17:13 PM (15 years ago)

Author:

[email protected]

Message:

Only one character lookahead should be enough for the lexer
​https://bugs.webkit.org/show_bug.cgi?id=41213

Reviewed by Oliver Hunt.

The lexer had 4 character lookahead before, which required
a complex shifting mechanism. This can be improved by using
only one character lookahead for most decisions, and a
peek() function as a fallback when it is absolutely necessary.

• parser/Lexer.cpp:

(JSC::Lexer::currentCharacter):
(JSC::Lexer::currentOffset):
(JSC::Lexer::setCode):
(JSC::Lexer::shift):
(JSC::Lexer::peek):
(JSC::Lexer::getUnicodeCharacter):
(JSC::Lexer::shiftLineTerminator):
(JSC::Lexer::lastTokenWasRestrKeyword):
(JSC::Lexer::lex):
(JSC::Lexer::scanRegExp):
(JSC::Lexer::skipRegExp):

• parser/Lexer.h:

File:

• trunk/JavaScriptCore/parser/Lexer.h (modified) (5 diffs)

trunk/JavaScriptCore/parser/Lexer.h

@@ -2 +2 @@
  *  Copyright (C) 1999-2000 Harri Porten ([email protected])
  *  Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Apple Inc. All rights reserved.
+ *  Copyright (C) 2010 Zoltan Herczeg ([email protected])
  *
  *  This library is free software; you can redistribute it and/or
@@ -27 +28 @@
 #include "SourceCode.h"
 #include <wtf/ASCIICType.h>
+#include <wtf/AlwaysInline.h>
 #include <wtf/SegmentedVector.h>
 #include <wtf/Vector.h>
@@ -67 +69 @@
         ~Lexer();
 
-        void shift1();
-        void shift2();
-        void shift3();
-        void shift4();
-        void shiftLineTerminator();
-
         void record8(int);
         void record16(int);
@@ -79 +75 @@
         void copyCodeWithoutBOMs();
 
-        int currentOffset() const;
-        const UChar* currentCharacter() const;
+        ALWAYS_INLINE void shift();
+        ALWAYS_INLINE int peek(int offset);
+        int getUnicodeCharacter();
+        void shiftLineTerminator();
 
-        const Identifier* makeIdentifier(const UChar* characters, size_t length);
+        ALWAYS_INLINE const UChar* currentCharacter() const;
+        ALWAYS_INLINE int currentOffset() const;
 
-        bool lastTokenWasRestrKeyword() const;
+        ALWAYS_INLINE const Identifier* makeIdentifier(const UChar* characters, size_t length);
+
+        ALWAYS_INLINE bool lastTokenWasRestrKeyword() const;
 
         static const size_t initialReadBufferCapacity = 32;
@@ -107 +108 @@
         // current and following unicode characters (int to allow for -1 for end-of-file marker)
         int m_current;
-        int m_next1;
-        int m_next2;
-        int m_next3;
-        
+
         IdentifierArena* m_arena;