Changeset 99436 in webkit for trunk/Source/JavaScriptCore/parser/ParserArena.h

Timestamp:

Nov 7, 2011, 9:54:15 AM (14 years ago)

Author:

[email protected]

Message:

Towards 8 Bit Strings: Templatize JSC::Lexer class by character type
​https://bugs.webkit.org/show_bug.cgi?id=71331

Source/JavaScriptCore:

Change the Lexer class to be a template class based on the character
type of the source. In the process updated the parseIdentifier()
and parseString() methods to create 8 bit strings where possible.
Also added some helper methods for accumulating temporary string
data in the 8 and 16 bit vectors.

Changed the SourceProvider::data() virtual method to return a
StringImpl* instead of a UChar*.

Updated the KeywordLookup generator to create code to match keywords
for both 8 and 16 bit source strings.

Due to a compiler bug (<rdar://problem/10194295>) moved enum
definition outside of Lexer class declaration. Remove second enum
no longer needed.

Reviewed by Darin Adler.

• KeywordLookupGenerator.py:
• interpreter/Interpreter.cpp:

(JSC::Interpreter::callEval):

• parser/Lexer.cpp:

(JSC::::Lexer):
(JSC::::~Lexer):
(JSC::::getInvalidCharMessage):
(JSC::::currentCharacter):
(JSC::::setCode):
(JSC::::internalShift):
(JSC::::shift):
(JSC::::peek):
(JSC::::getUnicodeCharacter):
(JSC::::shiftLineTerminator):
(JSC::::lastTokenWasRestrKeyword):
(JSC::::record8):
(JSC::::append8):
(JSC::::append16):
(JSC::::record16):
(JSC::::parseIdentifier):
(JSC::::parseIdentifierSlowCase):
(JSC::::parseString):
(JSC::::parseStringSlowCase):
(JSC::::parseHex):
(JSC::::parseOctal):
(JSC::::parseDecimal):
(JSC::::parseNumberAfterDecimalPoint):
(JSC::::parseNumberAfterExponentIndicator):
(JSC::::parseMultilineComment):
(JSC::::nextTokenIsColon):
(JSC::::lex):
(JSC::::scanRegExp):
(JSC::::skipRegExp):
(JSC::::clear):
(JSC::::sourceCode):

• parser/Lexer.h:

(JSC::Lexer::append16):
(JSC::Lexer::currentOffset):
(JSC::Lexer::setOffsetFromCharOffset):
(JSC::::isWhiteSpace):
(JSC::::isLineTerminator):
(JSC::::convertHex):
(JSC::::convertUnicode):
(JSC::::makeIdentifier):
(JSC::::setCodeStart):
(JSC::::makeIdentifierLCharFromUChar):
(JSC::::lexExpectIdentifier):

• parser/Parser.cpp:

(JSC::Parser::Parser):
(JSC::Parser::parseProperty):
(JSC::Parser::parseMemberExpression):

• parser/Parser.h:

(JSC::Parser::next):
(JSC::Parser::nextExpectIdentifier):

• parser/ParserArena.h:

(JSC::IdentifierArena::makeIdentifier):
(JSC::IdentifierArena::makeIdentifierLCharFromUChar):

• parser/SourceCode.h:

(JSC::SourceCode::subExpression):

• parser/SourceProvider.h:

(JSC::UStringSourceProvider::stringData):

• parser/SourceProviderCache.h:
• parser/SyntaxChecker.h:
• runtime/FunctionPrototype.cpp:

(JSC::insertSemicolonIfNeeded):

• runtime/Identifier.cpp:

(JSC::IdentifierTable::add):
(JSC::IdentifierLCharFromUCharTranslator::hash):
(JSC::IdentifierLCharFromUCharTranslator::equal):
(JSC::IdentifierLCharFromUCharTranslator::translate):
(JSC::Identifier::add8):

• runtime/Identifier.h:

(JSC::Identifier::Identifier):
(JSC::Identifier::createLCharFromUChar):
(JSC::Identifier::canUseSingleCharacterString):
(JSC::IdentifierCharBufferTranslator::hash):
(JSC::IdentifierCharBufferTranslator::equal):
(JSC::IdentifierCharBufferTranslator::translate):
(JSC::Identifier::add):
(JSC::Identifier::equal):
(JSC::IdentifierTable::add):

• runtime/JSGlobalObjectFunctions.cpp:

(JSC::decode):
(JSC::parseIntOverflow):
(JSC::globalFuncUnescape):

• runtime/JSGlobalObjectFunctions.h:

(JSC::parseIntOverflow):

• runtime/LiteralParser.cpp:

(JSC::LiteralParser::tryJSONPParse):
(JSC::LiteralParser::Lexer::lexString):

• wtf/text/StringImpl.h:

Source/WebCore:

Changed the SourceProvider::data() virtual method to return a
StringImpl* instead of a UChar*.
Changed Identifier() constructor to use JSGlobalData*.

Reviewed by Darin Adler.

No new tests - refactored SourceProvider class and sub-classes.

• bindings/js/CachedScriptSourceProvider.h:

(WebCore::CachedScriptSourceProvider::stringData):

• bindings/js/StringSourceProvider.h:

(WebCore::StringSourceProvider::stringData):

• bridge/qt/qt_runtime.cpp:

(JSC::Bindings::convertQVariantToValue):

Source/WebKit/qt:

Changed Identifier() constructor to use JSGlobalData*.

Reviewed by Darin Adler.

• Api/qwebframe.cpp:

(QWebFrame::addToJavaScriptWindowObject):

File:

• trunk/Source/JavaScriptCore/parser/ParserArena.h (modified) (3 diffs)

trunk/Source/JavaScriptCore/parser/ParserArena.h

@@ -43 +43 @@
         }
 
-        ALWAYS_INLINE const Identifier& makeIdentifier(JSGlobalData*, const UChar* characters, size_t length);
+        template <typename T>
+        ALWAYS_INLINE const Identifier& makeIdentifier(JSGlobalData*, const T* characters, size_t length);
+        ALWAYS_INLINE const Identifier& makeIdentifierLCharFromUChar(JSGlobalData*, const UChar* characters, size_t length);
+
         const Identifier& makeNumericIdentifier(JSGlobalData*, double number);
 
@@ -66 +69 @@
     };
 
-    ALWAYS_INLINE const Identifier& IdentifierArena::makeIdentifier(JSGlobalData* globalData, const UChar* characters, size_t length)
+    template <typename T>
+    ALWAYS_INLINE const Identifier& IdentifierArena::makeIdentifier(JSGlobalData* globalData, const T* characters, size_t length)
     {
         if (characters[0] >= MaximumCachableCharacter) {
@@ -87 +91 @@
     }
 
+    ALWAYS_INLINE const Identifier& IdentifierArena::makeIdentifierLCharFromUChar(JSGlobalData* globalData, const UChar* characters, size_t length)
+    {
+        if (characters[0] >= MaximumCachableCharacter) {
+            m_identifiers.append(Identifier::createLCharFromUChar(globalData, characters, length));
+            return m_identifiers.last();
+        }
+        if (length == 1) {
+            if (Identifier* ident = m_shortIdentifiers[characters[0]])
+                return *ident;
+            m_identifiers.append(Identifier(globalData, characters, length));
+            m_shortIdentifiers[characters[0]] = &m_identifiers.last();
+            return m_identifiers.last();
+        }
+        Identifier* ident = m_recentIdentifiers[characters[0]];
+        if (ident && Identifier::equal(ident->impl(), characters, length))
+            return *ident;
+        m_identifiers.append(Identifier::createLCharFromUChar(globalData, characters, length));
+        m_recentIdentifiers[characters[0]] = &m_identifiers.last();
+        return m_identifiers.last();
+    }
+    
     inline const Identifier& IdentifierArena::makeNumericIdentifier(JSGlobalData* globalData, double number)
     {