Changeset 163195 in webkit for trunk/Source/JavaScriptCore/parser/Lexer.cpp

Timestamp:

Jan 31, 2014, 1:34:38 PM (12 years ago)

Author:

[email protected]

Message:

Make it possible to implement JS builtins in JS
​https://bugs.webkit.org/show_bug.cgi?id=127887

Reviewed by Michael Saboff.

.:

CMake updates

• Source/cmake/gtest/CMakeLists.txt:

Source/JavaScriptCore:

This patch makes it possible to write builtin functions in JS.
The bindings, generators, and definitions are all created automatically
based on js files in the builtins/ directory. This patch includes one
such case: Array.prototype.js with an implementation of every().

There's a lot of refactoring to make it possible for CommonIdentifiers
to include the output of the generated files (DerivedSources/JSCBuiltins.{h,cpp})
without breaking the offset extractor. The result of this refactoring
is that CommonIdentifiers, and a few other miscellaneous headers now
need to be included directly as they were formerly captured through other
paths.

In addition this adds a flag to the Lookup table's hashentry to indicate
that a static function is actually backed by JS. There is then a lot of
logic to thread the special nature of the functon to where it matters.
This allows toString(), .caller, etc to mimic the behaviour of a host
function.

Notes on writing builtins:

• Each function is compiled independently of the others, and those implementations cannot currently capture all global properties (as that could be potentially unsafe). If a function does capture a global we will deliberately crash.
• For those "global" properties that we do want access to, we use the @ prefix, e.g. Object(this) becomes @Object(this). The @ identifiers are private names, and behave just like regular properties, only without the risk of adulteration. Again, in the @Object case, we explicitly duplicate the ObjectConstructor reference on the GlobalObject so that we have guaranteed access to the original version of the constructor.
• call, apply, eval, and Function are all rejected identifiers, again to prevent anything from accidentally using an adulterated object. Instead @call and @apply are available, and happily they completely drop the neq_ptr instruction as they're defined as always being the original call/apply functions.

These restrictions are just intended to make it harder to accidentally
make changes that are incorrect (for instance calling whatever has been
assigned to global.Object, instead of the original constructor function).
However, making a mistake like this should result in a purely semantic
error as fundamentally these functions are treated as though they were
regular JS code in the host global, and have no more privileges than
any other JS.

The initial proof of concept is Array.prototype.every, this shows a 65%
performance improvement, and that improvement is significantly hurt by
our poor optimisation of op_in.

As this is such a limited function, we have not yet exported all symbols
that we could possibly need, but as we implement more, the likelihood
of encountering missing features will reduce.

This did require breaking out a JSStringInlines header, and required
fixing a few objects that were trying to using PropertyName::publicName
rather than PropertyName::uid.

• API/JSCallbackObjectFunctions.h:

(JSC::JSCallbackObject<Parent>::getOwnPropertySlot):
(JSC::JSCallbackObject<Parent>::put):
(JSC::JSCallbackObject<Parent>::deleteProperty):
(JSC::JSCallbackObject<Parent>::getStaticValue):
(JSC::JSCallbackObject<Parent>::staticFunctionGetter):
(JSC::JSCallbackObject<Parent>::callbackGetter):

• CMakeLists.txt:
• DerivedSources.make:
• GNUmakefile.list.am:
• JavaScriptCore.vcxproj/JavaScriptCore.vcxproj:
• JavaScriptCore.vcxproj/JavaScriptCore.vcxproj.filters:
• JavaScriptCore.xcodeproj/project.pbxproj:
• builtins/Array.prototype.js:

(every):

• builtins/BuiltinExecutables.cpp: Added.

(JSC::BuiltinExecutables::BuiltinExecutables):
(JSC::BuiltinExecutables::createBuiltinExecutable):

• builtins/BuiltinExecutables.h:

(JSC::BuiltinExecutables::create):

• bytecode/CodeBlock.cpp:

(JSC::CodeBlock::CodeBlock):

• bytecode/CodeBlock.h:
• bytecode/ProfiledCodeBlockJettisoningWatchpoint.cpp:
• bytecode/UnlinkedCodeBlock.cpp:

(JSC::generateFunctionCodeBlock):
(JSC::UnlinkedFunctionExecutable::UnlinkedFunctionExecutable):
(JSC::UnlinkedFunctionExecutable::codeBlockFor):
(JSC::UnlinkedCodeBlock::UnlinkedCodeBlock):

• bytecode/UnlinkedCodeBlock.h:

(JSC::ExecutableInfo::ExecutableInfo):
(JSC::UnlinkedFunctionExecutable::create):
(JSC::UnlinkedFunctionExecutable::toStrictness):
(JSC::UnlinkedFunctionExecutable::isBuiltinFunction):
(JSC::UnlinkedCodeBlock::isBuiltinFunction):

• bytecompiler/BytecodeGenerator.cpp:

(JSC::BytecodeGenerator::BytecodeGenerator):

• bytecompiler/BytecodeGenerator.h:

(JSC::BytecodeGenerator::isBuiltinFunction):
(JSC::BytecodeGenerator::makeFunction):

• bytecompiler/NodesCodegen.cpp:

(JSC::CallFunctionCallDotNode::emitBytecode):
(JSC::ApplyFunctionCallDotNode::emitBytecode):

• create_hash_table:
• dfg/DFGOperations.cpp:
• generate-js-builtins: Added.

(getCopyright):
(getFunctions):
(generateCode):
(mangleName):
(FunctionExecutable):
(Identifier):
(JSGlobalObject):
(SourceCode):
(UnlinkedFunctionExecutable):
(VM):

• interpreter/Interpreter.cpp:
• interpreter/ProtoCallFrame.cpp:
• jit/JITOpcodes.cpp:
• jit/JITOpcodes32_64.cpp:
• jit/JITOperations.cpp:
• jit/JITPropertyAccess.cpp:
• jit/JITPropertyAccess32_64.cpp:
• jsc.cpp:
• llint/LLIntSlowPaths.cpp:
• parser/ASTBuilder.h:

(JSC::ASTBuilder::makeFunctionCallNode):

• parser/Lexer.cpp:

(JSC::Lexer<T>::Lexer):
(JSC::isSafeIdentifier):
(JSC::Lexer<LChar>::parseIdentifier):
(JSC::Lexer<UChar>::parseIdentifier):
(JSC::Lexer<T>::lex):

• parser/Lexer.h:

(JSC::isSafeIdentifier):
(JSC::Lexer<T>::lexExpectIdentifier):

• parser/Nodes.cpp:

(JSC::ProgramNode::setClosedVariables):

• parser/Nodes.h:

(JSC::ScopeNode::capturedVariables):
(JSC::ScopeNode::setClosedVariables):
(JSC::ProgramNode::closedVariables):

• parser/Parser.cpp:

(JSC::Parser<LexerType>::Parser):
(JSC::Parser<LexerType>::parseInner):
(JSC::Parser<LexerType>::didFinishParsing):
(JSC::Parser<LexerType>::printUnexpectedTokenText):

• parser/Parser.h:

(JSC::Scope::getUsedVariables):
(JSC::Parser::closedVariables):
(JSC::parse):

• parser/ParserModes.h:
• parser/ParserTokens.h:
• runtime/ArgList.cpp:
• runtime/Arguments.cpp:
• runtime/Arguments.h:
• runtime/ArgumentsIteratorConstructor.cpp:
• runtime/ArgumentsIteratorPrototype.cpp:
• runtime/ArrayPrototype.cpp:
• runtime/CodeCache.cpp:

(JSC::CodeCache::getFunctionExecutableFromGlobalCode):

• runtime/CommonIdentifiers.cpp:

(JSC::CommonIdentifiers::CommonIdentifiers):
(JSC::CommonIdentifiers::getPrivateName):
(JSC::CommonIdentifiers::getPublicName):

• runtime/CommonIdentifiers.h:
• runtime/CommonSlowPaths.cpp:
• runtime/CommonSlowPathsExceptions.cpp:
• runtime/ExceptionHelpers.cpp:

(JSC::createUndefinedVariableError):

• runtime/Executable.h:

(JSC::EvalExecutable::executableInfo):
(JSC::ProgramExecutable::executableInfo):
(JSC::FunctionExecutable::isBuiltinFunction):

• runtime/FunctionPrototype.cpp:

(JSC::functionProtoFuncToString):

• runtime/JSActivation.cpp:

(JSC::JSActivation::symbolTableGet):
(JSC::JSActivation::symbolTablePut):
(JSC::JSActivation::symbolTablePutWithAttributes):

• runtime/JSArgumentsIterator.cpp:
• runtime/JSArray.cpp:
• runtime/JSArrayIterator.cpp:
• runtime/JSCJSValue.cpp:
• runtime/JSCellInlines.h:
• runtime/JSFunction.cpp:

(JSC::JSFunction::createBuiltinFunction):
(JSC::JSFunction::calculatedDisplayName):
(JSC::JSFunction::sourceCode):
(JSC::JSFunction::isHostOrBuiltinFunction):
(JSC::JSFunction::isBuiltinFunction):
(JSC::JSFunction::callerGetter):
(JSC::JSFunction::getOwnPropertySlot):
(JSC::JSFunction::getOwnNonIndexPropertyNames):
(JSC::JSFunction::put):
(JSC::JSFunction::defineOwnProperty):

• runtime/JSFunction.h:
• runtime/JSGenericTypedArrayViewConstructorInlines.h:
• runtime/JSGenericTypedArrayViewInlines.h:
• runtime/JSGenericTypedArrayViewPrototypeInlines.h:
• runtime/JSGlobalObject.cpp:

(JSC::JSGlobalObject::reset):
(JSC::JSGlobalObject::visitChildren):

• runtime/JSGlobalObject.h:

(JSC::JSGlobalObject::objectConstructor):
(JSC::JSGlobalObject::symbolTableHasProperty):

• runtime/JSObject.cpp:

(JSC::getClassPropertyNames):
(JSC::JSObject::reifyStaticFunctionsForDelete):
(JSC::JSObject::putDirectBuiltinFunction):

• runtime/JSObject.h:
• runtime/JSPropertyNameIterator.cpp:
• runtime/JSPropertyNameIterator.h:
• runtime/JSString.h:
• runtime/JSStringInlines.h: Added.

(JSC::JSString::getStringPropertySlot):
(JSC::inlineJSValueNotStringtoString):
(JSC::JSValue::toWTFStringInline):

• runtime/JSSymbolTableObject.cpp:

(JSC::JSSymbolTableObject::getOwnNonIndexPropertyNames):

Don't report private names.

• runtime/JSSymbolTableObject.h:

(JSC::symbolTableGet):
(JSC::symbolTablePut):
(JSC::symbolTablePutWithAttributes):

• runtime/Lookup.cpp:

(JSC::setUpStaticFunctionSlot):

• runtime/Lookup.h:

(JSC::HashEntry::builtinGenerator):
(JSC::HashEntry::propertyGetter):
(JSC::HashEntry::propertyPutter):
(JSC::HashTable::entry):
(JSC::getStaticPropertySlot):
(JSC::getStaticValueSlot):
(JSC::putEntry):

• runtime/NativeErrorConstructor.cpp:

(JSC::NativeErrorConstructor::finishCreation):

• runtime/NativeErrorConstructor.h:
• runtime/PropertySlot.h:
• runtime/RegExpPrototype.cpp:
• runtime/SetConstructor.cpp:
• runtime/StringObject.cpp:
• runtime/Structure.cpp:
• runtime/VM.cpp:

(JSC::VM::VM):

• runtime/VM.h:

(JSC::VM::builtinExecutables):

Source/WebCore:

Updating for the newly required headers.

Test: js/regress/array-prototype-every.html

• ForwardingHeaders/runtime/JSStringInlines.h: Added.
• Modules/plugins/QuickTimePluginReplacement.cpp:
• bindings/js/JSIDBAnyCustom.cpp:
• bindings/js/JSIDBDatabaseCustom.cpp:
• bindings/js/JSIDBObjectStoreCustom.cpp:

Source/WebKit:

CMake updates

• CMakeLists.txt:

Source/WebKit2:

CMake updates

• CMakeLists.txt:

Tools:

CMake updates

• DumpRenderTree/CMakeLists.txt:
• WebKitTestRunner/CMakeLists.txt:
• WinCELauncher/CMakeLists.txt:

LayoutTests:

Updated the test results for new error messages (now that they're
actually helpful), and added a js-regress test to track performance.

• js/array-every-expected.txt:
• js/dom/array-prototype-properties-expected.txt:
• js/regress/array-prototype-every-expected.txt: Added.
• js/regress/array-prototype-every.html: Added.
• js/regress/script-tests/array-prototype-every.js: Added.

(test1):
(test2):
(test3):

File:

• trunk/Source/JavaScriptCore/parser/Lexer.cpp (modified) (13 diffs)

trunk/Source/JavaScriptCore/parser/Lexer.cpp

@@ -26 +26 @@
 #include "Lexer.h"
 
+#include "CommonIdentifiers.h"
+#include "Identifier.h"
 #include "JSFunctionInlines.h"
-
 #include "JSGlobalObjectFunctions.h"
-#include "Identifier.h"
 #include "NodeInfo.h"
 #include "Nodes.h"
@@ -91 +91 @@
     // Other types (only one so far)
     CharacterWhiteSpace,
+    CharacterPrivateIdentifierStart
 };
 
@@ -159 +160 @@
 /*  62 - >                  */ CharacterGreater,
 /*  63 - ?                  */ CharacterQuestion,
-/*  64 - @                  */ CharacterInvalid,
+/*  64 - @                  */ CharacterPrivateIdentifierStart,
 /*  65 - A                  */ CharacterIdentifierStart,
 /*  66 - B                  */ CharacterIdentifierStart,
@@ -487 +488 @@
 
 template <typename T>
-Lexer<T>::Lexer(VM* vm)
+Lexer<T>::Lexer(VM* vm, JSParserStrictness strictness)
     : m_isReparsing(false)
     , m_vm(vm)
+    , m_parsingBuiltinFunction(strictness == JSParseBuiltin)
 {
 }
@@ -754 +756 @@
     m_buffer16.append(static_cast<UChar>(c));
 }
-
+    
+#if !ASSERT_DISABLED
+bool isSafeIdentifier(VM& vm, const Identifier* ident)
+{
+    if (!ident)
+        return true;
+    /* Just block any use of suspicious identifiers.  This is intended to
+     * be used as a safety net while implementing builtins.
+     */
+    if (*ident == vm.propertyNames->call)
+        return false;
+    if (*ident == vm.propertyNames->apply)
+        return false;
+    if (*ident == vm.propertyNames->eval)
+        return false;
+    if (*ident == vm.propertyNames->Function)
+        return false;
+    return true;
+}
+#endif
+    
 template <>
 template <bool shouldCreateIdentifier> ALWAYS_INLINE JSTokenType Lexer<LChar>::parseIdentifier(JSTokenData* tokenData, unsigned lexerFlags, bool strictMode)
@@ -766 +788 @@
         }
     }
-
+    
+    bool isPrivateName = m_current == '@' && m_parsingBuiltinFunction;
+    if (isPrivateName)
+        shift();
+    
     const LChar* identifierStart = currentSourcePtr();
     unsigned identifierLineStart = currentLineStartOffset();
@@ -780 +806 @@
     const Identifier* ident = 0;
     
-    if (shouldCreateIdentifier) {
+    if (shouldCreateIdentifier || m_parsingBuiltinFunction) {
         int identifierLength = currentSourcePtr() - identifierStart;
         ident = makeIdentifier(identifierStart, identifierLength);
-
+        if (m_parsingBuiltinFunction) {
+            if (!isSafeIdentifier(*m_vm, ident) && !isPrivateName) {
+                m_lexErrorMessage = makeString("The use of '", ident->string(), "' is disallowed in builtin functions.");
+                return ERRORTOK;
+            }
+            if (isPrivateName)
+                ident = m_vm->propertyNames->getPrivateName(*ident);
+            else if (*ident == m_vm->propertyNames->undefinedKeyword)
+                tokenData->ident = &m_vm->propertyNames->undefinedKeywordPrivateName;
+            if (!ident)
+                return INVALID_PRIVATE_NAME_ERRORTOK;
+        }
         tokenData->ident = ident;
     } else
         tokenData->ident = 0;
 
-    if (UNLIKELY((remaining < maxTokenLength) && !(lexerFlags & LexerFlagsIgnoreReservedWords))) {
+    if (UNLIKELY((remaining < maxTokenLength) && !(lexerFlags & LexerFlagsIgnoreReservedWords)) && !isPrivateName) {
         ASSERT(shouldCreateIdentifier);
         if (remaining < maxTokenLength) {
@@ -815 +852 @@
         }
     }
+    
+    bool isPrivateName = m_current == '@' && m_parsingBuiltinFunction;
+    if (isPrivateName)
+        shift();
 
     const UChar* identifierStart = currentSourcePtr();
@@ -827 +868 @@
     
     if (UNLIKELY(m_current == '\\')) {
+        ASSERT(!isPrivateName);
         setOffsetFromSourcePtr(identifierStart, identifierLineStart);
         return parseIdentifierSlowCase<shouldCreateIdentifier>(tokenData, lexerFlags, strictMode);
@@ -838 +880 @@
     const Identifier* ident = 0;
     
-    if (shouldCreateIdentifier) {
+    if (shouldCreateIdentifier || m_parsingBuiltinFunction) {
         int identifierLength = currentSourcePtr() - identifierStart;
         if (isAll8Bit)
@@ -844 +886 @@
         else
             ident = makeIdentifier(identifierStart, identifierLength);
-        
+        if (m_parsingBuiltinFunction) {
+            if (!isSafeIdentifier(*m_vm, ident) && !isPrivateName) {
+                m_lexErrorMessage = makeString("The use of '", ident->string(), "' is disallowed in builtin functions.");
+                return ERRORTOK;
+            }
+            if (isPrivateName)
+                ident = m_vm->propertyNames->getPrivateName(*ident);
+            else if (*ident == m_vm->propertyNames->undefinedKeyword)
+                tokenData->ident = &m_vm->propertyNames->undefinedKeywordPrivateName;
+            if (!ident)
+                return INVALID_PRIVATE_NAME_ERRORTOK;
+        }
         tokenData->ident = ident;
     } else
         tokenData->ident = 0;
     
-    if (UNLIKELY((remaining < maxTokenLength) && !(lexerFlags & LexerFlagsIgnoreReservedWords))) {
+    if (UNLIKELY((remaining < maxTokenLength) && !(lexerFlags & LexerFlagsIgnoreReservedWords)) && !isPrivateName) {
         ASSERT(shouldCreateIdentifier);
         if (remaining < maxTokenLength) {
@@ -1659 +1712 @@
         FALLTHROUGH;
     case CharacterBackSlash:
+        parseIdent:
         if (lexerFlags & LexexFlagsDontBuildKeywords)
             token = parseIdentifier<false>(tokenData, lexerFlags, strictMode);
@@ -1671 +1725 @@
         m_lineStart = m_code;
         goto start;
+    case CharacterPrivateIdentifierStart:
+        if (m_parsingBuiltinFunction)
+            goto parseIdent;
+
+        FALLTHROUGH;
     case CharacterInvalid:
         m_lexErrorMessage = invalidCharacterMessage();