Ignore:
Timestamp:
Aug 4, 2015, 2:26:49 PM (10 years ago)
Author:
Yusuke Suzuki
Message:

[ES6] Support Module Syntax
​https://bugs.webkit.org/show_bug.cgi?id=147422

Reviewed by Saam Barati.

Source/JavaScriptCore:

This patch introduces ES6 Modules syntax parsing part.
In this patch, ASTBuilder just produces the corresponding nodes to the ES6 Modules syntax,
and this patch does not include the code generator part.

Modules require 2 phase parsing. In the first pass, we just analyze the dependent modules
and do not execute the body or construct the AST. And after analyzing all the dependent
modules, we will parse the dependent modules next.
After all analyzing part is done, we will start the second pass. In the second pass, we
will parse the module, produce the AST, and execute the body.
If we don't do so, we need to create all the ASTs in the module's dependent graph at first
because the given module can be executed after the all dependent modules are executed. It
means that we need to hold so many parser arenas. To avoid this, the first pass only extracts
the dependent modules' information.

In this patch, we don't add this analyzing part yet. This patch only implements the second pass.
This patch aims at just implementing the syntax parsing functionality correctly.
After this patch is landed, we will create the ModuleDependencyAnalyzer that inherits SyntaxChecker
to collect the dependent modules fast[1].

To test the parsing, we added the "checkModuleSyntax" function into jsc shell.
By using this, we can parse the given string as the module.

[1]: ​https://bugs.webkit.org/show_bug.cgi?id=147353

  • bytecompiler/NodesCodegen.cpp:

(JSC::ModuleProgramNode::emitBytecode):
(JSC::ImportDeclarationNode::emitBytecode):
(JSC::ExportAllDeclarationNode::emitBytecode):
(JSC::ExportDefaultDeclarationNode::emitBytecode):
(JSC::ExportLocalDeclarationNode::emitBytecode):
(JSC::ExportNamedDeclarationNode::emitBytecode):

  • jsc.cpp:

(GlobalObject::finishCreation):
(functionCheckModuleSyntax):

  • parser/ASTBuilder.h:

(JSC::ASTBuilder::createModuleSpecifier):
(JSC::ASTBuilder::createImportSpecifier):
(JSC::ASTBuilder::createImportSpecifierList):
(JSC::ASTBuilder::appendImportSpecifier):
(JSC::ASTBuilder::createImportDeclaration):
(JSC::ASTBuilder::createExportAllDeclaration):
(JSC::ASTBuilder::createExportDefaultDeclaration):
(JSC::ASTBuilder::createExportLocalDeclaration):
(JSC::ASTBuilder::createExportNamedDeclaration):
(JSC::ASTBuilder::createExportSpecifier):
(JSC::ASTBuilder::createExportSpecifierList):
(JSC::ASTBuilder::appendExportSpecifier):

  • parser/Keywords.table:
  • parser/NodeConstructors.h:

(JSC::ModuleSpecifierNode::ModuleSpecifierNode):
(JSC::ImportSpecifierNode::ImportSpecifierNode):
(JSC::ImportDeclarationNode::ImportDeclarationNode):
(JSC::ExportAllDeclarationNode::ExportAllDeclarationNode):
(JSC::ExportDefaultDeclarationNode::ExportDefaultDeclarationNode):
(JSC::ExportLocalDeclarationNode::ExportLocalDeclarationNode):
(JSC::ExportNamedDeclarationNode::ExportNamedDeclarationNode):
(JSC::ExportSpecifierNode::ExportSpecifierNode):

  • parser/Nodes.cpp:

(JSC::ModuleProgramNode::ModuleProgramNode):

  • parser/Nodes.h:

(JSC::ModuleProgramNode::startColumn):
(JSC::ModuleProgramNode::endColumn):
(JSC::ModuleSpecifierNode::moduleName):
(JSC::ImportSpecifierNode::importedName):
(JSC::ImportSpecifierNode::localName):
(JSC::ImportSpecifierListNode::specifiers):
(JSC::ImportSpecifierListNode::append):
(JSC::ImportDeclarationNode::specifierList):
(JSC::ImportDeclarationNode::moduleSpecifier):
(JSC::ExportAllDeclarationNode::moduleSpecifier):
(JSC::ExportDefaultDeclarationNode::declaration):
(JSC::ExportLocalDeclarationNode::declaration):
(JSC::ExportSpecifierNode::exportedName):
(JSC::ExportSpecifierNode::localName):
(JSC::ExportSpecifierListNode::specifiers):
(JSC::ExportSpecifierListNode::append):
(JSC::ExportNamedDeclarationNode::specifierList):
(JSC::ExportNamedDeclarationNode::moduleSpecifier):

  • parser/Parser.cpp:

(JSC::Parser<LexerType>::Parser):
(JSC::Parser<LexerType>::parseInner):
(JSC::Parser<LexerType>::parseModuleSourceElements):
(JSC::Parser<LexerType>::parseVariableDeclaration):
(JSC::Parser<LexerType>::parseVariableDeclarationList):
(JSC::Parser<LexerType>::createBindingPattern):
(JSC::Parser<LexerType>::tryParseDestructuringPatternExpression):
(JSC::Parser<LexerType>::parseDestructuringPattern):
(JSC::Parser<LexerType>::parseForStatement):
(JSC::Parser<LexerType>::parseFormalParameters):
(JSC::Parser<LexerType>::parseFunctionParameters):
(JSC::Parser<LexerType>::parseFunctionDeclaration):
(JSC::Parser<LexerType>::parseClassDeclaration):
(JSC::Parser<LexerType>::parseModuleSpecifier):
(JSC::Parser<LexerType>::parseImportClauseItem):
(JSC::Parser<LexerType>::parseImportDeclaration):
(JSC::Parser<LexerType>::parseExportSpecifier):
(JSC::Parser<LexerType>::parseExportDeclaration):
(JSC::Parser<LexerType>::parseMemberExpression):

  • parser/Parser.h:

(JSC::isIdentifierOrKeyword):
(JSC::ModuleScopeData::create):
(JSC::ModuleScopeData::exportedBindings):
(JSC::ModuleScopeData::exportName):
(JSC::ModuleScopeData::exportBinding):
(JSC::Scope::Scope):
(JSC::Scope::setIsModule):
(JSC::Scope::moduleScopeData):
(JSC::Parser::matchContextualKeyword):
(JSC::Parser::matchIdentifierOrKeyword):
(JSC::Parser::isofToken): Deleted.

  • parser/ParserModes.h:
  • parser/ParserTokens.h:
  • parser/SyntaxChecker.h:

(JSC::SyntaxChecker::createModuleSpecifier):
(JSC::SyntaxChecker::createImportSpecifier):
(JSC::SyntaxChecker::createImportSpecifierList):
(JSC::SyntaxChecker::appendImportSpecifier):
(JSC::SyntaxChecker::createImportDeclaration):
(JSC::SyntaxChecker::createExportAllDeclaration):
(JSC::SyntaxChecker::createExportDefaultDeclaration):
(JSC::SyntaxChecker::createExportLocalDeclaration):
(JSC::SyntaxChecker::createExportNamedDeclaration):
(JSC::SyntaxChecker::createExportSpecifier):
(JSC::SyntaxChecker::createExportSpecifierList):
(JSC::SyntaxChecker::appendExportSpecifier):

  • runtime/CommonIdentifiers.cpp:

(JSC::CommonIdentifiers::CommonIdentifiers):

  • runtime/CommonIdentifiers.h:
  • runtime/Completion.cpp:

(JSC::checkModuleSyntax):

  • runtime/Completion.h:
  • tests/stress/modules-syntax-error-with-names.js: Added.

(shouldThrow):

  • tests/stress/modules-syntax-error.js: Added.

(shouldThrow):
(checkModuleSyntaxError.checkModuleSyntaxError.checkModuleSyntaxError):

  • tests/stress/modules-syntax.js: Added.

(prototype.checkModuleSyntax):
(checkModuleSyntax):

  • tests/stress/tagged-templates-syntax.js:

LayoutTests:

'export' and 'import' are changed from FutureReservedWord to Keyword in ES6.
​http://www.ecma-international.org/ecma-262/6.0/#sec-keywords
And restrict 'super' use under the Script / Module contexts.

  • js/dom/reserved-words-as-property-expected.txt:
  • sputnik/Conformance/07_Lexical_Conventions/7.5_Tokens/7.5.3_Future_Reserved_Words/S7.5.3_A1.10-expected.txt:
  • sputnik/Conformance/07_Lexical_Conventions/7.5_Tokens/7.5.3_Future_Reserved_Words/S7.5.3_A1.16-expected.txt:
  • sputnik/Conformance/07_Lexical_Conventions/7.5_Tokens/7.5.3_Future_Reserved_Words/S7.5.3_A1.27-expected.txt:
File:
1 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/parser/Parser.h

    r187351 r187890  
    128128    return isEval(vm, ident) || isArguments(vm, ident);
    129129}
     130ALWAYS_INLINE static bool isIdentifierOrKeyword(const JSToken& token)
     131{
     132    return token.m_type == IDENT || token.m_type & KeywordTokenFlag;
     133}
     134
     135class ModuleScopeData : public RefCounted<ModuleScopeData> {
     136public:
     137    static Ref<ModuleScopeData> create() { return adoptRef(*new ModuleScopeData); }
     138
     139    const IdentifierSet& exportedBindings() const { return m_exportedBindings; }
     140
     141    bool exportName(const Identifier& exportedName)
     142    {
     143        return m_exportedNames.add(exportedName.impl()).isNewEntry;
     144    }
     145
     146    void exportBinding(const Identifier& localName)
     147    {
     148        m_exportedBindings.add(localName.impl());
     149    }
     150
     151private:
     152    IdentifierSet m_exportedNames { };
     153    IdentifierSet m_exportedBindings { };
     154};
    130155
    131156struct Scope {
    … …  
    165190        , m_loopDepth(rhs.m_loopDepth)
    166191        , m_switchDepth(rhs.m_switchDepth)
     192        , m_moduleScopeData(rhs.m_moduleScopeData)
    167193    {
    168194        if (rhs.m_labels) {
    … …  
    216242    }
    217243
     244    void setIsModule()
     245    {
     246        m_moduleScopeData = ModuleScopeData::create();
     247    }
     248
    218249    bool isFunction() const { return m_isFunction; }
    219250    bool isFunctionBoundary() const { return m_isFunctionBoundary; }
    … …  
    235266
    236267        return m_lexicalVariables;
     268    }
     269
     270    ModuleScopeData& moduleScopeData() const
     271    {
     272        ASSERT(m_moduleScopeData);
     273        return *m_moduleScopeData;
    237274    }
    238275
    … …  
    533570    IdentifierSet m_closedVariableCandidates;
    534571    IdentifierSet m_writtenVariables;
     572    RefPtr<ModuleScopeData> m_moduleScopeData { };
    535573};
    536574
    … …  
    848886    }
    849887   
    850     ALWAYS_INLINE bool isofToken()
    851     {
    852         return m_token.m_type == IDENT && *m_token.m_data.ident == m_vm->propertyNames->of;
     888    ALWAYS_INLINE bool matchContextualKeyword(const Identifier& identifier)
     889    {
     890        return m_token.m_type == IDENT && *m_token.m_data.ident == identifier;
     891    }
     892
     893    ALWAYS_INLINE bool matchIdentifierOrKeyword()
     894    {
     895        return isIdentifierOrKeyword(m_token);
    853896    }
    854897   
    … …  
    10081051    template <class TreeBuilder> TreeStatement parseStatementListItem(TreeBuilder&, const Identifier*& directive, unsigned* directiveLiteralLength);
    10091052    template <class TreeBuilder> TreeStatement parseStatement(TreeBuilder&, const Identifier*& directive, unsigned* directiveLiteralLength = 0);
     1053    enum class ExportType { Exported, NotExported };
    10101054#if ENABLE(ES6_CLASS_SYNTAX)
    1011     template <class TreeBuilder> TreeStatement parseClassDeclaration(TreeBuilder&);
     1055    template <class TreeBuilder> TreeStatement parseClassDeclaration(TreeBuilder&, ExportType = ExportType::NotExported);
    10121056#endif
    1013     template <class TreeBuilder> TreeStatement parseFunctionDeclaration(TreeBuilder&);
    1014     template <class TreeBuilder> TreeStatement parseVariableDeclaration(TreeBuilder&, DeclarationType);
     1057    template <class TreeBuilder> TreeStatement parseFunctionDeclaration(TreeBuilder&, ExportType = ExportType::NotExported);
     1058    template <class TreeBuilder> TreeStatement parseVariableDeclaration(TreeBuilder&, DeclarationType, ExportType = ExportType::NotExported);
    10151059    template <class TreeBuilder> TreeStatement parseDoWhileStatement(TreeBuilder&);
    10161060    template <class TreeBuilder> TreeStatement parseWhileStatement(TreeBuilder&);
    … …  
    10481092    template <class TreeBuilder> ALWAYS_INLINE bool parseFormalParameters(TreeBuilder&, TreeFormalParameterList, unsigned&);
    10491093    enum VarDeclarationListContext { ForLoopContext, VarDeclarationContext };
    1050     template <class TreeBuilder> TreeExpression parseVariableDeclarationList(TreeBuilder&, int& declarations, TreeDestructuringPattern& lastPattern, TreeExpression& lastInitializer, JSTextPosition& identStart, JSTextPosition& initStart, JSTextPosition& initEnd, VarDeclarationListContext, DeclarationType, bool& forLoopConstDoesNotHaveInitializer);
     1094    template <class TreeBuilder> TreeExpression parseVariableDeclarationList(TreeBuilder&, int& declarations, TreeDestructuringPattern& lastPattern, TreeExpression& lastInitializer, JSTextPosition& identStart, JSTextPosition& initStart, JSTextPosition& initEnd, VarDeclarationListContext, DeclarationType, ExportType, bool& forLoopConstDoesNotHaveInitializer);
    10511095    template <class TreeBuilder> TreeSourceElements parseArrowFunctionSingleExpressionBodySourceElements(TreeBuilder&);
    10521096    template <class TreeBuilder> TreeExpression parseArrowFunctionExpression(TreeBuilder&);
    1053     template <class TreeBuilder> NEVER_INLINE TreeDestructuringPattern createBindingPattern(TreeBuilder&, DestructuringKind, const Identifier&, int depth, JSToken, AssignmentContext, const Identifier** duplicateIdentifier);
    1054     template <class TreeBuilder> NEVER_INLINE TreeDestructuringPattern parseDestructuringPattern(TreeBuilder&, DestructuringKind, const Identifier** duplicateIdentifier = nullptr, bool* hasDestructuringPattern = nullptr, AssignmentContext = AssignmentContext::DeclarationStatement, int depth = 0);
     1097    template <class TreeBuilder> NEVER_INLINE TreeDestructuringPattern createBindingPattern(TreeBuilder&, DestructuringKind, ExportType, const Identifier&, int depth, JSToken, AssignmentContext, const Identifier** duplicateIdentifier);
     1098    template <class TreeBuilder> NEVER_INLINE TreeDestructuringPattern parseDestructuringPattern(TreeBuilder&, DestructuringKind, ExportType, const Identifier** duplicateIdentifier = nullptr, bool* hasDestructuringPattern = nullptr, AssignmentContext = AssignmentContext::DeclarationStatement, int depth = 0);
    10551099    template <class TreeBuilder> NEVER_INLINE TreeDestructuringPattern tryParseDestructuringPatternExpression(TreeBuilder&, AssignmentContext);
    10561100    template <class TreeBuilder> NEVER_INLINE TreeExpression parseDefaultValueForDestructuringPattern(TreeBuilder&);
     1101    template <class TreeBuilder> TreeSourceElements parseModuleSourceElements(TreeBuilder&);
     1102    enum class ImportSpecifierType { NamespaceImport, NamedImport, DefaultImport };
     1103    template <class TreeBuilder> typename TreeBuilder::ImportSpecifier parseImportClauseItem(TreeBuilder&, ImportSpecifierType);
     1104    template <class TreeBuilder> typename TreeBuilder::ModuleSpecifier parseModuleSpecifier(TreeBuilder&);
     1105    template <class TreeBuilder> TreeStatement parseImportDeclaration(TreeBuilder&);
     1106    template <class TreeBuilder> typename TreeBuilder::ExportSpecifier parseExportSpecifier(TreeBuilder& context, Vector<const Identifier*>& maybeLocalNames, bool& hasKeywordForLocalBindings);
     1107    template <class TreeBuilder> TreeStatement parseExportDeclaration(TreeBuilder&);
    10571108
    10581109    template <class TreeBuilder> NEVER_INLINE bool parseFunctionInfo(TreeBuilder&, FunctionRequirements, FunctionParseMode, bool nameIsInContainingScope, ConstructorKind, SuperBinding, int functionKeywordStart, ParserFunctionInfo<TreeBuilder>&, FunctionParseType);
    … …  
    11821233    CodeFeatures m_features;
    11831234    int m_numConstants;
     1235    JSParserCodeType m_codeType;
    11841236   
    11851237    struct DepthManager {
Note: See TracChangeset for help on using the changeset viewer.