Compiler design syntax analysis
Download as PPTX, PDF8 likes8,657 views
This document discusses syntax analysis in compiler design. It begins by explaining that the lexer takes a string of characters as input and produces a string of tokens as output, which is then input to the parser. The parser takes the string of tokens and produces a parse tree of the program. Context-free grammars are introduced as a natural way to describe the recursive structure of programming languages. Derivations and parse trees are discussed as ways to parse strings based on a grammar. Issues like ambiguity and left recursion in grammars are covered, along with techniques like left factoring that can be used to transform grammars.
Compiler design syntax analysis
- 1. COMPILER DESIGN SYNTAX ANALYSIS RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 1 Ms. RICHA SHARMA Assistant Professor [email protected] Lovely Professional University
- 2. SYNTAX ANALYSIS INTRODUCTION • LEXICAL PHASE IS IMPLEMENTED ON FINITE AUTOMATA & FINITE AUTOMATA CAN REALLY ONLY EXPRESS THINGS WHERE YOU CAN COUNT MODULUS ON K. • REGULAR LANGUAGES – THE WEAKEST FORMAL LANGUAGES WIDELY USED – MANY APPLICATIONS – CAN’T HANDLE ITERATION & NESTED LOOPS(NESTED IF ELSE ). TO SUMMARIZE, THE LEXER TAKES A STRING OF CHARACTER AS INPUT AND PRODUCES A STRING OF TOKENS AS OUTPUT. THAT STRING OF TOKENS IS THE INPUT TO THE PARSER WHICH TAKES A STRING OF TOKENS AND PRODUCES A PARSE TREE OF THE PROGRAM. SOMETIMES THE PARSE TREE IS ONLY IMPLICIT. SO THE, A COMPILER MAY NEVER ACTUALLY BUILD THE FULL PARSE TREE. RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 2
- 3. Lexical Analyzer Parser Source program token getNext Token Symbol table Parse tree Rest of Front End Intermediate representation ROLE OF SYNTAX ANALYSIS/PARSER RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 3
- 4. CONTEXT FREE GRAMMARS expression -> expression + term expression -> expression – term expression -> term term -> term * factor term -> term / factor term -> factor factor -> (expression) factor -> id S - IS A FINITE SET OF TERMINALS N - IS A FINITE SET OF NON-TERMINALS P - IS A FINITE SUBSET OF PRODUCTION RULES S - IS THE START SYMBOL G=(S ,N,P,S) • A GRAMMAR DERIVES STRINGS BY BEGINNING WITH START SYMBOL AND REPEATEDLY REPLACING A NON TERMINAL BY THE RIGHT HAND SIDE OF A PRODUCTION FOR THAT NON TERMINAL. • FROM THE START SYMBOL OF A GRAMMAR G FORM THE LANGUAGE L(G) DEFINED BY THE GRAMMAR THE STRINGS THAT CAN BE DERIVED . RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 4
- 5. RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 5 • PROGRAMMING LANGUAGES HAVE RECURSIVE STRUCTURE • CONTEXT-FREE GRAMMARS ARE A NATURAL NOTATION FOR THIS RECURSIVE STRUCTURE . NOT ALL STRINGS OF TOKENS ARE PROGRAMS . . . . . . PARSER MUST DISTINGUISH BETWEEN VALID AND INVALID STRINGS OF TOKENS WE NEED : – A LANGUAGE :FOR DESCRIBING VALID STRINGS OF TOKENS – A METHOD: FOR DISTINGUISHING VALID FROM INVALID STRINGS OF TOKENS CONTEXT FREE GRAMMARS
- 6. RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) E ::= T | E + T | E - T T ::= F | T * F |T / F F ::= id | (E) • ARITHMETIC EXPRESSIONS • STATEMENTS If Statement ::= if E then Statement else Statement CONTEXT FREE GRAMMAR EXAMPLES Steps: 1. Begin with a string with only the start symbol S 2. Replace any non-terminal X in the string by the right-hand side of some production X -> Y1…Yn 3. Repeat (2) until there are no non- terminals 6
- 7. DERIVATIONS • DERIVATION IS A SEQUENCE OF PRODUCTIONS SO BEGINNING WITH THE START SYMBOL. • WE CAN APPLY PRODUCTIONS ONE AT A TIME IN SEQUENCE & THAT WILL PRODUCES A DERIVATION. • A DERIVATION IS A SEQUENCE OF PRODUCTIONS A -> … -> … ->… -> … -> … • A DERIVATION CAN BE DRAWN AS A TREE – START SYMBOL IS THE TREE’S ROOT – FOR A PRODUCTION X -> Y1…Yn ADD CHILDREN Y1…Yn TO NODE X • GRAMMAR E -> E + E | E * E | (E) | ID • STRING ID *ID + ID RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 7
- 8. DERIVATIONS DERIVATIONS ARE OF TWO TYPES: • RIGHTMOST AND LEFTMOST DERIVATIONS • LETS DISCUSS WITH EXAMPLE GRAMMAR: E -> E + E | E * E | -E | (E) | ID STRING :(ID+ID) LEFT MOST DERIVATION RIGHT MOST DERIVATION E E = (E) = (E) = (E+E) = (E+E) = (ID+E) = (E+ID) =(ID+ID) =(ID+ID) RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 8
- 9. DERIVATIONS • NOW WE'RE GOING TO PARSE THIS STRING AND WE'RE GOING TO SHOW HOW TO PRODUCE A DERIVATION FOR THE STRING AND ALSO AT THE SAME TIME BUILD THE TREE. • PARSE TREES HAVE TERMINALS AT THE LEAVES AND NONTERMINALS AT THE INTERIOR NODES AND FURTHERMORE, IN-ORDER TRAVERSAL OF THE LEAVES IS THE ORIGINAL INPUT. • GRAMMAR E -> E + E | E * E | (E) | ID • STRING ID * ID + ID RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 9
- 10. LEFT MOST DERIVATION AND PARSE TREE E E RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 10
- 11. LEFT MOST DERIVATION AND PARSE TREE E E+E E E + E RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 11
- 12. LEFT MOST DERIVATION AND PARSE TREE E E+E E E*E+E E + E E * E RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 12
- 13. LEFT MOST DERIVATION AND PARSE TREE E E+E E E*E+E E + E id*E+E E * E id RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 13
- 14. LEFT MOST DERIVATION AND PARSE TREE E E+E E E*E+E E + E id*E+E E * E id*id+E id id RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 14
- 15. LEFT MOST DERIVATION AND PARSE TREE E E+E E E*E+E E + E id*E+E E * E id id*id+id id id RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 15
- 16. DERIVATIONS • A PARSE TREE HAS – TERMINALS AT THE LEAVES – NON-TERMINALS AT THE INTERIOR NODES • AN IN-ORDER TRAVERSAL OF THE LEAVES IS THE ORIGINAL INPUT • THE PARSE TREE SHOWS THE ASSOCIATION OF OPERATIONS, THE INPUT STRING DOES NOT . NOTE: THAT RIGHT-MOST AND LEFT-MOST DERIVATIONS HAVE THE SAME PARSE TREE IF NOT THEN THE GRAMMAR IS AMBIGUOUS GRAMMAR. RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 16
- 17. AMBIGUITY • IF STRING HAS TWO OR MORE RIGHT MOST DERIVATIONS OR TWO OR MORE LEFT DERIVATIONS THEN THAT STRING WILL HAVE TWO DISTINCT PARSE TREES AND HENCE GRAMMAR WILL BE AMBIGUOUS. • AMBIGUITY IS BAD: LEAVES MEANING OF SOME PROGRAMS ILL-DEFINED • MULTIPLE PARSE TREES FOR SOME PROGRAM THEN THAT ESSENTIALLY MEANS THAT YOU'RE LEAVING IT UP TO THE COMPILER TO PICK WHICH OF THOSE TWO POSSIBLE INTERPRETATIONS OF THE PROGRAM YOU WANT IT TO GENERATE CODE FOR AND THAT'S NOT A GOOD IDEA. • TO REMOVE AMBIGUITY WE NEED TO REWRITE THE RULES CHECKING OVER PRECEDENCE AND ASSOCIATIVITY . RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 17
- 18. AMBIGUITY RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 18 Eg: The string id +id* id produces two parse tree hence the grammar is ambiguous. One can remove the ambiguity by rewriting the grammar as introducing new non-terminal instead of r non-terminal , but it can result in left or right recursion .Hence we have to remove left recursion.
- 19. AMBIGUITY • IF WE HAVE AN AMBIGUOUS GRAMMAR: E →E * E E →NUM • AS THIS DEPENDS ON THE ASSOCIATIVITY OF *,WE USE DIFFERENT REWRITE RULES FOR DIFFERENT ASSOCIATIVITY . • IF * IS LEFT-ASSOCIATIVE, WE MAKE THE GRAMMAR LEFT-RECURSIVE BY HAVING A RECURSIVE REFERENCE TO THE LEFT ONLY OF THE OPERATOR SYMBOL. UNAMBIGUOUS GRAMMAR: E →E * E’ E →E’ E’→NUM RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 19
- 20. LEFT RECURSION • UNAMBIGUOUS GRAMMAR : E →E * E’ E →E’ E’→NUM • THIS GRAMMAR IS NOW LEFT RECURSIVE. LEFT RECURSIVE GRAMMAR IS ANY GRAMMAR THAT HAS A NON-TERMINAL WHERE IF YOU START WITH THAT NON- TERMINAL AND YOU DO SOME NON-EMPTY SEQUENCE OF RE-WRITES. • CONSIDER THE LEFT-RECURSIVE GRAMMAR S -> S a | b • S GENERATES ALL STRINGS STARTING WITH “a” AND FOLLOWED BY ANY NUMBER OF “b’S” • CAN REWRITE USING RIGHT-RECURSION • S ->bS’ S’ ->aS’ |€ RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 20
- 21. EXAMPLES OF LEFT RECURSION 1. E -> E + T | T T -> ID | (E) 2. S ->(L)|X L ->L,S|S 3. S ->S0S1S|01 RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 21
- 22. LEFT FACTORING • LEFT FACTORING IS A GRAMMAR TRANSFORMATION THAT IS USEFUL FOR PRODUCING A DETERMINISTIC GRAMMAR FROM NON- DETERMINISTIC GRAMMAR SUITABLE FOR PREDICTIVE OR TOP-DOWN PARSING. • CONSIDER FOLLOWING GRAMMAR: • STMT -> IF EXPR THEN STMT ELSE STMT • | IF EXPR THEN STMT • ON SEEING INPUT IF IT IS NOT CLEAR FOR THE PARSER WHICH PRODUCTION TO USE • WE CAN EASILY PERFORM LEFT FACTORING: • IF WE HAVE A->ΑΒ1 | ΑΒ2 THEN WE REPLACE IT WITH • A -> ΑA’ • A’ -> Β1 | Β2 RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 22
- 23. EXAMPLES OF LEFT FACTORING 1. S -> iEtS|iEtSES|a E ->b 2. S-> aSSbS|aSaSb|abb|b 3. S-> bSSaaS|bSSaSb|bSb|a RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 23
- 24. RICHA SHARMA (LOVELY PROFESSIONAL UNIVERSITY) 24