[ELF] A new Lexer for Linker Script

[hongyu-dev]

• delete peek2 [ELF] Delete peek2 in Lexer #99790
• delete consumeLabel [ELF] Remove consumeLabel in ScriptLexer #99567
• inExpr can have more states for example enum class State {Script, Expr, Wild }  #100462
• add new API for unquote(tok) for while statement #101035
• Improve hongyu-dev@4ad3dee
• update symbol assignment
• MEMORY, merge memory.s and memory-err.s
• improve testing for `DATA_SEGMENT_RELRO_END
• introduce state to remove some unquote and [ELF] Reject error-prone meta characters in input section description #84130 hack 30ec2bf
• fully stateful lexer; replace eager tokenize with on-demand lexing;with on-demand lexing, replace getColumnNumber (from Token) with Lexer::getColumnNumber [ELF] ScriptLexer: generate tokens lazily #100493
• replace mbs with an include stack (INCLUDE); replace getCurrentMB() [LLD] Invalid 'there is a cycle in linker script INCLUDEs' error #93947
• add a consume overload to accept Token. The existing consume(StringRef) is be a special case that expects identifier/quoted_string.
• add struct Token {StringRef str; TokenKind kind;}; vector<StringRef> tokens; => vector<Token> token; change next/peek to return Token #99921 [ELF] Added struct Token and changed next() and peek() to return Token #100180

We want to replace current lexer with a stateful one to report more errors and get rid of many hacks in current lexer such as (ScriptLexer::maybeSplitExpr is removed in #93947). The list of items is the plan to update lexer gradually.

[llvmbot]

@llvm/issue-subscribers-lld-elf

Author: Hongyu Chen (yugier)

- [x] ~~delete peek2~~ https://github.com//pull/99790 - [ ] delete consumeLabel - [ ] add `struct Token {StringRef str; TokenKind kind;}; vector<StringRef> tokens;` => `vector<Token> token;` change next/peek to return Token - [ ] add a consume overload to accept Token. The existing consume(StringRef) is be a special case that expects identifier/quoted_string. - [ ] introduce state to remove some unquote and https://github.com//pull/84130 hack - [ ] introduce state to replace inExpr - [ ] fully stateful lexer; replace eager tokenize with on-demand lexing;with on-demand lexing, replace getColumnNumber (from Token) with Lexer::getColumnNumber - [ ] replace mbs with an include stack (INCLUDE); replace getCurrentMB()

We want to replace current lexer with a stateful one to report more errors and get rid of many hacks in current lexer such as ScriptLexer::maybeSplitExpr. The list of items is the plan to update lexer gradually.