Handle negative literals in token_to_literal

[0xJonas]

token_to_literal does not consider the leading - for negative int/float literals, resulting in an error when expanding certain proc-macros:

use quote::quote;
use proc_macro2::Literal;
use proc_macro::TokenStream;

#[proc_macro]
pub fn foo(_: TokenStream) -> TokenStream {
    let t = Literal::i32_unsuffixed(-1);
    quote!(vec![#t]).into()
}

foo!(); // Error: expected Expr

The problem affects proc-macros, where the expansion contains a declarative macro, which contains a negative Literal.

[Veykril]

This might cause issues elsewhere for us, pretty sure our code so far assumes the minus token to be separate at all times. Would need to check more thoroughly

[0xJonas]

Would it be better if negative literals were split into minus+literal before parsing? The AST is technically missing a minus node.

[Veykril]

The AST is correct I believe. number literals do not contain a prefixed minus in rust's token model https://doc.rust-lang.org/reference/tokens.html#number-literals with the exception of proc-macros.

[Veykril]

I am having a look at t his right now, the tests from #19745 do pass so that's good. But I think this change might be a breaking change for the proc-macro server. Investigating that right now

[Veykril]

Hmm, looking more into this, have you verified that this fixes your issues you have been having? I don't think this touches the part responsible for what is happening in your code snippet?