proc_macro: Use ToTokens trait in quote macro

[SpriteOvO]

Tracking issues: #130977, #54722

This PR changed quote macro to use ToTokens trait instead of TokenStream::from, and added a test case for directly referencing an individual token.

r? @dtolnay
CC @tgross35

[tgross35]

I think the way this function works should be changed; instead of having a big tree that gets .collect()ed here, it should create a TokenStream at the beginning and then use ToTokens::to_tokens to append to it, rather than always cloning.

That should also let the loop be flattened a bit:

pub fn quote(stream: TokenStream) -> TokenStream {
    let mut output = TokenStream::new();
    if stream.is_empty() {
        return output;
    }
    let proc_macro_crate = quote!(crate);
    let mut after_dollar = false;

    for tree in stream {
        if after_dollar {
            after_dollar = false;
            match tree { /* ... */ }
            continue;
        }

        if let TokenTree::punct(ref tt) = tree {
            if tt.as_char() == '$' {
                after_dollar = true;
                continue;
            }
        }

        match tree {
            // similar to what exists except the arms will just use `to_tokens`
            // rather than cloning and returning something.
        }
    }
}

It may also be good to rename the macro_rules! quote in this file to quote_impl since reading all the quote! invocations in this function is somewhat misleading.

[tgross35]

Thanks for working on this!

Could you update the new test file with applicable tests from https://github.com/dtolnay/quote/blob/aafba72e10919ad47c05169271cb78e614fb2b9d/tests/test.rs? Because of token stream limitations we can't actually mark them #[test] or add the assert! (eventually we will, this should be a FIXME), but keeping the quote! invocations at least allows us to verify that the expected things do quote.

We can probably reuse the UI tests from https://github.com/dtolnay/quote/tree/aafba72e10919ad47c05169271cb78e614fb2b9d/tests/ui too. I think you can just make a new tests/ui/proc-macro/quote directory.