Track the number of tokens seen to metrics

[muellerzr]

What does this PR do?

This PR adds num_tokens_seen to the TrainerState allowing users to know how many tokens were passed in an individual batch. Uses gather to ensure that in DDP this can be known as well.

Fixes #27027

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@pacman100 @amyeroberts

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[muellerzr]

@amyeroberts question: these failures are all because they cannot find the main_input_name in the batch. Does this mean some of the models are wrong?

For instance, one of the failures in the examples script is from SpeechEncoderDecoderModel, which states main_input_name is "inputs", however when running the script it's using `"input_values":

self = {'input_values': tensor([[-0.0177, -0.0188, -0.0202,  ..., -0.0032, -0.0068, -0.0039],
        [-0.0196, -0.0556, -0.0...-100, -100, -100, -100, -100, -100, -100, -100, -100,
         -100, -100, -100, -100, -100, -100, -100, -100, -100]])}

[amyeroberts]

Thanks for adding this!

Main comment is about whether this should be running by default

[amyeroberts]

@muellerzr I think we don't have to worry about that if it's behind a flag in the training args. This way - if someone want to measure it then they have to make sure that main_input_name is properly set, but it won't break trainer for models which are currently compatible but don't have this correctly set.

[amyeroberts]

Thanks for adding and iterating!

Just some nits on the implementation

[muellerzr]

@amyeroberts failing tests seem to be unrelated

[amyeroberts]

Thanks!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.