Tests for more examples

[srush]

Add tests for more of the examples/

We need to have more testing code for examples. This is particularly true with NER which recently had a tokenizer issue.

(Self-assigning)

[sshleifer]

I agree. Tried for an hour last night to add coverage for run_bart_sum.py and got stuck on two things.

1. A circleci job that installs pytorch_lightning (and potentially other dependencies)
2. the ability to import examples/transformer_base.py

Suggested Approach:

• add the aforementioned circleci job
• a flag like @require_lightning to decorate some tests
• some code changes to get the imports working sanely.
• Checklist/Instructions for how new examples/ contributors can add test coverage.

Happy to help with this!
CC: @LysandreJik , @julien-c, @thomwolf @patrickvonplaten

[srush]

Cool. Most of these seem easy. Sharing code between examples is a bit harder. I am not sure if we should have and examples package, or symlink that shared file somehow. Currently we add it to the path before running the code.

[ethanjperez]

With the Pytorch-Lightning examples, I've been running into an issue loading trained models with --do_predict (when not also using --do_train), so it would be helpful to add some model-loading test as well :)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.