Updating run_swag script for new pytorch_transformers setup

[yakazimir]

https://github.com/huggingface/pytorch-transformers/blob/f2a3eb987e1fc2c85320fc3849c67811f5736b50/examples/single_model_scripts/run_swag.py#L35

It appears that WEIGHTS_NAME and CONFIG_NAME have moved out of {pytorch_transformers/pytorch_pretrained_bert}.file_utils (and instead can be imported directly from pytorch_transformers), as shown below :

>>> import pytorch_transformers as p
>>> p.__version__
'1.0.0'
>>> from pytorch_transformers.file_utils import WEIGHTS_NAME, CONFIG_NAME
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'WEIGHTS_NAME'
>>> from pytorch_transformers.file_utils import CONFIG_NAME
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'CONFIG_NAME'
>>> from pytorch_transformers import WEIGHTS_NAME
>>> 

This seems incorrect in the new run_swag script (as shown above) (if true, are there any other important imports that were overlooked here?)

[yakazimir]

I found a few other issues: 1) the script uses the old BertAdam (instead of AdamW)

https://github.com/huggingface/pytorch-transformers/blob/44dd941efb602433b7edc29612cbdd0a03bf14dc/examples/single_model_scripts/run_swag.py#L431

2. the train and test loops still use the old version of forward, i.e.,

https://github.com/huggingface/pytorch-transformers/blob/44dd941efb602433b7edc29612cbdd0a03bf14dc/examples/single_model_scripts/run_swag.py#L450
https://github.com/huggingface/pytorch-transformers/blob/44dd941efb602433b7edc29612cbdd0a03bf14dc/examples/single_model_scripts/run_swag.py#L525

In the former case, I added the following:

                #loss = model(input_ids, segment_ids, input_mask, label_ids) # line 450 in train
                outputs = model(input_ids, segment_ids, input_mask, label_ids)
                loss = outputs[0]

And the latter:

                ## tmp_eval_loss = model(input_ids, segment_ids, input_mask, label_ids) ## line 525
                output = model(input_ids, segment_ids, input_mask, label_ids)
                tmp_eval_loss,logits = output[:2]

I'd be curious if the last two versions are correct. I'm actually not able to reproduce the Swag results I was getting before the refactoring (reported here: https://github.com/huggingface/pytorch-transformers/blob/v0.6.2/README.md). Rather than getting around 80%, I'm stuck at around 78%.

[thomwolf]

Yes the run_swag script hasn't been updated to the new API yet.

Do you want to give it a look and submit a PR? I don't have plan to work on it in the short-term.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.