TF: use the correct config with (...)EncoderDecoder models

[gante]

What does this PR do?

Fixes #18071

Modifies unpack_inputs to ignore the config file for (...)EncoderDecoder models, mimicking the behavior in PT. If we don't ignore it, then unset options will get set with the config's default (False for most of them), causing the inner models to ignore their own config files.

⚠️ I've added a corresponding test for the EncoderDecoder models. I then noticed that other (...)EncoderFecoder tests have copy/pasted their own EncoderDecoderMixin, so I've left the other classes for a follow-up PR with the following question: should a common EncoderDecoderMixin be defined and shared across (...)EncoderDecoder tests, or should I add a similar test to all other classes individually?

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[sgugger]

Thanks for fixing! As for your question, it depends if they are all exact copies and can easily be refactored in one mixin or not.

[gante]

I believe they are -- going to give it a go afterwards if @ydshieh also agrees :)

[ydshieh]

I have limited connection at this moment in the mountain, so feel free to merge if you prefer. Regarding the common mixin, good for me. I see there are a few little things to address, like the input names (input_ids, pixel values etc).
Would be nice if you do this refactorization after merging my PR about the PT/TF equivalence tests, or incoperate the change in it 🙏

Thank you for the fix, @gante

[gante]

@ydshieh can I have a review plz 🙏

[ydshieh]

Thank you for the fix, @gante . Would you mind to rebase on main, and run test_pt_tf_model_equivalence for TF (Vision)EncoderDecoderModel? Thank you.

[ydshieh]

Just a nit, no need to feel strong:

I would personally prefer to use self.__name__ instead of str and test against EncoderDecoder. If I understand correctly, str gives the full path, which includes the module name like encoder_decoder or vision_encoder_decoder, right?

[gante]

I was trying self.__name__ on this line, and it seems like it isn't defined in some cases -- e.g. in tests/models/vision_encoder_decoder/test_modeling_tf_vision_encoder_decoder.py::TFViT2GPT2EncoderDecoderModelTest::test_encoder_decoder_model, it throws *** AttributeError: 'TFVisionEncoderDecoderModel' object has no attribute '__name__'

str(self) here includes the full import path for the object, like you wrote, which contains the class name -- '<transformers.models.vision_encoder_decoder.modeling_tf_vision_encoder_decoder.TFVisionEncoderDecoderModel object at 0x7fc7783b8a60>'

[gante]

Maybe we can update in the future, but since it is working for now I'm going with it :D

[ydshieh]

OK, thanks for the info.

[ydshieh]

Thanks for the improvement / cleanup!

[ydshieh]

nice addition!

[gante]

@ydshieh rebased with main and reran tests -- all working 👍