update the config.is_decoder=True before initialize the decoder

[zlinao]

Currently the PreTrainedEncoderDecoder class fails to initialize the "cross-attention layer" since it updates decoder.config.is_decoder = True after decoder initialization.

[codecov-io]

Codecov Report

Merging #2435 into master will decrease coverage by <.01%.
The diff coverage is 25%.

@@            Coverage Diff             @@
##           master    #2435      +/-   ##
==========================================
- Coverage   73.24%   73.24%   -0.01%     
==========================================
  Files          87       87              
  Lines       15001    15004       +3     
==========================================
+ Hits        10988    10989       +1     
- Misses       4013     4015       +2

Impacted Files	Coverage Δ
src/transformers/modeling_encoder_decoder.py	25.58% <25%> (+0.28%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9261c7f...b4418d3. Read the comment docs.

[LysandreJik]

Indeed, the cross attention is initialized in BertLayer and needs knowledge of the is_decoder boolean to ensure it is correctly initialized.

Looks good to me, thanks @zlinao

[zlinao]

Indeed, the cross attention is initialized in BertLayer and needs knowledge of the is_decoder boolean to ensure it is correctly initialized.

Looks good to me, thanks @zlinao

Yes, exactly.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.