Skip to content

Add support for returning the attention scores in encoder call. #1644

@mmilanifard

Description

@mmilanifard

Is your feature request related to a problem? Please describe.
Need to extract self attention scores from a call to the encoder block.

Describe the solution you'd like
Add "return_attention_scores" arg to the call method of encoder (similar to the signature of the attention layer call). Return self attention scores if indicated.

Describe alternatives you've considered
No other way to cleanly extract attention scores IIUC.

Additional context
I can create PR myself if it helps. Not sure if this needs to be done on the decoder model in the same PR.

Metadata

Metadata

Labels

stat:contributions welcomeAdd this label to feature request issues so they are separated out from bug reporting issuestype:featureNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions