Document and validate typical_p in generation #19128
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
src/transformers/generation_utils.py
Outdated
There was a problem hiding this comment.
This is a problem for several other options, such as top_k or length_penalty, which have no effect in some generation modes.
Despite it being a correct check, we are assessing how to best handle it. We would like to avoid having an if/else branch for each input variable, as it would make generate even longer and less readable. Since it is harmless to have typical_p set with do_sample=False and we don't want to promote further checks of this kind, I'm going to ask to remove it for now.
There was a problem hiding this comment.
OK, I do hope that the do_sample check can happen in the future because receiving typical_p but doing greedy or non-sampled beam decoding is, in my view, silently failing.
Removed this and committed requested changes ✅
mapmeld
force-pushed
the
validate_genparams
branch
from
c58f6f7 to
189b228
Compare
gante
left a comment
gante
left a comment
There was a problem hiding this comment.
Thank you for the changes <3
3 participants
What does this PR do?
Throws a
ValueErrorwhentypical_pargument is provided to text-generation, but its value ordo_sample=Falseprevent typical decoding from happening as intended. Adds a line documenting typical decoding.Most arguments to generate were previously covered in #18261 , but not
typical_p.