fix asr ut failures

[yao-matrix]

4 tests failed with AttributeError: 'AudioDecoder' object has no attribute 'copy':

pytest -rA tests/pipelines/test_pipelines_automatic_speech_recognition.py::AutomaticSpeechRecognitionPipelineTests::test_speculative_decoding_whisper_non_distil pytest -rA tests/test_pipeline_mixin.py::AutomaticSpeechRecognitionPipelineTests::test_speculative_decoding_whisper_non_distil pytest -rA tests/pipelines/test_pipelines_automatic_speech_recognition.py::AutomaticSpeechRecognitionPipelineTests::test_whisper_prompted pytest -rA tests/test_pipeline_mixin.py::AutomaticSpeechRecognitionPipelineTests::test_whisper_prompted

after fixing, they all passed

@SunMarc @ydshieh , pls help review, thx very much.

[yao-matrix]

In multi-device cases(like put 2 devices to run):
in current implementation, in assistant decoding case, assistant model will reuse main model's SuppressTokensLogitsProcessor, which place the suppress_tokens in the same device as input_tensor (which is device 0). assistant model will ingest encoder_outputs of the main model and do the decoder(in whisper case), while encoder_outputs may in device 1 but main model's suppress_tokens which is main model's is in device 0, so lead to RuntimeError:

RuntimeError: Expected all tensors to be on the same device, but got test_elements is on xpu:0, different from other tensors on xpu:1 (when checking argument in method wrapper_XPU_isin_Tensor_Tensor)

So based on current implementation(that assistant model shares main model's SuppressTokensLogitsProcessor), I move suppress_tokens to scores.device while doing isin.

[SunMarc]

Thanks for fixing ! cc @eustlb for final checks

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.