Skip to content

[CI] Fix red CI and ERROR failed should show #25995

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Sep 5, 2023
Merged

[CI] Fix red CI and ERROR failed should show #25995

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d46fd41
start with error too
ArthurZucker Sep 5, 2023
44c3814
fix ?
ArthurZucker Sep 5, 2023
8f0d437
start with nit
ArthurZucker Sep 5, 2023
517fe4c
one more path
ArthurZucker Sep 5, 2023
95c2306
use `job_name`
ArthurZucker Sep 5, 2023
3efe3e9
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker Sep 5, 2023
4baddad
mark pipeline test as slow
ArthurZucker Sep 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .circleci/create_circleci_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,16 +226,16 @@ def to_dict(self):
test_command += " || true"
steps.append({"run": {"name": "Run tests", "command": test_command}})

check_test_command = f'if [ -s reports/tests_{self.name}/failures_short.txt ]; '
check_test_command = f'if [ -s reports/{self.job_name}/failures_short.txt ]; '
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test_ is added to self.name in self.job_name based on a rule. self.jon_name is the one that is used when these files are created.

check_test_command += 'then echo "Some test failed!"; echo ""; '
check_test_command += f'cat reports/tests_{self.name}/failures_short.txt; '
check_test_command += f'cat reports/{self.job_name}/failures_short.txt; '
check_test_command += 'echo ""; echo ""; '

py_command = f'import os; fp = open("reports/tests_{self.name}/summary_short.txt"); failed = os.linesep.join([x for x in fp.read().split(os.linesep) if x.startswith("FAILED ")]); fp.close(); fp = open("summary_short.txt", "w"); fp.write(failed); fp.close()'
py_command = f'import os; fp = open("reports/{self.job_name}/summary_short.txt"); failed = os.linesep.join([x for x in fp.read().split(os.linesep) if x.startswith("FAILED ", "ERROR ")]); fp.close(); fp = open("summary_short.txt", "w"); fp.write(failed); fp.close()'
check_test_command += f"$(python3 -c '{py_command}'); "

check_test_command += f'cat summary_short.txt; echo ""; exit -1; '
check_test_command += f'elif [ -s reports/tests_{self.name}/stats.txt ]; then echo "All tests pass!"; '
check_test_command += f'elif [ -s reports/{self.job_name}/stats.txt ]; then echo "All tests pass!"; '

# return code `124` means the previous (pytest run) step is timeout
if self.name == "pr_documentation_tests":
Expand Down
1 change: 1 addition & 0 deletions tests/pipelines/test_pipelines_automatic_speech_recognition.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,7 @@ def test_torch_large(self):
self.assertEqual(output, {"text": "A MAN SAID TO THE UNIVERSE SIR I EXIST"})

@require_torch
@slow
Copy link
Contributor

@sanchit-gandhi sanchit-gandhi Sep 6, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this test is not related to the commit that was merged yesterday, which didn't touch this test: 8d51801

This probably failed due to using the whisper-tiny model in the test taking abnormally long, since we've not seen this failure before

Went through all the ASR pipeline tests and they're all correctly marked as slow / not slow now 👍

def test_return_timestamps_in_preprocess(self):
pipe = pipeline(
task="automatic-speech-recognition",
Expand Down