build: Selectively run ember tests #3571
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
size-limit report
|
HazAT
approved these changes
May 19, 2021
AbhiPrasad
force-pushed
the
abhi/skip-ember-tests
branch
from
0b786f0 to
6f0b9da
Compare
lobsterkatie
added a commit
that referenced
this pull request
Dec 1, 2021
Given that ember tests take longer to run than most other tests (and have been known to fail for reasons having nothing to do with whatever PR we're working on), in #3571 we stopped running them locally and on PRs*, instead only running them when merging to `master` or releasing. *Provided the PR didn't touch any of the `@sentry/ember` code As a result, we didn't catch a currently-failing ember test until later than we would have liked. To prevent that from happening again, this restores the running of ember tests (both locally and on PRs), but restricts which tests run in each circumstance, in order to prevent this change from slowing down the overall repo test suite more than necessary. Specifically: - Locally, we run tests only against the current version of ember. - In the context of a PR, we run tests against the current version, the LTS version, and a version using `embroider` (which is an optional ember compiler many people use). - When merging to `master` or creating a release, for greatest safety, we test against current, LTS, embroider, beta, and "classic" (a legacy ember version). Key changes: - Up until now, the logic controlling which set of ember tests to run has been split between a script (`run_tests.js`, which differentiated local runs from CI runs), and the GHA workflow config (`build.yml`, which differentiated PRs from merges into master and creation of releases). That work is now consolidated into the ember-try config (since ember-try is the runner which needs to know which tests to run in the first place!). - As a result of the above change, the method for forcing tests for all versions of ember to run needed to be modified. That option is now handled by a new script, `run-CI-tests.js`, which simply mimics the the third situation above by setting environment variables, thereby tricking ember-try into thinking it needs to run all tests.
AbhiPrasad
pushed a commit
that referenced
this pull request
Dec 1, 2021
Given that ember tests take longer to run than most other tests (and have been known to fail for reasons having nothing to do with whatever PR we're working on), in #3571 we stopped running them locally and on PRs*, instead only running them when merging to `master` or releasing. *Provided the PR didn't touch any of the `@sentry/ember` code As a result, we didn't catch a currently-failing ember test until later than we would have liked. To prevent that from happening again, this restores the running of ember tests (both locally and on PRs), but restricts which tests run in each circumstance, in order to prevent this change from slowing down the overall repo test suite more than necessary. Specifically: - Locally, we run tests only against the current version of ember. - In the context of a PR, we run tests against the current version, the LTS version, and a version using `embroider` (which is an optional ember compiler many people use). - When merging to `master` or creating a release, for greatest safety, we test against current, LTS, embroider, beta, and "classic" (a legacy ember version). Key changes: - Up until now, the logic controlling which set of ember tests to run has been split between a script (`run_tests.js`, which differentiated local runs from CI runs), and the GHA workflow config (`build.yml`, which differentiated PRs from merges into master and creation of releases). That work is now consolidated into the ember-try config (since ember-try is the runner which needs to know which tests to run in the first place!). - As a result of the above change, the method for forcing tests for all versions of ember to run needed to be modified. That option is now handled by a new script, `run-CI-tests.js`, which simply mimics the the third situation above by setting environment variables, thereby tricking ember-try into thinking it needs to run all tests.
onurtemizkan
pushed a commit
that referenced
this pull request
Dec 19, 2021
Given that ember tests take longer to run than most other tests (and have been known to fail for reasons having nothing to do with whatever PR we're working on), in #3571 we stopped running them locally and on PRs*, instead only running them when merging to `master` or releasing. *Provided the PR didn't touch any of the `@sentry/ember` code As a result, we didn't catch a currently-failing ember test until later than we would have liked. To prevent that from happening again, this restores the running of ember tests (both locally and on PRs), but restricts which tests run in each circumstance, in order to prevent this change from slowing down the overall repo test suite more than necessary. Specifically: - Locally, we run tests only against the current version of ember. - In the context of a PR, we run tests against the current version, the LTS version, and a version using `embroider` (which is an optional ember compiler many people use). - When merging to `master` or creating a release, for greatest safety, we test against current, LTS, embroider, beta, and "classic" (a legacy ember version). Key changes: - Up until now, the logic controlling which set of ember tests to run has been split between a script (`run_tests.js`, which differentiated local runs from CI runs), and the GHA workflow config (`build.yml`, which differentiated PRs from merges into master and creation of releases). That work is now consolidated into the ember-try config (since ember-try is the runner which needs to know which tests to run in the first place!). - As a result of the above change, the method for forcing tests for all versions of ember to run needed to be modified. That option is now handled by a new script, `run-CI-tests.js`, which simply mimics the the third situation above by setting environment variables, thereby tricking ember-try into thinking it needs to run all tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Only run ember tests if packages/ember or it's dependents have changed.
Ran
yarn init -y && yarn add @sentry/ember && npm ls --depth 4 --parseable | grep "@sentry/"to get dependentsBefore submitting a pull request, please take a look at our
Contributing guidelines and verify:
yarn lint) & (yarn test).