Skip to content

Conversation

RaisinTen
Copy link
Member

Fixes: #37222

@nodejs-github-bot nodejs-github-bot added the timers Issues and PRs related to the timers subsystem / setImmediate, setInterval, setTimeout. label Feb 4, 2021
@nodejs-github-bot
Copy link
Collaborator

@RaisinTen
Copy link
Member Author

RaisinTen commented Feb 4, 2021

Benchmark CI for timers: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/930/
Using SafeArrayIterator has resulted in a significant perf regression:

Benchmark results
19:56:10 timers/immediate.jstype='breadth4' n=5000000                         *     -4.65 %       ±3.97%  ±5.30%  ±6.92%
19:56:10 timers/immediate.jstype='depth1' n=5000000                           *      1.93 %       ±1.86%  ±2.49%  ±3.26%
19:56:10 timers/timers-breadth-args.jsn=1000000                             ***    -24.02 %       ±5.82%  ±7.78% ±10.19%

@nodejs-github-bot
Copy link
Collaborator

@RaisinTen
Copy link
Member Author

RaisinTen commented Feb 4, 2021

New Benchmark CI (using ReflectApply):
https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/932/

Benchmark results
                                                            confidence improvement accuracy (*)    (**)   (***)
timers/immediate.jstype='breadth1' n=5000000                                0.50 %       ±3.18%  ±4.23%  ±5.51%
timers/immediate.jstype='breadth4' n=5000000                               -1.48 %       ±4.55%  ±6.06%  ±7.89%
timers/immediate.jstype='breadth' n=5000000                                -1.31 %       ±3.90%  ±5.18%  ±6.75%
timers/immediate.jstype='clear' n=5000000                                   1.78 %       ±3.34%  ±4.45%  ±5.80%
timers/immediate.jstype='depth1' n=5000000                                  1.06 %       ±2.53%  ±3.37%  ±4.40%
timers/immediate.jstype='depth' n=5000000                                  -0.27 %       ±2.51%  ±3.35%  ±4.37%
timers/set-immediate-breadth-args.jsn=5000000                              -2.31 %       ±3.44%  ±4.57%  ±5.95%
timers/set-immediate-breadth.jsn=10000000                                  -1.88 %       ±4.10%  ±5.46%  ±7.11%
timers/set-immediate-depth-args.jsn=5000000                                 0.42 %       ±1.57%  ±2.09%  ±2.72%
timers/timers-breadth-args.jsn=1000000                                      1.50 %       ±6.38%  ±8.48% ±11.04%
timers/timers-breadth.jsn=5000000                                           1.18 %       ±3.17%  ±4.22%  ±5.50%
timers/timers-cancel-pooled.jsn=5000000                                     8.24 %      ±11.36% ±15.12% ±19.69%
timers/timers-cancel-unpooled.jsdirection='end' n=1000000                  -1.42 %       ±8.14% ±10.83% ±14.10%
timers/timers-cancel-unpooled.jsdirection='start' n=1000000                -2.38 %       ±5.38%  ±7.16%  ±9.32%
timers/timers-depth.jsn=1000                                                0.48 %       ±0.78%  ±1.04%  ±1.35%
timers/timers-insert-pooled.jsn=5000000                                     0.65 %       ±3.05%  ±4.06%  ±5.29%
timers/timers-insert-unpooled.jsdirection='end' n=1000000                  -2.16 %       ±3.85%  ±5.12%  ±6.66%
timers/timers-insert-unpooled.jsdirection='start' n=1000000                 1.73 %       ±5.02%  ±6.68%  ±8.70%
timers/timers-timeout-nexttick.jsn=50000                                   -0.24 %       ±2.17%  ±2.89%  ±3.76%
timers/timers-timeout-nexttick.jsn=5000000                                  0.25 %       ±6.66%  ±8.87% ±11.55%
timers/timers-timeout-pooled.jsn=10000000                                  -0.62 %       ±6.87%  ±9.15% ±11.91%
timers/timers-timeout-unpooled.jsn=1000000                           *    -12.70 %       ±9.87% ±13.19% ±17.28%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 22 comparisons, you can thus
expect the following amount of false-positive results:
  1.10 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.22 false positives, when considering a   1% risk acceptance (**, ***),
  0.02 false positives, when considering a 0.1% risk acceptance (***)

@aduh95
Copy link
Contributor

aduh95 commented Feb 4, 2021

Benchmark results are a bit weird, I kicked off a new benchmark CI to see if it's just a flakiness.
https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/934/

@Fishrock123
Copy link
Contributor

I think Reflect probably causes more overhead than just doing a more manual check.

Copy link
Contributor

@aduh95 aduh95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark CI results look good 🎉

                                                            confidence improvement accuracy (*)    (**)   (***)
timers/immediate.jstype='breadth1' n=5000000                                1.00 %       ±4.06%  ±5.41%  ±7.04%
timers/immediate.jstype='breadth4' n=5000000                               -1.00 %       ±4.15%  ±5.53%  ±7.21%
timers/immediate.jstype='breadth' n=5000000                                 3.12 %       ±4.05%  ±5.38%  ±7.01%
timers/immediate.jstype='clear' n=5000000                                  -0.04 %       ±2.15%  ±2.86%  ±3.72%
timers/immediate.jstype='depth1' n=5000000                                 -0.03 %       ±1.74%  ±2.31%  ±3.02%
timers/immediate.jstype='depth' n=5000000                                   0.75 %       ±2.22%  ±2.95%  ±3.85%
timers/set-immediate-breadth-args.jsn=5000000                               0.19 %       ±4.62%  ±6.16%  ±8.03%
timers/set-immediate-breadth.jsn=10000000                            *      4.00 %       ±3.89%  ±5.18%  ±6.74%
timers/set-immediate-depth-args.jsn=5000000                          *      2.03 %       ±1.80%  ±2.40%  ±3.14%
timers/timers-breadth-args.jsn=1000000                                     -5.37 %       ±5.51%  ±7.33%  ±9.54%
timers/timers-breadth.jsn=5000000                                           0.81 %       ±3.38%  ±4.50%  ±5.86%
timers/timers-cancel-pooled.jsn=5000000                                     1.91 %      ±10.69% ±14.22% ±18.51%
timers/timers-cancel-unpooled.jsdirection='end' n=1000000                  -0.92 %       ±5.95%  ±7.91% ±10.30%
timers/timers-cancel-unpooled.jsdirection='start' n=1000000                 0.18 %       ±6.67%  ±8.89% ±11.58%
timers/timers-depth.jsn=1000                                               -0.13 %       ±0.65%  ±0.87%  ±1.14%
timers/timers-insert-pooled.jsn=5000000                                    -0.54 %       ±2.80%  ±3.74%  ±4.88%
timers/timers-insert-unpooled.jsdirection='end' n=1000000                   1.44 %       ±4.47%  ±5.95%  ±7.74%
timers/timers-insert-unpooled.jsdirection='start' n=1000000                 0.55 %       ±5.13%  ±6.82%  ±8.88%
timers/timers-timeout-nexttick.jsn=50000                                    0.14 %       ±2.20%  ±2.93%  ±3.83%
timers/timers-timeout-nexttick.jsn=5000000                                  0.13 %       ±3.60%  ±4.80%  ±6.25%
timers/timers-timeout-pooled.jsn=10000000                            *      8.82 %       ±8.47% ±11.29% ±14.72%
timers/timers-timeout-unpooled.jsn=1000000                                  0.62 %      ±13.21% ±17.58% ±22.88%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 22 comparisons, you can thus
expect the following amount of false-positive results:
  1.10 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.22 false positives, when considering a   1% risk acceptance (**, ***),
  0.02 false positives, when considering a 0.1% risk acceptance (***)

@aduh95 aduh95 added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Feb 4, 2021
@nodejs-github-bot
Copy link
Collaborator

@benjamingr
Copy link
Member

Ping @nodejs/timers

Fixes: nodejs#37222

PR-URL: nodejs#37223
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
@Trott Trott force-pushed the timers/unsafe-array-iteration branch from 89f304f to c0e66e3 Compare February 6, 2021 13:18
@Trott
Copy link
Member

Trott commented Feb 6, 2021

Landed in c0e66e3

@Trott Trott merged commit c0e66e3 into nodejs:master Feb 6, 2021
@RaisinTen RaisinTen deleted the timers/unsafe-array-iteration branch February 6, 2021 13:36
danielleadams pushed a commit that referenced this pull request Feb 16, 2021
Fixes: #37222

PR-URL: #37223
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
This was referenced Feb 16, 2021
targos pushed a commit that referenced this pull request May 27, 2021
Fixes: #37222

PR-URL: #37223
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
targos pushed a commit that referenced this pull request May 30, 2021
Fixes: #37222

PR-URL: #37223
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
targos pushed a commit that referenced this pull request Jun 5, 2021
Fixes: #37222

PR-URL: #37223
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
targos pushed a commit that referenced this pull request Jun 11, 2021
Fixes: #37222

PR-URL: #37223
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs that have at least one approval, no pending requests for changes, and a CI started. timers Issues and PRs related to the timers subsystem / setImmediate, setInterval, setTimeout.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

investigate flaky test-repl-history-navigation

7 participants