Skip to content

Conversation

john-michaelburke
Copy link
Collaborator

@john-michaelburke john-michaelburke commented Mar 4, 2021

CPP-33

Added a backend memory benchmark which processes a 1min sbp file reading the memory usage of the process frequently then assert it is within a few bounds:

  • num_mem_readings >= min_allowed_mem_readings
  • mean + std - max_allowed <= max_allowed * threshold_rate
  • std <= max_allowed * max_allowed_std_dev_rate
  • mean - std >= absolute_mean

CPP-70

Merged

  • src/main/benches/frontend_bench.py
  • console_backend/benches/benchmarks.py
  • utils/disk_usage_benchmarks.py

Into

  • utils/bench_runner.py (Not sure if the length of this file warrants using some OOP yet but might be worth revisiting the idea)

All data was moved from scattered locations to console_backend/tests/data/

Other modification

  • The frontend benchmarks do not use cargo make in practice since they are copied to and run on self hosted runners (SHR). I added commands to the Makefile.toml in order to test they work locally similar to how they are run on the SHRs.

@john-michaelburke john-michaelburke force-pushed the johnmichael-burke/backend_mem_bench branch from ea3b172 to 25d325a Compare March 4, 2021 19:13
@john-michaelburke john-michaelburke changed the title Johnmichael burke/backend mem bench [CPP-33][CPP-70]Backend Memory Benchmark, Refactored Python Bench Runners Mar 4, 2021
@john-michaelburke john-michaelburke force-pushed the johnmichael-burke/backend_mem_bench branch 7 times, most recently from 8593c15 to 42d6504 Compare March 4, 2021 22:40
@john-michaelburke john-michaelburke force-pushed the johnmichael-burke/backend_mem_bench branch from 42d6504 to 74b02e4 Compare March 4, 2021 23:24
@john-michaelburke john-michaelburke marked this pull request as ready for review March 4, 2021 23:58
@john-michaelburke john-michaelburke requested a review from a team March 4, 2021 23:58
break;
}
}
let mems = ArrayView::from_shape(mem_readings_kb.len(), &mem_readings_kb).unwrap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we push this to a helper function?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Wasn't sure if you meant the process update step or the vec->arrayview so I abstracted both to helper functions.

@john-michaelburke john-michaelburke force-pushed the johnmichael-burke/backend_mem_bench branch from 6c4a949 to 04483df Compare March 5, 2021 22:02
@john-michaelburke john-michaelburke merged commit 72e0ff4 into main Mar 5, 2021
@john-michaelburke john-michaelburke deleted the johnmichael-burke/backend_mem_bench branch March 5, 2021 22:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants