Skip to content

Conversation

@vadorovsky
Copy link
Member

@vadorovsky vadorovsky commented Oct 15, 2021

Before this change, we assumed the worst case scenario about the number
of processes which can be schedules - the upper configurable limit for
x86_64 - which in theory is 4194304.

Allocating such a map size resulted in huge memory usage by lockc BPF
programs, over 500 MB, which is unacceptable for nodes with <=2 GB RAM
(which is usual both in IoT/ARM world and public clouds).

This change reads the currently configured limit by reading the
kernel.pid_max sysctl. On the most of systems it's configured to 32768.
That's way less than the maximum worst case scenario.

Memory usage before this change:

lockc-control-plane-0:~ # free -m
              total        used        free      shared  buff/cache   available
Mem:           1968         811         689           8         467        1011
Swap:             0           0           0
lockc-control-plane-0:~ # systemctl start lockcd
lockc-control-plane-0:~ # free -m
              total        used        free      shared  buff/cache   available
Mem:           1968        1420         154           8         393         339
Swap:             0           0           0

After this change:

lockc-control-plane-0:~ # free -m
              total        used        free      shared  buff/cache   available
Mem:           1968         793         220           3         955        1037
Swap:             0           0           0
lockc-control-plane-0:~ # systemctl start lockcd
lockc-control-plane-0:~ # free -m
              total        used        free      shared  buff/cache   available
Mem:           1968         803         210           3         955        1028
Swap:             0           0

Fixes: #82
Signed-off-by: Michal Rostecki [email protected]

Copy link
Collaborator

@mjura mjura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we should make PID_MAX as configuration option or explain how to tune system for it. On IoT devices there will be less than 2GB of memory.

Great job, thank you for fixing it

@vadorovsky
Copy link
Member Author

It's tuned through a sysctl - /proc/sys/kernel/pid_max. We could document that in case if someone has less than 1 GB RAM, I will push a change here.

Before this change, we assumed the worst case scenario about the number
of processes which can be schedules - the upper configurable limit for
x86_64 - which in theory is 4194304.

Allocating such a map size resulted in huge memory usage by lockc BPF
programs, over 500 MB, which is unacceptable for nodes with <=2 GB RAM
(which is usual both in IoT/ARM world and public clouds).

This change reads the currently configured limit by reading the
kernel.pid_max sysctl. On the most of systems it's configured to 32768.
That's way less than the maximum worst case scenario.

Memory usage before this change:

  lockc-control-plane-0:~ # free -m
                total        used        free      shared  buff/cache   available
  Mem:           1968         811         689           8         467        1011
  Swap:             0           0           0
  lockc-control-plane-0:~ # systemctl start lockcd
  lockc-control-plane-0:~ # free -m
                total        used        free      shared  buff/cache   available
  Mem:           1968        1420         154           8         393         339
  Swap:             0           0           0

After this change:

  lockc-control-plane-0:~ # free -m
                total        used        free      shared  buff/cache   available
  Mem:           1968         793         220           3         955        1037
  Swap:             0           0           0
  lockc-control-plane-0:~ # systemctl start lockcd
  lockc-control-plane-0:~ # free -m
                total        used        free      shared  buff/cache   available
  Mem:           1968         803         210           3         955        1028
  Swap:             0           0

Fixes: lockc-project#82
Signed-off-by: Michal Rostecki <[email protected]>
@vadorovsky
Copy link
Member Author

@mjura done, PTAL

Copy link
Collaborator

@mjura mjura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you Michal

@mjura mjura merged commit 2a2260f into lockc-project:main Oct 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

libbpf: map 'processes': failed to create: Cannot allocate memory(-12)

2 participants