Linux ARM64, align to 64k page size

[grantbeattie]

I'm seeing this when starting a node.js app using sharp 0.25.3 and the pre-built arm64 sharp/libvips on an Amazon c6g.large EC2 instance:

Error:
Something went wrong installing the "sharp" module

libglib-2.0.so.0: ELF load command alignment not page-aligned

- Remove the "node_modules/sharp" directory then run
  "npm install --ignore-scripts=false --verbose" and look for errors
- Consult the installation documentation at https://sharp.pixelplumbing.com/install
- Search for this error at https://github.com/lovell/sharp/issues

    at Object.<anonymous> (/app/node_modules/sharp/lib/constructor.js:34:9)
    at Module._compile (internal/modules/cjs/loader.js:1138:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
    at Module.load (internal/modules/cjs/loader.js:986:32)
    at Function.Module._load (internal/modules/cjs/loader.js:873:16)
    at Function.wrappedLoad [as _load] (/app/node_modules/newrelic/lib/shimmer.js:481:24)
    at Module.require (internal/modules/cjs/loader.js:1026:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/app/node_modules/sharp/lib/index.js:3:15)
    at Module._compile (internal/modules/cjs/loader.js:1138:30)

The app is running in Docker using Ubuntu 20.04 and node.js 12.18.

I haven't had a chance to dig into it any further yet, but when I do I'll update this issue.

[lovell]

The prebuilt binaries assume a standard Linux page size of 4096. These binaries are tested on Packet ARM64 instances e.g. https://travis-ci.org/github/lovell/sharp/jobs/698495901

What page size do the AWS Graviton2 instances use? Running the getconf PAGESIZE command should reveal this.

[grantbeattie]

$ getconf PAGESIZE
65536

This is also the case on a1.* instances (1st gen Graviton).

[lovell]

Thanks for confirming. Sadly page size is determined at Linux kernel compile time and is not configurable at runtime. Are you using the default/latest Amazon Linux AMI?

For now you'll either need to find and use an alternative AMI with the default 4k page size, or globally install libvips on the machine before installing sharp (but losing the ability to use prebuilt binaries).

If 64k page sizes are a "standard" on ARM64 then we'll probably want to start aligning to this by adding something like -z max-page-size=65536 to LDFLAGS in the linux-arm64v8 container. Happy to accept a PR if you're able.

[grantbeattie]

Thanks for confirming. Sadly page size is determined at Linux kernel compile time and is not configurable at runtime. Are you using the default/latest Amazon Linux AMI?

No, the host OS AMI is CentOS 7 altarch, with the default kernel.

If 64k page sizes are a "standard" on ARM64 then we'll probably want to start aligning to this by adding something like -z max-page-size=65536 to LDFLAGS in the linux-arm64v8 container. Happy to accept a PR if you're able.

Yeah, that seems to be somewhat common, from what I can tell. I'll see what I can do. Thanks :)

[grantbeattie]

Recent versions of ld on aarch64 align loadable segments to 64KB page size. The issue seems to be patchelf is adding a new loadable segment with 4KB alignment.

Before patchelf is run, the loadable segments are all aligned at 64KB:

$ readelf -l /target/lib/libglib*.so

Elf file type is DYN (Shared object file)
Entry point 0x1c2c0
There are 7 program headers, starting at offset 64

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000163e98 0x0000000000163e98  R E    0x10000
  LOAD           0x0000000000164938 0x0000000000174938 0x0000000000174938
                 0x00000000000013e8 0x0000000000001f48  RW     0x10000
  DYNAMIC        0x0000000000164c50 0x0000000000174c50 0x0000000000174c50
                 0x00000000000001f0 0x00000000000001f0  RW     0x8
  NOTE           0x00000000000001c8 0x00000000000001c8 0x00000000000001c8
                 0x0000000000000024 0x0000000000000024  R      0x4
  GNU_EH_FRAME   0x000000000013f3b0 0x000000000013f3b0 0x000000000013f3b0
                 0x000000000000434c 0x000000000000434c  R      0x4
  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000  RW     0x10
  GNU_RELRO      0x0000000000164938 0x0000000000174938 0x0000000000174938
                 0x00000000000006c8 0x00000000000006c8  R      0x1

 Section to Segment mapping:
  Segment Sections...
   00     .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame
   01     .init_array .fini_array .data.rel.ro .dynamic .got .got.plt .data .bss
   02     .dynamic
   03     .note.gnu.build-id
   04     .eh_frame_hdr
   05
   06     .init_array .fini_array .data.rel.ro .dynamic .got

$ find /target/lib -type f -name "*.so*" -exec sh -c "patchelf --set-rpath '\$ORIGIN' --force-rpath {}" \;

$ readelf -l /target/lib/libglib*.so
Elf file type is DYN (Shared object file)
Entry point 0x1c2c0
There are 8 program headers, starting at offset 64

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000163e98 0x0000000000163e98  R E    0x10000
  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000  RW     0x10
  NOTE           0x00000000000001c8 0x00000000000001c8 0x00000000000001c8
                 0x0000000000000024 0x0000000000000024  R      0x4
  GNU_EH_FRAME   0x000000000013f3b0 0x000000000013f3b0 0x000000000013f3b0
                 0x000000000000434c 0x000000000000434c  R      0x4
  LOAD           0x0000000000164938 0x0000000000174938 0x0000000000174938
                 0x00000000000013e8 0x0000000000001f48  RW     0x10000
  GNU_RELRO      0x0000000000164938 0x0000000000174938 0x0000000000174938
                 0x00000000000006c8 0x00000000000006c8  R      0x1
  DYNAMIC        0x0000000000170000 0x0000000000180000 0x0000000000180000
                 0x0000000000000200 0x0000000000000200  RW     0x8
  LOAD           0x0000000000170000 0x0000000000180000 0x0000000000180000
                 0x000000000000c6e8 0x000000000000c6e8  RW     0x1000

 Section to Segment mapping:
  Segment Sections...
   00     .dynsym .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame
   01
   02
   03     .eh_frame_hdr
   04     .init_array .fini_array .data.rel.ro .got .got.plt .data .bss
   05     .init_array .fini_array .data.rel.ro .got
   06     .dynamic
   07     .dynamic .dynstr .gnu.hash .note.gnu.build-id

I think the sane thing to do for the moment is pass --page-size 65536 via the arm64v8 Docker container's environment. I'll PR it.

[lovell]

Great detective work, thank you. It looks like patchelf has a, er, patch to fix this properly - see NixOS/patchelf#216

[kleisauke]

PR #44 could also fix this, as it removes the need for patchelf.

[lovell]

@kleisauke Thank you, I had forgotten that your magic had managed to remove the need for patchelf.

@grantbeattie I expect we'll merge #44 in time for libvips v8.10.0, so the time-honoured approach of waiting will also solve this problem.

[grantbeattie]

Awesome, I'll use my existing patch for now and shouldn't need it in the future 🎉

[grantbeattie]

In the mean time, I've published a fixed build for arm64 here:

https://github.com/grantbeattie/sharp-libvips/releases/tag/v8.9.1-64k