Skip to content

Conversation

vitaliylag
Copy link

@vitaliylag vitaliylag commented Jul 3, 2025

Description

This pull request adds compilation of Python, GCC and binutils (when needed).

Added time for compilation (6 cores @ ~3.2+ GHz)

Added time:

  • Python 3.9.23: 43 | 76 sec (added to compile older versions of Node.js)
  • Python 3.13.5: 41 | 86 sec (added because it may become needed in the future)
  • GCC 15.1.0: 450 | 852 sec

Comments:

  • First number is the time of recompilation (accelerated with ccache)
  • Second number is the time of the first compilation
  • All numbers include downloading time
  • The x86 recipe theoretically should be slightly slower, but I didn't notice a difference

Other tests:

  • GCC 12.2.0: 289 | 637 sec
  • binutils-2.44: 37 | 75 sec (binutils 2.35+ is required by Node.js v24.3.0)
  • installPrerequisites: 137 sec

Reduced time of compilation (6 cores @ ~3.2+ GHz)

  • Node.js v24.3.0:
    • First time:
      • 58 minutes (GCC 15.1.0)
      • Instead of 1 hour 55 minutes (GCC 12.1.1)

Info

  • Recipes are changed a lot (reviewing whole files is probably faster than comparing changes).
  • 3 changed recipes are almost copies. To review the code see recipes/x64-glibc-217 and then run bin/copy-recipes.sh. This pull request changes 31 files but actually recipes/x64-glibc-217 contains only 9 files (~20 lines per file).
    • Shared Docker image is used to reduce required space and time. Probably these recipes are about 2 times faster than previous ones.
  • FROM --platform=linux/386 centos:7 is used for x86 because V8 does not support x64→x86 compilation anymore (because emulator or scripts for snapshot creation are disabled for this pair). This image is not shared.
  • Requirements:
    • Docker 23+ (2023)
    • Or Docker 18+ (2019) (execute "export DOCKER_BUILDKIT=1" before start)
    • 16 GB of RAM is recommended

Tested on

My environment:

  • Intel (x64)
  • WSL 2
  • Debian 12
  • Docker 28.2.2

Test results

@rvagg
Copy link
Member

rvagg commented Jul 4, 2025

Sorry, could you give me a TL;DR for this, why this PR exists and why someone (me) should invest some time figuring this out and reviewing it? I don't mean to be rude, there's just a lot going on there, it's going to take an investment to review, and I'm not sure what problem we're solving.

@vitaliylag
Copy link
Author

These 3 recipes are broken since Node.js v23, this pull request makes them work again.

@rvagg
Copy link
Member

rvagg commented Jul 4, 2025

👌 ok, that's the context I needed. Good work I guess. I'll try and make time to have a look, but there may be others watching this that can pitch in.

The summary is that building some of the toolchain gets us the builds back?

@vitaliylag
Copy link
Author

vitaliylag commented Jul 4, 2025

Yes.

I am running tests now, it works good at the time. I will post the result of the tests as a table later.

@vitaliylag
Copy link
Author

vitaliylag commented Jul 7, 2025

All tests are done

  • All tests have been reexecuted after the last commit (9776ab2)
  • Only versions allowed by should-build.sh are tested
  • ccache was enabled

Results

Click to expand
Recipe Version Status Time
x64-glibc-217 v24.3.0 OK 202 sec
x64-glibc-217 v23.11.1 OK 194 sec
x64-glibc-217 v22.17.0 OK 191 sec
x64-glibc-217 v21.7.3 OK 162 sec
x64-glibc-217 v20.19.3 OK 154 sec
x64-glibc-217 v20.16.0 OK 155 sec
x64-glibc-217 v19.9.0 OK 161 sec
x64-glibc-217 v18.20.8 OK 169 sec
x64-glibc-217 v16.20.2 OK 126 sec
x64-glibc-217 v15.14.0 OK 111 sec
x64-glibc-217 v14.21.3 OK 110 sec
x64-glibc-217 v14.14.0 OK 111 sec
x64-glibc-217 v13.14.0 OK 101 sec
x64-glibc-217 v12.13.0 OK 87 sec
x64-glibc-217 v11.15.0 OK 73 sec
x64-glibc-217 v10.24.1 OK 75 sec
x64-glibc-217 v9.11.2 OK 68 sec
x64-glibc-217 v8.17.0 OK 71 sec
x64-glibc-217 v7.10.1 OK 51 sec
x64-glibc-217 v6.17.1 OK 52 sec
Recipe Version Status Time
x64-glibc-217 v24.2.0 OK 197 sec
x64-glibc-217 v24.0.0 OK 200 sec
x64-glibc-217 v23.0.0 OK 200 sec
x64-glibc-217 v22.3.0 OK 181 sec
x64-glibc-217 v22.0.0 OK 168 sec
x64-glibc-217 v21.3.0 OK 6238 sec
x64-glibc-217 v21.0.0 OK 171 sec
x64-glibc-217 v20.0.0 OK 2359 sec
x64-glibc-217 v19.0.0 OK 2143 sec
x64-glibc-217 v18.0.0 OK 282 sec
x64-glibc-217 v16.0.0 OK 330 sec
x64-glibc-217 v15.0.0 OK 1284 sec
x64-glibc-217 v14.0.0 OK 1129 sec
x64-glibc-217 v13.4.0 OK 2120 sec
x64-glibc-217 v13.0.0 OK 1638 sec
x64-glibc-217 v12.0.0 OK 260 sec
x64-glibc-217 v11.0.0 OK 205 sec
x64-glibc-217 v10.0.0 OK 206 sec
x64-glibc-217 v9.0.0 OK 178 sec
x64-glibc-217 v8.0.0 OK 189 sec
x64-glibc-217 v7.0.0 OK 141 sec
x64-glibc-217 v6.2.0 OK 105 sec
Recipe Version Status Time
x64-pointer-compression v24.3.0 OK 466 sec
x64-pointer-compression v22.17.0 OK 181 sec
x64-pointer-compression v21.7.3 OK 157 sec
x64-pointer-compression v20.16.0 OK 153 sec
x64-pointer-compression v19.9.0 OK 187 sec
x64-pointer-compression v18.20.8 OK 188 sec
x64-pointer-compression v16.20.2 OK 119 sec
x64-pointer-compression v15.14.0 OK 109 sec
x64-pointer-compression v14.21.3 OK 108 sec
x64-pointer-compression v14.14.0 OK 114 sec
x64-pointer-compression v13.14.0 OK 102 sec
Recipe Version Status Time
x64-pointer-compression v24.2.0 OK 195 sec
x64-pointer-compression v22.3.0 OK 183 sec
x64-pointer-compression v22.0.0 OK 172 sec
x64-pointer-compression v21.3.0 OK 6315 sec
x64-pointer-compression v20.0.0 OK 2355 sec
x64-pointer-compression v19.0.0 OK 2165 sec
x64-pointer-compression v18.0.0 OK 1996 sec
x64-pointer-compression v16.0.0 OK 1543 sec
x64-pointer-compression v15.0.0 OK 1307 sec
x64-pointer-compression v14.0.0 OK 1135 sec
x64-pointer-compression v13.4.0 OK 2020 sec
Recipe Version Status Time
x86 v24.3.0 OK 196 sec
x86 v23.11.1 OK 181 sec
x86 v22.17.0 OK 178 sec
x86 v21.7.3 OK 157 sec
x86 v20.19.3 OK 149 sec
x86 v20.16.0 OK 146 sec
x86 v19.9.0 OK 154 sec
x86 v18.20.8 OK 151 sec
x86 v17.9.1 OK 141 sec
x86 v16.20.2 OK 122 sec
x86 v15.14.0 OK 106 sec
x86 v14.21.3 OK 110 sec
x86 v14.14.0 OK 104 sec
x86 v13.14.0 OK 103 sec
x86 v12.13.0 OK 82 sec
x86 v11.15.0 OK 74 sec
x86 v10.24.1 OK 75 sec
x86 v9.11.2 OK 64 sec
x86 v8.17.0 OK 64 sec
x86 v7.10.1 OK 52 sec
x86 v6.17.1 OK 53 sec
Recipe Version Status Time
x86 v24.2.0 OK 187 sec
x86 v24.0.0 OK 184 sec
x86 v23.0.0 OK 173 sec
x86 v22.3.0 OK 174 sec
x86 v21.3.0 OK 2882 sec
x86 v21.0.0 OK 167 sec
x86 v20.0.0 OK 164 sec
x86 v19.0.0 OK 160 sec
x86 v18.0.0 OK 153 sec
x86 v17.0.0 OK 148 sec
x86 v16.0.0 OK 126 sec
x86 v15.0.0 OK 115 sec
x86 v14.0.0 OK 109 sec
x86 v13.4.0 OK 108 sec
x86 v13.0.0 OK 106 sec
x86 v12.0.0 OK 85 sec
x86 v11.0.0 OK 74 sec
x86 v10.0.0 OK 77 sec
x86 v9.0.0 OK 68 sec
x86 v8.0.0 OK 66 sec
x86 v7.0.0 OK 55 sec
x86 v6.2.0 OK 51 sec

@rvagg rvagg requested a review from Copilot July 8, 2025 02:06
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Refactors CentOS 7 build recipes for x86, x64-pointer-compression, and x64-glibc-217 to add Python 3.9/3.13, GCC 15.1, and binutils builds, unify install scripts, and modernize Dockerfiles.

  • Introduces shared scripts (installPrerequisites, installFromSourceCode) and splits version/variation logic into run_versions.sh & run_other.sh
  • Refactors each run.sh for consistent environment setup and parameterized compilation
  • Updates Dockerfiles for multi-platform builds, caching, and modular dependency installation

Reviewed Changes

Copilot reviewed 30 out of 31 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
recipes/x86/should-build.sh Expanded Node.js version checks for x86
recipes/x86/run_other.sh Applies source patches and SSE2 disable logic for x86
recipes/x86/run_versions.sh Selects appropriate Python/GCC based on Node.js version
recipes/x86/run.sh Unified setup for compiling Node.js under x86
recipes/x86/files/installPrerequisites Extracted Yum repo fixes & base package installs
recipes/x86/files/installFromSourceCode Extracted generic source-build helper
recipes/x86/files/opt__gcc15__enable GCC 15 enable script
recipes/x86/Dockerfile Switched to linux/386, modularized installs
recipes/x64-pointer-compression/{,run.sh,Dockerfile} Parallel refactor for pointer-compression variant
recipes/x64-glibc-217/{,run.sh,Dockerfile} Parallel refactor for glibc-2.17 variant
Comments suppressed due to low confidence (3)

recipes/x86/should-build.sh:12

  • [nitpick] The comment mentions a GCC version range, but the test is matching Node.js versions v22.0–22.2. Update the comment to clearly describe which Node.js versions are excluded.
[[ ! "$fullversion" =~ ^v22\.[0-2]\. ]]                             # GCC version between 9.3 and 15.1 is required but not installed

recipes/x64-pointer-compression/should-build.sh:13

  • [nitpick] This regex excludes Node.js v22.2.x, but the comment refers to GCC versions. Clarify whether you're gating on Node.js or GCC versions in the comment.
[[ ! "$fullversion" =~ ^v22\.2\. ]]                                   # GCC version between 12.1  and 15.1 is required but not installed

recipes/x64-glibc-217/should-build.sh:13

  • [nitpick] The exclusion regex is for Node.js v22.2.x, but the comment describes a GCC version gap. Update the comment to match the actual Node.js filter.
[[ ! "$fullversion" =~ ^v22\.2\. ]]                                  # GCC version between 12.1  and 15.1 is required but not installed

Comment on lines 10 to 12
[ "$major" -ge 7 ] || ( [ "$major" -eq 6 ] && [ "$minor" -ge 2 ] ) # Node.js v6.1- cannot download required files due to broken links

[[ ! "$fullversion" =~ ^v22\.[0-2]\. ]] # GCC version between 9.3 and 15.1 is required but not installed
Copy link
Preview

Copilot AI Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is on its own line, so its exit status isn't combined with the other conditions. You should join all version checks with && (or use set -e) so the script only exits zero when all criteria pass.

Suggested change
[ "$major" -ge 7 ] || ( [ "$major" -eq 6 ] && [ "$minor" -ge 2 ] ) # Node.js v6.1- cannot download required files due to broken links
[[ ! "$fullversion" =~ ^v22\.[0-2]\. ]] # GCC version between 9.3 and 15.1 is required but not installed
[ "$major" -ge 7 ] && ( [ "$major" -eq 6 ] && [ "$minor" -ge 2 ] ) && [[ ! "$fullversion" =~ ^v22\.[0-2]\. ]] # Ensure all version checks pass

Copilot uses AI. Check for mistakes.

decode "$fullversion"

test "$major" -ge "14" && test "$major" -lt "23"
[ "$major" -ge 14 ] || ( [ "$major" -eq 13 ] && [ "$minor" -ge 4 ] ) # Pointer compression is supported since Node.js v13.4
Copy link
Preview

Copilot AI Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above: this standalone test isn’t AND-chained with the others, so only the last command’s result determines the script’s exit code. Combine the build filters with && to enforce all checks.

Suggested change
[ "$major" -ge 14 ] || ( [ "$major" -eq 13 ] && [ "$minor" -ge 4 ] ) # Pointer compression is supported since Node.js v13.4
[ "$major" -ge 14 ] && ( [ "$major" -eq 13 ] && [ "$minor" -ge 4 ] ) # Pointer compression is supported since Node.js v13.4

Copilot uses AI. Check for mistakes.

decode "$fullversion"

test "$major" -ge "18"
[ "$major" -ge 7 ] || ( [ "$major" -eq 6 ] && [ "$minor" -ge 2 ] ) # Node.js v6.1- cannot download required files due to broken links
Copy link
Preview

Copilot AI Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This version check is not AND-chained with the subsequent lines, so the overall exit code is only based on the final test. Use && or enable set -e at the top to combine all conditions.

Suggested change
[ "$major" -ge 7 ] || ( [ "$major" -eq 6 ] && [ "$minor" -ge 2 ] ) # Node.js v6.1- cannot download required files due to broken links
[ "$major" -ge 7 ] && ( [ "$major" -eq 6 ] && [ "$minor" -ge 2 ] ) # Node.js v6.1- cannot download required files due to broken links

Copilot uses AI. Check for mistakes.


# x86 does not support _mm_cvtsi128_si64 instruction so forbid it's usage and fallback to non-SSE solution
cd "$nodeDir/deps/v8/src"
find . -name '*.cc' -type f -print0 | xargs -0 sed -i -e 's/#ifdef __SSE2__/#if false/g'
Copy link
Preview

Copilot AI Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If find returns no files, xargs will still run sed once and may error. Add --no-run-if-empty to the xargs invocation to skip sed when there are no matches.

Suggested change
find . -name '*.cc' -type f -print0 | xargs -0 sed -i -e 's/#ifdef __SSE2__/#if false/g'
find . -name '*.cc' -type f -print0 | xargs -0 --no-run-if-empty sed -i -e 's/#ifdef __SSE2__/#if false/g'

Copilot uses AI. Check for mistakes.

@vitaliylag
Copy link
Author

My comments on the AI review:

  • should-build.sh: #!/bin/bash -e is used so it should be OK. "&&" is used inside brackets. Because it is last expression inside the brackets the error will not be ignored.
  • AI did not notice all recipes are copied by bin/copy-recipes.sh.

@vitaliylag
Copy link
Author

vitaliylag commented Jul 8, 2025

Test results after the last commit (b7475f7) (all OK)

Click to expand
Recipe Version Status Time
x64-glibc-217 v6.2.0 OK 59 sec
x64-glibc-217 v7.0.0 OK 60 sec
x64-glibc-217 v8.0.0 OK 71 sec
x64-glibc-217 v9.0.0 OK 69 sec
x64-glibc-217 v12.0.0 OK 89 sec
x64-glibc-217 v13.0.0 OK 100 sec
x64-glibc-217 v14.0.0 OK 112 sec
x64-glibc-217 v15.0.0 OK 116 sec
x64-glibc-217 v16.0.0 OK 173 sec
x64-glibc-217 v18.0.0 OK 173 sec
x64-glibc-217 v19.0.0 OK 157 sec
x64-glibc-217 v20.0.0 OK 159 sec
x64-glibc-217 v21.0.0 OK 167 sec
x64-glibc-217 v21.3.0 OK 164 sec
x64-glibc-217 v22.0.0 OK 180 sec
x64-glibc-217 v22.3.0 OK 188 sec
x64-glibc-217 v23.0.0 OK 205 sec
x64-glibc-217 v24.0.0 OK 203 sec
x64-glibc-217 v24.2.0 OK 208 sec
Recipe Version Status Time
x64-glibc-217 v6.17.1 OK 56 sec
x64-glibc-217 v7.10.1 OK 54 sec
x64-glibc-217 v8.17.0 OK 69 sec
x64-glibc-217 v9.11.2 OK 69 sec
x64-glibc-217 v10.24.1 OK 83 sec
x64-glibc-217 v11.15.0 OK 78 sec
x64-glibc-217 v12.13.0 OK 94 sec
x64-glibc-217 v13.14.0 OK 110 sec
x64-glibc-217 v14.14.0 OK 117 sec
x64-glibc-217 v14.21.3 OK 168 sec
x64-glibc-217 v15.14.0 OK 121 sec
x64-glibc-217 v16.20.2 OK 131 sec
x64-glibc-217 v18.20.8 OK 199 sec
x64-glibc-217 v19.9.0 OK 358 sec
x64-glibc-217 v20.16.0 OK 505 sec
x64-glibc-217 v20.19.3 OK 483 sec
x64-glibc-217 v21.7.3 OK 721 sec
x64-glibc-217 v22.17.0 OK 686 sec
x64-glibc-217 v23.11.1 OK 831 sec
x64-glibc-217 v24.3.0 OK 731 sec
Recipe Version Status Time
x64-pointer-compression v13.4.0 OK 104 sec
x64-pointer-compression v14.0.0 OK 107 sec
x64-pointer-compression v15.0.0 OK 114 sec
x64-pointer-compression v16.0.0 OK 120 sec
x64-pointer-compression v18.0.0 OK 150 sec
x64-pointer-compression v19.0.0 OK 153 sec
x64-pointer-compression v20.0.0 OK 158 sec
x64-pointer-compression v21.3.0 OK 164 sec
x64-pointer-compression v22.0.0 OK 190 sec
x64-pointer-compression v22.3.0 OK 187 sec
x64-pointer-compression v24.2.0 OK 202 sec
Recipe Version Status Time
x64-pointer-compression v13.14.0 OK 111 sec
x64-pointer-compression v14.14.0 OK 114 sec
x64-pointer-compression v14.21.3 OK 118 sec
x64-pointer-compression v15.14.0 OK 119 sec
x64-pointer-compression v16.20.2 OK 128 sec
x64-pointer-compression v18.20.8 OK 156 sec
x64-pointer-compression v19.9.0 OK 167 sec
x64-pointer-compression v20.16.0 OK 158 sec
x64-pointer-compression v21.7.3 OK 170 sec
x64-pointer-compression v22.17.0 OK 194 sec
x64-pointer-compression v24.3.0 OK 197 sec
Recipe Version Status Time
x86 v6.2.0 OK 47 sec
x86 v7.0.0 OK 51 sec
x86 v8.0.0 OK 70 sec
x86 v9.0.0 OK 66 sec
x86 v10.0.0 OK 75 sec
x86 v11.0.0 OK 78 sec
x86 v12.0.0 OK 78 sec
x86 v13.0.0 OK 100 sec
x86 v13.4.0 OK 106 sec
x86 v14.0.0 OK 108 sec
x86 v15.0.0 OK 113 sec
x86 v16.0.0 OK 120 sec
x86 v17.0.0 OK 139 sec
x86 v18.0.0 OK 149 sec
x86 v19.0.0 OK 156 sec
x86 v20.0.0 OK 160 sec
x86 v21.0.0 OK 163 sec
x86 v21.3.0 OK 163 sec
x86 v22.3.0 OK 181 sec
x86 v23.0.0 OK 183 sec
x86 v24.0.0 OK 187 sec
x86 v24.2.0 OK 192 sec
Recipe Version Status Time
x86 v6.17.1 OK 56 sec
x86 v7.10.1 OK 54 sec
x86 v8.17.0 OK 69 sec
x86 v9.11.2 OK 70 sec
x86 v10.24.1 OK 80 sec
x86 v11.15.0 OK 80 sec
x86 v12.13.0 OK 86 sec
x86 v13.14.0 OK 107 sec
x86 v14.14.0 OK 112 sec
x86 v14.21.3 OK 111 sec
x86 v15.14.0 OK 110 sec
x86 v16.20.2 OK 128 sec
x86 v17.9.1 OK 146 sec
x86 v18.20.8 OK 160 sec
x86 v19.9.0 OK 172 sec
x86 v20.16.0 OK 154 sec
x86 v20.19.3 OK 159 sec
x86 v21.7.3 OK 164 sec
x86 v22.17.0 OK 194 sec
x86 v23.11.1 OK 190 sec
x86 v24.3.0 OK 187 sec

@vitaliylag
Copy link
Author

vitaliylag commented Jul 12, 2025

Benchmark results with empty ccache:

Recipe Version GCC 12.1 GCC 15.1
x64-glibc-217 v23.7.0 7428 sec 3192 sec
x64-glibc-217 v23.0.0 7336 sec 3106 sec
x64-glibc-217 v22.17.0 5371 sec 3091 sec
x64-pointer-compression v22.17.0 5444 sec 2705 sec


Also:

x86 performance
Version x64-glibc-217 x86
v23.7.0  (GCC 15.1) 3192 sec 3041 sec
v23.0.0  (GCC 15.1) 3106 sec 3043 sec
v22.17.0  (GCC 15.1) 3091 sec 2805 sec
Node.js v8~v23
Recipe Version GCC 12.1
x64-glibc-217 v23.7.0 7428 sec
x64-glibc-217 v23.0.0 7336 sec
x64-glibc-217 v22.0.0 5031 sec
x64-glibc-217 v21.0.0 5911 sec
x64-glibc-217 v20.0.0 2246 sec
x64-glibc-217 v19.0.0 2060 sec
x64-glibc-217 v18.0.0 1894 sec
x64-glibc-217 v16.0.0 1456 sec
x64-glibc-217 v15.0.0 1225 sec
x64-glibc-217 v14.0.0 1090 sec
x64-glibc-217 v13.0.0 1467 sec
x64-glibc-217 v12.0.0 632 sec
x64-glibc-217 v11.0.0 537 sec
x64-glibc-217 v10.0.0 486 sec
x64-glibc-217 v9.0.0 459 sec
x64-glibc-217 v8.0.0 435 sec

As I understood Docker can run x86 without emulation on x64 platform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants