Skip to content

Commit d4ddfaf

Browse files
incrypto32incrypto32
authored
gnd : Graph Node Dev (#6056)
* Refactor main function (#5980) * node: Refactor main execution flow and introduce launcher module * node/launcher: extract setup_configuration helper from run * node/launcher: extract setup_metrics helper from run * node/launcher: extract setup_store helper from run * node/launcher: extract build_blockchain_map helper from run * node/launcher: extract cleanup_ethereum_shallow_blocks helper from run * node/launcher: extract spawn_block_ingestor helper from run * node/launcher: extract deploy_subgraph_from_flag helper from run * node/launcher: extract spawn_contention_checker helper from run * node/launcher: extract build_graphql_server helper from run * node/launcher: extract build_subgraph_registrar helper from run * Implement a File Link Resolver (#5981) * graph: Add a new FIleLinkResolver * graph: remove `/ipfs/` prefix when using file link resolver * graph: Implement custom deserialise logic for Link to enable file link resolver * tests: Add runner test that uses file link resolver * graph: Conditionally disable deployment hash validation based on env var * graph: use constant for "/ipfs/" prefix in `remove_prefix` * graph: Simplify resolve_path by removing redundant path.is_absolute() check * graph: Remove leftover println from file_resolver tests * tests: Refactor runner tests extract test utils into recipe.rs * tests: Add a test for file_link_resolver * Graph node dev mode (#5982) * node: Create a new binary for graph node dev mode * graph, store: Add unassign_subgraph method to SubgraphStore * node: Add helpers for graph node dev for subgraph management * node: Add helper functions for watching files in dev mode * node: Wire file watching in dev mode to redeploy subgraphs * node: fix formatting * gnd: Support multiple subgraphs, grafting, subgraph composition in dev mode (#6000) * graph: Add clone_for_deployment to FileLinkResolver to create FileLinkResolver with the right base dir for a subgraph * graph: Add for_deployment to LinkResolverTrait * core, graph: use for_deployment to get properly scoped resolver * graph: Implement aliases for file link resolver * node: Make gnd work with multiple subgraphs * node: Support subgraph datasource in gnd * node: correct the default value for manfiest * core, node, graph: Ignore graft base in dev mode * node: Allow providing a postgres url for gnd * node: Do not use pgtemp in windows * store: enable `vendored` feature for openssl crate * chain/ethereum: Return error when ipc is used in non unix platform * node: Refactor launcher * node/dev : Better error message when database directory doesn't exist * node: refactor watcher * core, node, graph: Manipulate raw manifest instead of passing ignore_graft_base This reverts commit b5bbf93. * node: Correct comments on `redeploy_all_subgraphs` * node/gnd: Deploy all subgraphs first before wathcing files * core, graph : Refactor LinkResolver trait * Workflow to build the gnd binary (#6013) * .github: Create a workflow for building gnd binaries * .github: Codesign gnd binary for macOs * .github: notarize gnd binary for macOs * gnd: Integration tests (#6035) * node/gnd: Make ports configurable * node/gnd: Deploy all subgraphs on startup * tests: Refactor subgraph datasources in TestCase * tests: refactor Testcase method for source subgraphs * tests: Add integration tests for gnd * store: Use bundled pq-sys * gnd: remove temp database directory on exit * gnd: use pgtemp from graphprotocol org * gnd: add alias for pgtemp db for windows * gnd: use deep codesigning for macos binaries * update workflow to add entitlements.plist
1 parent 085f281 commit d4ddfaf

File tree

45 files changed

+3077
-810
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+3077
-810
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,13 @@ jobs:
108108
command: test
109109
args: --package graph-tests --test runner_tests
110110

111+
- name: Run file link resolver test
112+
id: file-link-resolver-test
113+
uses: actions-rs/cargo@v1
114+
with:
115+
command: test
116+
args: --package graph-tests --test file_link_resolver
117+
111118
integration-tests:
112119
name: Run integration tests
113120
runs-on: ubuntu-latest

‎.github/workflows/gnd-binary-build.yml‎

Lines changed: 189 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,189 @@
1+
name: Build gnd Binaries
2+
3+
on:
4+
workflow_dispatch:
5+
6+
jobs:
7+
build:
8+
name: Build gnd for ${{ matrix.target }}
9+
runs-on: ${{ matrix.runner }}
10+
strategy:
11+
fail-fast: false
12+
matrix:
13+
include:
14+
- target: x86_64-unknown-linux-gnu
15+
runner: ubuntu-latest
16+
asset_name: gnd-linux-x86_64
17+
- target: aarch64-unknown-linux-gnu
18+
runner: ubuntu-24.04-arm
19+
asset_name: gnd-linux-aarch64
20+
- target: x86_64-apple-darwin
21+
runner: macos-13
22+
asset_name: gnd-macos-x86_64
23+
- target: aarch64-apple-darwin
24+
runner: macos-latest
25+
asset_name: gnd-macos-aarch64
26+
- target: x86_64-pc-windows-msvc
27+
runner: windows-latest
28+
asset_name: gnd-windows-x86_64.exe
29+
30+
steps:
31+
- name: Checkout repository
32+
uses: actions/checkout@v4
33+
34+
- name: Install Rust toolchain
35+
run: |
36+
rustup toolchain install stable
37+
rustup target add ${{ matrix.target }}
38+
rustup default stable
39+
40+
- name: Rust Cache
41+
uses: Swatinem/rust-cache@v2
42+
with:
43+
key: ${{ matrix.target }}
44+
45+
- name: Install dependencies (Ubuntu)
46+
if: startsWith(matrix.runner, 'ubuntu')
47+
run: |
48+
sudo apt-get update
49+
sudo apt-get install -y libpq-dev protobuf-compiler musl-tools libssl-dev
50+
51+
- name: Install dependencies (macOS)
52+
if: startsWith(matrix.runner, 'macos')
53+
run: |
54+
brew install postgresql protobuf
55+
56+
- name: Install protobuf (Windows)
57+
if: startsWith(matrix.runner, 'windows')
58+
run: choco install protoc
59+
60+
- name: Cache vcpkg
61+
uses: actions/cache@v4
62+
if: startsWith(matrix.runner, 'windows')
63+
id: vcpkg-cache
64+
with:
65+
path: |
66+
${{ github.workspace }}/vcpkg
67+
C:/vcpkg/installed
68+
C:/vcpkg/packages
69+
key: ${{ runner.os }}-vcpkg-${{ hashFiles('**/Cargo.lock') }}
70+
restore-keys: |
71+
${{ runner.os }}-vcpkg-
72+
73+
- name: Install vcpkg and dependencies (Windows)
74+
if: startsWith(matrix.runner, 'windows') && steps.vcpkg-cache.outputs.cache-hit != 'true'
75+
run: |
76+
# Install vcpkg
77+
git clone https://github.com/microsoft/vcpkg.git
78+
cd vcpkg
79+
.\bootstrap-vcpkg.bat
80+
81+
# Install libpq using vcpkg
82+
.\vcpkg.exe install libpq:x64-windows
83+
shell: pwsh
84+
85+
- name: Set Windows environment variables
86+
if: startsWith(matrix.runner, 'windows')
87+
run: |
88+
echo "VCPKG_ROOT=${{ github.workspace }}/vcpkg" | Out-File -FilePath $env:GITHUB_ENV -Append
89+
echo "LIBPQ_DIR=${{ github.workspace }}/vcpkg/installed/x64-windows" | Out-File -FilePath $env:GITHUB_ENV -Append
90+
echo "RUSTFLAGS=-L ${{ github.workspace }}/vcpkg/installed/x64-windows/lib" | Out-File -FilePath $env:GITHUB_ENV -Append
91+
shell: pwsh
92+
93+
- name: Build gnd binary (Unix/Mac)
94+
if: ${{ !startsWith(matrix.runner, 'windows') }}
95+
run: cargo build --bin gnd --release --target ${{ matrix.target }}
96+
97+
- name: Build gnd binary (Windows)
98+
if: startsWith(matrix.runner, 'windows')
99+
run: cargo build --bin gnd --release --target ${{ matrix.target }}
100+
env:
101+
LIBPQ_DIR: ${{ format('{0}/vcpkg/installed/x64-windows', github.workspace) }}
102+
VCPKGRS_DYNAMIC: 1
103+
104+
- name: Sign macOS binary
105+
if: startsWith(matrix.runner, 'macos')
106+
uses: lando/code-sign-action@v3
107+
with:
108+
file: target/${{ matrix.target }}/release/gnd
109+
certificate-data: ${{ secrets.APPLE_CERT_DATA }}
110+
certificate-password: ${{ secrets.APPLE_CERT_PASSWORD }}
111+
certificate-id: ${{ secrets.APPLE_TEAM_ID }}
112+
options: --options runtime --entitlements entitlements.plist
113+
114+
- name: Notarize macOS binary
115+
if: startsWith(matrix.runner, 'macos')
116+
uses: lando/notarize-action@v2
117+
with:
118+
product-path: target/${{ matrix.target }}/release/gnd
119+
appstore-connect-username: ${{ secrets.NOTARIZATION_USERNAME }}
120+
appstore-connect-password: ${{ secrets.NOTARIZATION_PASSWORD }}
121+
appstore-connect-team-id: ${{ secrets.APPLE_TEAM_ID }}
122+
123+
- name: Prepare binary (Unix)
124+
if: ${{ !startsWith(matrix.runner, 'windows') }}
125+
run: |
126+
cp target/${{ matrix.target }}/release/gnd ${{ matrix.asset_name }}
127+
chmod +x ${{ matrix.asset_name }}
128+
gzip ${{ matrix.asset_name }}
129+
130+
- name: Prepare binary (Windows)
131+
if: startsWith(matrix.runner, 'windows')
132+
run: |
133+
copy target\${{ matrix.target }}\release\gnd.exe ${{ matrix.asset_name }}
134+
7z a -tzip ${{ matrix.asset_name }}.zip ${{ matrix.asset_name }}
135+
136+
- name: Upload artifact
137+
uses: actions/upload-artifact@v4
138+
with:
139+
name: ${{ matrix.asset_name }}
140+
path: |
141+
${{ matrix.asset_name }}.gz
142+
${{ matrix.asset_name }}.zip
143+
if-no-files-found: error
144+
145+
release:
146+
name: Create Release
147+
needs: build
148+
if: startsWith(github.ref, 'refs/tags/')
149+
runs-on: ubuntu-latest
150+
steps:
151+
- name: Checkout code
152+
uses: actions/checkout@v4
153+
154+
- name: Setup GitHub CLI
155+
run: |
156+
# GitHub CLI is pre-installed on GitHub-hosted runners
157+
gh --version
158+
env:
159+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
160+
161+
- name: Download all artifacts
162+
uses: actions/download-artifact@v4
163+
with:
164+
path: artifacts
165+
166+
- name: Display structure of downloaded artifacts
167+
run: ls -R artifacts
168+
169+
- name: Upload Assets to Release
170+
run: |
171+
# Extract version from ref (remove refs/tags/ prefix)
172+
VERSION=${GITHUB_REF#refs/tags/}
173+
174+
# Upload Linux x86_64 asset
175+
gh release upload $VERSION artifacts/gnd-linux-x86_64/gnd-linux-x86_64.gz --repo $GITHUB_REPOSITORY
176+
177+
# Upload Linux ARM64 asset
178+
gh release upload $VERSION artifacts/gnd-linux-aarch64/gnd-linux-aarch64.gz --repo $GITHUB_REPOSITORY
179+
180+
# Upload macOS x86_64 asset
181+
gh release upload $VERSION artifacts/gnd-macos-x86_64/gnd-macos-x86_64.gz --repo $GITHUB_REPOSITORY
182+
183+
# Upload macOS ARM64 asset
184+
gh release upload $VERSION artifacts/gnd-macos-aarch64/gnd-macos-aarch64.gz --repo $GITHUB_REPOSITORY
185+
186+
# Upload Windows x86_64 asset
187+
gh release upload $VERSION artifacts/gnd-windows-x86_64.exe/gnd-windows-x86_64.exe.zip --repo $GITHUB_REPOSITORY
188+
env:
189+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

0 commit comments

Comments
 (0)