Skip to content

Commit 0fbb2c2

Browse files
committed
feat: replace embedded yt-dlp/ffmpeg with on-demand first-run download (slimmer binary)
1 parent 1338ce7 commit 0fbb2c2

File tree

11 files changed

+264
-368
lines changed

11 files changed

+264
-368
lines changed

.vscode/tasks.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
{
1414
"label": "debug: cargo build",
1515
"type": "shell",
16-
"command": "cargo build --debug",
16+
"command": "cargo build",
1717
"problemMatcher": [
1818
"$rustc"
1919
],

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Thanks for your interest in contributing!
88

99
## Development setup
1010
- Install Rust (stable) and `cargo`.
11-
- Optional: `yt-dlp` in PATH if you want to test resolver behavior locally.
11+
- Optional: `yt-dlp` in PATH if you want; otherwise the node will auto-download tools to `~/.resonix/bin` on first run.
1212
- Build: `cargo build` | Format: `cargo fmt`
1313

1414
## Coding guidelines

Cargo.lock

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "resonix-node"
3-
version = "0.2.0"
3+
version = "0.2.5"
44
edition = "2021"
55
description = "High-performance audio node. HTTP/WebSocket server that resolves and streams audio using Symphonia and yt-dlp."
66
license = "BSD-3-Clause"
@@ -57,6 +57,9 @@ rspotify = { version = "0.15", default-features = false, features = [
5757
dotenvy = "0.15"
5858
base64 = "0.22"
5959
uuid = { version = "1", features = ["v4"] }
60+
zip = "0.4"
61+
tar = "0.4"
62+
xz2 = "0.1"
6063

6164
[badges]
6265
docsrs = {}
@@ -65,9 +68,6 @@ maintenance = { status = "actively-developed" }
6568
[build-dependencies]
6669
winres = "0.1"
6770
reqwest = { version = "0.12", default-features = false, features = ["blocking", "rustls-tls"] }
68-
zip = "0.4"
69-
tar = "0.4"
70-
xz2 = "0.1"
7171

7272
[package.metadata.bundle]
7373
name = "Resonix"

README.md

Lines changed: 11 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Features
1111
- Allow/block URL patterns via regex
1212
- Lightweight EQ and volume filters
1313
- Minimal authentication via static password header
14-
- Self-contained (optional): embeds `yt-dlp` and `ffmpeg` binaries inside the executable
14+
- First-run auto-download of `yt-dlp` and `ffmpeg` into a user cache (`~/.resonix/bin`) keeping the executable slim
1515

1616
### Status
1717
Early preview. APIs may evolve. See License (BSD-3-Clause).
@@ -109,35 +109,28 @@ Environment overrides
109109
- `RESOLVE_TIMEOUT_MS=...` → override timeout
110110
- `SPOTIFY_CLIENT_ID` / `SPOTIFY_CLIENT_SECRET` → fallback env vars if `[spotify]` section is omitted.
111111
- You can also set custom env var names and reference them from the config, e.g.: `client_id = "MY_APP_SPOTIFY_ID"` and then set `MY_APP_SPOTIFY_ID` in your `.env` or environment.
112-
- `RESONIX_EMBED_EXTRACT_DIR=...` → directory to which embedded binaries are written (default: OS temp dir / `resonix-embedded`)
112+
- (legacy) `RESONIX_EMBED_EXTRACT_DIR` is ignored now; tools are stored in `~/.resonix/bin`
113+
(runtime export `RESONIX_TOOLS_DIR` shows the resolved directory)
113114

114115
Runtime export (informational)
115116
- On startup the resolved paths are placed into `RESONIX_YTDLP_BIN` / `RESONIX_FFMPEG_BIN` env vars for child processes spawned by the node. Normally you do not need to set these manually.
116117

117-
### Embedded binaries (standalone mode)
118+
### Tool management (yt-dlp / ffmpeg)
118119

119-
Resonix can bundle `yt-dlp` and `ffmpeg` directly into the executable:
120+
On startup Resonix checks for `yt-dlp` and `ffmpeg`.
120121

121-
1. During build, `build.rs` downloads platform-appropriate binaries into `assets/bin` (if they are missing).
122-
2. Those files are embedded with `include_bytes!` so the final `resonix-node` can run without the tools installed system-wide.
123-
3. At runtime, if the configured / external tool paths fail validation, the embedded bytes are extracted to a writable folder (default temporary directory or `RESONIX_EMBED_EXTRACT_DIR`) and invoked from there.
124-
125-
Selection order for each tool:
122+
Resolution order (per tool):
126123
1. Explicit env (`YTDLP_PATH` / `FFMPEG_PATH`)
127-
2. Config value (`resolver.ytdlp_path` / `resolver.ffmpeg_path`)
128-
3. Embedded binary fallback (if present)
129-
130-
To force use of your own system tools, either:
131-
- Set `YTDLP_PATH` / `FFMPEG_PATH` to the desired executables, or
132-
- Provide paths in `Resonix.toml` and keep them working; embedded versions are only used if validation (`--version` / `-version`) fails.
124+
2. Config (`resolver.ytdlp_path` / `resolver.ffmpeg_path`)
125+
3. Auto-managed download to `~/.resonix/bin` (created if missing)
133126

134-
To update embedded versions, delete the corresponding file(s) in `assets/bin` and rebuild; they will be re-downloaded.
127+
If only one tool is missing, only that one is downloaded. Existing executables are left untouched. Delete a file to force re-download of the latest release.
135128

136-
To ship a smaller binary without embedding, remove the files from `assets/bin` before building (they will not be embedded if absent) and rely on external paths/env values.
129+
macOS: `ffmpeg` is not auto-downloaded (install via Homebrew: `brew install ffmpeg`).
137130

138131
Notes
139132
- The resolver downloads temporary audio files using `yt-dlp`. Ensure sufficient disk space and legal use in your jurisdiction.
140-
- For sources needing remux/extraction, `ffmpeg` is required. Embedded or external versions are acceptable.
133+
- For sources needing remux/extraction, `ffmpeg` is required.
141134

142135
---
143136

SECURITY.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,4 +15,4 @@ Handling and Disclosure
1515
Operational Guidance
1616
- If you enable authentication, keep your `server.password` secret and rotate it if leaked.
1717
- Avoid exposing the node directly to the public internet; prefer a private network or authentication proxy.
18-
- Keep dependencies and `yt-dlp` up to date if you use the resolver.
18+
- Keep dependencies and the auto-downloaded tools (`yt-dlp`, `ffmpeg`) up to date if you use the resolver. Delete them in `~/.resonix/bin` to force fresh download.

build.rs

Lines changed: 22 additions & 220 deletions
Original file line numberDiff line numberDiff line change
@@ -1,237 +1,39 @@
1-
use std::{
2-
env, fs, io,
3-
path::{Path, PathBuf},
4-
};
5-
6-
fn download_to(url: &str, dest: &Path) -> Result<(), String> {
7-
println!("cargo:warning=Downloading {} -> {}", url, dest.display());
8-
if let Some(parent) = dest.parent() {
9-
fs::create_dir_all(parent).map_err(|e| e.to_string())?;
10-
}
11-
let resp = reqwest::blocking::get(url).map_err(|e| e.to_string())?;
12-
if !resp.status().is_success() {
13-
return Err(format!("Request failed: {}", resp.status()));
14-
}
15-
let bytes = resp.bytes().map_err(|e| e.to_string())?;
16-
fs::write(dest, &bytes).map_err(|e| e.to_string())?;
17-
Ok(())
18-
}
19-
20-
fn ensure_yt_dlp(target_os: &str, out_dir: &Path) -> Result<PathBuf, String> {
21-
let (url, filename) = match target_os {
22-
"windows" => ("https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.exe", "yt-dlp.exe"),
23-
"macos" => ("https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_macos", "yt-dlp"),
24-
_ => ("https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp", "yt-dlp"), // linux & others
25-
};
26-
let dest = out_dir.join(filename);
27-
if !dest.exists() {
28-
download_to(url, &dest)?;
29-
}
30-
#[cfg(unix)]
31-
{
32-
use std::os::unix::fs::PermissionsExt;
33-
if let Ok(meta) = fs::metadata(&dest) {
34-
let mut perm = meta.permissions();
35-
perm.set_mode(0o755);
36-
let _ = fs::set_permissions(&dest, perm);
37-
}
38-
}
39-
Ok(dest)
40-
}
41-
42-
fn ensure_ffmpeg(target_os: &str, out_dir: &Path) -> Result<PathBuf, String> {
43-
if target_os == "macos" {
44-
return Err("Skipping ffmpeg embed on macOS".into());
45-
}
46-
47-
let target_arch = env::var("CARGO_CFG_TARGET_ARCH").unwrap_or_else(|_| "x86_64".into());
48-
let (platform_tag, is_zip) = match (target_os, target_arch.as_str()) {
49-
("windows", "aarch64") => ("winarm64", true),
50-
("windows", _) => ("win64", true),
51-
("linux", "aarch64") => ("linuxarm64", false),
52-
("linux", _) => ("linux64", false),
53-
(other, _) => return Err(format!("Unsupported OS for ffmpeg embedding: {other}")),
54-
};
55-
let archive_name = if is_zip { "ffmpeg.zip" } else { "ffmpeg.tar.xz" };
56-
let archive_url = format!(
57-
"https://github.com/BtbN/FFmpeg-Builds/releases/latest/download/ffmpeg-master-latest-{platform_tag}-gpl.{}",
58-
if is_zip { "zip" } else { "tar.xz" }
59-
);
60-
let bin_subpath = if target_os == "windows" {
61-
format!("ffmpeg-master-latest-{platform_tag}-gpl/bin/ffmpeg.exe")
62-
} else {
63-
format!("ffmpeg-master-latest-{platform_tag}-gpl/bin/ffmpeg")
64-
};
65-
let ffmpeg_bin = out_dir.join(if target_os == "windows" { "ffmpeg.exe" } else { "ffmpeg" });
66-
if ffmpeg_bin.exists() {
67-
// Assume prior full extraction already happened.
68-
return Ok(ffmpeg_bin);
69-
}
70-
71-
let archive_path = out_dir.join(archive_name);
72-
download_to(&archive_url, &archive_path)?;
73-
if is_zip {
74-
let file = fs::File::open(&archive_path).map_err(|e| e.to_string())?;
75-
let mut zip = zip::ZipArchive::new(file).map_err(|e| e.to_string())?;
76-
for i in 0..zip.len() {
77-
let mut f = zip.by_index(i).map_err(|e| e.to_string())?;
78-
let name = f.name().to_string();
79-
if let Some(bin_prefix) = bin_subpath.rsplit_once('/') {
80-
// (dir, file)
81-
let bin_dir_prefix = bin_prefix.0.to_string();
82-
if name.ends_with('/') {
83-
continue;
84-
}
85-
if name.contains(&bin_dir_prefix) {
86-
if let Some(filename) = name.split('/').last() {
87-
let out_path = out_dir.join(filename);
88-
let mut out_f = fs::File::create(&out_path).map_err(|e| e.to_string())?;
89-
io::copy(&mut f, &mut out_f).map_err(|e| e.to_string())?;
90-
}
91-
}
92-
}
93-
}
94-
} else {
95-
// tar.xz
96-
let file = fs::File::open(&archive_path).map_err(|e| e.to_string())?;
97-
let decompressor = xz2::read::XzDecoder::new(file);
98-
let mut archive = tar::Archive::new(decompressor);
99-
for entry in archive.entries().map_err(|e| e.to_string())? {
100-
let mut entry = entry.map_err(|e| e.to_string())?;
101-
if let Ok(path) = entry.path() {
102-
if let Some(path_str) = path.to_str() {
103-
if let Some((bin_dir_prefix, _file)) = bin_subpath.rsplit_once('/') {
104-
if path_str.contains(bin_dir_prefix) && !path_str.ends_with('/') {
105-
if let Some(filename) = path.file_name() {
106-
let out_path = out_dir.join(filename);
107-
entry.unpack(&out_path).map_err(|e| e.to_string())?;
108-
}
109-
}
110-
}
111-
}
112-
}
113-
}
114-
}
115-
let _ = fs::remove_file(&archive_path);
116-
#[cfg(unix)]
117-
{
118-
use std::os::unix::fs::PermissionsExt;
119-
if let Ok(meta) = fs::metadata(&ffmpeg_bin) {
120-
let mut perm = meta.permissions();
121-
perm.set_mode(0o755);
122-
let _ = fs::set_permissions(&ffmpeg_bin, perm);
123-
}
124-
}
125-
Ok(ffmpeg_bin)
126-
}
1+
use std::env;
2+
use std::path::Path;
1273

1284
fn main() {
129-
let target_os = env::var("CARGO_CFG_TARGET_OS").unwrap_or_else(|_| String::from("unknown"));
130-
println!("cargo:rerun-if-env-changed=CARGO_CFG_TARGET_OS");
1315
println!("cargo:rerun-if-changed=build.rs");
132-
println!("cargo:rustc-check-cfg=cfg(has_embedded_bins)");
133-
6+
let target_os = env::var("CARGO_CFG_TARGET_OS").unwrap_or_else(|_| String::from("unknown"));
1347
if target_os == "windows" {
1358
println!("cargo:rerun-if-changed=assets/app/avatar.ico");
1369
println!("cargo:rerun-if-changed=assets/app/icon.ico");
13710
let mut res = winres::WindowsResource::new();
11+
13812
for path in ["assets/app/icon.ico", "assets/app/avatar.ico"] {
13913
if Path::new(path).exists() {
14014
res.set_icon(path);
141-
let _ = res.compile();
14215
break;
14316
}
14417
}
145-
}
146-
147-
let bin_dir = Path::new("assets").join("bin");
148-
fs::create_dir_all(&bin_dir).expect("create bin dir");
149-
150-
if let Err(e) = ensure_yt_dlp(&target_os, &bin_dir) {
151-
println!("cargo:warning=Failed to ensure yt-dlp: {e}");
152-
}
153-
if let Err(e) = ensure_ffmpeg(&target_os, &bin_dir) {
154-
println!("cargo:warning=Failed to ensure ffmpeg: {e}");
155-
}
156-
157-
println!("cargo:rerun-if-changed={}", bin_dir.display());
158-
159-
println!("cargo:rustc-env=RESONIX_EMBED_OS_DIR={}", bin_dir.display());
160-
161-
let out_dir_fs = PathBuf::from(env::var("OUT_DIR").expect("OUT_DIR set"));
162-
let gen_path = out_dir_fs.join("embedded_bins.rs");
163-
let mut gen = String::new();
164-
gen.push_str("// @generated by build.rs\n");
165-
let yt = match target_os.as_str() {
166-
"windows" => bin_dir.join("yt-dlp.exe"),
167-
_ => bin_dir.join("yt-dlp"),
168-
};
169-
if yt.exists() {
170-
let rel = yt.strip_prefix(".").unwrap_or(&yt); // best-effort
171-
let rel_str = rel.to_string_lossy().replace('\\', "/");
172-
gen.push_str(&format!("pub const YT_DLP_PATH: &str = r#\"{}\"#;\n", yt.display()));
173-
gen.push_str(&format!(
174-
"pub const YT_DLP: &[u8] = include_bytes!(concat!(env!(\"CARGO_MANIFEST_DIR\"), r#\"/{}\"#));\n",
175-
rel_str
176-
));
177-
} else {
178-
gen.push_str("pub const YT_DLP_PATH: &str = \"\";\npub const YT_DLP: &[u8] = &[];\n");
179-
}
180-
let ff = if target_os == "windows" { bin_dir.join("ffmpeg.exe") } else { bin_dir.join("ffmpeg") };
181-
if ff.exists() {
182-
let rel = ff.strip_prefix(".").unwrap_or(&ff);
183-
let rel_str = rel.to_string_lossy().replace('\\', "/");
184-
gen.push_str(&format!("pub const FFMPEG_PATH: &str = r#\"{}\"#;\n", ff.display()));
185-
gen.push_str(&format!(
186-
"pub const FFMPEG: &[u8] = include_bytes!(concat!(env!(\"CARGO_MANIFEST_DIR\"), r#\"/{}\"#));\n",
187-
rel_str
188-
));
189-
} else {
190-
gen.push_str("pub const FFMPEG_PATH: &str = \"\";\npub const FFMPEG: &[u8] = &[];\n");
191-
}
19218

193-
// Generic embedding of every file present in assets/bin for completeness (so ffprobe, dlls, etc. are shipped).
194-
let mut embedded_list_entries = String::new();
195-
embedded_list_entries
196-
.push_str("pub struct EmbeddedFile { pub name: &'static str, pub bytes: &'static [u8] }\n");
197-
let mut array_items = Vec::new();
198-
if let Ok(read_dir) = fs::read_dir(&bin_dir) {
199-
for entry in read_dir.flatten() {
200-
if let Ok(ft) = entry.file_type() {
201-
if ft.is_dir() {
202-
continue;
203-
}
204-
}
205-
let path = entry.path();
206-
if let Some(fname) = path.file_name().and_then(|s| s.to_str()) {
207-
let rel = path.strip_prefix(".").unwrap_or(&path);
208-
let rel_str = rel.to_string_lossy().replace('\\', "/");
209-
// Sanitize identifier
210-
let mut ident = fname
211-
.chars()
212-
.map(|c| if c.is_ascii_alphanumeric() { c } else { '_' })
213-
.collect::<String>();
214-
if !ident.chars().next().map(|c| c.is_ascii_alphabetic() || c == '_').unwrap_or(false) {
215-
ident = format!("_{}", ident);
216-
}
217-
ident = ident.to_ascii_uppercase();
218-
embedded_list_entries.push_str(&format!("pub const EMBED_FILE_{ident}: &[u8] = include_bytes!(concat!(env!(\"CARGO_MANIFEST_DIR\"), r#\"/{}\"#));\n", rel_str));
219-
array_items
220-
.push(format!("EmbeddedFile {{ name: r#\"{fname}\"#, bytes: EMBED_FILE_{ident} }}"));
221-
}
19+
let company = env::var("RESONIX_COMPANY").unwrap_or_else(|_| "Resonix OSS Team".into());
20+
let product = env::var("RESONIX_PRODUCT").unwrap_or_else(|_| "Resonix".into());
21+
let copyright = env::var("RESONIX_COPYRIGHT")
22+
.unwrap_or_else(|_| "© 2025 Resonix OSS".into());
23+
24+
let version = env::var("CARGO_PKG_VERSION").unwrap_or_else(|_| "0.0.0".into());
25+
res.set("CompanyName", &company);
26+
res.set("FileDescription", "High-performance audio node");
27+
res.set("ProductName", &product);
28+
res.set("ProductVersion", &version);
29+
res.set("FileVersion", &version);
30+
res.set("OriginalFilename", "resonix-node.exe");
31+
res.set("InternalName", "resonix-node");
32+
res.set("LegalCopyright", &copyright);
33+
34+
if let Err(e) = res.compile() {
35+
eprintln!("Failed to embed Windows resources: {e}");
22236
}
22337
}
224-
embedded_list_entries.push_str("pub const EMBEDDED_FILES: &[EmbeddedFile] = &[\n");
225-
for item in &array_items {
226-
embedded_list_entries.push_str(" ");
227-
embedded_list_entries.push_str(item);
228-
embedded_list_entries.push_str(",\n");
229-
}
230-
embedded_list_entries.push_str("];");
231-
gen.push_str(&embedded_list_entries);
232-
if fs::write(&gen_path, gen).is_err() {
233-
println!("cargo:warning=Failed to write embedded_bins.rs");
234-
}
235-
println!("cargo:rustc-env=RESONIX_EMBED_BINS_RS={}", gen_path.display());
236-
println!("cargo:rustc-cfg=has_embedded_bins");
38+
// If not Windows or no icons, nothing else to do.
23739
}

0 commit comments

Comments
 (0)