Allow locally installed libraries in sketch profiles.

[cmaglie]

Please check if the PR fulfills these requirements

See how to contribute

• The PR has no duplicates (please search among the Pull Requests
  before creating one)
• The PR follows
  our contributing guidelines
• Tests for the changes have been added (for bug fixes / features)
• Docs have been added / updated (for bug fixes / features)
• UPGRADING.md has been updated with a migration guide (for breaking changes)
• configuration.schema.json updated if new parameters are added.

What kind of change does this PR introduce?

This change allows us to specify libraries in the sketch profile using a path to a local directory by prefixing the path with the string dir: .
It also provides this information in compile --dump-profile

What is the current behavior?

Only libraries in the Arduino Libraries Index could be specified in a sketch profile. A typical example may be the following:

profiles:
  uno:
    fqbn: arduino:avr:uno
    platforms:
      - platform: arduino:avr (1.8.6)
    libraries:
      - Adafruit SSD1306 (2.5.13)
      - Adafruit GFX Library (1.11.11)
      - Adafruit BusIO (1.17.0)

What is the new behavior?

A library installed in the filesystem could be specified as well:

profiles:
  uno:
    fqbn: arduino:avr:uno
    platforms:
      - platform: arduino:avr (1.8.6)
    libraries:
      - dir: libraries/MyLib
      - dir: /path/to/library/MyLibOutsideTheSketch 
      - Adafruit SSD1306 (2.5.13)
      - Adafruit GFX Library (1.11.11)
      - Adafruit BusIO (1.17.0)

In this case the two libraries:

• libraries/MyLib
• /path/to/library/MyLibOutsideTheSketch
  will be included during the compilation. In particular, a relative path refers to a path relative to the sketch path, in this case libraries/MyLib is to be intended as SKETCHPATH/libaries/MyLib.

Does this PR introduce a breaking change, and is titled accordingly?

No

Other information

[codecov]

Codecov Report

❌ Patch coverage is 65.95745% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.96%. Comparing base (cf62bee) to head (1a933cb).
⚠️ Report is 29 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/arduino/sketch/profiles.go	45.00%	6 Missing and 5 partials ⚠️
commands/instances.go	68.75%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2930      +/-   ##
==========================================
+ Coverage   67.91%   67.96%   +0.05%     
==========================================
  Files         238      238              
  Lines       22489    22524      +35     
==========================================
+ Hits        15273    15309      +36     
+ Misses       6017     6013       -4     
- Partials     1199     1202       +3     

Flag	Coverage Δ
unit	67.96% <65.95%> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[alessio-perugini]

The implementation looks good, I'd just add a test to assert the compilation given a sketch.yaml with also relative lib.

[egnor]

This is amazing! I will finally be able to get rid of my various hacky uses of src/ directory symlinks!

[alessio-perugini]

@egnor 😄 If you want to give it a try, before we merge this, you can find the test binaries here: https://github.com/arduino/arduino-cli/actions/runs/15586877233
Or you can git-checkout this PR, and run go build .

Feedbacks are always welcomed 🤓

[mike-lloyd03]

I cannot get this to work. I'm trying to share some code between two sketches. I have this sketch file:

default_fqbn: esp32:esp32:esp32s3:CDCOnBoot=cdc

profiles:
  main:
    port: /dev/ttyACM1
    platforms:
      - platform: esp32:esp32 (3.2.0)
    libraries:
      - Adafruit BusIO (1.17.2)
      - Adafruit GFX Library (1.12.1)
      - Adafruit ST7735 and ST7789 Library (1.11.0)
      - ESP32Encoder (0.11.7)
      - NimBLE-Arduino (2.3.2)
      - OneButton (2.6.1)
      - dir: ../common

With this file structure:

project_dir/
    common/
        fs.h
    sketch1/
        sketch1.ino
        sketch.yaml
    sketch2/
        sketch2.ino
        sketch.yaml

Inside sketch1.ino:

#include "fs.h"

Compiler throws an error:

$ cd sketch 1
$ arduino-cli compile

project_dir/sketch1/sketch1.ino:1:10: fatal error: fs.h: No such file or directory
   10 | #include "fs.h"
      |          ^~~~~~
compilation terminated.

When I run with the --json flag, the common library is absent from the builder_result.used_libraries array. Looks like it's not even being picked up.

[egnor]

It should probably be generating an error, but I suspect your problem is that the library directory can't just include files, it has to be a proper library in the newer format with a library.properties file and code in the src/ subdirectory. It does work for me fwiw.

[mike-lloyd03]

I actually tried exactly that. Code in the src directory with a library.properties file in the library root. Same issue. No error and library not included in compilation.

I also passed a non-existent directory in the libraries list and got no error. So it seems like the cli is parsing the file correctly but not doing anything with the directory being included.

[egnor]

I actually tried exactly that. Code in the src directory with a library.properties file in the library root. Same issue. No error and library not included in compilation.

I also passed a non-existent directory in the libraries list and got no error. So it seems like the cli is parsing the file correctly but not doing anything with the directory being included.

Ok, file a bug and we can debug there instead of commenting on the PR? Ideally include a zip file (or github pointer) for a directory layout that reproduces the issue?