PDL Optimizer #941

claudiosv · 2025-05-05T22:06:59Z

Undo PEP604 changes in pdl_ast and pdl_interpreter
Unexpected wikipedia package install failure
Figure out a solution for the skipped tests (mock?)

mandel

That looks very good!

My main question is to know if it would be possible to move the dataset specific Python code in the examples folder instead of src/pdl?

I have a few other questions and comments:

Would it be easy to do the formatting changes in a different commint to make the review easier?
Can you also update the UI with the new ipython block? It is probably mostly re-generating the types in pdl-live-react (npm run types).
I suggested to use match instead of ifs in the PDL programs.
Check if you have to skip some examples in test_examples_run.py.

examples/optimizer/evalplus.pdl

examples/optimizer/fever.pdl

This also updates the tauri cli to rename runr -> run (run previously invoked the python interpreter). And adds stub support for --data, --data-file, --trace command line options (not implemented yet). Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

This also removes the beeai compiler test in that workflow, as we now cover that in the core rust interpreter tests. Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Claudio Spiess <[email protected]>

Rather than duplicating this "metadata" logic -- the stuff common to all non-literal blocks. This just starts the migration. The main trick is to use the serde `flatten` capability, so that we can maintain a separate Metadata struct in Rust, but have it flattened into the enclosing object for serde. Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

This also adds initial scaffolding for timing, and adds a ModelBlock and ArrayBlockBuilder. Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

And start populating the timing info (incomplete). Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

… model_input trace field Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Claudio Spiess <[email protected]>

* Initial prompt library Signed-off-by: Claudio Spiess <[email protected]> * Documentation and fixes Signed-off-by: Claudio Spiess <[email protected]> * Finish prompt lib docs Signed-off-by: Claudio Spiess <[email protected]> * Address feedback Signed-off-by: Claudio Spiess <[email protected]> --------- Signed-off-by: Claudio Spiess <[email protected]>

It wasn't set previously because importlib searches for the distribution name and not the module's top-level name. Fallbacks are in place to revert to searching for 'pdl', and if that fails, it fallsback to the hardcoded version in _version.py Signed-off-by: Abi Ullattil <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

* independent implementation Signed-off-by: Mandana Vaziri <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Signed-off-by: Claudio Spiess <[email protected]>

claudiosv · 2025-05-12T23:16:03Z

.pre-commit-config.yaml

    rev: 'v1.15.0'
    hooks:
      - id: mypy
+        args: [--explicit-package-bases]


Without this line, mypy reports this error:

src/pdl/_version.py: error: Source file found twice under different module names: "optimizer.fever_evaluator" and "examples.optimizer.fever_evaluator" src/pdl/_version.py: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules for more info src/pdl/_version.py: note: Common resolutions include: a) adding `__init__.py` somewhere, b) using `--explicit-package-bases` or adjusting MYPYPATH Found 1 error in 1 file (errors prevented further checking)

This is because examples.optimizer.fever_evaluator is imported in test_optimizer.py as from examples.optimizer.fever_evaluator import FEVEREvaluator, but also from within examples/optimizer/optimize.py as from .fever_evaluator import FEVEREvaluator

Signed-off-by: Claudio Spiess <[email protected]>

mandel reviewed May 7, 2025

View reviewed changes

examples/optimizer/evalplus.pdl Show resolved Hide resolved

examples/optimizer/fever.pdl Show resolved Hide resolved

starpit and others added 29 commits May 9, 2025 17:49

chore: bump rust dependencies to resolve alert

1379876

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

test: tauri github actions test should apt install the deb

8cb0ae8

This also removes the beeai compiler test in that workflow, as we now cover that in the core rust interpreter tests. Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

feat: support for --data and --data-file in rust interpreter

cb0b34b

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Add optimizer module

8f122b6

Signed-off-by: Claudio Spiess <[email protected]>

Remove BAM

72d0e64

Signed-off-by: Claudio Spiess <[email protected]>

Initial prompt library

3d773a8

Signed-off-by: Claudio Spiess <[email protected]>

Clean up & bring in changes

0f31164

Signed-off-by: Claudio Spiess <[email protected]>

AST dump fixes

f08326c

Signed-off-by: Claudio Spiess <[email protected]>

Documentation and fixes

36cc142

Signed-off-by: Claudio Spiess <[email protected]>

Finish prompt lib docs

566d52c

Signed-off-by: Claudio Spiess <[email protected]>

Address feedback

b6b9f2a

Signed-off-by: Claudio Spiess <[email protected]>

Lint

dfab1e8

Signed-off-by: Claudio Spiess <[email protected]>

chore: update granite-io dependency (#896)

ade9c69

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

refactor: move def attr into Metadata (rust interpreter)

d73622c

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

feat: introduce Expr typing and apply it to IfBlock.condition

087e1d2

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

feat: update rust Call AST to use Expr for condition attr

0ffa988

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

chore: bump to rust 2024 edition

0962c0d

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

feat: continue to flesh out block metadata structure in rust

a4f935f

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

refactor: add metadata attr to remaining rust block asts

a3a9849

This also adds initial scaffolding for timing, and adds a ModelBlock and ArrayBlockBuilder. Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

feat: update rust Repeat AST to use Expr for for attr (#904)

a0232f9

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

refactor: introduce Advanced enum to rust AST

71bd1da

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

refactor: refactor rust ast to place metadata in common struct

23eff68

And start populating the timing info (incomplete). Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

fix: improve deserialization of python-generated model block traces

98e5944

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

fix: in rust ast, allow ModelBlock model to be an expr

2df4341

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

feat: initial pdl__id and --trace support for rust interpreter

a92ce92

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

fix: update rust interpreter to create Data blocks for expr eval, and…

ba861d3

… model_input trace field Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

fix: populate trace context field in rust interpreter

0ca9930

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

claudiosv and others added 18 commits May 9, 2025 17:51

Lint

c01605f

Signed-off-by: Claudio Spiess <[email protected]>

Skip tests for now

6e7747f

Signed-off-by: Claudio Spiess <[email protected]>

Update schema

66e1b61

Signed-off-by: Claudio Spiess <[email protected]>

chore: update pre-commit tools (#937)

c3f635f

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Use granite-io async interface (#936)

311b59b

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Lint

1361fc8

Signed-off-by: Claudio Spiess <[email protected]>

chore: bump ui dependences

091b68d

Signed-off-by: Nick Mitchell <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

fix: skip failing execution tests (#938)

3a43a4d

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

independent implementation (#934)

c388836

* independent implementation Signed-off-by: Mandana Vaziri <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

feat: add a parse_dict function to pdl_parser (#943)

8c5fbce

Signed-off-by: Louis Mandel <[email protected]> Signed-off-by: Claudio Spiess <[email protected]>

Address feedback 1

bb5367e

Signed-off-by: Claudio Spiess <[email protected]>

Address feedback 2

ec21e7e

Signed-off-by: Claudio Spiess <[email protected]>

Lint & fix mypy module warning

6c7ae94

Signed-off-by: Claudio Spiess <[email protected]>

Skip tests again

1f8a748

Signed-off-by: Claudio Spiess <[email protected]>

Try to resolve wikipedia package in ci

cf8d805

Signed-off-by: Claudio Spiess <[email protected]>

Fix rebase

eb7878e

Signed-off-by: Claudio Spiess <[email protected]>

claudiosv force-pushed the optimizer-merge branch from cd40344 to eb7878e Compare May 10, 2025 00:56

claudiosv added 4 commits May 9, 2025 17:58

Fix pyproject

c706ea7

Signed-off-by: Claudio Spiess <[email protected]>

Merge remote-tracking branch 'origin/main' into optimizer-merge

ff5b999

Signed-off-by: Claudio Spiess <[email protected]>

Move multiprocess to optional

2b3a607

Signed-off-by: Claudio Spiess <[email protected]>

Add ipython to pdl-live

0913a18

Signed-off-by: Claudio Spiess <[email protected]>

claudiosv marked this pull request as ready for review May 12, 2025 15:27

Fix trace metadata

32f511c

Signed-off-by: Claudio Spiess <[email protected]>

claudiosv commented May 12, 2025

View reviewed changes

claudiosv added 3 commits May 14, 2025 17:06

Add documentation

5001f92

Signed-off-by: Claudio Spiess <[email protected]>

Add optimizer test PDL

ee45d5e

Signed-off-by: Claudio Spiess <[email protected]>

Add config and expand docs

91408e0

Signed-off-by: Claudio Spiess <[email protected]>

claudiosv merged commit bcfabe8 into main May 19, 2025
10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

PDL Optimizer #941

PDL Optimizer #941

Uh oh!

claudiosv commented May 5, 2025 •

edited

Loading

Uh oh!

mandel left a comment

Uh oh!

Uh oh!

Uh oh!

claudiosv May 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

PDL Optimizer #941

PDL Optimizer #941

Uh oh!

Conversation

claudiosv commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mandel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

claudiosv May 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

claudiosv commented May 5, 2025 •

edited

Loading