Skip to content

Conversation

ricardoV94
Copy link
Member

@ricardoV94 ricardoV94 commented May 31, 2025

There were some odd errors cropping up in pymc-marketing and a private project.

The issue happened when you created an Arange with float64 and another with np.dtype("float64") (or any combo of dtype and string), which evaluate to the same thing, but hash differently (see below why). The PyTensor C-cache has a sanity-check mechanism to make sure the eq and hash of our Ops is correctly implemented, as the validity of the cache depends on this.

PyTensor uses the properties of Op to define equality and hash automatically, and for Arange that is it's dtype argument. Surprisingly though, np.dtype("float64") == "float64" and hash(np.dtype("float64")) != hash("float64"), which seems to be a fundamental bug in Numpy: numpy/numpy#17864

This only became a problem once we added a C-implementation to Arange in #1392 as this is where the automatic checks for hash/eq are performed.

I tried to look for other Ops parametrized by dtype of some form or another, and make sure those are always converted to the equivalent strings.

CC @lucianopaz and @juanitorduz


📚 Documentation preview 📚: https://pytensor--1436.org.readthedocs.build/en/1436/

@ricardoV94 ricardoV94 added bug Something isn't working C-backend labels May 31, 2025
@ricardoV94 ricardoV94 changed the title Coerce dtype __props__ to string due to invalid hash of np.dtype() objects Coerce dtype __props__ to string due to invalid hash of np.dtype() objects May 31, 2025
@ricardoV94 ricardoV94 force-pushed the numpy_dtype_hash_bug branch 2 times, most recently from eacddf1 to a118b03 Compare June 2, 2025 10:38
@ricardoV94 ricardoV94 force-pushed the numpy_dtype_hash_bug branch from a118b03 to fd371c3 Compare June 2, 2025 10:40
Copy link

codecov bot commented Jun 2, 2025

Codecov Report

❌ Patch coverage is 80.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.12%. Comparing base (92eef5e) to head (fd371c3).
⚠️ Report is 131 commits behind head on main.

Files with missing lines Patch % Lines
pytensor/tensor/random/op.py 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1436   +/-   ##
=======================================
  Coverage   82.12%   82.12%           
=======================================
  Files         211      211           
  Lines       49751    49757    +6     
  Branches     8817     8819    +2     
=======================================
+ Hits        40857    40862    +5     
- Misses       6714     6715    +1     
  Partials     2180     2180           
Files with missing lines Coverage Δ
pytensor/sparse/sandbox/sp2.py 82.79% <100.00%> (ø)
pytensor/tensor/basic.py 91.69% <100.00%> (+0.01%) ⬆️
pytensor/tensor/elemwise.py 90.01% <100.00%> (ø)
pytensor/tensor/io.py 77.41% <100.00%> (ø)
pytensor/tensor/random/op.py 92.51% <0.00%> (-1.01%) ⬇️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ricardoV94 ricardoV94 merged commit ff98ab8 into pymc-devs:main Jun 2, 2025
72 of 73 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working C-backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants