Bug: DuckDB List Validation assertion failure

[connortsui20]

On develop, if we run all 3 of these tests with VX_DUCKDB_DEBUG=1, we get an assertion failure when DuckDB validates the list view entries. So for now, we should ignore them.

For context, I have been migrating List to use ListViewArray instead of ListArray, and I reimplemented export under vortex-duckdb/src/exporter/list.rs to export directly from a ListView.

It seems like we might be interpreting memory incorrectly? Here is the relevant code (on develop):

vortex-duckdb/src/exporter/list.rs

pub struct duckdb_list_entry {
    pub offset: u64,
    pub length: u64,
}

<-- snip -->

let offsets_slice: &mut [cpp::duckdb_list_entry] =
    unsafe { vector.as_slice_mut::<cpp::duckdb_list_entry>(len) };

<-- snip -->

// Ordering doesn't really seem to matter here but maybe it does?
vector.list_vector_reserve(sum_of_list_lens)?;
let mut elements_vector = vector.list_vector_get_child();
vector.list_vector_set_size(sum_of_list_lens)?;

self.elements_exporter.export(
    usize::try_from(start_offset)?,
    usize::try_from(sum_of_list_lens)?,
    &mut elements_vector,
)

Here is an example assertion failure:

---- e2e_test::vortex_scan_test::test_vortex_scan_list_of_ints stdout ----

thread 'e2e_test::vortex_scan_test::test_vortex_scan_list_of_ints' panicked at vortex-duckdb/src/e2e_test/vortex_scan_test.rs:488:10:
called `Result::unwrap()` on an `Err` value: Failed to execute query:
>   INTERNAL Error: Assertion triggered in file "/Users/connor/spiral/vortex-data/vortex-develop/target/duckdb-source-v1.4.0/duckdb-1.4.0/src/common/types/vector.cpp"
>   on line 1802: le.offset + le.length <= child_size

<-- snip -->
4        duckdb::InternalException::InternalException<char const*, int, char const*>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, char const*, int, char const*) + 80
5        duckdb::DuckDBAssertInternal(bool, char const*, char const*, int) + 124
6        duckdb::Vector::Verify(duckdb::Vector&, duckdb::SelectionVector const&, unsigned long long) + 4572
7        duckdb::Vector::Verify(unsigned long long) + 56
8        duckdb::DataChunk::Verify() + 152
9        duckdb::PipelineExecutor::EndOperator(duckdb::PhysicalOperator&, duckdb::optional_ptr<duckdb::DataChunk, true>) + 84
<-- snip -->

I double checked that the values that we write into that vector are correct (both on develop and on my branch where the list view offsets and sizes are definitely correct). So it is likely the case that we are writing correct values, but writing it in the wrong place (or at least duckdb is expecting it somewhere else).

I'd like to go into lldb and figure out what exactly le.offset and le.length are, but I wasn't able to figure out how to do that easily since a new process is spawned.

CC @0ax1 @danking

---

Also adds a test that is slightly simpler than test_vortex_scan_nested_fixed_size_list_utf8, simply removes the outer fixed-size list layer (and the other assertion error child.GetVectorType() == VectorType::FLAT_VECTOR still triggers)

---

Edit: Somewhat resolved in comments below?

Something of note that is VERY weird that happened to me more than once (but not every time):

• If I do a clean build it fails immediately.
• If I make a trivial change anywhere (add dbg!(""); somewhere) and run with only VX_DUCKDB_DEBUG, it sometimes stops failing, pretty consistently
• If I add VX_DUCKDB_ASAN and compile tests again, it starts to fail again

There seems to be something strange going on with the builds as well because in certain cases the tests will start passing again seemingly randomly, and I cannot replicate it consistently.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 111 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.16%. Comparing base (e1f043d) to head (42e2e2e).
⚠️ Report is 2 commits behind head on develop.

Files with missing lines	Patch %	Lines
vortex-duckdb/src/e2e_test/vortex_scan_test.rs	0.00%	111 Missing ⚠️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[0ax1]

Something of note that is VERY weird that happened to me more than once (but not every time):

Only explanation I can come up with is that your editor races on the build without having set VX_DUCKDB_DEBUG.
After the editor does a non-debug build, you trigger the next test run, which does not trigger a DuckDB rebuild.

In general, changing the DuckDB env vars is picked up by our vortex-duckdb/build.rs https://github.com/vortex-data/vortex/blob/develop/vortex-duckdb/build.rs#L281. I think setting the asan env var makes the build aware to rebuild. Probably the same would happen if you set VX_DUCKDB_DEBUG=0 and back to 1 again. So my hunch is that your editor somehow manages to slip through a non-debug build. I don't think the issue is in DuckDB's C++ build system.

I'll have a look and see if I can repro this.

---

Edit: We could repro this on Joe's machine. Oddly, it was linking against the debug lib though. When you run into this issue again, can you check for the output of otool -L target/duckdb-lib/libduckdb.dylib?

For debug it'll give you:

otool -L target/duckdb-lib/libduckdb.dylib
target/duckdb-lib/libduckdb.dylib:
        @rpath/libduckdb.0.0.dylib (compatibility version 0.0.0, current version 0.0.1)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 2000.63.0)
        @rpath/libclang_rt.tsan_osx_dynamic.dylib (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1356.0.0)

where .0.0 is the confirmation that we're linking against the debug build.

[connortsui20]

You're probably right with it being something to do with the editor, I tried making trivial changes with vim and no LSP and wasn't able to reproduce (tests still failed), and then when I opened zed and allowed rust-analyzer to finish building, the tests passed!

I will note though that I have my rust-analyzer set up to use a separate build directory in ./target/rust-analyzer/ so that my editor doesn't take the lock on the actual build directory when I want to do other stuff. Seems like duckdb-lib is just under target/, not under target/duckdb-lib, so my rust-analyzer is probably rebuilding DuckDB incorrectly?