Bad response when inserting rows. `DB::Exception: Cannot read all data. Bytes read: N. Bytes expected: N`

[QazCetelic]

Describe the bug

Insert fails with error DB::Exception: Cannot read all data. Bytes read: N. Bytes expected: N. No rows are inserted into the table.

Steps to reproduce

1. Create table:

create table category_counts
(
    subject  String,
    category String,
    time     DATETIME32,
    count    UInt32
)
    engine = ReplacingMergeTree
    order by (subject, category, time);

2. Try to attempt to insert data
   Sample data below

clickhouse = { version = "0.13.3", features = ["chrono"] }

let mut category_count_insert = ch_conn.insert("category_counts")?;

#[derive(Debug, Row, Serialize)]
pub struct CategoryCountRow {
    pub subject: String,
    pub category: String,
    pub time: NaiveDateTime,
    pub count: u32,
}

let counts = extract_category_counts(&doc)?;
for category_count in counts {
    let category_subject = category_count.subject;
    for (key, count) in category_count.values {
        let row = CategoryCountRow {
            subject: category_subject.clone(),
            category: key.clone(),
            time: now.clone().naive_utc(),
            count: count,
        };
        category_count_insert.write(&row).await.expect("Failed to insert category_count");
    }
}
category_count_insert.end().await?;

3. See error bad response: Code: 33. DB::Exception: Cannot read all data. Bytes read: 107. Bytes expected: 111.: (at row 151) : While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 25.4.2.31 (official build))

Expected behaviour

It should insert the value (or throw a clearer error if the code is wrong)

Code example

See above

Error log

3. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000f383629
4. DB::SerializationString::deserializeBinary(DB::IColumn&, DB::ReadBuffer&, DB::FormatSettings const&) const @ 0x000000001254d2dc
5. DB::BinaryRowInputFormat<false>::readRow(std::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn>>>&, DB::RowReadExtension&) @ 0x0000000014a38884
6. DB::IRowInputFormat::read() @ 0x0000000014a1a02a
7. DB::IInputFormat::generate() @ 0x00000000149b85d6
8. DB::ISource::tryGenerate() @ 0x00000000149914de
9. DB::ISource::work() @ 0x00000000149910e7
10. DB::ExecutionThreadContext::executeTask() @ 0x00000000149ad496
11. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x00000000149a0ea3
12. DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001499f9a5
13. DB::CompletedPipelineExecutor::execute() @ 0x000000001499e1ef
14. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&, String const&, std::shared_ptr<DB::Context const> const&, std::optional<DB::FormatSettings> const&)>) @ 0x000000001359bf34
15. DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional<DB::CurrentThread::QueryScope>&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x0000000014861401
16. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x0000000014865969
17. DB::HTTPServerConnection::run() @ 0x000000001491bb2a
18. Poco::Net::TCPServerConnection::start() @ 0x00000000180162e7
19. Poco::Net::TCPServerDispatcher::run() @ 0x0000000018016739
20. Poco::PooledThread::run() @ 0x0000000017fe1a5b
21. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000017fdff3d
22. ? @ 0x00007f2ed6270ac3
23. ? @ 0x00007f2ed6301a04

2025.08.01 07:42:20.482001 [ 84 ] {388b11df-d1d2-42c0-9c20-0de378ea4075} <Error> DynamicQueryHandler: Code: 33. DB::Exception: Cannot read all data. Bytes read: 107. Bytes expected: 111.: (at row 151)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA), Stack trace (when copying this message, always include the lines below):

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000f2bff3b
1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000009c74bec
2. DB::Exception::Exception<unsigned long&, String>(int, FormatStringHelperImpl<std::type_identity<unsigned long&>::type, std::type_identity<String>::type>, unsigned long&, String&&) @ 0x000000000b4ea70b
3. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000f383629
4. DB::SerializationString::deserializeBinary(DB::IColumn&, DB::ReadBuffer&, DB::FormatSettings const&) const @ 0x000000001254d2dc
5. DB::BinaryRowInputFormat<false>::readRow(std::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn>>>&, DB::RowReadExtension&) @ 0x0000000014a38884
6. DB::IRowInputFormat::read() @ 0x0000000014a1a02a
7. DB::IInputFormat::generate() @ 0x00000000149b85d6
8. DB::ISource::tryGenerate() @ 0x00000000149914de
9. DB::ISource::work() @ 0x00000000149910e7
10. DB::ExecutionThreadContext::executeTask() @ 0x00000000149ad496
11. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x00000000149a0ea3
12. DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000001499f9a5
13. DB::CompletedPipelineExecutor::execute() @ 0x000000001499e1ef
14. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&, String const&, std::shared_ptr<DB::Context const> const&, std::optional<DB::FormatSettings> const&)>) @ 0x000000001359bf34
15. DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional<DB::CurrentThread::QueryScope>&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x0000000014861401
16. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x0000000014865969
17. DB::HTTPServerConnection::run() @ 0x000000001491bb2a
18. Poco::Net::TCPServerConnection::start() @ 0x00000000180162e7
19. Poco::Net::TCPServerDispatcher::run() @ 0x0000000018016739
20. Poco::PooledThread::run() @ 0x0000000017fe1a5b
21. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000017fdff3d
22. ? @ 0x00007f2ed6270ac3
23. ? @ 0x00007f2ed6301a04
 (version 25.4.2.31 (official build))

Query log

n/a

Configuration

Environment

• Client version: clickhouse-rs v0.13.3 with chrono feature flag
• Server version: 25.4.2.31
• OS: Debian 6.1.140-1 (2025-05-22) x86_64 GNU/Linux

ClickHouse server

• ClickHouse Server version:
• ClickHouse Server non-default settings, if any:
• CREATE TABLE statements for tables involved:
• Sample data: category_counts.json

[pravic]

+1, also happens.

[mshustov]

should be closed by #244
Could you test rbwnat-insert branch?

and just in case, I don't know what NaiveDateTime but you can use this example for reference https://github.com/ClickHouse/clickhouse-rs/blob/main/examples/data_types_derive_simple.rs#L136-L137

[QazCetelic]

TLDR: rbwnat-insert fixes the issue

It's the DateTime variant of the NaiveDate of your example. I tried using DateTime<Utc> too but that didn't solve it. The rbwnat-insert branch #244 produced another error:

thread 'tokio-runtime-worker' panicked at /home/user/.cargo/git/checkouts/clickhouse-rs-f71a5193e2df2922/170a545/src/rowbinary/validation.rs:98:21:
While processing column CategoryCountCH.time: attempting to (de)serialize ClickHouse type DateTime as &str which is not compatible
stack backtrace:
   0: __rustc::rust_begin_unwind

Struct in question:

#[derive(Debug, Row, Serialize, Deserialize)]
pub struct CategoryCountCH {
    pub subject: String,
    pub category: String,
    pub time: DateTime<Utc>,
    pub count: u32,
}

EDIT: I had to explicitly add #[serde(with = "clickhouse::serde::chrono::datetime")] to the struct. That solved the issue.

#[derive(Debug, Row, Serialize, Deserialize)]
pub struct CategoryCountCH {
    pub subject: String,
    pub category: String,
    #[serde(with = "clickhouse::serde::chrono::datetime")] // <- This was missing
    pub time: DateTime<Utc>,
    pub count: u32,
}

Also, moving the data from Postgres to ClickHouse reduced the size from 5MB to 30KB 😄

[slvrtrn]

I will think about how to make errors with missing serde attributes or custom wrappers even clearer. Perhaps it could be implemented in the scope of #236.

While processing column CategoryCountCH.time: attempting to (de)serialize ClickHouse type DateTime as &str which is not compatible

Of course, this is already a lot better than just Cannot read all data (insert) or NotEnoughData (select), but still does not give a valid pointer to resolve the issue.