Call-mods Error: thread 'main' panicked at 'called `Option::unwrap()` on a `None` value'

[OOM-Killer415]

Hi,

I’ve been trying to run the call-mods module on a .bam file to ‘filter out’ reads containing lower methylation probabilities prior to visualization yet keep encountering the following error when running this command :
modkit call-mods 221110_ys18_Me_Naked_R10_all_6mAcalled.sorted.bam filt.bam --mod-threshold A:0.8 --mod-threshold a:0.9

thread 'main' panicked at 'called Option::unwrap() on a None value', /home/boegersome/.cargo/registry/src/gitproxy.zycloud.tk-1ecc6299db9ec823/rust-htslib-0.40.2/src/bam/header.rs:95:27

I’m admittedly very new to bioinformatics but it's unclear what the error could be from this message. I’ve also noticed that the ‘call-mods’ command works fine when running on a .bam file with little to no methylation signal, but always errors when running on a sufficiently modified sample.

[ArtRand]

Hello @OOM-Killer415,
Could you reproduce the error with the environment variable RUST_BACKTRACE=full and post the traceback, for example:

$ RUST_BACKTRACE=full modkit ...

This error is coming from the bam-parsing library (rust-htslib), looks like it's failing to parse the records in the header. Could you post and/or inspect the headers from a BAM that works and one that doesn't? (samtools view -H <bam> will print out just the header)

[OOM-Killer415]

Thanks for the reply @ArtRand !

Here’s the full trace:

thread 'main' panicked at 'called Option::unwrap() on a None value', /home/boegersome/.cargo/registry/src/gitproxy.zycloud.tk-1ecc6299db9ec823/rust-htslib-0.40.2/src/bam/header.rs:95:27
stack backtrace:
0: 0x55c472dd4143 - std::backtrace_rs::backtrace::libunwind::trace::he0156af2558114c2
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
1: 0x55c472dd4143 - std::backtrace_rs::backtrace::trace_unsynchronized::h1e2672bcf5105eb5
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x55c472dd4143 - std::sys_common::backtrace::_print_fmt::haa919a14d8d859ec
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/sys_common/backtrace.rs:66:5
3: 0x55c472dd4143 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hf15e8f9e6884dd5f
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/sys_common/backtrace.rs:45:22
4: 0x55c472df11fc - core::fmt::write::he42254d9e3c27115
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/core/src/fmt/mod.rs:1202:17
5: 0x55c472dbf1f5 - std::io::Write::write_fmt::hfb37e0ab3a125c66
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/io/mod.rs:1679:15
6: 0x55c472dc1774 - std::sys_common::backtrace::_print::h8078bdb0e2e92b53
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/sys_common/backtrace.rs:48:5
7: 0x55c472dc1774 - std::sys_common::backtrace::print::h09fd65486fb9c4f7
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/sys_common/backtrace.rs:35:9
8: 0x55c472dc1774 - std::panicking::default_hook::{{closure}}::hb89b98c578903f40
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panicking.rs:295:22
9: 0x55c472dc13b4 - std::panicking::default_hook::h27aa44be03b01ac8
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panicking.rs:314:9
10: 0x55c472dc1d53 - std::panicking::rust_panic_with_hook::he7013d2ea706cde0
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panicking.rs:698:17
11: 0x55c472dd4709 - std::panicking::begin_panic_handler::{{closure}}::hd66fe345adc7bed2
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panicking.rs:586:13
12: 0x55c472dd4284 - std::sys_common::backtrace::__rust_end_short_backtrace::ha2b47ffe9f27aed9
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/sys_common/backtrace.rs:138:18
13: 0x55c472dc1a42 - rust_begin_unwind
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panicking.rs:584:5
14: 0x55c4728cc053 - core::panicking::panic_fmt::hbfca70bd08ba6212
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/core/src/panicking.rs:142:14
15: 0x55c4728cbf1d - core::panicking::panic::hc1bbfa9fa05e63d3
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/core/src/panicking.rs:48:5
16: 0x55c472a1a73d - rust_htslib::bam::header::Header::to_hashmap::h210b72452ef9435e
17: 0x55c4729e0d25 - mod_kit::util::add_modkit_pg_records::hf57c42d61222df73
18: 0x55c4728dfabf - mod_kit::commands::CallMods::run::h451572734f8d00e8
19: 0x55c4728d6454 - mod_kit::commands::Commands::run::h2e6d4d0c96d66e35
20: 0x55c4728d1a45 - modkit::main::h40fded74d8b5c29f
21: 0x55c4728d1fa6 - std::sys_common::backtrace::__rust_begin_short_backtrace::hcbd872a0139785e5
22: 0x55c4728cde25 - std::rt::lang_start::{{closure}}::h3039ac1eaa34d45f
23: 0x55c472dbe628 - core::ops::function::impls::<impl core::ops::function::FnOnce for &F>::call_once::h62af992155415807
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/core/src/ops/function.rs:283:13
24: 0x55c472dbe628 - std::panicking::try::do_call::hcfafbbba7d4f6a6c
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panicking.rs:492:40
25: 0x55c472dbe628 - std::panicking::try::h7ee1bfcad42abe9b
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panicking.rs:456:19
26: 0x55c472dbe628 - std::panic::catch_unwind::h009aa132eb8dd7d2
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panic.rs:137:14
27: 0x55c472dbe628 - std::rt::lang_start_internal::{{closure}}::h93ae259af980c7d0
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/rt.rs:148:48
28: 0x55c472dbe628 - std::panicking::try::do_call::ha0627c997265a210
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panicking.rs:492:40
29: 0x55c472dbe628 - std::panicking::try::h64a0afc1377cc785
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panicking.rs:456:19
30: 0x55c472dbe628 - std::panic::catch_unwind::h1c61ea510b397b89
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/panic.rs:137:14
31: 0x55c472dbe628 - std::rt::lang_start_internal::ha75927e1903320fe
at /build/rustc-Oic09u/rustc-1.65.0+dfsg0ubuntu1/library/std/src/rt.rs:148:20
32: 0x55c4728d1f98 - main
33: 0x7fcc04629d90 - __libc_start_call_main
at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
34: 0x7fcc04629e40 - __libc_start_main_impl
at ./csu/../csu/libc-start.c:392:3
35: 0x55c4728cc285 - _start
36: 0x0 -

I also took a look at a few headers from working and not-working .bams – here are a few examples:

.bam that worked:

@hd VN:1.6 SO:coordinate
@pg ID:basecaller PN:dorado VN:0.3.0+88df11b+dirty CL:dorado basecaller /mnt/hdd/nanopore2/dorado_0.3.0/dorado-0.3.0-linux-x64/bin/dna_r10.4.1_e8.2_400bps_sup@v3.5.2 /media/nanopore/Raw_data/221119_Mock_Me_HMWNaked_r10/no_sample/20221119_1210_MN20223_FAV42347_ae64e5f1/fast5 --modified-bases-models /mnt/hdd/nanopore2/dorado_0.3.0/dorado-0.3.0-linux-x64/bin/dna_r10.4.1_e8.2_400bps_sup@v4.2.0_6mA@v2 --device cuda:0 --reference /media/nanopore/genome/sacCer3.fa
@pg ID:samtools PN:samtools PP:basecaller VN:1.13 CL:samtools sort -o 221119_ys18_Mock_Me_Naked_r10_6mAcalled.sorted.bam 221119_ys18_Mock_Me_Naked_r10_6mAcalled.bam
@pg ID:samtools.1 PN:samtools PP:samtools VN:1.13 CL:samtools view -H 221119_ys18_Mock_Me_Naked_r10_6mAcalled.sorted.bam
@sq SN:chrI LN:230218
@sq SN:chrII LN:813184
@sq SN:chrIII LN:316620
@sq SN:chrIV LN:1531933
@sq SN:chrIX LN:439888
@sq SN:chrV LN:576874
@sq SN:chrVI LN:270161
@sq SN:chrVII LN:1090940
@sq SN:chrVIII LN:562643
@sq SN:chrX LN:745751
@sq SN:chrXI LN:666816
@sq SN:chrXII LN:1078177
@sq SN:chrXIII LN:924431
@sq SN:chrXIV LN:784333
@sq SN:chrXV LN:1091291
@sq SN:chrXVI LN:948066
@sq SN:chrM LN:85779

.bam that errored:

@hd VN:1.6 SO:coordinate
@pg ID:basecaller PN:dorado VN:0.3.0+88df11b+dirty CL:dorado basecaller /mnt/hdd/nanopore2/dorado_0.3.0/dorado-0.3.0-linux-x64/bin/dna_r10.4.1_e8.2_400bps_sup@v3.5.2 /media/nanopore/Raw_data/221110_ys18_Me_Naked_R10_all/pod5/ --modified-bases-models /mnt/hdd/nanopore2/dorado_0.3.0/dorado-0.3.0-linux-x64/bin/dna_r10.4.1_e8.2_400bps_sup@v4.2.0_6mA@v2 --device cuda:0 --reference /media/nanopore/genome/sacCer3.fa -I
@pg ID:samtools PN:samtools PP:basecaller VN:1.13 CL:samtools sort -o 221110_ys18_Me_Naked_R10_all_6mAcalled.sorted.bam 221110_ys18_Me_Naked_R10_all_6mAcalled.bam
@pg ID:samtools.1 PN:samtools PP:samtools VN:1.13 CL:samtools view -H 221110_ys18_Me_Naked_R10_all_6mAcalled.sorted.bam
@rg ID:319f86a76c843af447039f7fb442a71ad95a924a_dna_r10.4.1_e8.2_400bps_sup@v3.5.2 PU:FAV43215 PM: DT:2022-11-11T00:04:19.698+00:00 PL:ONT DS:basecall_model=dna_r10.4.1_e8.2_400bps_sup@v3.5.2 runid=319f86a76c843af447039f7fb442a71ad95a924a LB:no_sample SM:no_sample
@rg ID:98f8158941feeafca0d9a3a2fe46d8d4c3b2d070_dna_r10.4.1_e8.2_400bps_sup@v3.5.2 PU:FAV43215 PM: DT:2022-11-09T23:04:02.612+00:00 PL:ONT DS:basecall_model=dna_r10.4.1_e8.2_400bps_sup@v3.5.2 runid=98f8158941feeafca0d9a3a2fe46d8d4c3b2d070 LB:no_sample SM:no_sample
@rg ID:72f2a08aadadd1499d2e4d940132fcd063b86c1a_dna_r10.4.1_e8.2_400bps_sup@v3.5.2 PU:FAV43215 PM: DT:2022-11-09T00:41:45.299+00:00 PL:ONT DS:basecall_model=dna_r10.4.1_e8.2_400bps_sup@v3.5.2 runid=72f2a08aadadd1499d2e4d940132fcd063b86c1a LB:no_sample SM:no_sample
@sq SN:chrI LN:230218
@sq SN:chrII LN:813184
@sq SN:chrIII LN:316620
@sq SN:chrIV LN:1531933
@sq SN:chrIX LN:439888
@sq SN:chrV LN:576874
@sq SN:chrVI LN:270161
@sq SN:chrVII LN:1090940
@sq SN:chrVIII LN:562643
@sq SN:chrX LN:745751
@sq SN:chrXI LN:666816
@sq SN:chrXII LN:1078177
@sq SN:chrXIII LN:924431
@sq SN:chrXIV LN:784333
@sq SN:chrXV LN:1091291
@sq SN:chrXVI LN:948066
@sq SN:chrM LN:85779

[ArtRand]

@OOM-Killer415
Are you sure these headers are correct? Usually the record type code (e.g. @SQ) are capital letters. Just double checking. If it's not too hard to attach a subset of a failed BAM I can get to the bottom of this. I'd be keen to get an example file so I can fix the dependency.

[OOM-Killer415]

Its likely a copy-paste issue. When viewing the headers with samtools I see capital type codes.

Here's the subset of a failed BAM:

error.subset1.bam.gz.gz

[ArtRand]

Yup, I didn't realize that github mardown would be so "friendly" :)

The problem is that the PM tag in the RG records is missing a value (I didn't spot this before). If you fix or remove them, that should fix the problem. I don't like how this causes a panic, however, so I'm going to take care of this in the next release. Thanks for bringing this to my attention!

[ArtRand]

@OOM-Killer415 version 0.1.9 shouldn't have this problem.