Unknown log stream results in a backtrace

[vytas7]

Following the Docker Quickstart, I forgot to create a log stream.

When trying to ingest some logs, I would expect a response along the lines of HTTP 404, but instead the HTTP worker panics:

thread 'actix-rt|system:0|arbiter:0' panicked at 'called `Result::unwrap()` on an `Err` value: StreamMetaNotFound("demo")', server/src/handlers/http/ingest.rs:99:37
stack backtrace:
   0: rust_begin_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panicking.rs:64:14
   2: core::result::unwrap_failed
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/result.rs:1791:5
   3: parseable::handlers::http::ingest::push_logs::{{closure}}
   4: actix_web::handler::handler_service::{{closure}}::{{closure}}
   5: <actix_web::resource::Resource<T> as actix_web::service::HttpServiceFactory>::register::{{closure}}::{{closure}}
   6: <actix_web::scope::Scope<T> as actix_web::service::HttpServiceFactory>::register::{{closure}}::{{closure}}
   7: <actix_web::scope::Scope<T> as actix_web::service::HttpServiceFactory>::register::{{closure}}::{{closure}}
   8: <actix_web_httpauth::middleware::AuthenticationMiddleware<S,F,T> as actix_service::Service<actix_web::service::ServiceRequest>>::call::{{closure}}
   9: <actix_web::scope::Scope<T> as actix_web::service::HttpServiceFactory>::register::{{closure}}::{{closure}}
  10: <actix_web_prometheus::MetricsResponse<S,B> as core::future::future::Future>::poll
  11: <actix_web::middleware::logger::LoggerResponse<S,B> as core::future::future::Future>::poll
  12: <actix_web::middleware::compress::CompressResponse<S,B> as core::future::future::Future>::poll
  13: <actix_cors::middleware::CorsMiddleware<S> as actix_service::Service<actix_web::service::ServiceRequest>>::call::{{closure}}
  14: actix_http::h1::dispatcher::InnerDispatcher<T,S,B,X,U>::poll_response
  15: <actix_http::h1::dispatcher::Dispatcher<T,S,B,X,U> as core::future::future::Future>::poll
  16: <actix_http::service::HttpServiceHandlerResponse<T,S,B,X,U> as core::future::future::Future>::poll
  17: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
  18: tokio::runtime::task::core::Core<T,S>::poll
  19: tokio::runtime::task::harness::Harness<T,S>::poll
  20: tokio::task::local::LocalSet::tick
  21: std::thread::local::LocalKey<T>::with
  22: <core::pin::Pin<P> as core::future::future::Future>::poll
  23: tokio::runtime::scheduler::current_thread::Context::enter
  24: tokio::macros::scoped_tls::ScopedKey<T>::set
  25: tokio::runtime::runtime::Runtime::block_on

[nitisht]

Thanks we’ll take a look soon. How are you trying to ingest logs? I mean is there a client agent here or simple HTTP call from curl etc, either way - details will be helpful

[vytas7]

Hi @nitisht, I believe what I was doing was basically exactly the same sequence as the Docker Quick Start (except problems with mounting volumes, got the same OS Error 13 as mentioned on another closed issue), just I forgot to issue a PUT to create a log stream, but jumped directly onto the following POST.

Edit: like this:

docker run --rm -it -p 8000:8000 -e RUST_BACKTRACE=1 parseable/parseable:latest parseable local-store 
<...>
thread 'actix-rt|system:0|arbiter:0' panicked at 'called `Result::unwrap()` on an `Err` value: StreamMetaNotFound("demo")', server/src/handlers/http/ingest.rs:99:37
stack backtrace:
   0: rust_begin_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panicking.rs:64:14
   2: core::result::unwrap_failed
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/result.rs:1791:5
   3: parseable::handlers::http::ingest::push_logs::{{closure}}
   4: actix_web::handler::handler_service::{{closure}}::{{closure}}
   5: <actix_web::resource::Resource<T> as actix_web::service::HttpServiceFactory>::register::{{closure}}::{{closure}}
   6: <actix_web::scope::Scope<T> as actix_web::service::HttpServiceFactory>::register::{{closure}}::{{closure}}
   7: <actix_web::scope::Scope<T> as actix_web::service::HttpServiceFactory>::register::{{closure}}::{{closure}}
   8: <actix_web_httpauth::middleware::AuthenticationMiddleware<S,F,T> as actix_service::Service<actix_web::service::ServiceRequest>>::call::{{closure}}
   9: <actix_web::scope::Scope<T> as actix_web::service::HttpServiceFactory>::register::{{closure}}::{{closure}}
  10: <actix_web_prometheus::MetricsResponse<S,B> as core::future::future::Future>::poll
  11: <actix_web::middleware::logger::LoggerResponse<S,B> as core::future::future::Future>::poll
  12: <actix_web::middleware::compress::CompressResponse<S,B> as core::future::future::Future>::poll
  13: <actix_cors::middleware::CorsMiddleware<S> as actix_service::Service<actix_web::service::ServiceRequest>>::call::{{closure}}
  14: actix_http::h1::dispatcher::InnerDispatcher<T,S,B,X,U>::poll_response
  15: <actix_http::h1::dispatcher::Dispatcher<T,S,B,X,U> as core::future::future::Future>::poll
  16: <actix_http::service::HttpServiceHandlerResponse<T,S,B,X,U> as core::future::future::Future>::poll
  17: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
  18: tokio::runtime::task::core::Core<T,S>::poll
  19: tokio::runtime::task::harness::Harness<T,S>::poll
  20: tokio::task::local::LocalSet::tick
  21: std::thread::local::LocalKey<T>::with
  22: <core::pin::Pin<P> as core::future::future::Future>::poll
  23: tokio::runtime::scheduler::current_thread::Context::enter
  24: tokio::macros::scoped_tls::ScopedKey<T>::set
  25: tokio::runtime::runtime::Runtime::block_on

On the localhost

curl --location --request POST 'http://localhost:8000/api/v1/logstream/demo' --header 'X-P-META-meta1: value1' --header 'X-P-TAGS-tag1: value1' --header 'Authorization: Basic YWRtaW46YWRtaW4=' --header 'Content-Type: application/json' --data-raw '[
    {
        "id": "434a5f5e-2f5f-11ed-a261-0242ac120002",
        "datetime": "2023-01-05T07:20:50.52Z",
        "host": "153.10.110.81", 
        "user-identifier": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) Firefox/64.0", 
        "method": "PUT", 
        "status": 500, 
        "referrer": "http://www.google.com/"
    }
]'
curl: (52) Empty reply from server

[nitisht]

Thanks @vytas7 let us check and get back

[nitisht]

The issue with panic is fixed with #388 . Thanks for reporting @vytas7 . This was a regression introduced in last release. We're adding an integration test in https://github.com/parseablehq/quest to make sure we catch this now on.

BTW, there is another endpoint api/v1/ingest where Parseable doesn't expect the stream to exist, instead you can send the stream name in header X-P-Stream (and the stream is created if doesn't exist). Refer the doc here: https://www.parseable.io/docs/api/log-ingestion

Are you still facing the issue with #377 (comment) ? I changed the readme to make it simpler: https://github.com/parseablehq/parseable#white_check_mark-getting-started. Please let me know and we'll see how to address that.

[vytas7]

Thanks for getting back @nitisht, yes, I still have the same issue. I don't think using the $HOME directory would make any difference either.

I've tried using /tmp as in the previous rendition of the docs, but after getting the error, I modified the permissions of /tmp/parseable to a+rw, then it works.
After storing some data, this is what has been created when listing from the host system

total 0
drwxr-xr-x 3 65532 65532 60 maj  1 19:14 'date=2023-05-01'

Basically everything is stored under an arbitrary UID (65532), which in general doesn't translate well to mapping in directory volumes from the host. I think the containers should be updated to simply use root (probably not the best practice for other reasons...), or start with root to set up the correct permissions, and then drop privileges, or simply document how to use Docker volume containers.

[nitisht]

Thanks @vytas7 for the detailed response. Let me try this on a Linux machine and document how to use the Docker volumes with containers properly.