Kaleido:TNG failure using python:3.12-alpine and python:3.12-slim-bookworm

[gvwilson]

A user reports, "I tried to run [the test program] in a container using python:3.12-alpine and python:3.12-slim-bookworm images. I installed chromium (apk add —no-cache chromium; apt update && apt install chromium). I get an error regarding "Close() was called before the browser finished opening”".

Test program:

import plotly.express as px
fig = px.scatter(px.data.iris(), x="sepal_length", y="sepal_width", color="species")
fig.write_image("figure.png", engine="kaleido")

Error message:

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/choreographer/browser.py", line 225, in _open_async
    raise BrowserFailedError("The browser seemed to close immediately after starting. Perhaps adding debug_browser=True will help.") from e
choreographer.browser.BrowserFailedError: The browser seemed to close immediately after starting. Perhaps adding debug_browser=True will help.
Traceback (most recent call last):
  File "//test.py", line 3, in <module>
    fig.write_image("figure.png", engine="kaleido")
  File "/usr/local/lib/python3.12/site-packages/plotly/basedatatypes.py", line 3835, in write_image
    return pio.write_image(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/plotly/io/_kaleido.py", line 266, in write_image
    img_data = to_image(
               ^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/plotly/io/_kaleido.py", line 143, in to_image
    img_bytes = scope.transform(
                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/kaleido/scopes/plotly.py", line 197, in transform
    img = kaleido.to_image_block(spec, Path(self._tempfile.name).absolute(), self._topojson, self._mapbox_access_token)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/kaleido/__init__.py", line 27, in to_image_block
    return asyncio.run(to_image(spec, f, topojson, mapbox_token))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/kaleido/__init__.py", line 30, in to_image
    async with Browser(headless=True) as browser:
choreographer.browser.BrowserFailedError: Close() was called before the browser finished opening- maybe it crashed?

dtdoctor output attacheddtdoctor.txt

[gvwilson]

@mschoettle has provided a container for us to try: https://github.com/mschoettle/plotly-image-export-alpine - thank you

[ayjayt]

Interesting, probably this:

[1023/201538.372535:ERROR:zygote_host_impl_linux.cc(101)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

causing chromium to shutdown immediately.

ill submit something before EOD

[vfilter]

Will this be included in the next RC?

[ayjayt]

I'm sorry I didn't see this earlier, the next RC should be released today or tomorrow and yes, I believe this issue was fixed.

In any case, all of the debugging output is being improved in this RC which will help us further support our our users.

Thanks for your patience.

[ayjayt]

Hi @vfilter

A new release candidate is available on pypi. The issue of running as root was solved, so if that was your issue and you run pip install --pre kaleido, it should be fixed.

The release candidate is currently going through a cycle of alpha testing, and includes options for debugging that will be documented to further help debug issues upon release.

Kaleido is under very activate development, I appreicate your patience.

[vfilter]

@ayjayt thank you for your kind response. What we thought is related to the original issue of this PR has actually not been solved in the latest prerelease for us. Happy to move this into a new PR if unrelated to this.

This works when running locally, but does not when running it on k8s in a container.

By the looks of it, Kaleido's init.py instantiates a Browser() class from the choreographer package, which is either looking for the BROWSER_PATH env variable, or trying to get it from this function in the system package.

If I understand this correctly, we need to have chrome installed in the docker image? If that's the case, I can simply add chrome to the Docker image and add the BROWSER_PATH env variable. I shall try that next.

BrowserFailedError: Could not find an acceptable browser. Please set environmental variable BROWSER_PATH or pass path=/path/to/browser into the Browser() constructor.

File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling
    result = func()
             ^^^^^^
File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 579, in code_to_exec
    exec(code, module.__dict__)
File "/app/app.py", line 10, in <module>
    pg.run()
File "/usr/local/lib/python3.11/site-packages/streamlit/navigation/page.py", line 303, in run
    exec(code, module.__dict__)
File "/app/simulations.py", line 424, in <module>
    summary_svg = summary_chart.to_image(format="svg", width=896, height=432, scale=1)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/plotly/basedatatypes.py", line 3772, in to_image
    return pio.to_image(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/plotly/io/_kaleido.py", line 143, in to_image
    img_bytes = scope.transform(
                ^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/kaleido/scopes/plotly.py", line 218, in transform
    img = kaleido.to_image_block(spec, Path(self._tempfile.name).absolute(), self._topojson, self._mapbox_access_token, debug=debug)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/kaleido/__init__.py", line 43, in to_image_block
    return asyncio.run(to_image(spec, f, topojson, mapbox_token, debug=debug))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/kaleido/__init__.py", line 53, in to_image
    Browser(headless=True, debug=debug, debug_browser=sys.stderr) as browser,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/choreographer/browser.py", line 106, in __init__
    raise BrowserFailedError(

Dockerfile

FROM python:3.11-slim AS build
COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv

ARG MOTHERDUCK_TOKEN=${MOTHERDUCK_TOKEN}
ENV MOTHERDUCK_TOKEN=${MOTHERDUCK_TOKEN}

ARG APP_DIR="/app/"
WORKDIR ${APP_DIR}

COPY ./src/build.py ${APP_DIR}

RUN uv pip install --system duckdb==1.1.2
RUN uv run build.py


FROM python:3.11-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv

ARG APP_DIR="/app/"
WORKDIR ${APP_DIR}

COPY ./uv.lock /app/
COPY ./pyproject.toml /app/
COPY ./src/ /app/

RUN uv pip install --system -r pyproject.toml

COPY --from=build ${APP_DIR}/powerplants.duckdb ${APP_DIR}

EXPOSE 8501

HEALTHCHECK CMD curl --fail http://localhost:8501/_stcore/health

ENTRYPOINT ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]

[vfilter]

Following up here. I've updated the Dockerfile in the following way:

FROM python:3.11-slim AS build
COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv

ARG MOTHERDUCK_TOKEN=${MOTHERDUCK_TOKEN}
ENV MOTHERDUCK_TOKEN=${MOTHERDUCK_TOKEN}

ARG APP_DIR="/app/"
RUN mkdir -p ${APP_DIR}
WORKDIR ${APP_DIR}

COPY ./src/build.py ${APP_DIR}

RUN uv pip install --system duckdb==1.1.2
RUN uv run build.py


FROM python:3.11-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv

ARG APP_DIR="/app/"
ENV BROWSER_PATH=/opt/chrome/chrome-linux64/chrome
RUN mkdir -p ${APP_DIR}
WORKDIR ${APP_DIR}

# Install dependencies
RUN apt-get update -y && apt-get install -y wget xvfb unzip jq curl

# Install Google Chrome dependencies
RUN apt-get install -y libxss1 libappindicator1 libgconf-2-4 \
    fonts-liberation libasound2 libnspr4 libnss3 libx11-xcb1 libxtst6 lsb-release xdg-utils \
    libgbm1 libnss3 libatk-bridge2.0-0 libgtk-3-0 libx11-xcb1 libxcb-dri3-0

# Fetch the latest version numbers and URLs for Chrome and ChromeDriver
RUN curl -s https://googlechromelabs.github.io/chrome-for-testing/last-known-good-versions-with-downloads.json > /tmp/versions.json

RUN CHROME_URL=$(jq -r '.channels.Stable.downloads.chrome[] | select(.platform=="linux64") | .url' /tmp/versions.json) && \
    wget -q --continue -O /tmp/chrome-linux64.zip $CHROME_URL && \
    unzip /tmp/chrome-linux64.zip -d /opt/chrome

RUN chmod +x /opt/chrome/chrome-linux64/chrome

# Clean up
RUN rm /tmp/chrome-linux64.zip /tmp/versions.json

COPY ./uv.lock /app/
COPY ./pyproject.toml /app/
COPY ./src/ /app/

RUN uv pip install --system -r pyproject.toml

COPY --from=build ${APP_DIR}/powerplants.duckdb ${APP_DIR}

EXPOSE 8501

HEALTHCHECK CMD curl --fail http://localhost:8501/_stcore/health

ENTRYPOINT ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]

However, now I'm actually getting an error similar to what this thread was originally started about:

BrowserFailedError: Close() was called before the browser finished opening- maybe it crashed?

File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling
    result = func()
             ^^^^^^
File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 579, in code_to_exec
    exec(code, module.__dict__)
File "/app/app.py", line 10, in <module>
    pg.run()
File "/usr/local/lib/python3.11/site-packages/streamlit/navigation/page.py", line 303, in run
    exec(code, module.__dict__)
File "/app/simulations.py", line 424, in <module>
    summary_svg = summary_chart.to_image(format="svg", width=896, height=432, scale=1)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/plotly/basedatatypes.py", line 3772, in to_image
    return pio.to_image(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/plotly/io/_kaleido.py", line 143, in to_image
    img_bytes = scope.transform(
                ^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/kaleido/scopes/plotly.py", line 218, in transform
    img = kaleido.to_image_block(spec, Path(self._tempfile.name).absolute(), self._topojson, self._mapbox_access_token, debug=debug)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/kaleido/__init__.py", line 43, in to_image_block
    return asyncio.run(to_image(spec, f, topojson, mapbox_token, debug=debug))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/kaleido/__init__.py", line 52, in to_image
    async with (

[mschoettle]

@vfilter Are you using kaleido==0.4.0rc3? Your code snippets are not showing the pyproject.toml where this could be seen.

[ayjayt]

If I understand this correctly, we need to have chrome installed in the docker image? If that's the case, I can simply add chrome to the Docker image and add the BROWSER_PATH env variable. I shall try that next.

This analysis was correct @vfilter, but you shouldn't necessarily need to add the BROWSER_PATH variable, as linux is pretty good about finding chrome by itself. Although I'm not sure given how you're pulling it in.

@mschoettle is also correct in that we cannot see if you are using the must updated version of kaleido, there have been a flurry of updates to deal with a wide range of bugs.

BrowserFailedErrors like the one you're receiving are very interesting to me because generally it's a configuration/platform match that we need to address- I will see later today if I can provoke it with the docker-file you supplied (give me two hours, actually).

That said, you can also also set the environmental variable KALEIDO-DEBUG to anything, and it will generate an ENORMOUS amount of logs that I can read.