Question: How to capture the stdout of the python interpreter

[sebschmi]

Please see the following question I asked on stackoverflow: https://stackoverflow.com/questions/69203700/how-to-capture-the-stdout-of-the-python-interpreter-inside-the-rust-program-when

How can I achieve this with pyo3?

The use case is to envelope the output with the logger used by the Rust application.

[birkenfeld]

Same as in Python, you replace sys.stdout (and maybe sys.stderr too) with a custom object whose write() method does what you want.

[birkenfeld]

Ok, now that I can actually run and test something, here's an example:

use pyo3::prelude::*;

#[pyclass]
struct LoggingStdout;

#[pymethods]
impl LoggingStdout {
    fn write(&self, data: &str) {
        println!("stdout from python: {:?}", data);
    }
}

fn main() -> PyResult<()> {
    Python::with_gil(|py| {
        let sys = py.import("sys")?;
        sys.setattr("stdout", LoggingStdout.into_py(py))?;
        py.run("print('some', 'output')", None, None)?;
        Ok(())
    })
}

The output will be:

stdout from python: "some"
stdout from python: " "
stdout from python: "output"
stdout from python: "\n"

As you can see, depending on how the output is written to sys.stdout by the Python side, you may not get full lines in one go - you can accept that, add a line buffer on the Rust side, or make sure that the Python side uses write with full lines.

[sebschmi]

Thank you very much, this looks simple!

[aminya]

Here's another approach if you'd like to collect the logs

use std::sync::RwLock;

use anyhow::Error;
use pyo3::prelude::*;

pub(crate) static PY_LOGS: RwLock<Vec<String>> = RwLock::new(Vec::new());

pub(crate) fn clear_py_logs() {
    PY_LOGS.write().unwrap().clear();
}

#[pyclass]
#[derive(Clone)]
struct LoggingStdout;

#[pymethods]
impl LoggingStdout {
    fn write(&self, data: &str) {
        PY_LOGS.write().unwrap().push(data.to_string());
    }
}

pub(crate) fn attach_logger(py: Python<'_>) -> Result<(), Error> {
    let sys = py.import("sys")?;
    let logger = LoggingStdout {};
    sys.setattr("stdout", logger.clone())?;
    sys.setattr("stderr", logger)?;

    return Ok(());
}