Skip to content

Conversation

@john-michaelburke
Copy link
Collaborator

@john-michaelburke john-michaelburke commented Jul 17, 2021

Implements

  • Added drop for Server and ServerEndpoint(might be unnecessary). Drop for Server attempts to join the main backend thread
  • Added a timeout to the fetch_message rust -> python function to prevent locking up on join initiated by the frontend.
  • Adds an IS_RUNNING global to the Frontend receive_messages function thread in order to join after the app begins to close.
  • Added a mutex and join when frontend begins to close the app.

Concerns

  • Not actual sure if this resolved the issue there is only so much manual testing I can do. But no failures after ~30-40 tries. Might be worth setting up a one off test rig which simulates hitting the close button and checking the return code.
  • I set the timeout to 1 millisecond and it does burn cycles, might need to set this to just under whatever device max data rate is. I see on piksi 10-20kb/s not sure what the average SBP message size is but could calculate an optimal timeout in theory.

@john-michaelburke john-michaelburke force-pushed the john-michaelburke/wait_for_backend branch 8 times, most recently from 62a017f to 0c3914e Compare July 19, 2021 20:43
@john-michaelburke john-michaelburke force-pushed the john-michaelburke/wait_for_backend branch from 0c3914e to cf1c7b1 Compare July 19, 2021 20:44

if let Some(recv) = self.client_recv.take() {
drop(recv);
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also had a join here for the thread spawned in "fetch_message" but it causes an infinite hang on the Macos frontend benchmark whenever we try to kill the app although it works fine with linux/windows 😢 . Nonetheless I think I am seeing less crashes without the join. I am setting things such that the thread should end but definitely not ideal without the join.

@john-michaelburke john-michaelburke marked this pull request as ready for review July 19, 2021 21:32
struct ServerEndpoint {
server_send: Option<mpsc::Sender<Vec<u8>>>,
}
impl Drop for ServerEndpoint {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need any explicit drop implementations on the Server/SeverEndpoint type because fundamentally as long as the Server object is held by the Python code, this drop implementation isn't going to run. It order for drop to run, the Python code would need to explicitly detect the object, a better option is to just request termination, and then to do a blocking wait for termination to occur.

impl Drop for ServerEndpoint {
fn drop(&mut self) {
if let Some(sender) = self.server_send.take() {
drop(sender);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be in a helper function exposed to Python, so that Python can indicate that it's done with the server

fn drop(&mut self) {
self.is_running.set(false);

if let Some(recv) = self.client_recv.take() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to explicitly drop the receiver, the close of the sender should be reflected as an error when we attempt to receive, then we can break of the receive loop on that error condition

if !is_running.get() {
break;
}
let buf = server_recv.recv();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think all you need to do is add a timeout here, and then move the drop for ServerEndpoint into a method exposed to Python, no need for the is_running variable, just use the closure of the recv endpoint to signal that the thread should exit

}
};
match message {
m::message::Status(Ok(cv_in)) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be necessary if we just close the sender as mentioned above

recv_thread.start()
app.exec_()
data_model.status(ApplicationStates.CLOSE)
with running_lock:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be able to call something like messages_main.stop_server() then just call the recv_thread.join(), closing the sender end of the mpsc should cause the loop in fetch_messages to terminate

@john-michaelburke
Copy link
Collaborator Author

This work was moved to #93.

Closing this PR.

@john-michaelburke john-michaelburke deleted the john-michaelburke/wait_for_backend branch July 21, 2021 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants