[BUG] state change for long_callback can cause infinite loop

[kwikwag]

Thank you so much for developing Dash!

• pip list | grep dash

dash                          2.5.1
dash-auth                     1.4.1
dash-bootstrap-components     1.2.0

• not frontend related

Describe the bug

Consider the following setup:

app.layout = h.Div([
    h.Button('Click me', id='button'),
    h.Div('', id='data'),
])

@app.callback(
    output=Output('data', 'children'),
    inputs=[Input('button', 'n_clicks')],
    state=[State('data', 'children')]
)
def clicked(n_clicks, data):
    if n_clicks is None:
        return str(0)
    return str(int(data) + 1)

Every time the user clicks the button, the callback will be called once and count will increase. Note that state and output point to the same variable.

However, if you replace @app.callback() with @app.long_callback() you will get an infinite loop. The reason for this is that Dash.long_callback() uses the input arguments -- including the 'state' arguments -- to create a unique key to identify the pending job. When the state changes, this key changes, and the job gets resubmitted.

The above is similar but not quite my use-case: I have a long-running multi-part job. I want the first part to be calculated only once, i.e. when the user first clicks the button, but not re-calculated every time the button is hit. For this reason I share 'state' and 'output' - the result of the first calculation gets cached in a store. In my case, the result of the first calculation doesn't change upon re-invoking the job, so the callback gets invoked exactly twice.

Expected behavior

@app.long_callback() should behave the same as @app.callback(), specifically, invocations due to 'state' changes should be avoided as they are not 'inputs' changes.

Screenshots

[alexcjohnson]

Oh interesting - State values definitely should be included in the cache keys for long callbacks, but you're right that it shouldn't trigger the job again when the State value changes mid-job. @T4rk1n can you comment on whether #2039 gets this right by both of these criteria, or if this is still an issue?

[T4rk1n]

I think it is fixed, before it was triggered by the interval callback which would collect the current state and generate the cache key on that callback. Job ids/cache keys are now saved on the renderer store so it wouldn't change the job_id on state changes. On input change it match the old ids and cancel the old jobs.

[alexcjohnson]

OK great - let's leave this issue open for the moment then, and when we have a new release (very soon!) @kwikwag perhaps you can retest and see if it is indeed fixed?

[kwikwag]

OK great - let's leave this issue open for the moment then, and when we have a new release (very soon!) @kwikwag perhaps you can retest and see if it is indeed fixed?

Will do

[yuval-gvs]

It seems to work well now, thanks for the fix!
One important comment, however: long_callback() seems to be deprecated now, but the documentation is not up to date, so it's unclear how my code should be modified. In fact the deprecation notice instructs the user to add long=True when it in fact should be background=True (long=True) results in an error.

P.S. I modified the above code to work slightly clearer by adding n_clicks to the output, given that when clicking the button multiple times previous callbacks get cancelled.

@app.callback(
    background=True,
    output=Output('data', 'children'),
    inputs=[Input('button', 'n_clicks')],
    state=[State('data', 'children')]
)
def clicked(n_clicks, data):
    if n_clicks is None:
        return json.dumps(dict(data=0, n_clicks=n_clicks))
    
    state = json.loads(data)
    return json.dumps(dict(data=state['data'] + 1, n_clicks=n_clicks))