bokeh callback key error

[havok2063]

There is some kind of KeyError that occurs on a Bokeh plot callback. This happens when you create a plot or two. The error does not crash the app but it happens every time and outputs a loong traceback. There are also no such comm messages that I'm not sure where they are coming from or if they're related.

Exception ignored in: <function Widget.__del__ at 0x16f519bc0>
Traceback (most recent call last):
  File "/Users/brian/anaconda3/envs/vspy12/lib/python3.12/site-packages/ipywidgets/widgets/widget.py", line 516, in __del__
    self.close()
  File "/Users/brian/anaconda3/envs/vspy12/lib/python3.12/site-packages/jupyter_bokeh/widgets.py", line 92, in close
    self._document.remove_on_change(self)
  File "/Users/brian/anaconda3/envs/vspy12/lib/python3.12/site-packages/bokeh/document/document.py", line 592, in remove_on_change
    self.callbacks.remove_on_change(*callbacks)
  File "/Users/brian/anaconda3/envs/vspy12/lib/python3.12/site-packages/bokeh/document/callbacks.py", line 331, in remove_on_change
    del self._change_callbacks[callback]
        ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: BokehModel(render_bundle={'docs_json': {'74e9a707-b102-477c-b7ea-df83d3935c93': {'version': '3.6.3', 'title': 'Bokeh Application', 'roots': [{'type': 'object', 'name': 'Plot', 'id': 'p1121', 'attributes': {'name': '0', 'subscribed_events': {'type': 'set', 'entries': ['reset']}, 

Full traceback

No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: 40f6c61ea7fd453786458fe239e7a868
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
No such comm: a65e7f69bd424c879b0deb94ef3705ae
Exception ignored in: <function Widget.__del__ at 0x16f519bc0>
Traceback (most recent call last):
  File "/Users/brian/anaconda3/envs/vspy12/lib/python3.12/site-packages/ipywidgets/widgets/widget.py", line 516, in __del__
    self.close()
  File "/Users/brian/anaconda3/envs/vspy12/lib/python3.12/site-packages/jupyter_bokeh/widgets.py", line 92, in close
    self._document.remove_on_change(self)
  File "/Users/brian/anaconda3/envs/vspy12/lib/python3.12/site-packages/bokeh/document/document.py", line 592, in remove_on_change
    self.callbacks.remove_on_change(*callbacks)
  File "/Users/brian/anaconda3/envs/vspy12/lib/python3.12/site-packages/bokeh/document/callbacks.py", line 331, in remove_on_change
    del self._change_callbacks[callback]
        ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: BokehModel(render_bundle={'docs_json': {'74e9a707-b102-477c-b7ea-df83d3935c93': {'version': '3.6.3', 'title': 'Bokeh Application', 'roots': [{'type': 'object', 'name': 'Plot', 'id': 'p1121', 'attributes': {'name': '0', 'subscribed_events': {'type': 'set', 'entries': ['reset']}, 'context_menu': {'type': 'object', 'name': 'Menu', 'id': 'p1118', 'attributes': {'styles': {'type': 'map', 'entries': [('color', 'black'), ('font-size', '16px')]}, 'items': [{'type': 'object', 'name': 'ActionItem', 'id': 'p1128', 'attributes': {'name': 'menu-clear', 'label': 'Clear selection', 'disabled': True, 'action': {'type': 'object', 'name': 'CustomJS', 'id': 'p1177', 'attributes': {'args': {'type': 'map', 'entries': [('source', {'type': 'object', 'name': 'ColumnDataSource', 'id': 'p1115', 'attributes': {'selected': {'type': 'object', 'name': 'Selection', 'id': 'p1116', 'attributes': {'indices': [], 'line_indices': []}}, 'selection_policy': {'type': 'object', 'name': 'UnionRenderers', 'id': 'p1117'}, 'data': {'type': 'map', 'entries': [('centers', {'type': 'ndarray', 'array': {'type': 'bytes', 'data': 'ZsbZSxkAIkATx8ITCwA7QHpVzMAEgEZAake39wOAT0CtHFGXAUBUQKWVxjIBwFhAnQ48zgBAXUDKw9g0AOBgQEaAkwIAIGNAwjxO0P9fZUA++Qie/59nQLq1w2v/32lANnJ+Of8fbECyLjkH/19uQJf1eWr/T3BA1VNXUf9vcUATsjQ4/49yQFEQEh//r3NAj27vBf/PdEDNzMzs/u91QA=='}, 'shape': [20], 'dtype': 'float64', 'order': 'little'}), ('left', {'type': 'ndarray', 'array': {'type': 'bytes', 'data': 'AAAAYHPlKj9mFiDZCwAyQGY+wx8FAEJAmXH2UgQAS0Bm0hTDAQBSQABsrlwBgFZAmQVI9gAAW0Ayn+GPAIBfQGacvRQAAGJAM2mK4f8/ZEAANleu/39mQMwCJHv/v2hAmc/wR///akBmnL0U/z9tQDJpiuH+f29AAJsrV//fcEBmAZI9//9xQMxn+CP/H3NAM85eCv8/dECZNMXw/l91QA=='}, 'shape': [20], 'dtype': 'float64', 'order': 'little'}), ('right', {'type': 'ndarray', 'array': {'type': 'bytes', 'data': 'ZhYg2QsAMkBmPsMfBQBCQJlx9lIEAEtAZtIUwwEAUkAAbK5cAYBWQJkFSPYAAFtAMp/hjwCAX0BmnL0UAABiQDNpiuH/P2RAADZXrv9/ZkDMAiR7/79oQJnP8Ef//2pAZpy9FP8/bUAyaYrh/n9vQACbK1f/33BAZgGSPf//cUDMZ/gj/x9zQDPOXgr/P3RAmTTF8P5fdUAAmyvX/n92QA=='}, 'shape': [20], 'dtype': 'float64', 'order': 'little'}), ('y', {'type': 'ndarray', 'array': {'type': 'bytes', 'data': 'daoAAEOVAACRnwAACvYAANVHAQCFjwEAPFMBAOUpAQB77AAAF9UAADylAAAptQAAhskAAEshAQA9lgEAmi4CAJvAAQAi2wAAAbAAANyVAAA='}, 'shape': [20], 'dtype': 'int32', 'order': 'little'})]}}})]}, 'code': '\n             source.selected.indices = [];\n             source.selected.indices.change.emit();\n             '}}}}, {'type': 'object', 'name': 'ActionItem', 'id': 'p1130', 'attributes': {'name': 'reset-view', 'label': 'Reset plot', 'action': {'type': 'object', 'name': 'CustomJS', 'id': 'p1129', 'attributes': {'args': {'type': 'map', 'entries': [('p', {'id': 'p1121'})]}, 'code': 'p.reset.emit()'}}}}]}}, 'height': 360, 'width_policy': 'max', 'height_policy': 'fixed', 'x_range': {'type': 'object', 'name': 'DataRange1d', 'id': 'p1119'}, 'y_range': {'type': 'object', 'name': 'DataRange1d', 'id': 'p1120', 'attributes': {'start': 0}}, 'x_scale': {'type': 'object', 'name': 'LinearScale', 'id': 'p1124'}, 'y_scale': {'type': 'object', 'name': 'LinearScale', 'id': 'p1125'}, 'extra_x_ranges': {'type': 'map', 'entries': [('lin', {'type': 'object', 'name': 'DataRange1d', 'id': 'p1135'}), ('log', {'type': 'object', 'name': 'DataRange1d', 'id': 'p1136'})]}, 'extra_y_ranges': {'type': 'map', 'entries': [('lin', {'type': 'object', 'name': 'DataRange1d', 'id': 'p1137'}), ('log', {'type': 'object', 'name': 'DataRange1d', 'id': 'p1138'})]}, 'extra_x_scales': {'type': 'map', 'entries': [('lin', {'type': 'object', 'name': 'LinearScale', 'id': 'p1131'}), ('log', {'type': 'object', 'name': 'LogScale', 'id': 'p1132'})]}, 'extra_y_scales': {'type': 'map', 'entries': [('lin', {'type': 'object', 'name': 'LinearScale', 'id': 'p1133'}), ('log', {'type': 'object', 'name': 'LogScale', 'id': 'p1134'})]}, 'title': {'type': 'object', 'name': 'Title', 'id': 'p1126'}, 'renderers': [{'type': 'object', 'name': 'GlyphRenderer', 'id': 'p1150', 'attributes': {'data_source': {'id': 'p1115'}, 'view': {'type': 'object', 'name': 'CDSView', 'id': 'p1151', 'attributes': {'filter': {'type': 'object', 'name': 'AllIndices', 'id': 'p1152'}}}, 'glyph': {'type': 'object', 'name': 'Quad', 'id': 'p1149', 'attributes': {'left': {'type': 'field', 'field': 'left'}, 'right': {'type': 'field', 'field': 'right'}, 'bottom': {'type': 'value', 'value': 0}, 'top': {'type': 'field', 'field': 'y'}, 'fill_color': {'type': 'value', 'value': 'skyblue'}}}}}], 'toolbar': {'type': 'object', 'name': 'Toolbar', 'id': 'p1127', 'attributes': {'autohide': True, 'tools': [{'type': 'object', 'name': 'PanTool', 'id': 'p1154'}, {'type': 'object', 'name': 'BoxZoomTool', 'id': 'p1155', 'attributes': {'overlay': {'type': 'object', 'name': 'BoxAnnotation', 'id': 'p1156', 'attributes': {'syncable': False, 'line_color': 'black', 'line_alpha': 1.0, 'line_width': 2, 'line_dash': [4, 4], 'fill_color': 'lightgrey', 'fill_alpha': 0.5, 'level': 'overlay', 'visible': False, 'left': {'type': 'number', 'value': 'nan'}, 'right': {'type': 'number', 'value': 'nan'}, 'top': {'type': 'number', 'value': 'nan'}, 'bottom': {'type': 'number', 'value': 'nan'}, 'left_units': 'canvas', 'right_units': 'canvas', 'top_units': 'canvas', 'bottom_units': 'canvas', 'handles': {'type': 'object', 'name': 'BoxInteractionHandles', 'id': 'p1162', 'attributes': {'all': {'type': 'object', 'name': 'AreaVisuals', 'id': 'p1161', 'attributes': {'fill_color': 'white', 'hover_fill_color': 'lightgray'}}}}}}}}, {'type': 'object', 'name': 'BoxSelectTool', 'id': 'p1164', 'attributes': {'renderers': 'auto', 'overlay': {'type': 'object', 'name': 'BoxAnnotation', 'id': 'p1165', 'attributes': {'syncable': False, 'line_color': 'black', 'line_alpha': 1.0, 'line_width': 2, 'line_dash': [4, 4], 'fill_color': 'lightgrey', 'fill_alpha': 0.5, 'level': 'overlay', 'visible': False, 'left': {'type': 'number', 'value': 'nan'}, 'right': {'type': 'number', 'value': 'nan'}, 'top': {'type': 'number', 'value': 'nan'}, 'bottom': {'type': 'number', 'value': 'nan'}, 'editable': True, 'handles': {'type': 'object', 'name': 'BoxInteractionHandles', 'id': 'p1171', 'attributes': {'all': {'type': 'object', 'name': 'AreaVisuals', 'id': 'p1170', 'attributes': {'fill_color': 'white', 'hover_fill_color': 'lightgray'}}}}}}}}, {'type': 'object', 'name': 'LassoSelectTool', 'id': 'p1172', 'attributes': {'renderers': 'auto', 'continuous': False, 'overlay': {'type': 'object', 'name': 'PolyAnnotation', 'id': 'p1173', 'attributes': {'syncable': False, 'level': 'overlay', 'visible': False, 'xs': [], 'ys': [], 'editable': True, 'line_color': 'black', 'line_alpha': 1.0, 'line_width': 2, 'line_dash': [4, 4], 'fill_color': 'lightgrey', 'fill_alpha': 0.5}}}}, {'type': 'object', 'name': 'HoverTool', 'id': 'p1153', 'attributes': {'visible': False, 'renderers': 'auto', 'tooltips': '\n    <div>\n        <div style="display: table; border-spacing: 2px;">\n            \n            <div style="display: table-row">\n                <div style="display: table-cell; color: #0D47A1; text-align: right;">\n                    ra:\n                </div>\n                <div style="display: table-cell;">\n                    @centers{0}\n                </div>\n            </div>\n        \n            <div style="display: table-row">\n                <div style="display: table-cell; color: #0D47A1; text-align: right;">\n                    count(ra):\n                </div>\n                <div style="display: table-cell;">\n                    @y{0}\n                </div>\n            </div>\n        \n        </div>\n    </div>\n    <style>\n        div.bk-tooltip-content > div > div:not(:first-child) {\n            display:none !important;\n        }\n    </style>', 'formatters': {'type': 'map', 'entries': [('@centers', {'type': 'object', 'name': 'CustomJSHover', 'id': 'p1176', 'attributes': {'code': 'return value.toFixed(4);'}})]}, 'point_policy': 'follow_mouse'}}, {'type': 'object', 'name': 'WheelZoomTool', 'id': 'p1163', 'attributes': {'renderers': 'auto'}}, {'type': 'object', 'name': 'SaveTool', 'id': 'p1175'}, {'type': 'object', 'name': 'ResetTool', 'id': 'p1174'}], 'active_scroll': {'id': 'p1163'}}}, 'toolbar_location': 'above', 'left': [{'type': 'object', 'name': 'LinearAxis', 'id': 'p1143', 'attributes': {'ticker': {'type': 'object', 'name': 'BasicTicker', 'id': 'p1144', 'attributes': {'mantissas': [1, 2, 5]}}, 'formatter': {'type': 'object', 'name': 'BasicTickFormatter', 'id': 'p1145'}, 'axis_label': 'count(ra)', 'major_label_policy': {'type': 'object', 'name': 'AllLabels', 'id': 'p1146'}}}], 'below': [{'type': 'object', 'name': 'LinearAxis', 'id': 'p1139', 'attributes': {'ticker': {'type': 'object', 'name': 'BasicTicker', 'id': 'p1140', 'attributes': {'mantissas': [1, 2, 5]}}, 'formatter': {'type': 'object', 'name': 'BasicTickFormatter', 'id': 'p1141'}, 'axis_label': 'ra', 'major_label_policy': {'type': 'object', 'name': 'AllLabels', 'id': 'p1142'}}}], 'center': [{'type': 'object', 'name': 'Grid', 'id': 'p1147', 'attributes': {'ticker': {'id': 'p1140'}}}, {'type': 'object', 'name': 'Grid', 'id': 'p1148', 'attributes': {'dimension': 1, 'ticker': {'id': 'p1144'}}}], 'lod_factor': 20000, 'lod_threshold': 1000, 'lod_timeout': 10000, 'output_backend': 'webgl', 'reset_policy': 'event_only'}}]}}, 'render_items': [{'docid': '74e9a707-b102-477c-b7ea-df83d3935c93', 'roots': {'p1121': 'dca457ca-14de-429b-9352-840f29af48a9'}, 'root_ids': ['p1121']}], 'div': '\n  <div id="dca457ca-14de-429b-9352-840f29af48a9" data-root-id="p1121" style="display: contents;"></div>\n'})

[rileythai]

This is because I think the BokehModel under the hood literally hashes the entire object to use as a key for removing a callback. I didn't end up looking into this but I think when I remove or change callbacks, or change the object, the document can't find them anymore because they've literally been modified.

Fixing this requires a fix in the jupyter_bokeh code to support this kind of interactivity, which that BokehModel class isn't built for.

[rileythai]

I have given this some thought, I think there are two things at play here:

1. No such comm seems to only occur when you add/delete plots. This is presumably because the effects are setting up/trying to activate before the plot is ready. When deleting a plot it makes 4 messages, and when adding it makes about 16.

2. The KeyError seems to be in removing the callback on deletion (Exception ignored in: <function Widget.__del__ at 0x7caf2d8cad40>). We don't actually "delete" the objects (replaced with Card), so this must be the garbage collector or something in Solara calling this. The tree seems to be Widget.__del__ -> self.close() [ipywidgets] -> self._document.remove_on_change [jupyter_bokeh] -> self.callbacks.remove_on_change(*callbacks) [bokeh]. This is reproducible in the docker by just adding/deleting/moving around histograms until it appears.

[havok2063]

Thanks for these thoughts. I'll try to digest this and poke around after the data release at the end of July. So far, this hasn't impacted functionality or performances as far as I can tell.