Bug: Exception during replay or visualization

[abrichr]

Occasionally, attempting to replay or visualize a recording will result in the following error:

2023-04-22 19:23:42.140 | INFO     | puterbot.events:merge_consecutive_input_events:570 - name='mouse_move' num_events_removed=370
2023-04-22 19:23:42.140 | ERROR    | puterbot.events:process_events:621 - (<function merge_consecutive_mouse_move_events at 0x13632f0a0>, InputEvent(id=2778, name='type', timestamp=1683820640.0885906, recording_timestamp=1682204625.970328, screenshot_timestamp=1683820641.2495313, window_event_timestamp=1683820634.120639, text='h-i-s', canonical_text='h-i-s'), InputEvent(id=2779, name='type', timestamp=1683820639.0881405, recording_timestamp=1682204625.970328, screenshot_timestamp=1683820641.4406426, window_event_timestamp=1683820634.120639, text='<space>-i-s', canonical_text='<49>-i-s'))
Traceback (most recent call last):

  File "/Users/abrichr/MLDSAI/src/puterbot/puterbot/visualize.py", line 226, in <module>
    main()
    └ <function main at 0x13b1f7520>

  File "/Users/abrichr/MLDSAI/src/puterbot/puterbot/visualize.py", line 135, in main
    input_events = get_events(recording, process=PROCESS_EVENTS, meta=meta)
                   │          │                  │                    └ {}
                   │          │                  └ True
                   │          └ Recording(id=14, timestamp=1682204625.970328, monitor_width=1512, monitor_height=982, double_click_interval_seconds=0.5, doub...
                   └ <function get_events at 0x13632ed40>

  File "/Users/abrichr/MLDSAI/src/puterbot/puterbot/events.py", line 53, in get_events
    input_events, window_events, screenshots = process_events(
    │             │                            └ <function process_events at 0x13b1f72e0>
    │             └ [WindowEvent(id=24, recording_timestamp=1682204625.970328, timestamp=1683820634.120639, title='Terminal puterbot — Python ◂ P...
    └ [InputEvent(id=2366, name='move', timestamp=1683820639.0447876, recording_timestamp=1682204625.970328, screenshot_timestamp=1...

> File "/Users/abrichr/MLDSAI/src/puterbot/puterbot/events.py", line 617, in process_events
    assert prev_event.timestamp <= event.timestamp, (
           │          │            │     └ <sqlalchemy.orm.attributes.InstrumentedAttribute object at 0x13633b830>
           │          │            └ InputEvent(id=2779, name='type', timestamp=1683820639.0881405, recording_timestamp=1682204625.970328, screenshot_timestamp=16...
           │          └ <sqlalchemy.orm.attributes.InstrumentedAttribute object at 0x13633b830>
           └ InputEvent(id=2778, name='type', timestamp=1683820640.0885906, recording_timestamp=1682204625.970328, screenshot_timestamp=16...

AssertionError: (<function merge_consecutive_mouse_move_events at 0x13632f0a0>, InputEvent(id=2778, name='type', timestamp=1683820640.0885906, recording_timestamp=1682204625.970328, screenshot_timestamp=1683820641.2495313, window_event_timestamp=1683820634.120639, text='h-i-s', canonical_text='h-i-s'), InputEvent(id=2779, name='type', timestamp=1683820639.0881405, recording_timestamp=1682204625.970328, screenshot_timestamp=1683820641.4406426, window_event_timestamp=1683820634.120639, text='<space>-i-s', canonical_text='<49>-i-s'))
> /Users/abrichr/MLDSAI/src/puterbot/puterbot/events.py(615)process_events()
    614         # TODO: keep events in which window_event_timestamp is updated
--> 615         for prev_event, event in zip(input_events, input_events[1:]):
    616             try:

ipdb> 

Steps to reproduce:

1. Run python puterbot/record.py "testing out puterbot"
2. Type some text, move the mouse, resize a window
3. Hit CTRL+C and wait for the events to flush
4. Run python puterbot/visualize.py or python puterbot/replay.py NaiveReplayStratevy

[abrichr]

Closing as part of architecture transition. Legacy frozen at v0.46.0. See PR #960.