It ended up being a bug with Cloudflare.
It appears that if you perform any async tasks (like interacting with DO storage) after accepting the WebSocket but before you return the response, containing the client-portion of the WebSocket pair, it confuses it with an "outgoing WebSocket connection".