Fix OpalServer-EvenBroadcaster integration & other OpalServer refactors

[roekatz]

1. Retry initial EventBroadcaster connection so not to restart process (as a result of recent fix not ignoring connection failure of EventBroadcaster) - This relies on this PR of fastapi_websocket_pubsub
2. Concentrate all uses of PubSubEndpoint & EventBroadcaster (from fastapi_websocket_pubsub) in opal-server's PubSub (use the shared abstraction rather than the LL objects)
3. Make PubSub responsible for detecting broadcaster disconnection an triggering the disconnect callbacks (instead of each component taking care of it separately - e.g statistics etc).
4. Reuse task management & cleanup functionality
5. Remove basically redundant code of topics/publisher.py (most of each is just a layer that adds nothing, useful stuff like the PeriodicPublisher was moved to more appropriate location).
6. Clean Spaghetti code of OpalServer managing background tasks.

[netlify]

✅ Deploy Preview for opal-docs canceled.

Name	Link
🔨 Latest commit	376acd6
🔍 Latest deploy log	https://app.netlify.com/sites/opal-docs/deploys/66e03ee4fee18f0008dbc59f

[roekatz]

This will be fixed to 0.3.9 once we release fastapi_websocket_pubsub with that fix - https://github.com/permitio/fastapi_websocket_pubsub/pull/82/files

[zeevmoney]

Thanks for this, @roekatz. This draft has been open since July 2024, now conflicts with master, and depends on an unmerged upstream change (permitio/fastapi_websocket_pubsub#82). In the meantime master gained the ignore_broadcaster_disconnected / BROADCAST_CONN_LOSS_BUGFIX_EXPERIMENT_ENABLED path, and the broadcaster reconnect/consistency problem this PR set out to fix is now addressed by the newer, tested PR #915 (PER-15065), which adds pubsub_resilience.py plus reconnect and consistency integration tests. The remaining value here is the OpalServer refactor work (deleting opal_common/topics/publisher.py, removing the redundant PUBLISHER_ENABLED/init_publisher, the server.py task-management cleanup, and the small async_utils/confi helpers) — worth keeping, but a separate, downstream-sensitive change that needs a fresh rebase. We're closing this in favor of #915 for the reconnect fix. Please reopen if you disagree.