refactor: remove Flink/Arroyo/legacy Kafka-ingest infra from asap-tools (#721)

* refactor(rust): remove Flink/Arroyo/legacy-Kafka streaming engine support

Collapses StreamingEngine to a single Precompute variant across
asap-planner-rs and asap-query-engine, removes IngestConfig::Kafka and
the Kafka consumer driver, and rewires OTLP ingest to pair with
streaming_engine=precompute instead of the removed arroyo gate.

Also removes the now-unreachable Arroyo wire-format cluster:
PrecomputedOutput::deserialize_from_json_arroyo, each accumulator's
serialize_to_bytes_arroyo/deserialize_from_bytes_arroyo, the
test_offline_precomputes debug binary, the dump_precomputes config
knob, and the orphaned datafusion_summary_library crate (its only
consumer was the Arroyo/Kafka ingest path).

Part of #709.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3YXZoukBCkyRf7otqs7ow

* refactor(asap-tools): remove Flink/Arroyo/legacy Kafka-ingest infrastructure

Deletes asap-summary-ingest/ (the Arroyo pipeline generator) and all
asap-tools infra that only served Flink/Arroyo/Kafka: the installation
scripts, docker-compose includes, experiment service classes
(KafkaService, FlinkService, ArroyoService, PrometheusKafkaAdapterService,
ArroyoThroughputMonitor, DumbKafkaConsumerService), the standalone
Kafka/producer/benchmark-pipeline execution-utilities, and the
corresponding Hydra config knobs (streaming.engine choices,
use_kafka_ingest, profiling.flink/arroyo, use_container.arroyo,
dump_precomputes, replace_query_engine_with_dumb_consumer).

`precompute` (HTTP remote write into asap-query-engine) is now the only
supported streaming/ingest path.

Also removes other infra that only served the removed pipeline and had
no other caller: asap-tools/installation/kafka/, fake_kafka_exporter/
(a real Kafka producer), the ArroyoThroughputAnalyzer mode in
analyze_throughput.py, the confluent_kafka Python dependency, and the
now-dead asap-summary-ingest pytest job in the Python CI workflow.

Closes #709.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3YXZoukBCkyRf7otqs7ow

* docs: update architecture and how-to guides for Flink/Arroyo/Kafka removal

Rewrites the architecture diagrams and component descriptions in
docs/01-getting-started, docs/02-components, asap-tools/docs, and
asap-query-engine/docs to describe the actual current pipeline
(Prometheus remote write -> QueryEngine precompute engine directly,
no Arroyo/Flink/Kafka hop), and updates the manual-stack-run and
add-new-sketch how-to guides to the current config-file-based
query_engine_rust interface.

Part of #709.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3YXZoukBCkyRf7otqs7ow

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

Author: milindsrivastava1997

.github/workflows/python.yml

@@ -8,7 +8,6 @@ on:
       - 'asap-tools/**'
       - 'asap-tools/data-sources/prometheus-exporters/**'
       - 'asap-tools/execution-utilities/**'
-      - 'asap-summary-ingest/**'
       - 'asap-common/dependencies/py/**'
       - '.github/workflows/python.yml'
   pull_request:
@@ -19,7 +18,6 @@ on:
       - 'asap-tools/**'
       - 'asap-tools/data-sources/prometheus-exporters/**'
       - 'asap-tools/execution-utilities/**'
-      - 'asap-summary-ingest/**'
       - 'asap-common/dependencies/py/**'
       - '.github/workflows/python.yml'
   workflow_dispatch:
@@ -41,7 +39,6 @@ jobs:
       utilities: ${{ steps.filter.outputs.utilities }}
       prometheus_exporters: ${{ steps.filter.outputs.prometheus_exporters }}
       execution_utilities: ${{ steps.filter.outputs.execution_utilities }}
-      summary_ingest: ${{ steps.filter.outputs.summary_ingest }}
     steps:
       - uses: actions/checkout@v4
       - uses: dorny/paths-filter@v2
@@ -60,9 +57,6 @@ jobs:
             execution_utilities:
               - 'asap-tools/execution-utilities/**'
               - 'asap-common/dependencies/py/**'
-            summary_ingest:
-              - 'asap-summary-ingest/**'
-              - 'asap-common/dependencies/py/**'
 
   test-prometheus-client:
     needs: detect-changes
@@ -206,21 +200,3 @@ jobs:
         flake8 . --config=../../.flake8 --count --select=E9,F63,F7,F82 --show-source --statistics
         # Exit-zero treats all errors as warnings
         flake8 . --config=../../.flake8 --count --exit-zero --max-complexity=10 --statistics
-
-  test-summary-ingest:
-    needs: detect-changes
-    if: needs.detect-changes.outputs.summary_ingest == 'true'
-    runs-on: ubuntu-latest
-    steps:
-    - uses: actions/checkout@v4
-    - name: Set up Python
-      uses: actions/setup-python@v4
-      with:
-        python-version: '3.11'
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        pip install -e asap-common/dependencies/py/promql_utilities/
-        pip install pytest pyyaml jinja2 loguru requests
-    - name: Run pytest
-      run: python -m pytest asap-summary-ingest/tests/ -v