feat: add optional local web UI

MANIFEST.in

@@ -7,6 +7,7 @@ include scrapling/*.db-*
 include scrapling/py.typed
 include scrapling/.scrapling_dependencies_installed
 include .scrapling_dependencies_installed
+recursive-include scrapling/webui *.html *.css *.js
 
 recursive-exclude * __pycache__
-recursive-exclude * *.py[co]
+recursive-exclude * *.py[co]

docs/development/web_ui_rfc.md

@@ -0,0 +1,94 @@
+# RFC: Optional local Web UI
+
+**Status:** Proof of concept  
+**Proposed by:** Ibrahim Khan Jagwal
+
+## Summary
+
+Add an optional, local-first Web UI for users who do not use MCP and want to operate Scrapling without writing code. The UI should remain a thin interface over Scrapling's public fetcher and parser APIs.
+
+The proof of concept introduces:
+
+- `scrapling ui`
+- HTTP, dynamic, and stealth fetching modes
+- CSS and XPath extraction
+- Text, HTML, and Markdown representations
+- Indexed job history in SQLite
+- JSON and CSV exports
+- Localhost-only binding and private-network target blocking by default
+
+## Motivation
+
+Scrapling currently serves Python developers, command-line users, and MCP-compatible AI clients. A browser interface makes selector experimentation and one-off extraction accessible to users who do not belong to those groups.
+
+The UI is not intended to replace Python spiders or become a hosted scraping service. Its first role is a local extraction workbench.
+
+## Architecture
+
+```text
+Browser UI
+    |
+Local Starlette application
+    |-- Fetcher
+    |-- DynamicFetcher
+    |-- StealthyFetcher
+    |-- Selector engine
+    `-- SQLite job index
+```
+
+MCP remains an independent transport. A later iteration can extract common operation services shared by the UI and MCP adapters when doing so removes proven duplication.
+
+## Packaging
+
+The UI belongs behind an optional dependency:
+
+```bash
+pip install "scrapling[ui]"
+```
+
+Core parser users should not receive Starlette, Uvicorn, or browser dependencies unless they request the UI.
+
+## Data model
+
+The initial index stores one row per extraction:
+
+- stable job identifier
+- creation timestamp
+- requested and final URLs
+- fetch mode
+- selector and selector type
+- output representation
+- response status
+- duration
+- title
+- item count
+- extracted values
+- error information
+
+SQLite is deliberately sufficient for a local proof of concept. A future crawler-oriented index should use a storage abstraction before adding page graphs, content hashes, full-text search, or schema inference.
+
+## Security
+
+The default design:
+
+- binds to `127.0.0.1`
+- accepts only `http` and `https` URLs
+- resolves target hosts and rejects non-global IP addresses
+- limits request timeouts
+- escapes extracted values before rendering
+- exposes only controlled database exports
+- does not accept arbitrary output paths
+
+Before supporting public deployment, the project would need authentication, CSRF protection, request and crawl quotas, stronger DNS-rebinding and redirect validation, secret handling, and an explicit remote-deployment threat model.
+
+## Follow-up work
+
+1. Saved projects and reusable extraction schemas.
+2. Crawl progress, cancellation, and checkpoint controls.
+3. Page-link graph and content-hash indexing.
+4. Full-text search across extracted records.
+5. Interactive selector highlighting in a sandboxed preview.
+6. Configurable structured field mappings.
+7. Authentication for deliberate remote deployments.
+
+These should be discussed and reviewed separately instead of being bundled into the initial UI contribution.

docs/ui/getting-started.md

@@ -0,0 +1,63 @@
+# Scrapling Web UI
+
+The optional Web UI is a local visual workspace for people who want to fetch pages, try selectors, inspect values, and export results without first writing a Python scraper or configuring an MCP client.
+
+## Installation
+
+Install the UI and fetcher dependencies:
+
+```bash
+pip install "scrapling[ui]"
+```
+
+Browser-based modes also require Chromium:
+
+```bash
+scrapling install
+```
+
+## Starting the UI
+
+```bash
+scrapling ui
+```
+
+Open [http://127.0.0.1:8001](http://127.0.0.1:8001) in a browser. The default listener is local-only.
+
+The database is stored at `~/.scrapling/ui.db`. Choose another location when needed:
+
+```bash
+scrapling ui --database ./workspace.db
+```
+
+## Extraction modes
+
+| Mode | Use case |
+| --- | --- |
+| HTTP | Fast extraction from static HTML |
+| Dynamic | Pages that require JavaScript rendering |
+| Stealth | Browser-rendered pages with stronger bot protection |
+
+Every successful extraction is added to the local index with its URL, final URL, response status, selector, duration, and extracted values. Results can be exported as JSON or CSV.
+
+## Network safety
+
+The UI rejects localhost, private-network, and non-HTTP targets by default. This reduces the risk of a webpage or another user turning the interface into a server-side request forgery proxy.
+
+For deliberate local development, private targets can be enabled explicitly:
+
+```bash
+scrapling ui --allow-private-targets
+```
+
+Do not use that option on a shared or publicly reachable machine. If the UI is exposed beyond localhost, place it behind an authenticated reverse proxy.
+
+## Relationship to MCP
+
+The Web UI and MCP server are separate interfaces:
+
+- `scrapling ui` is for direct human interaction in a browser.
+- `scrapling mcp` exposes tools to compatible AI clients.
+- Python applications can continue using Scrapling's fetchers and spiders directly.
+
+The UI uses the same fetchers and selector engine; it does not reimplement scraping behavior.

pyproject.toml

@@ -91,8 +91,14 @@ shell = [
     "markdownify>=1.2.0",
     "scrapling[fetchers]",
 ]
+ui = [
+    "starlette>=0.27",
+    "uvicorn>=0.31.1",
+    "markdownify>=1.2.0",
+    "scrapling[fetchers]",
+]
 all = [
-    "scrapling[ai,shell]",
+    "scrapling[ai,shell,ui]",
 ]
 
 [project.urls]
@@ -115,6 +121,9 @@ include-package-data = true
 where = ["."]
 include = ["scrapling*"]
 
+[tool.setuptools.package-data]
+scrapling = ["webui/*.html", "webui/*.css", "webui/*.js"]
+
 [tool.mypy]
 python_version = "3.10"
 warn_unused_configs = true

scrapling/cli.py

@@ -185,6 +185,33 @@ def mcp(http, host, port, executable_path, auth_token, allowed_host):
     server.serve(http, host, port, allowed_hosts=allowed_host)
 
 
+@command(help="Run Scrapling's local Web UI.")
+@option("--host", type=str, default="127.0.0.1", show_default=True, help="Host address for the Web UI")
+@option("--port", type=int, default=8001, show_default=True, help="Port for the Web UI")
+@option(
+    "--database",
+    type=str,
+    default=None,
+    help="SQLite history database path (default: ~/.scrapling/ui.db)",
+)
+@option(
+    "--allow-private-targets",
+    is_flag=True,
+    default=False,
+    help="Allow scraping localhost and private-network URLs (unsafe on shared systems)",
+)
+def ui(host, port, database, allow_private_targets):
+    """Start the optional browser-based interface."""
+    try:
+        from scrapling.core.ui import ScraplingWebUI
+    except (ImportError, ModuleNotFoundError) as error:
+        raise ModuleNotFoundError(
+            'You need to install Scrapling with the UI extra first: pip install "scrapling[ui]"'
+        ) from error
+
+    ScraplingWebUI(database=database, allow_private_targets=allow_private_targets).serve(host, port)
+
+
 @command(help="Interactive scraping console")
 @option(
     "-c",
@@ -696,3 +723,4 @@ def main():
 main.add_command(shell)
 main.add_command(extract)
 main.add_command(mcp)
+main.add_command(ui)