28 Commits

  • refac: refuse a compressed body when fetching a remote URL (#33271)
    Remote fetches now ask for an uncompressed response: web pages, documents, remote images, profile pictures from a login provider and webhook deliveries. A response that comes back compressed anyway is refused, except for a webhook reply, which is only logged and is left unread instead.
    
    A page or file whose host echoes a stored `Content-Encoding` regardless of what the client asks for (an S3 or MinIO object uploaded with that metadata) is no longer read, and a profile picture that arrives that way falls back to the default avatar.
  • perf: build debug log messages lazily so disabled debug logs cost nothing (#27834)
    GLOBAL_LOG_LEVEL defaults to INFO, so every log.debug(...) in the backend is discarded, but the message is built first: 187 call sites interpolate their payload into an f-string before the logging call runs, so the work happens on every request and the result is thrown away. The worst one sits in process_chat_payload and stringifies the whole request body, full conversation history included, once per chat completion.
    
    That one line with DEBUG disabled, CPython 3.12:
    
    | conversation | payload | before   | after   |
    | ------------ | ------- | -------- | ------- |
    | 4 messages   | 1.2 kB  | 3.4 us   | 0.07 us |
    | 20 messages  | 17 kB   | 24.8 us  | 0.07 us |
    | 60 messages  | 123 kB  | 216.6 us | 0.07 us |
    
    The lazy form log.debug('form_data: %s', form_data) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. With DEBUG enabled the emitted lines are byte-identical, f'{x=}' sites included: those map to %r. MistralLoader._debug_log callers get the same treatment, since that wrapper already forwards *args.
  • perf(images): offload validate_url() DNS resolution with asyncio.to_thread (#25825)
    validate_url() calls socket.getaddrinfo() for SSRF protection, which
    blocks the event loop for 100-700ms per DNS lookup. This affects:
    
    - Image generation (get_image_data) — every external image URL
    - Image editing (load_url_image) — every external image URL
    - OAuth profile pictures (_process_picture_url) — every login
    - Webhook delivery (post_webhook) — every notification
    - Image base64 conversion (get_image_base64_from_url) — chat images
    
    Wrap all 5 async call sites in asyncio.to_thread() so DNS resolution
    runs in the thread pool. The event loop remains free to serve other
    requests during the lookup.
    
    Benchmark (3 domains, 3 trials averaged):
    - BEFORE: max jitter 479ms, 1 blocked ping per trial
    - AFTER:  max jitter 1ms, 0 blocked pings (324x improvement)
    
    Co-authored-by: Tim Baek <tim@openwebui.com>
  • fix: block private-IP webhook URLs to close SSRF on caller-controlled URL (#24587)
    * fix: block private-IP webhook URLs to close SSRF on caller-controlled URL
    
    post_webhook(url, ...) in utils/webhook.py forwards the URL straight to
    aiohttp.ClientSession.post with no SSRF gate. The URL is caller-controlled
    on two surfaces:
    
    - User notification settings under ENABLE_USER_WEBHOOKS=true — any
      authenticated user can set the URL their notifications POST to.
    - Automation notification triggers (calendar alerts, etc.).
    
    Without a gate, the URL can target cloud metadata (169.254.169.254 /
    fd00:ec2::254), localhost-bound services, RFC1918 internal hosts, or any
    other private address reachable from the server process. Blind SSRF — no
    response body returned to the caller — but enough to enumerate internal
    services via response timing / status codes, and on cloud deployments
    enough to issue requests against IMDSv1 if available.
    
    Call validate_url() at the top of post_webhook. The function blocks
    private/reserved IPs when ENABLE_RAG_LOCAL_WEB_FETCH is False (the
    default), is the project's chosen SSRF gate, and is already applied to
    the equivalent fetch surfaces (retrieval, image-load, OAuth profile
    picture). Operators who legitimately need to webhook to private IPs
    (internal monitoring, self-hosted Slack alternatives, etc.) can set
    ENABLE_RAG_LOCAL_WEB_FETCH=True — same opt-out as the other gated
    surfaces.
    
    Scope intentionally limited to webhooks. The OAuth discovery and
    external reranker paths cwanglab also flagged are admin-configured with
    intentional private-IP defaults (reranker defaults to
    http://localhost:8080/v1/rerank) and are out of scope per Rule 9 — the
    admin owns the URL choice and the operator opt-out exists for them too.
    
    Reported by cwanglab in GHSA-5x9f-85cg-w3hf (cluster canonical with six
    closed siblings: g36v-23gj-j69x, 6j8f-h58v-xgmw, xpwv-52pm-p8hj,
    v9gp-hv2c-9qv8, fw7w-jrw7-p3v9, x7xq-74rg-m8mf).
    
    Co-authored-by: cwanglab <cwanglab@users.noreply.github.com>
    
    * fix: also pass allow_redirects=False on webhook post_webhook session.post
    
    Companion to the previous commit. validate_url() only validates the
    initial URL; aiohttp's default allow_redirects=True would still follow
    a 302 to a private-IP target. Same redirect-bypass class as the rh5x
    cluster's five call sites, sixth call site to receive the same gate.
    
    Co-authored-by: cwanglab <cwanglab@users.noreply.github.com>
    
    ---------
    
    Co-authored-by: cwanglab <cwanglab@users.noreply.github.com>
  • fix: prevent TypeError in Teams webhook when user data is missing (#22444)
    json.loads(event_data.get("user", {})) crashes with TypeError when
    the "user" key is absent because the default value {} is a dict, not
    a JSON string. json.loads expects str/bytes, not dict.
    
    Also handle the case where "user" is already a dict (not serialized
    JSON) to make the webhook more robust.
    
    Co-authored-by: gambletan <ethanchang32@gmail.com>
  • chore/perf: Remove old SRC level log env vars with no impact (#20045)
    * Update openai.py
    
    * Update env.py
    
    * Merge pull request open-webui#19030 from open-webui/dev (#119)
    
    Co-authored-by: Tim Baek <tim@openwebui.com>
    Co-authored-by: Claude <noreply@anthropic.com>
    
    ---------
    
    Co-authored-by: Tim Baek <tim@openwebui.com>
    Co-authored-by: Claude <noreply@anthropic.com>