Threat Intelligence Database

This is the same issue as CVE-2024-3651, however the original remediation in 2024 was not a complete fix. Payloads such as `"\u0660" * N` or `"\u30fb" * N + "\u6f22"` utilize the `valid_contexto` function prior to length rejection, and for high values of `N` will take a long time to process. ### Impact A specially crafted argument to the `idna.encode()` function could consume significant resources. This may lead to a denial-of-service. ### Patches Starting in version 3.14, the function rejects long inputs as soon as practicable prior to any further processing to minimize resource consumption. In version 3.15, this approach was extended to lesser used alternate functions (i.e. per-label conversions and codec support). ### Workarounds Domain names cannot exceed 253 characters in length, if this length limit is enforced prior to passing the domain to the `idna.encode()` function it should no longer consume significant resources. This is triggered by arbitrarily large inputs that would not occur in normal usage, but may be passed to the library assuming there is no preliminary input validation by the higher-level application.

urllib3 handles redirects and retries using the same mechanism, which is controlled by the `Retry` object. The most common way to disable redirects is at the request level, as follows: ```python resp = urllib3.request("GET", "https://httpbin.org/redirect/1", redirect=False) print(resp.status) # 302 ``` However, it is also possible to disable redirects, for all requests, by instantiating a `PoolManager` and specifying `retries` in a way that disable redirects: ```python import urllib3 http = urllib3.PoolManager(retries=0) # should raise MaxRetryError on redirect http = urllib3.PoolManager(retries=urllib3.Retry(redirect=0)) # equivalent to the above http = urllib3.PoolManager(retries=False) # should return the first response resp = http.request("GET", "https://httpbin.org/redirect/1") ``` However, the `retries` parameter is currently ignored, which means all the above examples don't disable redirects. ## Affected usages Passing `retries` on `PoolManager` instantiation to disable redirects or restrict their number. By default, requests and botocore users are not affected. ## Impact Redirects are often used to exploit SSRF vulnerabilities. An application attempting to mitigate SSRF or open redirect vulnerabilities by disabling redirects at the PoolManager level will remain vulnerable. ## Remediation You can remediate this vulnerability with the following steps: * Upgrade to a patched version of urllib3. If your organization would benefit from the continued support of urllib3 1.x, please contact [[email protected]](mailto:[email protected]) to discuss sponsorship or contribution opportunities. * Disable redirects at the `request()` level instead of the `PoolManager()` level.

When using urllib3's proxy support with `ProxyManager`, the `Proxy-Authorization` header is only sent to the configured proxy, as expected. However, when sending HTTP requests *without* using urllib3's proxy support, it's possible to accidentally configure the `Proxy-Authorization` header even though it won't have any effect as the request is not using a forwarding proxy or a tunneling proxy. In those cases, urllib3 doesn't treat the `Proxy-Authorization` HTTP header as one carrying authentication material and thus doesn't strip the header on cross-origin redirects. Because this is a highly unlikely scenario, we believe the severity of this vulnerability is low for almost all users. Out of an abundance of caution urllib3 will automatically strip the `Proxy-Authorization` header during cross-origin redirects to avoid the small chance that users are doing this on accident. Users should use urllib3's proxy support or disable automatic redirects to achieve safe processing of the `Proxy-Authorization` header, but we still decided to strip the header by default in order to further protect users who aren't using the correct approach. ## Affected usages We believe the number of usages affected by this advisory is low. It requires all of the following to be true to be exploited: * Setting the `Proxy-Authorization` header without using urllib3's built-in proxy support. * Not disabling HTTP redirects. * Either not using an HTTPS origin server or for the proxy or target origin to redirect to a malicious origin. ## Remediation * Using the `Proxy-Authorization` header with urllib3's `ProxyManager`. * Disabling HTTP redirects using `redirects=False` when sending requests. * Not using the `Proxy-Authorization` header.

### Summary The CSS selector parser in soupsieve (the CSS selector engine for Beautiful Soup 4) allocates unbounded memory when compiling large comma-separated selector lists. An attacker who can supply a crafted CSS selector string to `soupsieve.compile()` or Beautiful Soup's `.select()` / `.select_one()` can cause the application to allocate hundreds of megabytes of heap memory from a relatively small input, leading to memory exhaustion and denial of service. To be completely transparent, AI tools helped surface this issue. However, it was independently reproduced and carefully validated. Researchers follow responsible disclosure practices and originally shared this report privately. A **500 KB** selector string triggers allocation of approximately **244 MB** of heap memory - a 488x— amplification ratio**. ### Details **Affected code:** `soupsieve/css_parser.py`, lines ~204, 925, 1106 The soupsieve CSS parser splits comma-separated selector lists and creates one `CSSSelector` object per list item. Each `CSSSelector` object contains parsed selector data structures including `SelectorList`, `Selector`, and associated tag/attribute/pseudo-class metadata. When a selector string such as `a,a,a,...` (with 250,000 comma-separated items) is passed to `sv.compile()`, the parser: 1. Tokenises the entire string and identifies each comma-delimited segment (line ~1106) 2. Parses each segment into a full `Selector` object with all associated metadata (line ~925) 3. Stores all parsed selectors in a `SelectorList` (line ~204) **Root cause:** No limit is enforced on the number of selectors in a comma-separated list. The parser will attempt to parse and store an arbitrary number of selectors, with each selector object consuming approximately **976 bytes** of heap memory. The total allocation scales linearly with the number of list items, but the amplification ratio (output memory / input bytes) is extremely high because each single-character selector like `a` expands into a complex object graph. **Attack surface:** Any application that passes user-supplied CSS selectors to `soupsieve.compile()` or Beautiful Soup's `.select()` / `.select_one()`. ### Proof of Concept ```python import tracemalloc import soupsieve as sv tracemalloc.start() # Build a 500 KB selector string: "a,a,a,...,a" (250,000 items) count = 250_000 selector = ",".join("a" for _ in range(count)) print(f"Selector string size: {len(selector):,} bytes ({len(selector) / 1024:.0f} KB)") # Compile the selector — this allocates ~244 MB compiled = sv.compile(selector) current, peak = tracemalloc.get_traced_memory() tracemalloc.stop() print(f"Compiled selector count: {len(compiled.selectors):,}") print(f"Current memory: {current / 1024 / 1024:.1f} MB") print(f"Peak memory: {peak / 1024 / 1024:.1f} MB") print(f"Amplification ratio: {peak / len(selector):.0f}x") # Expected output: # Selector string size: 499,999 bytes (488 KB) # Compiled selector count: 250,000 # Current memory: ~244 MB # Peak memory: ~244 MB # Amplification ratio: ~488x ``` ### Impact **Severity: High** An attacker can exhaust available memory on any server-side Python application that compiles user-supplied CSS selectors via soupsieve. This can cause: - **OOM kills** in containerised deployments (Kubernetes pods, Docker containers) with memory limits - **Swap thrashing** on bare-metal servers, degrading performance for all co-located processes - **Process termination** via Python's `MemoryError` exception if the system runs out of addressable memory | Parameter | Value | |---|---| | Input size | ~500 KB selector string | | Memory allocated | ~244 MB | | Amplification ratio | ~488× | | Per-object overhead | ~976 bytes per selector | | Authentication required | None | | User interaction required | None | **Scalability of attack:** The memory allocation scales linearly - doubling the selector count doubles memory usage. An attacker can tune the payload to exactly exhaust a target's memory limits. Multiple concurrent requests multiply the effect. **Downstream exposure:** soupsieve is an automatic dependency of `beautifulsoup4`, one of the most widely installed Python packages. Any web application accepting CSS selectors from users (e.g., web scraping APIs, content filtering tools, CMS preview features) is potentially affected. --- ### Credit Discovered by a security research team from the University of Sydney, focused on detecting open source software vulnerabilities. Liyi Zhou: https://lzhou1110.github.io/ Ziyue Wang: https://zyy0530.github.io/ Strick: https://str1ckl4nd.github.io/ Maurice: https://maurice.busystar.org/ Chenchen Yu: https://7thparkk.github.io/

### Summary The CSS selector parser in soupsieve (the CSS selector engine for Beautiful Soup 4) contains a regular expression vulnerable to catastrophic backtracking. When processing an attribute selector with an unterminated quoted value, the `VALUE` regex pattern in `css_parser.py` enters exponential backtracking. A payload of only **300 bytes** causes the regex engine to hang for **over 3 seconds**, enabling a trivial Regular Expression Denial of Service (ReDoS) attack. To be completely transparent, AI tools helped surface this issue. However, this was independently reproduced and carefully validated. Any application that passes untrusted CSS selector strings to `soupsieve.compile()` or Beautiful Soup's `.select()` / `.select_one()` is affected. ### Details **Affected code:** `soupsieve/css_parser.py`, line ~121 - `RE_VALUES` / `VALUE` regex pattern The soupsieve CSS parser uses a compiled regular expression to tokenise attribute selector values. This pattern matches both quoted strings (`"value"` or `'value'`) and unquoted identifiers. The regex contains alternation branches for: 1. Double-quoted strings: `"[^"\\]*(?:\\.[^"\\]*)*"` 2. Single-quoted strings: `'[^'\\]*(?:\\.[^'\\]*)*'` 3. Unquoted identifiers When an attribute selector contains an **unterminated quoted value** - e.g., `[a="xxxx...` (opening `"` but no closing `"`) -” the regex engine attempts to match the quoted-string branch. After that branch fails (no closing quote), the engine backtracks and attempts to match the remaining input against subsequent alternation branches and parent patterns. The structure of the pattern causes **catastrophic backtracking** where the number of backtracking steps grows exponentially with the length of the content between the opening quote and the end of the string. **Root cause:** The regex pattern does not anchor or guard against the case where a quoted string is never terminated. The overlapping character classes across alternation branches create exponential backtracking when the quoted-string branch fails on long input. **Key characteristics:** - **Input size:** Only 300 bytes are needed to trigger a >3 second hang - **Amplification:** Each additional character approximately doubles the backtracking time - **No memory impact:** The attack consumes CPU only (regex backtracking is compute-bound) ### Proof of Concept ```python import time import soupsieve as sv PAYLOAD_LEN = 300 # Control: well-formed selector with terminated quote (completes instantly) well_formed = '[a="' + ('x' * PAYLOAD_LEN) + '"]' start = time.perf_counter() try: sv.compile(well_formed) except Exception: pass control_time = time.perf_counter() - start print(f"Well-formed selector ({len(well_formed)} bytes): {control_time:.4f}s") # Exploit: unterminated quote triggers catastrophic regex backtracking malformed = '[a="' + ('x' * PAYLOAD_LEN) start = time.perf_counter() try: sv.compile(malformed) # WARNING: This will hang for >3 seconds except Exception: pass exploit_time = time.perf_counter() - start print(f"Malformed selector ({len(malformed)} bytes): {exploit_time:.4f}s") slowdown = exploit_time / max(control_time, 1e-9) print(f"Slowdown: {slowdown:.0f}x") # Expected output: # Well-formed selector (306 bytes): ~0.001s # Malformed selector (304 bytes): >3.0s (may need to be killed) # Slowdown: >3000x # # NOTE: On some systems the malformed selector may hang indefinitely. # Use a timeout mechanism (signal.alarm, threading.Timer) when testing. ``` **Safe testing variant with timeout:** ```python import signal import soupsieve as sv def timeout_handler(signum, frame): raise TimeoutError("ReDoS confirmed: regex backtracking exceeded timeout") PAYLOAD_LEN = 300 malformed = '[a="' + ('x' * PAYLOAD_LEN) signal.signal(signal.SIGALRM, timeout_handler) signal.alarm(3) # 3-second timeout try: sv.compile(malformed) print("Selector compiled (not vulnerable)") except TimeoutError as e: print(f"VULNERABLE: {e}") except Exception as e: print(f"Other error: {e}") finally: signal.alarm(0) # Cancel the alarm ``` ### Impact **Severity: High** An attacker can cause CPU exhaustion on any server-side Python application that compiles user-supplied CSS selectors via soupsieve. The attack is particularly dangerous because: 1. **Tiny payload:** Only 300 bytes are needed - well within typical URL parameter, form field, or API request limits 2. **No special characters:** The payload consists entirely of printable ASCII characters (`[a="xxx...`) 3. **Exponential scaling:** Each additional byte approximately doubles the backtracking time, making the attack easily tuneable 4. **Thread blocking:** The regex engine blocks the calling thread with no opportunity for interruption (except via OS signals) | Parameter | Value | |---|---| | Input size | 300 bytes | | CPU time consumed | >3 seconds (exponential with payload length) | | Memory consumed | Negligible (CPU-only attack) | | Authentication requir

Pallets Click, versions 8.3.2 and below, contain a command injection vulnerability in the click.edit() function, allowing attackers to pass arbitrary OS commands from an unprivileged account.

### Impact A specially crafted argument to the `idna.encode()` function could consume significant resources. This may lead to a denial-of-service. ### Patches The function has been refined to reject such strings without the associated resource consumption in version 3.7. ### Workarounds Domain names cannot exceed 253 characters in length, if this length limit is enforced prior to passing the domain to the `idna.encode()` function it should no longer consume significant resources. This is triggered by arbitrarily large inputs that would not occur in normal usage, but may be passed to the library assuming there is no preliminary input validation by the higher-level application. ### References * https://huntr.com/bounties/93d78d07-d791-4b39-a845-cbfabc44aadb

urllib3, when used in a Pyodide runtime environment such as browsers or Node.js, does not enforce its redirect control parameters (`retries` and `redirect`). This is because the runtime itself controls redirect behavior, ignoring urllib3's settings. This can lead to applications relying on urllib3 for redirect control remaining vulnerable to SSRF or open redirect attacks if they depend on disabling redirects. A patched version of urllib3 is available for Node.js usage, but browsers inherently lack mechanisms to control redirects via XMLHttpRequest or the Fetch API, resulting in default redirect behavior.

This is the same issue as CVE-2024-3651, however the original remediation in 2024 was not a complete fix. Payloads such as `"\u0660" * N` or `"\u30fb" * N + "\u6f22"` utilize the `valid_contexto` function prior to length rejection, and for high values of `N` will take a long time to process. ### Impact A specially crafted argument to the `idna.encode()` function could consume significant resources. This may lead to a denial-of-service. ### Patches Starting in version 3.14, the function rejects long inputs as soon as practicable prior to any further processing to minimize resource consumption. In version 3.15, this approach was extended to lesser used alternate functions (i.e. per-label conversions and codec support). ### Workarounds Domain names cannot exceed 253 characters in length, if this length limit is enforced prior to passing the domain to the `idna.encode()` function it should no longer consume significant resources. This is triggered by arbitrarily large inputs that would not occur in normal usage, but may be passed to the library assuming there is no preliminary input validation by the higher-level application.

### Impact urllib3's [streaming API](https://urllib3.readthedocs.io/en/2.7.0/advanced-usage.html#streaming-and-i-o) is designed for the efficient handling of large HTTP responses by reading the content in chunks, rather than loading the entire response body into memory at once. urllib3 can perform decompression based on the HTTP `Content-Encoding` header (e.g., `gzip`, `deflate`, `br`, or `zstd`). When using the streaming API since version 2.6.0, the library decompresses only the necessary bytes, enabling partial content consumption. However, urllib3 before version 2.7.0 could still decompress the whole response instead of the requested portion in two cases: 1. During the second `HTTPResponse.read(amt=N)` call when the response was decompressed using the official [Brotli](https://pypi.org/project/brotli/) library. 2. When `HTTPResponse.drain_conn()` was called after the response had been read and decompressed partially (compression algorithm did not matter here). These issues could cause urllib3 to fully decode a small amount of highly compressed data in a single operation. This could result in excessive resource consumption (high CPU usage and massive memory allocation for the decompressed data; CWE-409) on the client side. ### Affected usages Applications and libraries using urllib3 versions earlier than 2.7.0 may be affected when streaming compressed responses from untrusted sources in either of these cases, unless decompression is explicitly disabled: 1. A response encoded with `br` is read incrementally with at least two `HTTPResponse.read(amt=N)` or `HTTPResponse.stream(amt=N)` calls while using the official [Brotli](https://pypi.org/project/brotli/) library. 2. `HTTPResponse.drain_conn()` is called after response decompression has already started. ### Remediation Upgrade to at least urllib3 version 2.7.0 in which the library: 1. Is more efficient for reads with Brotli. 2. Always skips decompression for `HTTPResponse.drain_conn()`. If upgrading is not immediately possible, the following workarounds may reduce exposure in specific cases: 1. For the Brotli-specific issue only, switch from [brotli](https://pypi.org/project/brotli/) to [brotlicffi](https://pypi.org/project/brotlicffi/) until you can upgrade urllib3; the official Brotli package is affected because of https://github.com/google/brotli/issues/1396. 2. If your code explicitly calls `HTTPResponse.drain_conn()`, call `HTTPResponse.close()` instead when connection reuse is not important. ### Credits The Brotli-specific issue was reported by @kimkou2024. `HTTPResponse.drain_conn()` inefficiency was reported by @Cycloctane.