The vulnerability in Netty's WebSocketServerExtensionHandler arises because it maintains an unbounded queue per connection for valid WebSocket extensions. For every inbound HttpRequest, an entry is added to this queue, but entries are only removed when the application writes an HttpResponse. An attacker can exploit this by sending HTTP/1.1 pipelined requests faster than the server responds, causing the queue to grow indefinitely. This can lead to JVM heap exhaustion and an OutOfMemoryError, crashing the server. Since WebSocketServerCompressionHandler inherits from this handler, servers enabling permessage-deflate are also vulnerable on their plain HTTP ports before WebSocket upgrades or authentication occur. The issue affects Netty versions 4.1.88.Final through 4.1.137.Final and 4.2.0.Final through 4.2.17.Final and is resolved in 4.1.138.Final and 4.2.18.Final.

A remote unauthenticated attacker can cause a denial of service by exhausting the JVM heap memory through unbounded queue growth, leading to server crashes (OutOfMemoryError). There is no impact on confidentiality or integrity reported.

Upgrade affected Netty versions to 4.1.138.Final or later, or 4.2.18.Final or later, where the issue is fixed. No other mitigation is indicated by the vendor advisory.