### Summary A remote, unauthenticated peer can leak one direct `ByteBuf` per HTTP/2 `DATA` frame in applications that enable HTTP/2 content decompression via `DelegatingDecompressorFrameListener`. When a `DATA` frame is processed for a stream whose decompressor has already been closed, `Http2Decompressor.decompress(...)` retains the frame buffer but never releases it on the error path, so its reference count never returns to zero. Repeating this over a long-lived HTTP/2 connection exhausts direct memory and crashes the JVM with `OutOfMemoryError` — a denial of service. ### Details In `codec-http2/src/main/java/io/netty/handler/codec/http2/DelegatingDecompressorFrameListener.java`, `Http2Decompressor.decompress(...)` does: ```java // around line 433 decompressor.writeInbound(data.retain()); ``` The argument `data.retain()` is evaluated **before** `writeInbound(...)` executes, incrementing the buffer's reference count (`refCnt: 1 -> 2`). The very first statement of `EmbeddedChannel.writeInbound(...)` is `ensureOpen()` (`EmbeddedChannel.java:360`), which throws `ClosedChannelException` when the decompressor's internal `EmbeddedChannel` has already been closed. When that happens: - the `DATA` payload has been `retain()`ed but never entered the pipeline, so the decoder's `finally { release() }` never runs; - the surrounding `catch (Throwable t)` block in `decompress(...)` (around line 451) does **not** release the extra reference; - the input buffer therefore can never reach refCnt 0, and its (typically direct) memory is leaked. The decompressor channel is closed on a reachable path: `Http2Connection` `onStreamRemoved` → `Http2Decompressor.cleanup()` → `EmbeddedChannel.finishAndReleaseAll()` (`DelegatingDecompressorFrameListener.java:125-133` and `418-420`). A peer that sends `DATA` frames for a stream whose decompressor has already been cleaned up (e.g. continuing to send `DATA` after `END_STREAM` / stream removal) thus leaks one direct `ByteBuf` per frame. **Affected code**: `DelegatingDecompressorFrameListener.java`, method `Http2Decompressor.decompress(...)` — the `decompressor.writeInbound(data.retain())` call (line ~433) and its `catch (Throwable t)` block (line ~451), which lacks a `data.release()` rollback. **Suggested fix**: track whether `writeInbound` succeeded and roll back the extra `retain()` only when the data never entered the pipeline: ```java boolean writeSucceeded = false; try { decompressor.writeInbound(data.retain()); writeSucceeded = true; // pipeline now owns the release if (endOfStream) { decompressor.finish(); } return 0; } catch (Throwable t) { if (!writeSucceeded) { data.release(); // roll back the extra retain(); data never entered pipeline } if (t instanceof Http2Exception) { throw (Http2Exception) t; } throw streamError(stream.id(), INTERNAL_ERROR, t, ...); } ``` | Case | writeSucceeded | catch action | Reason | |------|:---:|---|---| | `ensureOpen()` throws (this bug) | `false` | `data.release()` | data never entered pipeline | | handler throws internally | `true` | no release | decoder `finally` already released | | `finish()` throws | `true` | no release | `writeInbound` already succeeded | ### PoC Reproduced against the official, unmodified `netty-codec-http2-4.2.15.Final.jar` from Maven Central, using real netty classes and measuring `ByteBuf.refCnt()` directly (the leaking logic is not mocked). Reproduction steps: 1. Download the official artifacts and their dependencies from Maven Central (version `4.2.15.Final`): `netty-common`, `netty-buffer`, `netty-transport`, `netty-resolver`, `netty-handler`, `netty-codec-base`, `netty-codec`, `netty-codec-http`, `netty-codec-http2`, `netty-codec-compression`. 2. Build a real `Http2Decompressor` wrapping a real gzip decoder `EmbeddedChannel` (`ZlibCodecFactory.newZlibDecoder(ZlibWrapper.GZIP)`). 3. Close the internal decompressor channel (equivalent to the end state of `cleanup()` / `finishAndReleaseAll()`). 4. Encode a real gzip `DATA` payload with `ZlibCodecFactory.newZlibEncoder(GZIP)` (`refCnt = 1`). 5. Call `decompress(...)` on the closed channel. 6. Observe: `writeInbound(...)` throws `ClosedChannelException` at its `ensureOpen()` entry (`EmbeddedChannel.java:360`), reached from `DelegatingDecompressorFrameListener.java:433`; `data.refCnt()` is now `2`. 7. Release once as the frame reader would; `refCnt` stays at `1` (`release()` returns `false`) → leaked. Observed reference-count trace: ``` gzipData initial refCnt = 1 decompress -> data.retain() -> refCnt = 2 (retain applied, never rolled back) caller releases once -> refCnt = 1 (release() returns false; not deallocated) => buffer never reaches 0 -> direct memory leaked ``` Observed exception stack (confirms the leak point): ``` java.nio.channels.ClosedChannelException at io.netty.channel.embedded.EmbeddedChannel.checkO

AI Analysis