Scrapy's handling of system proxy settings is scheme-specific, meaning different proxies can be set for HTTP and HTTPS URLs. However, during HTTP redirects that change the URL scheme, Scrapy did not update the proxy accordingly. For example, an HTTP request using an HTTP proxy redirected to an HTTPS URL would still use the HTTP proxy instead of switching to the HTTPS proxy. This flaw could expose URLs visited via one proxy to another proxy provider, undermining security configurations that rely on scheme-specific proxies. The vulnerability affects Scrapy versions >=2.9.0 and <2.11.2 and is resolved by upgrading to version 2.11.2.

The vulnerability can lead to unintended proxy usage during redirects, potentially exposing URLs to proxy providers that should not have access to them. This compromises the confidentiality of browsing activity when different proxies are used for HTTP and HTTPS for security reasons. There is no indication of integrity or availability impact. No known exploits in the wild have been reported.

Upgrade Scrapy to version 2.11.2 or later, where this issue is fixed. Alternatively, implement custom middleware replacements for RedirectMiddleware, MetaRefreshMiddleware, and HttpProxyMiddleware that incorporate the fix from Scrapy 2.11.2. Verify the custom solution correctly handles scheme-specific proxy switching during redirects.