A tor relay's memory allocation during channel establishment: security implications for DDoS attacks

This master's thesis analyzes the memory allocation behavior of Tor relays during the channel establishment process, including TCP connections and TLS handshakes, to support the investigation of a distributed denial-of-service attack against two Tor relays that resulted in memory exhaustion of both Tor processes in 2024. This attack differs greatly from other known DoS attacks on Tor since no Tor channels were established during the incident. The thesis introduces event tracing methods (e.g., custom programs) that enable a detailed analysis of Tor's per-allocation events during accepting a TCP connection, handling TLS handshakes compliant with Tor's link protocol versions, and connection teardown. Additionally, the thesis assesses the DoSConnectionMaxConcurrentCount countermeasure, which limits the number of concurrent TCP connections per client, by analyzing Tor's source code and conducting practical tests. The results of the thesis indicate that the 2024 distributed denial-of-service attack was not caused by an adversary establishing or maintaining large numbers of concurrent TCP connections, performing TLS handshakes compliant with Tor's link protocol, or leveraging residual memory allocations after connection closure for tested connection types.