TCP in System Design: Reliable Connection Guide

TL;DR

TCP (Transmission Control Protocol) is a connection-oriented transport protocol that guarantees reliable, ordered delivery of data between applications over IP networks. It uses a three-way handshake to establish connections, acknowledgments and retransmissions for reliability, and sophisticated flow control and congestion control algorithms to optimize throughput while preventing network collapse. TCP is the foundation for most internet applications (HTTP, SSH, databases) where data integrity matters more than minimal latency.

Background

Before TCP, early network protocols like NCP (Network Control Program) in ARPANET struggled with reliability and congestion. When a packet was lost, applications had to detect and handle it themselves. In 1974, Vint Cerf and Bob Kahn published the TCP/IP specification, splitting network responsibilities into layers: IP for routing packets across networks, and TCP for ensuring those packets arrived correctly and in order. This separation became the foundation of the modern internet.

TCP solves a fundamental problem: unreliable networks. Packets can be lost, duplicated, corrupted, or arrive out of order due to routing changes. Applications shouldn’t need to handle these network-level failures. TCP provides an abstraction—a reliable byte stream—that hides this complexity. When you write data to a TCP socket, TCP guarantees it will either arrive intact at the destination or you’ll get an error indicating the connection failed. This guarantee is why TCP became the default choice for most internet applications.

The protocol operates at Layer 4 (Transport) of the OSI model. See Communication Overview for TCP’s role in the protocol stack. Unlike UDP, which trades reliability for speed (see UDP for connectionless transport), TCP prioritizes correctness. This design choice shaped the internet: reliable protocols like HTTP, SMTP, and SSH all build on TCP’s guarantees.

Architecture

TCP’s architecture centers on connection-oriented communication. Before any data flows, two endpoints must establish a connection through a three-way handshake. This creates state on both sides: sequence numbers, window sizes, and congestion control parameters.

The key architectural components are:

Connection State Machine: Each TCP connection progresses through states: CLOSED → LISTEN → SYN-SENT → SYN-RECEIVED → ESTABLISHED → FIN-WAIT → TIME-WAIT → CLOSED. The state machine ensures both sides agree on connection lifecycle events. The TIME-WAIT state (typically 2 minutes) prevents old duplicate packets from interfering with new connections using the same port pair.

Sequence and Acknowledgment Numbers: Every byte transmitted gets a sequence number. The receiver sends back acknowledgments (ACKs) indicating which bytes it has received. This bidirectional numbering system enables TCP to detect missing or duplicate packets. Initial sequence numbers are randomized to prevent attacks and avoid confusion with old connections.

Send and Receive Buffers: The operating system maintains buffers for each connection. The send buffer holds data the application has written but TCP hasn’t yet transmitted or hasn’t been acknowledged. The receive buffer holds data TCP has received but the application hasn’t read yet. Buffer sizes directly impact throughput and memory usage.

Sliding Window Protocol: TCP uses a sliding window for both flow control and reliability. The sender can transmit multiple packets before waiting for acknowledgments, up to the window size. As ACKs arrive, the window slides forward, allowing more data to be sent. This pipelining is crucial for high-throughput connections, especially over high-latency links where waiting for each ACK would waste bandwidth.

Timers: TCP uses multiple timers: retransmission timers (when to resend unacknowledged data), persist timers (when the receive window is zero), keepalive timers (detecting dead connections), and TIME-WAIT timers (preventing port reuse issues). The retransmission timeout (RTO) is dynamically calculated based on measured round-trip time (RTT) using algorithms like Jacobson/Karels.

TCP Connection Architecture and State Components

graph TB
    subgraph Application Layer
        App["Application<br/><i>write() / read()</i>"]
    end
    
    subgraph TCP Layer - Sender Side
        SendBuf["Send Buffer<br/><i>Unacknowledged data</i>"]
        SeqNum["Sequence Number<br/><i>Tracks bytes sent</i>"]
        CongWin["Congestion Window<br/><i>cwnd - network capacity</i>"]
        RetransTimer["Retransmission Timer<br/><i>RTO calculation</i>"]
    end
    
    subgraph TCP Layer - Receiver Side
        RecvBuf["Receive Buffer<br/><i>Out-of-order data</i>"]
        AckNum["Acknowledgment Number<br/><i>Next expected byte</i>"]
        RecvWin["Receive Window<br/><i>Available buffer space</i>"]
    end
    
    subgraph Connection State
        StateMachine["State Machine<br/><i>LISTEN/ESTABLISHED/etc</i>"]
        Timers["Timers<br/><i>Keepalive, TIME-WAIT</i>"]
    end
    
    subgraph Network Layer
        IP["IP Layer<br/><i>Packet routing</i>"]
    end
    
    App -->|"write()"| SendBuf
    SendBuf --> SeqNum
    SeqNum --> CongWin
    CongWin -->|"Min(cwnd, recv_win)"| IP
    RetransTimer -.->|"Timeout"| SendBuf
    
    IP -->|"Incoming segments"| RecvBuf
    RecvBuf --> AckNum
    AckNum -->|"ACK packets"| IP
    RecvWin -->|"Advertise in ACK"| IP
    RecvBuf -->|"read()"| App
    
    StateMachine -.-> SendBuf
    StateMachine -.-> RecvBuf
    Timers -.-> StateMachine

TCP maintains per-connection state including send/receive buffers, sequence and acknowledgment numbers, congestion and receive windows, and multiple timers. The actual sending rate is limited by the minimum of the congestion window (network capacity) and receive window (receiver buffer space), ensuring both network and endpoint constraints are respected.

Internals

TCP’s reliability and performance come from several sophisticated mechanisms working together.

Three-Way Handshake: Connection establishment requires three packets. The client sends SYN (synchronize) with its initial sequence number. The server responds with SYN-ACK, acknowledging the client’s sequence number and providing its own. The client sends ACK, acknowledging the server’s sequence number. Only after this handshake can data flow. This adds one full round-trip time (RTT) of latency before any application data is transmitted—a critical consideration for short-lived connections. Modern optimizations like TCP Fast Open (TFO) allow data in the initial SYN packet for repeat connections.

Reliability Mechanisms: TCP achieves reliability through acknowledgments and retransmissions. Each segment includes a checksum to detect corruption. The receiver sends cumulative ACKs indicating the next expected sequence number. If the sender doesn’t receive an ACK within the retransmission timeout, it resends the segment. Selective Acknowledgment (SACK) is an extension that allows receivers to acknowledge non-contiguous blocks of data, enabling faster recovery from multiple packet losses.

TCP uses duplicate ACKs as an early warning system. If the receiver gets packets out of order, it sends duplicate ACKs for the last in-order byte. Three duplicate ACKs trigger fast retransmit—resending the missing segment immediately without waiting for the timeout. This significantly reduces recovery time.

Flow Control: TCP’s flow control prevents a fast sender from overwhelming a slow receiver. The receiver advertises a receive window in every ACK, indicating how much buffer space it has available. The sender cannot send more unacknowledged data than this window size. If the receiver’s buffer fills up, it advertises a zero window, pausing transmission until space becomes available. This per-connection flow control is distinct from congestion control, which manages network capacity.

Congestion Control: TCP’s congestion control algorithms prevent network collapse when too many senders compete for bandwidth. The sender maintains a congestion window (cwnd) separate from the receive window. The actual sending rate is limited by the minimum of these two windows.

TCP congestion control has several phases:

• Slow Start: New connections start with cwnd = 1 MSS (maximum segment size, typically 1460 bytes). For each ACK received, cwnd increases by 1 MSS, causing exponential growth. This quickly ramps up to available bandwidth.
• Congestion Avoidance: Once cwnd reaches the slow start threshold (ssthresh), growth becomes linear—increasing by 1 MSS per RTT. This probes for additional bandwidth more cautiously.
• Fast Recovery: When packet loss is detected via duplicate ACKs (not timeout), TCP enters fast recovery. It reduces cwnd by half, retransmits the missing segment, and continues transmitting new data. This maintains some throughput during recovery.
• Timeout: If a retransmission timeout occurs, TCP assumes severe congestion. It sets ssthresh to half the current cwnd, resets cwnd to 1 MSS, and restarts slow start.

Modern variants like TCP Cubic (Linux default) and BBR (developed by Google) improve on classic Reno/NewReno algorithms. BBR focuses on measuring bottleneck bandwidth and RTT rather than inferring congestion from packet loss, achieving better throughput on lossy networks.

Nagle’s Algorithm: To reduce overhead from tiny packets, Nagle’s algorithm buffers small writes until either enough data accumulates for a full segment or an ACK arrives for previous data. This improves efficiency but can add latency for interactive applications. Many applications disable Nagle (TCP_NODELAY socket option) for request-response protocols where latency matters.

TCP Three-Way Handshake Connection Establishment

sequenceDiagram
    participant Client
    participant Server
    
    Note over Client,Server: Connection Establishment (3-Way Handshake)
    Client->>Server: 1. SYN (seq=x)
    Note over Client: State: SYN-SENT
    Note over Server: State: SYN-RECEIVED
    Server->>Client: 2. SYN-ACK (seq=y, ack=x+1)
    Note over Client: State: ESTABLISHED
    Client->>Server: 3. ACK (seq=x+1, ack=y+1)
    Note over Server: State: ESTABLISHED
    
    Note over Client,Server: Data Transfer Phase
    Client->>Server: Data (seq=x+1)
    Server->>Client: ACK (ack=x+1+len)
    
    Note over Client,Server: Connection Termination (4-Way Handshake)
    Client->>Server: 4. FIN (seq=x+n)
    Note over Client: State: FIN-WAIT-1
    Server->>Client: 5. ACK (ack=x+n+1)
    Note over Client: State: FIN-WAIT-2
    Server->>Client: 6. FIN (seq=y+m)
    Note over Server: State: LAST-ACK
    Client->>Server: 7. ACK (ack=y+m+1)
    Note over Client: State: TIME-WAIT (2 minutes)
    Note over Server: State: CLOSED
    Note over Client: State: CLOSED

TCP connection lifecycle showing the three-way handshake for establishment (SYN, SYN-ACK, ACK), data transfer phase, and four-way handshake for graceful termination. The TIME-WAIT state prevents port reuse conflicts and handles delayed packets from the old connection.

TCP Sliding Window Flow Control Mechanism

sequenceDiagram
    participant Sender
    participant Network
    participant Receiver
    
    Note over Receiver: Receive Buffer: 8KB available
    Receiver->>Sender: Window Size = 8KB
    
    Note over Sender: Can send up to 8KB unacknowledged
    Sender->>Network: Segment 1 (2KB, seq=1000)
    Sender->>Network: Segment 2 (2KB, seq=3000)
    Sender->>Network: Segment 3 (2KB, seq=5000)
    Sender->>Network: Segment 4 (2KB, seq=7000)
    Note over Sender: Window full - must wait
    
    Network->>Receiver: Segment 1 arrives
    Network->>Receiver: Segment 2 arrives
    Note over Receiver: App reads 4KB<br/>Buffer: 4KB available
    Receiver->>Sender: ACK=5000, Window=4KB
    
    Note over Sender: Window slides forward<br/>Can send 4KB more
    Sender->>Network: Segment 5 (2KB, seq=9000)
    Sender->>Network: Segment 6 (2KB, seq=11000)
    
    Network->>Receiver: Segment 3 arrives
    Network->>Receiver: Segment 4 arrives
    Note over Receiver: Buffer full<br/>App hasn't read yet
    Receiver->>Sender: ACK=9000, Window=0
    Note over Sender: Zero window - pause sending<br/>Start persist timer

TCP’s sliding window protocol enables pipelining multiple segments before receiving acknowledgments, maximizing throughput. The receiver advertises available buffer space, and the sender respects this window to prevent overflow. When the window reaches zero, transmission pauses until the application reads data and frees buffer space.

TCP Congestion Control State Machine

stateDiagram-v2
    [*] --> SlowStart: Connection starts<br/>cwnd = 1 MSS
    
    SlowStart --> CongestionAvoidance: cwnd >= ssthresh<br/>(threshold reached)
    SlowStart --> FastRecovery: 3 duplicate ACKs<br/>(packet loss detected)
    SlowStart --> SlowStart: ACK received<br/>cwnd += 1 MSS<br/>(exponential growth)
    
    CongestionAvoidance --> FastRecovery: 3 duplicate ACKs<br/>ssthresh = cwnd/2<br/>cwnd = ssthresh
    CongestionAvoidance --> CongestionAvoidance: ACK received<br/>cwnd += 1 MSS per RTT<br/>(linear growth)
    CongestionAvoidance --> SlowStart: Timeout<br/>ssthresh = cwnd/2<br/>cwnd = 1 MSS
    
    FastRecovery --> CongestionAvoidance: New ACK received<br/>(recovery complete)
    FastRecovery --> FastRecovery: Duplicate ACK<br/>cwnd += 1 MSS<br/>(inflate window)
    FastRecovery --> SlowStart: Timeout<br/>ssthresh = cwnd/2<br/>cwnd = 1 MSS
    
    note right of SlowStart
        Exponential growth phase
        Quickly probe for bandwidth
        cwnd doubles per RTT
    end note
    
    note right of CongestionAvoidance
        Linear growth phase
        Cautiously probe for more bandwidth
        Additive increase
    end note
    
    note right of FastRecovery
        Recover from single packet loss
        Maintain partial throughput
        Avoid slow start penalty
    end note

TCP congestion control prevents network collapse through three phases: Slow Start (exponential growth), Congestion Avoidance (linear growth), and Fast Recovery (quick recovery from packet loss). Timeouts trigger aggressive backoff to slow start, while duplicate ACKs allow faster recovery through fast retransmit.

Performance Characteristics

TCP’s performance profile reflects its reliability-first design. Understanding these characteristics helps you make informed architectural decisions.

Latency: TCP adds latency through multiple mechanisms. The three-way handshake adds one RTT before any data flows. For a connection from California to New York (~70ms RTT), that’s 70ms of setup overhead. For short requests (like DNS queries or microservice calls), this overhead dominates total latency. Connection pooling and keep-alive connections amortize this cost across multiple requests.

Retransmissions add variable latency. A lost packet triggers either fast retransmit (1 RTT delay) or timeout-based retransmission (typically 200ms-3s). On a 1% packet loss network, you’ll see periodic latency spikes as TCP recovers.

Throughput: TCP’s throughput depends on bandwidth, RTT, and packet loss. The bandwidth-delay product (BDP) determines how much data can be “in flight” simultaneously. For a 1 Gbps link with 50ms RTT, BDP = 1,000,000,000 bits/sec × 0.05 sec = 50 Mb = 6.25 MB. Your TCP window must be at least this large to fully utilize the link.

The Mathis equation estimates TCP throughput: Throughput ≤ (MSS / RTT) × (1 / √packet_loss). For MSS=1460 bytes, RTT=50ms, and 0.1% loss: Throughput ≤ (1460 / 0.05) × (1 / √0.001) ≈ 924 KB/sec. This shows why packet loss devastates throughput and why low-latency networks achieve higher speeds.

Connection Overhead: Each TCP connection consumes kernel memory for buffers and state. Linux defaults to 87 KB per connection (send + receive buffers). A server with 100,000 concurrent connections uses ~8.7 GB just for TCP buffers. This is why high-concurrency servers like NGINX carefully tune buffer sizes and use connection pooling for backend connections.

Connection establishment and teardown have CPU costs. At Google’s scale, TCP handshake processing consumed significant CPU. This motivated QUIC development, which reduces handshake overhead. See HTTP for how HTTP/3 uses QUIC.

Scalability: TCP scales well for moderate connection counts but faces challenges at extreme scale. The C10K problem (handling 10,000 concurrent connections) drove innovations in event-driven I/O (epoll, kqueue). Modern servers routinely handle millions of connections using techniques like SO_REUSEPORT (distributing connections across CPU cores) and careful memory management.

Trade-offs

TCP makes deliberate trade-offs that make it ideal for some use cases and problematic for others.

Reliability vs. Latency: TCP’s reliability guarantees come at a latency cost. Head-of-line blocking means if packet N is lost, packets N+1, N+2, etc. cannot be delivered to the application even if they arrived successfully. The application waits while TCP retransmits packet N. For video streaming or gaming, this delay is worse than occasional missing data. This trade-off is why real-time applications often prefer UDP.

Congestion Control vs. Throughput: TCP’s congestion control is conservative by design, backing off when it detects congestion. This fairness prevents network collapse but can underutilize available bandwidth. On high-bandwidth, high-latency networks (like satellite links), TCP’s slow start takes many RTTs to ramp up to full speed. Newer algorithms like BBR address this, but classic TCP can leave bandwidth unused.

Connection State vs. Simplicity: TCP’s connection state enables reliability but creates complexity. Servers must track state for every connection, consuming memory and creating attack surfaces (SYN flood attacks exploit handshake state). Stateless protocols like UDP are simpler and more resilient to certain attacks, though they push complexity to the application layer.

Ordered Delivery vs. Parallelism: TCP delivers bytes in strict order within a connection. If you need to transfer multiple independent objects (like web page resources), a single TCP connection serializes them. HTTP/2 multiplexes multiple streams over one TCP connection but still suffers head-of-line blocking at the TCP layer. HTTP/3 switched to QUIC specifically to enable true stream independence.

Universality vs. Optimization: TCP is a general-purpose protocol, not optimized for any specific use case. Specialized protocols can outperform TCP in their niche. For example, RDMA (Remote Direct Memory Access) bypasses TCP for data center networks, achieving microsecond latencies. But TCP’s universality—working across any IP network—makes it the safe default choice.

TCP vs UDP Protocol Comparison

graph TB
    subgraph TCP Characteristics
        TCP_Conn["Connection-Oriented<br/><i>3-way handshake required</i>"]
        TCP_Rel["Reliable Delivery<br/><i>ACKs + retransmissions</i>"]
        TCP_Order["Ordered Delivery<br/><i>Sequence numbers</i>"]
        TCP_Flow["Flow Control<br/><i>Sliding window</i>"]
        TCP_Cong["Congestion Control<br/><i>Prevents network collapse</i>"]
        TCP_Over["Higher Overhead<br/><i>20-60 byte headers</i>"]
        TCP_Lat["Higher Latency<br/><i>Handshake + retransmissions</i>"]
    end
    
    subgraph UDP Characteristics
        UDP_Connless["Connectionless<br/><i>No handshake</i>"]
        UDP_Unrel["Unreliable Delivery<br/><i>No guarantees</i>"]
        UDP_Unorder["Unordered Delivery<br/><i>Packets independent</i>"]
        UDP_NoFlow["No Flow Control<br/><i>App responsibility</i>"]
        UDP_NoCong["No Congestion Control<br/><i>Can overwhelm network</i>"]
        UDP_Low["Lower Overhead<br/><i>8 byte header</i>"]
        UDP_Fast["Lower Latency<br/><i>Immediate transmission</i>"]
    end
    
    subgraph Use Cases
        TCP_Use["TCP: Web, Email, File Transfer<br/>Databases, APIs, SSH<br/><i>Data integrity critical</i>"]
        UDP_Use["UDP: Video Streaming, Gaming<br/>VoIP, DNS, IoT Sensors<br/><i>Speed over reliability</i>"]
    end
    
    TCP_Conn --> TCP_Use
    TCP_Rel --> TCP_Use
    TCP_Order --> TCP_Use
    
    UDP_Connless --> UDP_Use
    UDP_Fast --> UDP_Use
    UDP_Low --> UDP_Use

TCP and UDP represent opposite trade-offs in transport protocols. TCP prioritizes reliability, ordering, and congestion control at the cost of higher latency and overhead—ideal for applications where data integrity matters. UDP minimizes latency and overhead by eliminating guarantees—suitable for real-time applications where speed trumps perfect delivery.

When to Use (and When Not To)

Choose TCP when data integrity and ordered delivery are essential. This includes most traditional internet applications.

Use TCP for:

• Web applications: HTTP(S) traffic where every byte of HTML, CSS, JavaScript, and images must arrive correctly. See HTTP for how HTTP uses TCP connections.
• Database connections: SQL queries and results require perfect reliability. A corrupted byte in a financial transaction could be catastrophic.
• File transfers: FTP, SFTP, rsync, and similar tools depend on TCP to ensure files arrive intact.
• Email: SMTP, IMAP, and POP3 use TCP because email messages must be delivered completely and correctly.
• Remote access: SSH and RDP require reliable, ordered delivery for terminal sessions and remote desktop protocols.
• API communication: RESTful APIs and RPC frameworks (gRPC over HTTP/2) typically use TCP for request-response reliability.

Consider alternatives when:

• Real-time communication: Voice/video calls, gaming, live streaming benefit from UDP’s lower latency. Occasional packet loss is acceptable; delay is not. See UDP for connectionless transport.
• High-frequency trading: Microsecond latencies matter more than guaranteed delivery. Custom UDP-based protocols or kernel bypass techniques (DPDK) are common.
• IoT sensor data: Periodic sensor readings where losing occasional samples is acceptable. UDP reduces overhead on constrained devices.
• Multicast/broadcast: TCP is point-to-point only. UDP supports one-to-many communication patterns.
• Very short messages: DNS queries use UDP because the entire request and response fit in single packets, avoiding TCP’s handshake overhead.

Hybrid approaches: Many systems use both. Netflix uses TCP for control plane (API requests, metadata) and adaptive streaming protocols that can tolerate some TCP retransmission delays. WebRTC uses UDP for media streams but TCP for signaling. The key is matching the protocol to each data flow’s requirements.

Real-World Examples

company: Netflix context: Video streaming platform serving 200+ million subscribers implementation: Netflix uses TCP for delivering video streams via HTTP adaptive streaming (DASH/HLS). While UDP might seem better for video, TCP’s reliability ensures every video segment arrives intact, and adaptive bitrate algorithms compensate for TCP’s throughput variations. Netflix’s Open Connect CDN maintains persistent TCP connections to edge servers, using connection pooling to amortize handshake costs. They tune TCP buffer sizes and congestion control parameters per network path, achieving 95%+ bandwidth utilization on most links. interesting_detail: Netflix contributed to TCP BBR development at Google. BBR improved streaming performance on lossy networks (mobile, Wi-Fi) by 20-30% compared to classic TCP congestion control, reducing rebuffering events significantly.

company: Stripe context: Payment processing API handling billions of dollars in transactions implementation: Stripe’s API exclusively uses HTTPS over TCP for all payment requests. The reliability guarantees are non-negotiable—a lost packet in a payment authorization could mean a failed transaction or, worse, a duplicate charge. Stripe maintains connection pools from API servers to backend databases (PostgreSQL) and internal services, reusing TCP connections across requests. They monitor TCP retransmission rates as a key infrastructure health metric; spikes indicate network issues requiring investigation. interesting_detail: Stripe’s infrastructure team found that TCP TIME-WAIT state accumulation on load balancers became a bottleneck during traffic spikes. They tuned tcp_tw_reuse and implemented connection draining strategies to handle 100,000+ connections per server without exhausting ephemeral ports.

company: Slack context: Real-time messaging platform with millions of concurrent WebSocket connections implementation: Slack uses TCP for WebSocket connections that deliver messages in real-time. Each connected client maintains a persistent TCP connection to Slack’s edge servers. The challenge is scaling to millions of concurrent connections while keeping memory usage reasonable. Slack tunes TCP buffer sizes aggressively (smaller than defaults) since most messages are small. They use TCP keepalive to detect dead connections and reclaim resources, with keepalive intervals tuned to balance resource usage against false positives on mobile networks. interesting_detail: Slack’s engineering team discovered that TCP slow start significantly impacted message delivery latency for new connections. They implemented TCP Fast Open (TFO) for repeat connections, reducing first-message latency by 15-30ms—noticeable in a real-time messaging context where every millisecond of perceived lag matters.

---

Interview Essentials

Mid-Level

Explain the TCP three-way handshake step-by-step. What happens if the final ACK is lost?

How does TCP detect packet loss? Describe both timeout-based and fast retransmit mechanisms.

What is the difference between flow control and congestion control in TCP?

Why does TCP use sequence numbers? How does this enable reliability?

What is the TIME-WAIT state and why does it exist? What problems does it solve?

Senior

Design a connection pooling strategy for a microservices architecture. How would you size pools and handle connection failures?

Explain TCP’s congestion control phases (slow start, congestion avoidance, fast recovery). How do modern variants like BBR differ?

You’re seeing high latency spikes in your API. How would you diagnose if TCP retransmissions are the cause? What metrics would you check?

When would you choose to disable Nagle’s algorithm (TCP_NODELAY)? What are the trade-offs?

How does TCP’s head-of-line blocking affect HTTP/2 performance? Why did HTTP/3 switch to QUIC?

Staff+

Design a high-throughput data transfer system between data centers (1000 miles apart, 1 Gbps links). How would you optimize TCP for this scenario? Calculate expected throughput given typical packet loss rates.

Your service handles 500,000 concurrent TCP connections. Walk through the memory and CPU implications. How would you optimize kernel parameters and application architecture?

Compare TCP congestion control algorithms (Reno, Cubic, BBR) for different network conditions (data center, mobile, satellite). Which would you choose for each and why?

Design a system to detect and mitigate SYN flood attacks while minimizing impact on legitimate traffic. Consider both network and application-layer defenses.

Explain how TCP’s reliability guarantees interact with distributed system failure modes. When does TCP’s ‘reliable delivery’ abstraction break down in practice?

Common Interview Questions

Why is TCP called ‘connection-oriented’? What state is maintained?

How does TCP ensure data arrives in order?

What causes TCP to retransmit data?

Explain the sliding window protocol in TCP.

When would you use TCP vs. UDP?

Red Flags to Avoid

Confusing flow control with congestion control—they solve different problems

Not understanding that TCP guarantees delivery OR connection failure, not infinite retries

Claiming TCP is always slower than UDP without context (depends on use case)

Ignoring the memory cost of TCP connections at scale

Not knowing that TCP’s reliability is per-connection, not end-to-end across system failures

---

Key Takeaways

TCP provides reliable, ordered, connection-oriented delivery through a three-way handshake, sequence numbers, acknowledgments, and retransmissions. This reliability comes at the cost of higher latency and overhead compared to UDP.

Flow control (receiver’s buffer management) and congestion control (network capacity management) are distinct mechanisms. Congestion control algorithms like slow start, congestion avoidance, and fast recovery prevent network collapse while maximizing throughput.

TCP’s performance is bounded by the bandwidth-delay product and packet loss rate. High-latency or lossy networks significantly impact throughput. Connection pooling and keep-alive strategies amortize handshake costs for repeated communication.

Choose TCP when data integrity matters more than minimal latency: web applications, databases, file transfers, APIs. Consider UDP for real-time applications (gaming, VoIP) where occasional loss is acceptable but delay is not.

At scale, TCP connection state consumes significant memory and CPU. Tuning kernel parameters (buffer sizes, TIME-WAIT handling, congestion control algorithms) and architectural patterns (connection pooling, load balancing) are essential for high-performance systems.

---