Using tcpdump to Troubleshoot “ssh: connect to host port 22: Operation timed out”

When you encounter the frustrating “ssh: connect to host port 22: Operation timed out” error, it means your SSH client sent a connection request to a server, but received no response.

While the cause can range from firewalls to network misconfigurations, the tcpdump command-line utility is a powerful tool to see exactly what’s happening on the network layer.

This guide will walk you through using tcpdump to diagnose this specific SSH issue.

Understanding the SSH Handshake

Before diving into tcpdump, it’s helpful to understand the basic TCP three-way handshake that initiates an SSH connection:

1. SYN: Your client sends a SYN (synchronize) packet to the server on port 22 to initiate a connection.
2. SYN-ACK: The server, if it’s listening and accepts the connection, responds with a SYN-ACK (synchronize-acknowledge) packet.
3. ACK: Your client completes the handshake by sending an ACK (acknowledge) packet back to the server.

The “Operation timed out” error typically occurs when this handshake fails, most often because the client’s SYN packet goes unanswered.

The Strategy: Capture on Both Ends

The most effective way to troubleshoot with tcpdump is to run it on both the client (your local machine) and the server (the remote machine you’re trying to connect to) simultaneously. This allows you to see if packets are leaving the client and if they are arriving at the server.

You will need shell access to the server, which might require using a web-based console or another access method if SSH is not working.

Step 1: Capturing Packets on the Client

On your local machine, open a terminal window and start tcpdump to listen for traffic related to the SSH connection attempt.

The Command:

sudo tcpdump -i any -n 'port 22 and host SERVER_IP'

Let’s break down this command:

• sudo: tcpdump requires root privileges to capture network packets.
• i any: This tells tcpdump to listen on all network interfaces (e.g., Wi-Fi, Ethernet).
• n: This option prevents tcpdump from resolving IP addresses to hostnames, which can speed up the output.
• 'port 22 and host SERVER_IP': This is the crucial filter. It instructs tcpdump to only show packets that are destined for or coming from port 22 and involve the specified server’s IP address. Replace SERVER_IP with the actual IP address of the server you’re trying to connect to.

Execution and Analysis:

1. Run the tcpdump command above in one terminal.
2. Open a second terminal window and attempt your SSH connection: ssh user@SERVER_IP.
3. Observe the output in the first terminal.

What You Should See (The Ideal Scenario):

You should see an outgoing packet from your client to the server. The key indicator is the S flag, which denotes a SYN packet.

10:30:01.123456 IP YOUR_CLIENT_IP.54321 > SERVER_IP.22: Flags [S], seq 1234567890, win 65535, options [mss 1460,sackOK,TS val 123 ecr 0,nop,wscale 7], length 0

This output confirms that your client is successfully sending the initial connection request.

Step 2: Capturing Packets on the Server

Now, perform a similar capture on the server to see if it’s receiving the client’s packets. You’ll need to be logged into your server through a console or an alternative method.

The Command:

sudo tcpdump -i any -n 'port 22 and host CLIENT_IP'

Notice the filter is now looking for traffic involving your client’s IP address. You can find your public client IP by searching “what is my IP” in a browser.

Execution and Analysis:

1. Run the tcpdump command on the server.
2. From your client machine, attempt the SSH connection again.
3. Observe the tcpdump output on the server.

Step 3: Interpreting the Results

By comparing the output from both the client and the server, you can pinpoint the location of the problem.

Scenario 1: SYN packet sent by client, but NOT received by server.

• Client tcpdump shows: An outgoing SYN packet (with the [S] flag).
• Server tcpdump shows: No incoming packets from the client’s IP.
• Conclusion: The packet is being lost or blocked somewhere between your client and the server.
• Likely Causes:
  • A firewall on your local network or your ISP is blocking outbound traffic on port 22.
  • A network firewall in front of the server (like an AWS Security Group, Google Cloud firewall rule, or a corporate firewall) is blocking inbound traffic on port 22.
  • An incorrect routing configuration in the network.

Scenario 2: SYN packet sent by client AND received by server, but no reply.

• Client tcpdump shows: An outgoing SYN packet, and possibly some retransmissions.
• Server tcpdump shows: An incoming SYN packet from your client’s IP.
• Conclusion: The server received the request but is not responding.
• Likely Causes:
  • The SSH service (sshd) is not running on the server.
  • A firewall on the server itself (like ufw or firewalld) is blocking incoming connections on port 22, even though the packet reached the server’s network interface.
  • The SSH service is configured to listen on a different port.

Scenario 3: Server sends a SYN-ACK, but it’s not received by the client.

• Client tcpdump shows: Only the outgoing SYN packet being sent repeatedly.
• Server tcpdump shows: An incoming SYN, followed by an outgoing SYN-ACK (with the [S.] flag).
• Conclusion: The server is responding, but the reply is not making it back to your client.
• Likely Causes:
  • A firewall is blocking the return traffic. This is common in misconfigured network ACLs or stateful firewall rules.
  • Asymmetric routing issues where the return path is different and problematic.

By using tcpdump to visualize the fundamental packet exchange of an SSH connection, you can move beyond guesswork and gather concrete evidence to identify the root cause of the “Operation timed out” error.