In a large environment, troubleshooting problems with network packet traces usually means you're logged into a remote host running tcpdump. Even after you develop some skill with
pcap-filter syntax, wielding tcpdump is clunky and it usually looks like you're trying to view The Matrix encoded.
There are other console based tools like tshark, but few of them are as useful and as user-friendly as
Wireshark which can render and parse network packets in an extremely readable and comprehensive fashion.
The problem is that Wireshark is a graphical interface. Running it on a remote host means you'll have to install it and all supporting dependencies and libraries on the remote host and then ssh X tunneling it back to your desktop. For many reasons, this may not work well. Or, you may not even be able to install Wireshark on the remote host for any number of reasons.
One workaround used by a lot of people is to capture some network output with tcpdump writing to a file, then fetch that capture file to your desktop and open it up in Wireshark. It's definitely handy that pcap is so portable that this is possible, but this method lacks the ability to watch network traffic in real-time.
So how can you achieve the holy grail and use Wireshark locally on your desktop to watch live traffic on a remote host?
Enter
socat - Multipurpose relay.
The socat utility is a swiss army knife of basically all possible types if input/ouput. One of its supported i/o types is named pipes.
In short, we can use socat as the middleman to read from a remote named pipe to a local named pipe. Then, we take advantage of Wireshark's ability to read right from a named pipe and read that local named file.
Here's the steps using the example username
jsmith, example remote host name
srv1, and example network interface name
eth0.
On the remote host:
- Create a temp dir for your named pipe file.
- sudo mkdir /tmp/fifo
- sudo chown jsmith /tmp/fifo
- sudo chmod 700 /tmp/fifo
- Create the named pipe
- sudo mkfifo /tmp/fifo/pcappipe
- Kick off tcpdump, writing to that pipe.
- sudo tcpdump -i eth0 -s 0 -U -w /tmp/fifo/pcappipe not port 22
Notice the temp dir permissions. You need to be able to read the named pipe as the non-root user with which you're going to use to log in.
Also notice the pcap filter 'not port 22'. You can alter this of course, but if you don't specifically exclude your ssh traffic, tcpdump is going to pick up all of the traffic from you being logged in as well as the part where we remotely read from the named pipe which takes place over ssh.
Next, on your local desktop, run socat like so:
socat -b 67108864 \
EXEC:"stdbuf -i0 -o0 -e0 ssh -x -C -t srv1 cat /tmp/fifo/pcappipe",pty,raw \
PIPE:/home/jsmith/localpcappipe,wronly=1,noatime
This tells socat to ssh into the remote host and cat the named pipe (sending the data to STDOUT). It reads from that and writes it to the named pipe file in your home directory.
The buffer tuning was important to making it as live as possible as well as more stable. Plus, this can be somewhat of a brittle process and socat can end up crashing easily. The buffer tuning helps make things much more stable and reliable.
Next, run wireshark, as root.
sudo wireshark -s 0 -k -i /home/jsmith/localpcappipe
Profit!
Normal ssh rules apply. So, if you can't ssh directly to your remote host, configure your .ssh/config file accordingly.
I need to tunnel through an intermediary jump host as well, so this is what I do in my .ssh/config file:
Host srv1
ProxyCommand ssh jumpsrv1 /usr/bin/nc %h 22
User jsmith
IdentityFile ~/.ssh/id_rsa
Compression yes
PubkeyAuthentication yes
Port 22
Protocol 2
EscapeChar none
ServerAliveInterval 30
(I know I know, there's a new ProxyJump directive... I don't change my .ssh/config that often.)
