Most of the time, a GUI-based protocol analyzer is the primary weapon in the fight against network issues. It is to the analyst as the Light Saber is to the Jedi. But there is also a lesser known dark side (so called for its typical black background). It goes by many names: the Shell, Terminal, Console, DOS Prompt, or Command Line. This article will introduce you to the power of the dark side, and describe how I was seduced in the heat of one analysis battle in particular.
One of Bitcricket’s recent Enterprise clients was suffering from intermittent printer problems, which was described as everything from printers being slow to printing garbage to widespread outages. Together we identified several potentially interesting printers to be monitored. The client had little in the way of protocol analyzers so using the free Wireshark to capture these printers’ traffic was a no-brainer. Several Wireshark-enabled laptops were attached to switch SPAN ports and configured to capture to disk in ring buffer mode, saving files in 32 MB segments.
What’s that you say? You aren’t a Wireshark user? This article doesn’t apply to you? I disagree. Read on.
The first printer that went down under my watch was one of nine attached to the same switch. This means that nine printer ports were SPANed to a single capture, and therefore nine copies of broadcast and multicast packets were sent to the SPAN port. Due to buffering limitations in this particular switch model, a maximum of five copies made it out of the SPAN port and into the trace files. In addition, the client’s network was suffering from a separate unicast flooding issue, so not only were broadcasts and multicasts duplicated, or more accurately quintupled, but so were some unicasts. Nonetheless, it was four more copies than I needed.
So in my post-capture analysis I had the displeasure of looking at dozens of 32 MB files containing packets from several switch ports, many of them quintupled. For each of these files I needed to filter out unrelated traffic, remove duplicate packets (my own personal Clone Wars), and concatenate the results back into a single file for easier viewing in the analyzer GUI. After going through the process manually in Wireshark, I felt that I should automate it. Fortunately, Wireshark’s command line tools are plentiful and capable. Four of them proved useful in this scenario:
- dumpcap – This utility does one thing and does it well; it captures to disk. While I didn’t use dumpcap in my post-capture analysis, it was used to initiate the round-the-clock capturing and saving of printer traffic.
- editcap – Manipulates capture files, mostly by removing packets but has other uses as well, such as converting between trace file formats. I used editcap’s remove duplicate packets option to reduce quintuplets to a single packet. It also came in handy for slicing packets to a shorter length to reduce file sizes when transferring files via email.
- tshark – Stands for Terminal-based Wireshark. It is the spunky R2D2 of the command line tools and does everything short of projecting a holographic distress call from the Princess. The feature I found most useful in this case was the ability to read an input trace file, apply a Wireshark display filter, and write an output file containing only the packets that passed the filter.
- mergecap – Merges or appends multiple input trace files into a single output trace file. After reducing the original 32 MB-segmented trace files using editcap and tshark, I used megecap to recombine the results into a single file.
To learn more about these command line tools or for some usage examples, check out these external resources:
- Laura Chappell, the Obi Wan Kenobi of Wireshark, recently started the BitSpitters series, which are short, instructional videos posted on YouTube. (Think of scaled down versions of her Animated Articles.) “Edit Trace File Timestamps” is one that demonstrates the use of editcap to adjust timestamps between a client-side capture and a server-side capture.
- Sake Blok is the Yoda of the Wireshark command line, although his name already sounds like that of a Star Wars character. He led a presentation at Sharkfest 2008 titled “Advanced Scripting and Command Line Usage with tshark and Related Utilities”. In his slide deck, he discusses a similar client/server synchronization situation and provides a shell script that automatically synchronizes the timestamps between two trace files based on PING packets contained therein.
I assembled these utilities in a script to processes dozens of the aforementioned 32 MB files. It saved me a great deal of time, resulted in a single 3 MB file that was a breeze to view in the analyzer, and allowed me to more easily uncover some interesting clues to the problem. I was happy enough to raise my arms over my head and utter Chewbacca-like sounds of joy. For those who are curious about the script, or would like to use it as a jumping point for their own dark side analysis, look at it here.
What does this have to do with non-Wireshark users? Well, much like C-3PO, the, ahem, protocol droid who boasts his fluency in over 6 million forms of communication, these utilities support the same extensive set of input file formats and output file formats as Wireshark. Most likely they can be used with your analyzer’s native file format too, or at least read and write a common denominator file format.
The next time you find yourself wading through a heap of data, remember the power of the dark side. Like a Light Saber, it may be a life saver. Or at least a time saver.
May the shell be with you.