See Raw Headers with Netcat

October 5th, 2015
tech
Getting response headers is easy: run curl -D- -o/dev/null -sS [url] or load the site in a browser with the networking tab open. Getting request headers is harder, because webservers don't have a way to dump this exactly as they received it. Yes, you can set them to log specific headers, but logging all headers exactly as they came over the wire isn't something I see in nginx or apache. Netcat can help here. Start it up on some unused port:
    nc -lp 8070
Now make the thing you're debugging visit yoursite:8070. Netcat will print out what headers you got:
    GET / HTTP/1.1
    Host: www.jefftk.com:8070
    Referer: http://www.jefftk.com/...
    Accept-Encoding: gzip,deflate
    Cache-Control: no-store, no-cache
    Accept-Language: en-US
    Connection: Keep-alive
    Accept: */*
    Accept-Encoding: gzip,deflate
    ...
See the duplicate Accept-Encoding header? It turned out that A was adding an extra encoding header which then made B disable gzip. [1] Once I had the raw headers as sent it was pretty clear what the problem was.

(Because the problem was a duplicated header I'm glad I didn't go with a higher level tool that might have masked that problem by storing headers in hashtable or something.)


[1] Sniffing the traffic would have been ideal, but I wasn't in a position to get in between them.

Comment via: google plus, facebook, substack

Recent posts on blogs I like:

Information control, isolation, and ideological abuse

[I have freelanced for a number of effective altruist organizations, such as the Centre for Effective Altruism and 80,000 Hours.

via Thing of Things March 11, 2026

The Newest Technology in Frozen

There are lots of different things in Frozen that are new-ish, but my dad and I were wondering: what is the actual newest thing in Frozen? This led me to watch Frozen a lot while taking notes. Some of the things I found included: Elastic hair-ties A safety …

via Lily Wise's Blog Posts March 1, 2026

2025-26 New Year review

This is an annual post reviewing the last year and setting intentions for next year. I look over different life areas (work, health, parenting, effectiveness, etc) and analyze my life tracking data. Highlights include a minimal group house, the usefulness…

via Victoria Krakovna January 19, 2026

more     (via openring)