vcl

A colleague of mine found out that many static resource requests which should've been cached upstream by a CDN were not being cached, and the reason was an extra Vary http header being sent with the response—in this case Host.

It was hard to reproduce the issue, but in the end we found out it was related to Apache bug #58231. Basically, since we used some RewriteConds that evaluated the HTTP_HOST value before a RewriteRule, we ran into a bug where Apache would dump a Vary: Host header into the request response. When this was set, it effectively bypassed Varnish's cache, as well as our upstream CDN... and since it applied to all image, css, js, xml, etc. requests, we saw a lot of unexpected volume hitting the backend Apache servers.

To fix the issue, at least until the upstream bug is fixed in Debian, we decided to strip Host from the Vary header inside our Varnish default.vcl. Inside the vcl_backend_response, we added:

If you're a Drupal or PHP developer used to debugging or troubleshooting some code by adding a print $variable; or dpm($object); to your PHP, and then refreshing the page to see the debug message (or using XDebug, or using watchdog logging...), debugging Varnish's VCL language can be intimidating.

VCL uses C-like syntax, and is compiled when varnish starts, so you can't just modify a .vcl file and refresh to see changes or debug something. And there are only a few places where you can simply stick a debug statement. So, I'll explain four different ways I use to debug VCLs in this post (note: don't do this on a production server!):

Simple Error statements (like print in PHP)

Sometimes, all you need to do is see the output of a variable, like req.http.Cookie, inside vcl_recv(). In these cases, you can just add an error statement to throw an error in Varnish and output the contents of a string, like the Cookie: