Header Problems with vary: User-Agent.

I ran into a interesting situation where the server was running super heavy on apache, maxing it’s connections, and server load.  It turned out varnish  was not caching items when it should, and was not giving a expiration time either.
In varnishstat it would show the miss count almost as great as what it was attempting to cache.

I was able to tell that it was using vary: User-Agent from looking at the content in the  Apache Access logs, and running a curl against a item that was static curl -Iv http://blah.com/image.jpg
In that you would see the Header, and I was missing an expiration date as well which was a bit strange.
We checked out his .htaccess file and saw that W3 Total Cache for WordPress was setting the header to !dont-vary.

<IfModule mod_deflate.c>
<IfModule mod_headers.c>
Header append Vary User-Agent env=!dont-vary
 Commenting out this line helped solve this problem, varnishstat seems to have near 100 percent hit rate after this point. 

A note on what vary: User-Agent does with varnish is store different cache for every different browser/client and browser/client version you are using. So you are going to see a ton of misses, if everyone is using a different version of firefox for instance, and varnish is creating a cache for each.

jim has written 83 articles

2 thoughts on “Header Problems with vary: User-Agent.

  1. Unknown says:

    I just spent three hours figuring this out, too! Wish I had known to look for the Vary header but stumbled into at the end. Same set up: Varnish with W3TC (on Apache). I was going to kill this line, too in my .htaccess. Did you have any negative issues by doing so? Thanks!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>