nginx error – upstream sent too big header

We recently decided to test nginx with our CDN.  It seems that lighttpd just isn’t up the the task of serving high-connection rate services.  It was grinding to a halt on our systems as they started processing 400 – 600 connections per second.  We ended up running multiple lighttpd’s on single servers to alleviate this problem, but in doing so we were losing all the benefits of lighttpd such as stat caching etc.

Enter nginx.  Nginx is another web server similar in nature to lighttpd.  We thought we’d give it a go with just one of our clients which processes in the vicinity of 800 connections per second.  Here is our commentary:
1) nginx seems very well designed and coded – I find the configuration absolutely brilliant and very self explanatory to a developer.  I think a non-developer might have a bit of a hard time with it however, but I’m I’m sure they could work it out.  One of the biggest hates in all the forums is how badly nginx is documented – it’s NOT true.  I found the core features to be very well documented (in English) albeit in various locations.

2) Speed – The improvement was immediately noticable.  This webserver software is indeed faster than lighttpd.  Anybody wishing to argue – go home.  It’s faster under both small loads and large loads.

3) High-volume of connections – Handles it smoothly.  Lighttpd really had problems here.  Nginx doesn’t seem to care.

We did get one error which didnt seem documented anywhere so we ended up going through the source code to track down the issue.  The error message we were getting was:

“upstream sent too big header while reading response header from upstream”

Which would cause 502 Bad Gateway errors.  Apparantly this is an expected error with the default fastcgi configuration as the buffer size is not big enough for processing our headers (not sure why considering I think the default is 4k?).  Anyway – here is our fastcgi paramaters that appear to be currently working:

fastcgi_connect_timeout 60;
fastcgi_send_timeout 180;
fastcgi_read_timeout 180;
fastcgi_buffer_size 128k;
fastcgi_buffers 4 256k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
fastcgi_intercept_errors on

Overall, we’re going to let nginx run for a few weeks and see if we run into any more little problems.  If we don’t, we’ll look into changing our entire CDN over to nginx as lighttpd just isn’t cutting it performance wise, and the bug support seems almost non existent these days.  Personally, I think lighttpd’s days are numbered and I’m expecting its distribution to start reducing sometime over the next 6 months….

3 Comments

• 

  By jdvogt, March 22, 2009 @ 6:03 am

  Thank you thank you!! I encountered the same error and was getting nervous as we are just about to launch our service (on demand mental health counseling).

  I decided to roll with nginx to proxy out to our app server and to our cms. So far I’m really impressed, but there certainly aren’t the years and years of blog posts and forum posts of people troubleshooting nginx like you find with a product like apache.
• 

  By admin, April 11, 2009 @ 5:30 pm

  Nginx has been great for us. It has been almost a year since we wrote this blog post, and the amount of documentation on Nginx I think has quadrupled in that time. We’re still happily using it for several fairly large projects. There’s still the occasional issue that we’re running in to (I don’t believe it is 100% bug-free), but I do feel it is significantly more bug-free than the immediate competitors to it such as lighttpd.

  It’s funny that 12 months ago when Nginx was barely a dot on the WWW, and lighttpd was quickly growing, we made the call that Lighttpd was doomed due to poor bug support and development process. The latest NetCraft surveys justifies these comments by indicating lighttpd’s market share is in decline, and Nginx now powers three times as many “large” websites than Lighttpd. Kinda sad to see from the Lighttpd perspective (it wasn’t all bad)….but kinda good for the rest of us that chose the ‘winning’ technology to build our businesses upon!

Other Links to this Post

1. Nginx error – upstream sent too big header | Nginx Lighttpd Tutorial — August 25, 2011 @ 12:21 am

RSS feed for comments on this post. TrackBack URI

Leave a comment

You must be logged in to post a comment.