Project

General

Profile

Feature #933

lighty should buffer responses (after it grows above certain size) on disk

Added by Anonymous over 9 years ago. Updated about 1 month ago.

Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
core
Target version:
1.4.40
Start date:
Due date:
% Done:

100%

Missing in 1.5.x:

Description

Hi, nginx <http://nginx.net/> provides the ability to specify how much of the request/response is buffered in memory and request/responses larger than that are buffered on filesystems
This would be useful for large POST requests (file-uploads) from slow clients or slow backends becayse lighty would manage the client handling and the backend would slurp the large request from the relatively quick connection between it and lighty

When you configure nginx, it defaults to providing the following parameters

nginx http client request body temporary files: $prefix/client_body_temp
nginx http proxy temporary files: $prefix/proxy_temp

http://wiki.codemongers.com/NginxHttpCoreModule#client_body_temp_path
http://wiki.codemongers.com/NginxHttpProxyModule

-- yusufg

Related issues

Related to Bug #949: fastcgi, cgi, flush, php5 problem. Fixed

Associated revisions

Revision 5a91fd4b (diff)
Added by gstrauss about 1 month ago

[core] buffer large responses to tempfiles (fixes #758, fixes #760, fixes #933, fixes #1387, #1283, fixes #2083)

This replaces buffering entire response in memory which might lead to
huge memory footprint and possibly to memory exhaustion.

use tempfiles of fixed size so disk space is freed as each file sent

update callers of http_chunk_append_mem() and http_chunk_append_buffer()
to handle failures when writing to tempfile.

x-ref:
"memory fragmentation leads to high memory usage after peaks"
https://redmine.lighttpd.net/issues/758
"Random crashing on FreeBSD 6.1"
https://redmine.lighttpd.net/issues/760
"lighty should buffer responses (after it grows above certain size) on disk"
https://redmine.lighttpd.net/issues/933
"Memory usage increases when proxy+ssl+large file"
https://redmine.lighttpd.net/issues/1283
"lighttpd+fastcgi memory problem"
https://redmine.lighttpd.net/issues/1387
"Excessive Memory usage with streamed files from PHP"
https://redmine.lighttpd.net/issues/2083

History

#1 Updated by jakabosky over 9 years ago

I like this feature. It can help free up the backends when a slow client is sending a large request body.

#2 Updated by jakabosky over 9 years ago

request content > 64Kbyte will be cached to disk.

cache location:


server.upload-dirs

large response content will be cached the same way.

#3 Updated by gstrauss about 1 month ago

  • Related to Bug #949: fastcgi, cgi, flush, php5 problem. added

#4 Updated by gstrauss about 1 month ago

  • Description updated (diff)
  • Status changed from New to Patch Pending
  • Assignee deleted (jan)
  • Target version set to 1.4.40

New: asynchronous, bidirectional streaming support for request and response
Submitted pull request: https://github.com/lighttpd/lighttpd1.4/pull/66

included in the pull request is buffering large responses to temporary files instead of keeping it all in memory

#5 Updated by gstrauss about 1 month ago

  • Status changed from Patch Pending to Fixed
  • % Done changed from 0 to 100

Also available in: Atom