Chapter 4. Downloads and Streams

A huge section of the Internet is reserved for downloadable and streamable content. Music, images, movies, programs, and things we have not thought about yet are made available (maybe for a small fee or through their advertisements) as a direct download. Music and movies (the latter often in flash video or short flv format) can also be streamed. Sites like YouTube, using Lighttpd to serve video files, show that Lighttpd has got what it takes to do this job.

Large files present a special scenario to our Lighttpd: we will have less, but bigger requests. Some users will use download managers that create a number of HTTP range requests in parallel to optimize against bandwidth restrictions some providers operate with. We may or may not want to allow that, especially if our throughput is high. In that case each additional connection from one user will take resources away from all the others. On the other hand, we may want to allow range requests, to let our customers resume broken downloads.

If we want to serve large files, we need to set the write timeouts higher than usual—to compensate for the longer time the transfer will need. We may also want to use the writev network backend or one of the aio backends in the new version.