383 lines
12 KiB
HTML
383 lines
12 KiB
HTML
<HTML
|
|
><HEAD
|
|
><TITLE
|
|
>How it Works</TITLE
|
|
><META
|
|
NAME="GENERATOR"
|
|
CONTENT="Modular DocBook HTML Stylesheet Version 1.7"><LINK
|
|
REL="HOME"
|
|
TITLE="ADSL Bandwidth Management HOWTO"
|
|
HREF="index.html"><LINK
|
|
REL="PREVIOUS"
|
|
TITLE="Background"
|
|
HREF="background.html"><LINK
|
|
REL="NEXT"
|
|
TITLE="Implementation"
|
|
HREF="implementation.html"></HEAD
|
|
><BODY
|
|
CLASS="SECT1"
|
|
BGCOLOR="#FFFFFF"
|
|
TEXT="#000000"
|
|
LINK="#0000FF"
|
|
VLINK="#840084"
|
|
ALINK="#0000FF"
|
|
><DIV
|
|
CLASS="NAVHEADER"
|
|
><TABLE
|
|
SUMMARY="Header navigation table"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
CELLPADDING="0"
|
|
CELLSPACING="0"
|
|
><TR
|
|
><TH
|
|
COLSPAN="3"
|
|
ALIGN="center"
|
|
>ADSL Bandwidth Management HOWTO</TH
|
|
></TR
|
|
><TR
|
|
><TD
|
|
WIDTH="10%"
|
|
ALIGN="left"
|
|
VALIGN="bottom"
|
|
><A
|
|
HREF="background.html"
|
|
ACCESSKEY="P"
|
|
>Prev</A
|
|
></TD
|
|
><TD
|
|
WIDTH="80%"
|
|
ALIGN="center"
|
|
VALIGN="bottom"
|
|
></TD
|
|
><TD
|
|
WIDTH="10%"
|
|
ALIGN="right"
|
|
VALIGN="bottom"
|
|
><A
|
|
HREF="implementation.html"
|
|
ACCESSKEY="N"
|
|
>Next</A
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><HR
|
|
ALIGN="LEFT"
|
|
WIDTH="100%"></DIV
|
|
><DIV
|
|
CLASS="SECT1"
|
|
><H1
|
|
CLASS="SECT1"
|
|
><A
|
|
NAME="HOW-IT-WORKS"
|
|
></A
|
|
>3. How it Works</H1
|
|
><P
|
|
>There are two basic steps to optimize upstream bandwidth. First we have to find a way to
|
|
prevent the ADSL modem from queuing packets since we have no control over how it handles
|
|
the queue. In order to do this we will throttle the amount of data the router sends out eth0
|
|
to be slightly less than the total upstream bandwidth of the ADSL modem. This will result
|
|
in the router having to queue packets that arrive from the Local Network faster than it is allowed
|
|
to send them.</P
|
|
><P
|
|
>The second step is to set up priority queuing discipline on the router.
|
|
We'll investigate a queue that can be configured to give priority to interactive
|
|
traffic such as telnet and multi-player games.</P
|
|
><TABLE
|
|
CLASS="SIDEBAR"
|
|
BORDER="1"
|
|
CELLPADDING="5"
|
|
><TR
|
|
><TD
|
|
><DIV
|
|
CLASS="SIDEBAR"
|
|
><A
|
|
NAME="AEN112"
|
|
></A
|
|
><P
|
|
></P
|
|
><P
|
|
> By using the HTB queue we can accomplish bandwidth shaping and priority queuing at the same time
|
|
while also assuring that no priority class is starved by another. Avoiding starvation wasn't
|
|
possible using the method outlined in the 0.1 revision of this document.</P
|
|
><P
|
|
></P
|
|
></DIV
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><P
|
|
>The final step is to configure the firewall to prioritize packets by using fwmark.</P
|
|
><DIV
|
|
CLASS="SECT2"
|
|
><H2
|
|
CLASS="SECT2"
|
|
><A
|
|
NAME="AEN115"
|
|
></A
|
|
>3.1. Throttling Outbound Traffic with Linux HTB</H2
|
|
><P
|
|
>Although the connection between the router and the modem is at 10Mbit/s, the modem
|
|
is only able to send data at 128kbit/s. Any data sent in excess of that rate will be queued
|
|
at the modem. Thus, a ping packet sent from the router may go to the modem immediately, but
|
|
may take a few seconds to actually get sent out to the Internet if the queue in the modem
|
|
has any packets in it. Unfortunately most ADSL modems provide no mechanism to specify how
|
|
packets are dequeued or how large the queue is, so our first objective is to move the place
|
|
where the outbound packets are queued to somewhere where we have more control over the queue.</P
|
|
><P
|
|
>We'll do this by using the HTB queue to limit
|
|
the rate at which we send packets to the ADSL modem. Even though our upstream bandwidth may be
|
|
128kbit/s we'll have to limit the rate at which we send packets to be slightly below that. If
|
|
we want to lower the latency we have to be SURE that not a single packet is ever queued at the
|
|
modem. Through experimentation I have found that limiting the outbound traffic to about 90kbit/s
|
|
gives me almost 95% of the bandwidth I could achieve without HTB rate control. With HTB enabled at this
|
|
rate, we've prevented the ADSL modem from queuing packets.</P
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECT2"
|
|
><H2
|
|
CLASS="SECT2"
|
|
><A
|
|
NAME="AEN119"
|
|
></A
|
|
>3.2. Priority Queuing with HTB</H2
|
|
><TABLE
|
|
CLASS="SIDEBAR"
|
|
BORDER="1"
|
|
CELLPADDING="5"
|
|
><TR
|
|
><TD
|
|
><DIV
|
|
CLASS="SIDEBAR"
|
|
><A
|
|
NAME="AEN121"
|
|
></A
|
|
><P
|
|
></P
|
|
><P
|
|
>Note: previous claims in this section (originally named N-band priority queuing) were
|
|
later found to be incorrect. It actually WAS possible to classify packets into the individual bands
|
|
of the priority queue by only using the fwmark field, however it was poorly documented at the writing
|
|
of version 0.1 of this document</P
|
|
><P
|
|
></P
|
|
></DIV
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><P
|
|
>At this point we still haven't realized any change in the performance. We've merely moved the
|
|
FIFO queue from the ADSL modem to the router. In fact, with Linux configured to a default queue
|
|
size of 100 packets we've probably made our problem worse at this point! But not for long...</P
|
|
><P
|
|
>Each neighbor class in an HTB queue can be assigned a priority. By placing different types
|
|
of traffic in different classes and then assigning these classes different priorities, we can
|
|
control the order in which packets are dequeued and sent. HTB makes it possible to do this while still
|
|
avoiding starvation of any one class, since we're able to specify a minimum guaranteed rate for each
|
|
class. In addition to this, HTB allows for us to tell a class that it may use any unused bandwidth
|
|
from other classes up to a certain ceiling.</P
|
|
><P
|
|
>Once we have our classes set up, we set up filters to place traffic in classes. There are several
|
|
ways to do this, but the method described in this document uses the familiar iptables/ipchains to
|
|
mark packets with an fwmark. The filters place traffic into the classes of the HTB queue based on
|
|
their fwmark. This way, we're able to set up matching rules in iptables to send certain types of
|
|
traffic to certain classes.</P
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECT2"
|
|
><H2
|
|
CLASS="SECT2"
|
|
><A
|
|
NAME="AEN126"
|
|
></A
|
|
>3.3. Classifying Outbound Packets with iptables</H2
|
|
><TABLE
|
|
CLASS="SIDEBAR"
|
|
BORDER="1"
|
|
CELLPADDING="5"
|
|
><TR
|
|
><TD
|
|
><DIV
|
|
CLASS="SIDEBAR"
|
|
><A
|
|
NAME="AEN128"
|
|
></A
|
|
><P
|
|
></P
|
|
><P
|
|
>Note: originally this document used ipchains to classify packets. The newer iptables
|
|
is now used.</P
|
|
><P
|
|
></P
|
|
></DIV
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><P
|
|
>The final step in configuring your router to give priority to interactive traffic is
|
|
to set up the firewall to define how traffic should be classified. This is done by setting the
|
|
packet's fwmark field.</P
|
|
><P
|
|
>Without getting into too much detail, here is a simplified description of how outbound packets
|
|
might be classified into 4 classes with the highest priority class being 0x00:</P
|
|
><P
|
|
></P
|
|
><OL
|
|
TYPE="1"
|
|
><LI
|
|
><P
|
|
>Mark ALL packets as 0x03. This places all packets, by default, into the lowest priority queue.</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
>Mark ICMP packets as 0x00. We want ping to show the latency for the highest priority packets.</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
>Mark all packets that have a destination port 1024 or less as 0x01. This gives priority to system
|
|
services such as Telnet and SSH. FTP's control port will also fall into this range however FTP data transfer
|
|
takes place on high ports and will remain in the 0x03 band.</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
>Mark all packets that have a destination port of 25 (SMTP) as 0x03. If someone sends an email with
|
|
a large attachment we don't want it to swamp interactive traffic.</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
>Mark all packets that are going to a multi-player game server as 0x02. This will give gamers low latency but
|
|
will keep them from swamping out the the system applications that require low latency.</P
|
|
><P
|
|
>Mark any "small" packets as 0x02. Outbound ACK packets from inbound downloads should be sent promptly
|
|
to assure efficient downloads. This is possible using the iptables length module.</P
|
|
></LI
|
|
></OL
|
|
><P
|
|
>Obviously, this can be customized to fit your needs.</P
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECT2"
|
|
><H2
|
|
CLASS="SECT2"
|
|
><A
|
|
NAME="AEN145"
|
|
></A
|
|
>3.4. A few more tweaks...</H2
|
|
><P
|
|
>There are two more things that you can do to improve your latency. First, you can set the Maximum Transmittable
|
|
Unit (mtu) to be lower than the default of 1500 bytes. Lowering this number will lower the average time you have
|
|
to wait to send a priority packet if there is already a full-sized low-priority packet being sent. Lowering this
|
|
number will also slightly decrease your throughput because each packet contains at least 40 bytes worth of IP and
|
|
TCP header information.</P
|
|
><P
|
|
>The other thing you can do to improve latency even on your low-priority traffic is to lower your queue length
|
|
from the default of 100, which on an ADSL line could take as much as 10 seconds to empty with a 1500 byte mtu.</P
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECT2"
|
|
><H2
|
|
CLASS="SECT2"
|
|
><A
|
|
NAME="AEN149"
|
|
></A
|
|
>3.5. Attempting to Throttle Inbound Traffic</H2
|
|
><P
|
|
>By using the Intermediate Queuing Device (IMQ), we can run all incoming packets through a queue in the same
|
|
way that we queue outbound packets. Packet priority is much simpler in this case. Since we can only (attempt to)
|
|
control inbound TCP traffic, we'll put all non-TCP traffic in the 0x00 class, and all TCP traffic in the 0x01 class.
|
|
We'll also place "small" TCP packets in the 0x00 class since these are most likely ACK packets for outbound data that
|
|
has already been sent. We'll set up a standard FIFO queue on the 0x00 class, and we'll set up a Random Early Drop (RED)
|
|
queue on the 0x01 class. RED is better than a FIFO (tail-drop) queue at controlling TCP because it will drop packets
|
|
before the queue overflows in an attempt to slow down transfers that look like they're about to get out of control.
|
|
We'll also rate-limit both classes to some maximum inbound rate which is less than your true inbound speed over the
|
|
ADSL modem.</P
|
|
><DIV
|
|
CLASS="SECT3"
|
|
><H3
|
|
CLASS="SECT3"
|
|
><A
|
|
NAME="AEN152"
|
|
></A
|
|
>3.5.1. Why Inbound Traffic Limiting isn't all That Good</H3
|
|
><P
|
|
>We want to limit our inbound traffic to avoid filling up the queue at the ISP, which can sometimes buffer
|
|
as much as 5 seconds worth of data. The problem is that currently the only way to limit inbound TCP traffic
|
|
is to drop perfectly good packets. These packets have already taking up some share of bandwidth on the ADSL modem
|
|
only to be dropped by the Linux box in an effort to slow down future packets. These dropped packets will eventually
|
|
be retransmitted consuming more bandwidth. When we limit traffic, we are limiting the rate of packets which we
|
|
will accept into our network. Since the <EM
|
|
>actual</EM
|
|
> inbound data rate is somewhere above this because
|
|
of the packets we drop, we'll actually have to limit our downstream to <EM
|
|
>much</EM
|
|
> lower than the
|
|
actual rate of the ADSL modem in order to assure low latency. In practice I had to limit my 1.5mbit/s downstream
|
|
ADSL to 700kbit/sec in order to keep the latency acceptable with 5 concurrent downloads. The more TCP sessions
|
|
you have, the more bandwidth you'll waste with dropped packets, and the lower you'll have to set your limit rate.</P
|
|
><P
|
|
>A much better way to control inbound TCP traffic would be TCP window manipulation, but as of this writing
|
|
there exists no (free) implementation of it for Linux (that I know of...).</P
|
|
></DIV
|
|
></DIV
|
|
></DIV
|
|
><DIV
|
|
CLASS="NAVFOOTER"
|
|
><HR
|
|
ALIGN="LEFT"
|
|
WIDTH="100%"><TABLE
|
|
SUMMARY="Footer navigation table"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
CELLPADDING="0"
|
|
CELLSPACING="0"
|
|
><TR
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="left"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="background.html"
|
|
ACCESSKEY="P"
|
|
>Prev</A
|
|
></TD
|
|
><TD
|
|
WIDTH="34%"
|
|
ALIGN="center"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="index.html"
|
|
ACCESSKEY="H"
|
|
>Home</A
|
|
></TD
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="right"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="implementation.html"
|
|
ACCESSKEY="N"
|
|
>Next</A
|
|
></TD
|
|
></TR
|
|
><TR
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="left"
|
|
VALIGN="top"
|
|
>Background</TD
|
|
><TD
|
|
WIDTH="34%"
|
|
ALIGN="center"
|
|
VALIGN="top"
|
|
> </TD
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="right"
|
|
VALIGN="top"
|
|
>Implementation</TD
|
|
></TR
|
|
></TABLE
|
|
></DIV
|
|
></BODY
|
|
></HTML
|
|
> |