IP AccountingTCP/IP (Transmission Control Protocol/Internet Protocol)accounting
In today’s world of commercial Internet service, it is becoming
increasingly important to know
how much data you are transmitting and receiving on your network connections.
If you are an Internet Service Provider and you charge your customers by
volume, this will be essential to your business. If you are a customer
of an Internet Service Provider that charges by data volume, you will
find it useful to collect your own data to ensure the accuracy of your
Internet charges.
There are other uses for network accounting that have nothing to do with
dollars and bills. If you manage a server that offers a number of different
types of network services, it might be useful to you to know exactly
how much data is being generated by each one. This sort of information
could assist you in making decisions, such as what hardware to buy or how
many servers to run.
The Linux kernel provides a facility that allows you to collect all sorts
of useful information about the network traffic it sees. This facility
is called IP accounting.
Configuring the Kernel for IP AccountingIP accountingkernel configurationconfiguringkernelfor IP accounting
The Linux IP accounting feature is very closely related to the Linux
firewall software. The places you want to collect
accounting data are the same places that you would be interested in
performing firewall filtering: into and out of a network host, and in the
software that does the routing of datagrams. If you haven't read the section
on firewalls, now is probably a good time to do so, as we will be using some of
the concepts described in .
2.0 kernelsIP accounting2.2 kernelsIP accounting
To activate the Linux IP accounting feature, you should first see if your
Linux kernel is configured for it. Check to see if the /proc/net/ip_acct file exists. If it does, your kernel already supports IP accounting. If it doesn't, you must build a new kernel, ensuring that you answer “Y” to the options in 2.0 and 2.2 series kernels:
Networking options --->
[*] Network firewalls
[*] TCP/IP networking
...
[*] IP: accounting
2.4 kernelsIP accounting
or in 2.4 series kernels:
Networking options --->
[*] Network packet filtering (replaces ipchains)
Configuring IP AccountingIP accountingconfiguringconfiguringIP accountingipfwadm commandconfiguring IP accountingipchains commandconfiguring IP accountingiptables commandconfiguring IP accounting
Because IP accounting is closely related to IP firewall, the same tool
was designated to configure it, so ipfwadm,
ipchains or iptables are used to configure IP accounting. The command syntax is very similar to
that of the firewall rules, so we won't focus on it, but we will discuss
what you can discover about the nature of your network traffic using this
feature.
The general syntax for IP accounting with ipfwadm is:
# ipfwadm -A [direction] [command] [parameters]
The direction argument is new. This is simply coded as
in,
out, or
both.
These directions are from the perspective of the linux machine itself, so
in means data coming into the machine from a network
connection and out means data that is being transmitted by
this host on a network connection. The both direction is the
sum of both the incoming and outgoing directions.
The general command syntax for ipchains
and iptables is:
# ipchains -A chainrule-specification
# iptables -A chainrule-specification
The ipchains and iptables
commands allow you to specify direction in a manner more consistent
with the firewall rules. IP Firewall Chains doesn't allow you to
configure a rule that aggregates both directions, but it does allow you
to configure rules in the forward chain that the
older implementation did not. We'll see the difference that makes in
some examples a little later.
The commands are much the same as firewall rules, except that the
policy rules do not apply here. We can add, insert, delete, and list
accounting rules. In the case of ipchains and
iptables, all valid rules are accounting rules, and
any command that doesn't specify the -j option
performs accounting only.
The rule specification parameters for IP accounting are the same as
those used for IP firewall. These are what we use to define precisely
what network traffic we wish to count and total.
Accounting by AddressIP accountingby address
Let's work with an example to illustrate how we'd use IP accounting.
Imagine we have a Linux-based router that serves two departments
at the Virtual Brewery. The router has two Ethernet devices,
eth0 and eth1, each of which
services a department; and a PPP device, ppp0, that
connects us via a high-speed serial link to the main campus of the
Groucho Marx University.
Let's also imagine that for billing purposes we want to know the total
traffic generated by each of the departments across the serial link, and
for management purposes we want to know the total traffic generated
between the two departments.
The following table shows the interface addresses we will use in our
example:
ifaceaddressnetmasketh0172.16.3.0255.255.255.0eth1172.16.4.0255.255.255.0
To answer the question, “How much data does each department
generate on the PPP link?”, we could use a rule that looks like
this:
# ipfwadm -A both -a -W ppp0 -S 172.16.3.0/24 -b
# ipfwadm -A both -a -W ppp0 -S 172.16.4.0/24 -b
or:
# ipchains -A input -i ppp0 -d 172.16.3.0/24
# ipchains -A output -i ppp0 -s 172.16.3.0/24
# ipchains -A input -i ppp0 -d 172.16.4.0/24
# ipchains -A output -i ppp0 -s 172.16.4.0/24
and with iptables:
# iptables -A FORWARD -i ppp0 -d 172.16.3.0/24
# iptables -A FORWARD -o ppp0 -s 172.16.3.0/24
# iptables -A FORWARD -i ppp0 -d 172.16.4.0/24
# iptables -A FORWARD -o ppp0 -s 172.16.4.0/24
The first half of each of these set of rules say, “Count all data
traveling in either direction across the interface named ppp0 with a source
or destination (remember the function of the -b flag in
ipfwadm and iptables) address of
172.16.3.0/24.” The second half of each ruleset is
the same, but for the second Ethernet network at our site.
To answer the second question, “How much data travels between the
two departments?”, we need a rule that looks like this:
# ipfwadm -A both -a -S 172.16.3.0/24 -D 172.16.4.0/24 -b
or:
# ipchains -A forward -s 172.16.3.0/24 -d 172.16.4.0/24 -b
or:
# iptables -A FORWARD -s 172.16.3.0/24 -d 172.16.4.0/24
# iptables -A FORWARD -s 172.16.4.0/24 -d 172.16.3.0/24
These rules will count all datagrams with a source address belonging
to one of the department networks and a destination address belonging
to the other.
Accounting by Service PortIP accountingby service port
Okay, let's suppose we also want a better idea of exactly what sort of traffic
is being carried across our PPP link. We might, for example, want to know
how much of the link the FTP, smtp, and World Wide Web services are consuming.
A script of rules to enable us to collect this information might look like:
#!/bin/sh
# Collect FTP, smtp and www volume statistics for data carried on our
# PPP link using ipfwadm
#
ipfwadm -A both -a -W ppp0 -P tcp -S 0/0 ftp ftp-data
ipfwadm -A both -a -W ppp0 -P tcp -S 0/0 smtp
ipfwadm -A both -a -W ppp0 -P tcp -S 0/0 www
or:
#!/bin/sh
# Collect ftp, smtp and www volume statistics for data carried on our
# PPP link using ipchains
#
ipchains -A input -i ppp0 -p tcp -s 0/0 ftp-data:ftp
ipchains -A output -i ppp0 -p tcp -d 0/0 ftp-data:ftp
ipchains -A input -i ppp0 -p tcp -s 0/0 smtp
ipchains -A output -i ppp0 -p tcp -d 0/0 smtp
ipchains -A input -i ppp0 -p tcp -s 0/0 www
ipchains -A output -i ppp0 -p tcp -d 0/0 www
or:
#!/bin/sh
# Collect ftp, smtp and www volume statistics for data carried on our
# PPP link using iptables.
#
iptables -A FORWARD -i ppp0 -m tcp -p tcp --sport ftp-data:ftp
iptables -A FORWARD -o ppp0 -m tcp -p tcp --dport ftp-data:ftp
iptables -A FORWARD -i ppp0 -m tcp -p tcp --sport smtp
iptables -A FORWARD -o ppp0 -m tcp -p tcp --dport smtp
iptables -A FORWARD -i ppp0 -m tcp -p tcp --sport www
iptables -A FORWARD -o ppp0 -m tcp -p tcp --dport www
There are a couple of interesting features to this
configuration. Firstly, we've specified the protocol. When we specify
ports in our rules, we must also specify a protocol because TCP and UDP
provide separate sets of ports. Since all of these services are
TCB-based, we've specified it as the protocol. Secondly, we've
specified the two services ftp and
ftp-data in one command. ipfwadm
allows you to specify single ports, ranges of ports, or arbitrary lists
of ports. The ipchains command allows either single
ports or ranges of ports, which is what we've used here. The syntax
"ftp-data:ftp" means "ports ftp-data (20) through
ftp (21)," and is how we encode ranges of ports in both
ipchains and iptables. When you
have a list of ports in an accounting rule, it means that any data
received for any of the ports in the list will cause the data to be
added to that entry's totals. Remembering that the FTP service uses
two ports, the command port and the data transfer port, we've added
them together to total the FTP traffic. Lastly, we've specified the
source address as “0/0,” which is
special notation that matches all addresses and is required by both
the ipfwadm and ipchains
commands in order to specify ports.
We can expand on the second point a little to give us a different view
of the data on our link. Let's now imagine that we class FTP, SMTP,
and World Wide Web traffic as essential traffic, and all other traffic
as nonessential. If we were interested in seeing the ratio of
essential traffic to nonessential traffic, we could do something like:
# ipfwadm -A both -a -W ppp0 -P tcp -S 0/0 ftp ftp-data smtp www
# ipfwadm -A both -a -W ppp0 -P tcp -S 0/0 1:19 22:24 26:79 81:32767
If you have already examined your /etc/services file, you
will see that the second rule covers all ports except (ftp, ftp-data, smtp,
and www).
How do we do this with the ipchains or
iptables commands, since they allow only one argument in
their port specification? We can exploit user-defined chains in accounting
just as easily as in firewall rules. Consider the following approach:
# ipchains -N a-essent
# ipchains -N a-noness
# ipchains -A a-essent -j ACCEPT
# ipchains -A a-noness -j ACCEPT
# ipchains -A forward -i ppp0 -p tcp -s 0/0 ftp-data:ftp -j a-essent
# ipchains -A forward -i ppp0 -p tcp -s 0/0 smtp -j a-essent
# ipchains -A forward -i ppp0 -p tcp -s 0/0 www -j a-essent
# ipchains -A forward -j a-noness
Here we create two user-defined chains, one called
a-essent, where we capture accounting data for
essential services and another called a-noness, where we
capture accounting data for nonessential services. We then add rules to
our forward chain that match our essential services and jump to the
a-essent chain, where we have just one rule that accepts
all datagrams and counts them. The last rule in our forward chain is a rule
that jumps to our a-noness chain, where again we have just
one rule that accepts all datagrams and counts them. The rule that jumps to the
a-noness chain will not be reached by any of our essential
services, as they will have been accepted in their own chain. Our tallies for
essential and nonessential services will therefore be available in the
rules within those chains. This is just one approach you could take; there
are others. Our iptables implementation of the same
approach would look like:
# iptables -N a-essent
# iptables -N a-noness
# iptables -A a-essent -j ACCEPT
# iptables -A a-noness -j ACCEPT
# iptables -A FORWARD -i ppp0 -m tcp -p tcp --sport ftp-data:ftp -j a-essent
# iptables -A FORWARD -i ppp0 -m tcp -p tcp --sport smtp -j a-essent
# iptables -A FORWARD -i ppp0 -m tcp -p tcp --sport www -j a-essent
# iptables -A FORWARD -j a-nonessdatagramsfragmentation offragmentation, datagram
This looks simple enough. Unfortunately, there is a small but unavoidable
problem when trying to do accounting by service type. You will remember that
we discussed the role the MTU plays in TCP/IP networking in an earlier
chapter. The MTU defines the largest datagram that will be transmitted on
a network device. When a datagram is received by a router that is larger than
the MTU of the interface that needs to retransmit it, the router performs a
trick called fragmentation. The router breaks the large
datagram into small pieces no longer than the MTU of the interface and then
transmits these pieces. The router builds new headers to put in front of each
of these pieces, and these are what the remote machine uses to reconstruct the
original data. Unfortunately, during the fragmentation process the port
is lost for all but the first fragment. This means that the IP accounting
can't properly count fragmented datagrams. It can reliably count only
the first fragment, or unfragmented datagrams. There is a small trick permitted
by ipfwadm that ensures that while we won't be able to know
exactly what port the second and later fragments were from, we can still
count them. An early version of Linux accounting software assigned
the fragments a fake port number, 0xFFFF, that we could count. To ensure that
we capture the second and later fragments, we could use a rule like:
# ipfwadm -A both -a -W ppp0 -P tcp -S 0/0 0xFFFF
The IP chains implementation has a slightly more sophisticated solution, but
the result is much the same. If using the ipchains command
we'd instead use:
# ipchains -A forward -i ppp0 -p tcp -f
and with iptables we'd use:
# iptables -A FORWARD -i ppp0 -m tcp -p tcp -f
These won't tell us what the original port for this data was, but at least we are able to see how much of our data is fragments, and be able to account
for the volume of traffic they consume.
In 2.2 kernels you can select a kernel compile-time option that
negates this whole issue if your Linux machine is acting as the single
access point for a network. If you enable the option when you compile your
kernel, all received datagrams will be reassembled by the Linux router
before routing and retransmission. This operation is performed before
the firewall and accounting software sees the datagram, and thus you
will have no fragments to deal with. In 2.4 kernels you compile and
load the netfilterforward-fragment module.
Accounting of ICMP DatagramsIP accountingof ICMP datagramsICMP (Internet Control Message Protocol)datagram accountingping flooding
The ICMP protocol does not use service port numbers and is therefore a little
bit more difficult to collect details on. ICMP uses a number of different
types of datagrams. Many of these are harmless and normal, while others
should only be seen under special circumstances. Sometimes people with too
much time on their hands attempt to maliciously disrupt the network
access of a user by generating large numbers of ICMP messages. This is
commonly called ping flooding. While IP accounting
cannot do anything to prevent this problem (IP firewalling can help, though!)
we can at least put accounting rules in place that will show us if anybody
has been trying.
ICMP doesn't use ports as TCP and UDP do. Instead ICMP has ICMP message
types. We can build rules to account for each ICMP message type. To do this,
we place the ICMP message and type number in place of the port field in the
ipfwadm accounting commands. We listed the ICMP message
types in ,” so refer to it
if you need to remember what they are.
An IP accounting rule to collect information about the volume of ping data
that is being sent to you or that you are generating might look like:
# ipfwadm -A both -a -P icmp -S 0/0 8
# ipfwadm -A both -a -P icmp -S 0/0 0
# ipfwadm -A both -a -P icmp -S 0/0 0xff
or, with ipchains:
# ipchains -A forward -p icmp -s 0/0 8
# ipchains -A forward -p icmp -s 0/0 0
# ipchains -A forward -p icmp -s 0/0 -f
or, with iptables:
# iptables -A FORWARD -m icmp -p icmp --sports echo-request
# iptables -A FORWARD -m icmp -p icmp --sports echo-reply
# iptables -A FORWARD -m icmp -p icmp -f
The first rule collects information about the
“ICMP Echo Request” datagrams (ping requests), and the
second rule collects information about the “ICMP Echo Reply”
datagrams (ping replies). The third rule collects information about ICMP
datagram fragments. This is a trick similar to that described for fragmented
TCP and UDP datagrams.
If you specify source and/or destination addresses in your rules, you can
keep track of where the pings are coming from, such as whether they originate
inside or outside your network. Once you've determined where the rogue
datagrams are coming from, you can decide whether you want to put firewall
rules in place to prevent them or take some other action, such as
contacting the owner of the offending network to advise them of the problem,
or perhaps even legal action if the problem is a malicious act.
Accounting by ProtocolIP accountingby protocol
Let's now imagine that we are interested in knowing how much of the
traffic on our link is TCP, UDP, and ICMP. We would use rules like
the following:
# ipfwadm -A both -a -W ppp0 -P tcp -D 0/0
# ipfwadm -A both -a -W ppp0 -P udp -D 0/0
# ipfwadm -A both -a -W ppp0 -P icmp -D 0/0
or:
# ipchains -A forward -i ppp0 -p tcp -d 0/0
# ipchains -A forward -i ppp0 -p udp -d 0/0
# ipchains -A forward -i ppp0 -p icmp -d 0/0
or:
# iptables -A FORWARD -i ppp0 -m tcp -p tcp
# iptables -A FORWARD -o ppp0 -m tcp -p tcp
# iptables -A FORWARD -i ppp0 -m udp -p udp
# iptables -A FORWARD -o ppp0 -m udp -p udp
# iptables -A FORWARD -i ppp0 -m icmp -p icmp
# iptables -A FORWARD -o ppp0 -m icmp -p icmp
With these rules in place, all of the traffic flowing across the
ppp0 interface will be analyzed to determine
whether it is TCP, UDP, or IMCP traffic, and the appropriate counters will
be updated for each. The iptables example splits incoming
flow from outgoing flow as its syntax demands it.
Using IP Accounting ResultsIP accountingusing results of
It is all very well to be collecting this information, but how do we
actually get to see it? To view the collected accounting data and the
configured accounting rules, we use our firewall configuration
commands, asking them to list our rules. The packet and byte counters
for each of our rules are listed in the output.
The ipfwadm, ipchains, and
iptables commands differ in how accounting data is handled,
so we will treat them independently.
Listing Accounting Data with ipfwadmipfwadm commandlisting accounting data with
The most basic means of listing our accounting data with the
ipfwadm command is to use it like this:
# ipfwadm -A -l
IP accounting rules
pkts bytes dir prot source destination ports
9833 2345K i/o all 172.16.3.0/24 anywhere n/a
56527 33M i/o all 172.16.4.0/24 anywhere n/a
This will tell us the number of packets sent in each direction. If we use
the extended output format with the option (not shown here because the
output is too wide for the page), we are also supplied the options and
applicable interface names. Most of the fields in the output will be
self-explanatory, but the following may not:
dir
The direction in which the rule applies. Expected values here are
in, out,
or i/o, meaning both ways.
prot
The protocols to which the rule applies.
opt
A coded form of the options we use when invoking
ipfwadm.
ifname
The name of the interface to which the rule applies.
ifaddress
The address of the interface to which the rule applies.
By default, ipfwadm displays the packet and byte
counts in a shortened form, rounded to the nearest thousand (K) or million
(M). We can ask it to display the collected data in exact units by using the
expanded option as follows:
# ipfwadm -A -l -e -xListing Accounting Data with ipchainsipchains commandlisting accounting data with
The ipchains command will not display our accounting data
(packet and byte counters) unless we supply it the -v
argument. The simplest means of listing our accounting data with the
ipchains is to use it like this:
# ipchains -L -v
Again, just as with ipfwadm, we can display the packet and
byte counters in units by using the expanded output mode. The
ipchains uses the -x argument for this:
# ipchains -L -v -xListing Accounting Data with iptablesiptables commandlisting accounting data with
The iptables command behaves very similarly to the
ipchains command. Again, we must use the -v
when listing tour rules to see the accounting counters. To list our accounting
data, we would use:
# iptables -L -v
Just as for the ipchains command, you can use the
-x argument to show the output in expanded
format with unit figures.
Resetting the CountersIP accountingresetting the counters
The IP accounting counters will overflow if you leave them long enough.
If they overflow, you will have difficulty determining the
value they actually represent. To avoid this problem, you should read
the accounting data periodically, record it, and then reset the counters
back to zero to begin collecting accounting information for the next
accounting interval.
The ipfwadm and ipchains commands provide
you with a means of doing this quite simply:
# ipfwadm -A -z
or:
# ipchains -Z
or:
# iptables -Z
You can even combine the list and zeroing actions together to ensure that
no accounting data is lost in between:
# ipfwadm -A -l -z
or:
# ipchains -L -Z
or:
# iptables -L -Z -v
These commands will first list the accounting data and then immediately
zero the counters and begin counting again. If you are interested in
collecting and using this information regularly, you would probably want to
put this command into a script that recorded the output and stored it
somewhere, and execute the script periodically using the
cron command.
Flushing the RulesetIP accountingflushing the rules
One last command that might be useful allows you to flush all the IP
accounting rules you have configured. This is most useful when you want to
radically alter your ruleset without rebooting the machine.
The -f argument in combination with the
ipfwadm command will flush all of the rules of the type
you specify. ipchains supports the -F
argument, which does the same:
# ipfwadm -A -f
or:
# ipchains -F
or:
# iptables -F
This flushes all of your configured IP accounting rules, removing them all
and saving you having to remove each of them individually. Note that flushing
the rules with ipchains does not cause any user-defined
chains to be removed, only the rules within them.
Passive Collection of Accounting DataIP accountingpassive collection
One last trick you might like to consider: if your Linux machine is
connected to an Ethernet, you can apply accounting rules to all of the data
from the segment, not only that which it is transmitted by or destined for it.
Your machine will passively listen to all of the data on the segment and
count it.
You should first turn IP forwarding off on your Linux machine so
that it doesn't try to route the datagrams it
receives.
This isn't a good thing to do if your Linux machine serves as a router. If
you disable IP forwarding, it will cease to route! Do this only on a
machine with a single physical network interface.
In the 2.0.36 and 2.2 kernels, this is a matter of:
# echo 0 >/proc/sys/net/ipv4/ip_forwardifconfig command
You should then enable promiscuous mode on your Ethernet interface using the
ifconfig command. Now you can establish accounting
rules that allow you to collect information about the datagrams flowing
across your Ethernet without involving your Linux in the route at all.