mirror of https://github.com/tLDP/LDP
159 lines
8.9 KiB
Plaintext
159 lines
8.9 KiB
Plaintext
<section> <title>What is the Usenet?</title>
|
|
|
|
<section> <title>Discussion groups </title>
|
|
<para>The Usenet is a huge worldwide collection of discussion
|
|
groups. Each discussion group has a name, <emphasis>e.g.</emphasis>
|
|
<literal>comp.os.linux.announce</literal>, and a collection of messages.
|
|
These messages, usually called <emphasis>articles</emphasis>, are posted
|
|
by readers like you and me who have access to Usenet servers, and are
|
|
then stored on the Usenet servers.</para>
|
|
|
|
<para>This ability to both read and write into a Usenet newsgroup makes
|
|
the Usenet very different from the bulk of what people today call ``the
|
|
Internet.'' The Internet has become a colloquial term to refer to the
|
|
World Wide Web, and the Web is (largely) read-only. There are online
|
|
discussion groups with Web interfaces, and there are mailing lists, but
|
|
Usenet is probably more convenient than either of these for most large
|
|
discussion communities. This is because the articles get replicated to
|
|
your local Usenet server, thus allowing you to read and post articles
|
|
without accessing the global Internet, something which is of great value
|
|
for those with slow Internet links. Usenet articles also conserve
|
|
bandwidth because they do not come and sit in each member's mailbox, unlike
|
|
email
|
|
based mailing lists. This way, twenty members of a mailing list in one
|
|
office will have twenty copies of each message copied to their
|
|
mailboxes. However, with a Usenet discussion group and a local Usenet
|
|
server, there's just one copy of each article, and it does not fill up
|
|
anyone's mailbox.</para>
|
|
|
|
<para>Another nice feature of having your own local Usenet server is
|
|
that articles stay on the server even after you've read them. You can't
|
|
accidentally delete a Usenet articles the way you can delete a message
|
|
from your mailbox. This way, a Usenet server is an
|
|
<emphasis>excellent</emphasis> way to archive articles of a group
|
|
discussion on a local server without placing the onus of archiving on
|
|
any group member. This makes local Usenet servers very valuable as
|
|
archives of internal discussion messages within corporate Intranets,
|
|
provided the article expiry configuration of the Usenet server software
|
|
has been set up for sufficiently long expiry periods.</para>
|
|
</section>
|
|
|
|
<section> <title>How it works, loosely speaking</title>
|
|
<para> Usenet news works by the reader first firing up a Usenet news
|
|
program, which in today's GUI world will highly likely be something like
|
|
Netscape Messenger or Microsoft's Outlook Express. There are a lot of
|
|
proven, well-designed character-based Usenet news readers, but a proper
|
|
review of the user agent software is outside the scope of this HOWTO, so
|
|
we will just assume that you are using whatever software you like. The
|
|
reader then selects a Usenet newsgroup from the hundreds or thousands of
|
|
newsgroups which are hosted by her local server, and accesses all unread
|
|
articles. These articles are displayed to her. She can then decide to
|
|
respond to some of them.</para>
|
|
|
|
<para>When the reader writes an article, either in response to an
|
|
existing one or as a start of a brand-new thread of discussion, her
|
|
software <emphasis>posts</emphasis> this article to the Usenet server.
|
|
The article contains a list of newsgroups into which it is to be posted.
|
|
Once it is accepted by the server, it becomes available for other users
|
|
to read and respond to. The article is automatically
|
|
<emphasis>expired</emphasis> or deleted by the server from its internal
|
|
archives based on expiry policies set in its software; the author of the
|
|
article usually can do little or nothing to control the expiry of her
|
|
articles.</para>
|
|
|
|
<para>A Usenet server rarely works on its own. It forms a part of
|
|
a collection of servers, which automatically exchange articles with
|
|
each other. The flow of articles from one server to another is called a
|
|
<emphasis>newsfeed</emphasis>. In a simplistic case, one can imagine a
|
|
worldwide network of servers, all configured to replicate articles with
|
|
each other, busily passing along copies across the network as soon as one
|
|
of them receives a new articles posted by a human reader. This replication
|
|
is done by powerful and fault-tolerant processes, and gives the Usenet
|
|
network its power. Your local Usenet server literally has a copy of all
|
|
current articles in all relevant newsgroups.</para>
|
|
|
|
</section>
|
|
|
|
<section> <title>About sizes, volumes, and so on </title>
|
|
<para>Any would-be Usenet server administrator or creator
|
|
<emphasis>must</emphasis> read the <quote>Periodic Posting about the basic steps
|
|
involved in configuring a machine to store Usenet news,</quote> also known as
|
|
the Site Setup FAQ, available from
|
|
<literal>ftp://rtfm.mit.edu/pub/usenet/news.answers/usenet/site-setup</literal>
|
|
or
|
|
<literal>ftp://ftp.uu.net/usenet/news.answers/news/site-setup.Z</literal>.
|
|
It was last updated in 1997, but trends haven't changed much since
|
|
then, though absolute volume figures have.</para>
|
|
|
|
<para>If you want your Usenet server to be a repository for all articles
|
|
in all newsgroups, you will probably not be reading this HOWTO, or even
|
|
if you do, you will rapidly realise that anyone who needs to read this
|
|
HOWTO may not be ready to set up such a server. This is because the
|
|
volumes of articles on the Usenet have reached a point where very
|
|
specialised networks, very high end servers, and large disk arrays
|
|
are required for handling such Usenet volumes. Those setups are called
|
|
``carrier-class'' Usenet servers, and will be discussed a bit later on in
|
|
this HOWTO. Administering such an array of hardware may not be the job
|
|
of the new Usenet administrator, for which this HOWTO (and most Linux
|
|
HOWTO's) are written.</para>
|
|
|
|
<para>Nevertheless, it may be interesting to understand what volumes we
|
|
are talking about. Usenet news article volumes have been doubling every
|
|
fourteen months or so, going by what we hear in comments from
|
|
carrier class Usenet administrators. In the beginning of 1997, this
|
|
volume was 1.2 GBytes of articles a day. Thus, the volumes should have
|
|
roughly done five doublings, or grown 32 times, by the time we reach
|
|
mid-2002, at the time of this writing. This gives us a volume of 38.4
|
|
GBytes per day. Assume that this transfer happens using uncompressed
|
|
NNTP (the norm), and add 50% extra for the overheads of NNTP, TCP,
|
|
and IP. This gives you a raw data transfer volume of 57.6 GBytes/day or
|
|
about 460 Gbits/day. If you have to transfer such volumes of data in 24
|
|
hours (86400 seconds), you'll need raw bandwidth of about 5.3 Mbits per
|
|
second just to <emphasis>receive all these articles</emphasis>. You'll
|
|
need more bandwidth to send out feeds to other neighbouring Usenet
|
|
servers, and then you'll need bandwidth to allow your readers to access
|
|
your servers and read and post articles in retail quantities. Clearly,
|
|
these volume figures are outside the network bandwidths of most
|
|
corporate organisations or educational institutions, and therefore only
|
|
those who are in the business of offering Usenet news can afford
|
|
it.</para>
|
|
|
|
<para>At the other end of the scale, it is perfectly feasible for a
|
|
small office to subscribe to a well-trimmed subset of Usenet newsgroups,
|
|
and exclude most of the high-volume newsgroups. Starcom Software, where
|
|
the authors of this HOWTO work, has worked with a fairly large subset of
|
|
600 newsgroups, which is still a tiny fraction of the 15,000+ newsgroups
|
|
that the carrier class services offer. Your office or college may not
|
|
even need 600 groups. And our company had excluded specific high-volume
|
|
but low-usefulness newsgroups like the <literal>talk</literal>,
|
|
<literal>comp.binaries</literal>, and <literal>alt</literal>
|
|
hierarchies. With the pruned subset, the total volume of articles per
|
|
day may amount to barely a hundred MBytes a day or so, and can be easily
|
|
handled by most small offices and educational institutions. And in such
|
|
situations, a single Intel Linux server can deliver excellent performance
|
|
as a Usenet server.</para>
|
|
|
|
<para>Then there's the <emphasis>internal</emphasis> Usenet service. By
|
|
internal here, we mean a private set of Usenet newsgroups, not a private
|
|
computer network. Every company or university which runs a Usenet news
|
|
service creates its own hierarchy of internal newsgroups, whose articles
|
|
never leave the campus or office, and which therefore do not consume
|
|
Internet bandwidth. These newsgroups are often the ones most hotly
|
|
accessed, and will carry more <emphasis>internally generated</emphasis>
|
|
traffic than all the ``public'' newsgroups you may subscribe to, within your
|
|
organisation. After all, how often does a guy have something to say
|
|
which is relevant to the world at large, unless he's discussing a globally
|
|
relevant topic like ``Unix rules!''? If such internal newsgroups are the
|
|
focus of your Usenet servers, then you may find that fairly modest
|
|
hardware and Internet bandwidth will suffice, depending on the size of
|
|
your organisation.</para>
|
|
|
|
<para>The new Usenet server administrator has to undertake a sizing
|
|
exercise to ensure that he does not bite off more than he, or his
|
|
network resources, can chew. We hope we have provided sufficient
|
|
information for him to get started with the right questions.</para>
|
|
|
|
</section>
|
|
|
|
</section>
|