diff --git a/LDP/howto/docbook/Usenet-News-HOWTO/swhistory.sgml b/LDP/howto/docbook/Usenet-News-HOWTO/swhistory.sgml new file mode 100644 index 00000000..2f9dd583 --- /dev/null +++ b/LDP/howto/docbook/Usenet-News-HOWTO/swhistory.sgml @@ -0,0 +1,343 @@ +
+Usenet software: a historical perspective + +This section comprises excerpts from a well-known Usenet Periodic +Posting document which was last changed in Feb 1998. Our copy of that old +document was picked up from + +ftp://rtfm.mit.edu/pub/usenet-by-hierarchy/news/software/b/Usenet_Software:_History_and_Sources + +We suspect other copies will also be found elsewhere. The physical +file on the FTP server appears to have been touched last on 29 Dec +1999. The first few lines of the archived file provide information about +the origin of this document and its authors: + + + +Expires: Fri, 28 Jan 2000 09:00:19 GMT +Message-ID: +From: netannounce@deshaw.com (Mark Moraes) +Subject: Usenet Software: History and Sources +Newsgroups: news.admin.misc,news.announce.newusers,news.software.readers,news.software.b,news.answers +Followup-To: news.newusers.questions +Approved: netannounce@deshaw.com (Mark Moraes) + +Archive-name: usenet/software/part1 +Original-from: spaf@cs.purdue.edu (Gene Spafford) +Comment: edited until 5/93 by spaf@cs.purdue.edu (Gene Spafford) +Last-change: 9 Feb 1998 by netannounce@deshaw.com (Mark Moraes) +Changes-posted-to: news.admin.misc,news.misc,news.software.readers,news.software.b,news.answers +]]> + + +We have been seeing this document as a periodic posting in +news.announce.newusers since the early nineties, and +it has always been our final reference on the history of Usenet +server software. We reproduce excerpts below, retaining the portions +which discuss server software, and removing discussions of client +software, newsreaders, software for non-Unix operating systems, +etc. All quoted portions are reproduced unedited +other than changing FTP file paths to the modern URL format. We have +added our comments emphasised, in separate paragraphs. We feel the +information captured here is essential reading for anyone interested in +Usenet server software. + + +If anyone can point us to a fresher version of this document, in +case it is still maintained, we will be happy to refer to that version +instead of this one, though we suspect the reader will not suffer due to +the four-year gap; most of the information reproduced below is +historical anyway. + + +
The quoted excerpts + +Currently, Usenet readers interact with the news using a number of +software packages and programs. This article mentions the important +ones and a little of their history, gives pointers where you can look +for more information and ends with some special notes about ``foreign'' +and ``obsolete'' software. At the very end is a list of sites from which +current versions of the Usenet software may be obtained. + + + ... + + +
History + +Usenet came into being in late 1979, shortly after the release of V7 +Unix with UUCP. Two Duke University grad students in North Carolina, +Tom Truscott and Jim Ellis, thought of hooking computers together to +exchange information with the Unix community. Steve Bellovin, a grad +student at the University of North Carolina, put together the first +version of the news software using shell scripts and installed it on the +first two sites: unc and duke. At +the beginning of 1980 the network consisted of those two sites and +phs (another machine at Duke), and was described +at the January Usenix conference. Steve Bellovin later rewrote +the scripts into C programs, but they were never released beyond +unc and duke. Shortly thereafter, +Steve Daniel did another implementation in C for public distribution. +Tom Truscott made further modifications, and this became the ``A'' news +release. + +In 1981 at U. C. Berkeley, grad student Mark Horton and high school +student Matt Glickman rewrote the news software to add functionality +and to cope with the ever increasing volume of news -- ``A'' News was +intended for only a few articles per group per day. This rewrite was +the ``B'' News version. The first public release was version 2.1 in +1982; the 1.* versions were all beta test. As the net grew, the news +software was expanded and modified. The last version maintained and +released primarily by Mark was 2.10.1. + +Rick Adams, at the Center for Seismic Studies, took over +coordination of the maintenance and enhancement of the B News software +with the 2.10.2 release in 1984. By this time, the increasing volume +of news was becoming a concern, and the mechanism for moderated groups +was added to the software at 2.10.2. Moderated groups were inspired by +ARPA mailing lists and experience with other bulletin board systems. +In late 1986, version 2.11 of B News was released, including a number +of changes to support a new naming structure for newsgroups, enhanced +batching and compression, enhanced ihave/sendme +control messages, and other features. + +The final release of B News was 2.11, patchlevel 19. B News has +been declared ``dead'' by a number of people, including Rick Adams, and +is unlikely to be upgraded further; most Usenet sites are using C News +or INN (see next paragraphs). + +In March 1986 a package was released implementing news transmission, +posting, and reading using the Network News Transfer Protocol (NNTP) +(as specified in RFC 977). This protocol allows hosts to exchange +articles via TCP/IP connections rather than using the traditional UUCP. +It also permits users to read and post news (using a modified news user +agent) from machines which cannot or choose not to install the Usenet +news software. Reading and posting are done using TCP/IP messages to a +server host which does run the Usenet software. Sites which have many +workstations like the Sun and SGI, and HP products find this a convenient +way to allow workstation users to read news without having to store +articles on each system. Many of the Usenet hosts that are also on the +Internet exchange news articles using NNTP because the load impact of NNTP +is much lower than UUCP (and NNTP ensures much faster propagation). + + Our comments: This remark about relative loadings + of UUCP and NNTP is no longer applicable with faster machines and + networks, and with hugely increased traffic volumes. Today's desktop + computers, let alone servers, can all handle both NNTP and UUCP loads + effortlessly, if traffic volumes can be restricted. This is partly + due to performance enhancements to UUCP as embodied in Taylor UUCP, + and partly due to vastly faster processors. + +NNTP grew out of independent work in 1984-1985 by Brian Kantor +at U. C. San Diego and Phil Lapsley at U. C. Berkeley. Primary +development was done at U. C. Berkeley by Phil Lapsley with help from +Erik Fair, Steven Grady, and Mike Meyer, among others. The NNTP package +(now called the reference implementation) was distributed on the 4.3BSD +release tape (although that was version 1.2a and out-of-date) and is +also available on many major hosts by anonymous FTP. The current +version is 1.5.12.2. It includes NOV (News Overview -- see below) +support and runs on a wide variety of systems. It is available from +ftp.academ.com:/pub/nntp1.5/nntp.1.5.12.2.tar.gz. +For those with access to the World-Wide Web on the Internet, the +WWW page http://www.academ.com/academ/nntp.html +contains a description and news about NNTP. A different +variant, called nntp-t5, implements many of the extensions +provided by INN (including NOV support). It is available from +ftp.uu.net:/networking/news/nntp/nntp-t5.tar.gz. + + +One widely-used version of news, known as C News, was developed +at the University of Toronto by Geoff Collyer and Henry Spencer. This +version is a rewrite of the lowest levels of news to increase article +processing speed, decrease article expiration processing and improve the +reliability of the news system through better locking, etc. The package +was released to the net in the autumn of 1987. For more information, +see the paper ``News Need Not Be Slow,'' published in The Winter 1987 +Usenix Technical Conference proceedings. This paper is also available +from ftp://ftp.cs.toronto.edu/doc/programming/c-news.*, +and is recommended reading for all news software programmers. The most +recent version of C News is the Sept 1994 ``Cleanup Release.'' C News +can be obtained by anonymous ftp from its official archive site, +ftp.cs.toronto.edu:pub/c-news/c-news.tar.Z. + + Our comments: C News is no longer maintained by + anyone that we know, other than ourselves. However, after fixing + the remaining bugs in the source, we have not found the need for + further maintenance. NNTPd from Brian Kantor and Phil Lapsley is + in the same state, but we are working on enhancements to the source + for access control and other functionality. + +Another Usenet system, known as InterNetNews, or INN, was written by +Rich Salz (rsalz@uunet.uu.net). INN is designed to run +on Unix hosts that have a socket interface. It is optimized for larger +hosts where most traffic uses NNTP, but it does provide full UUCP support. +INN is very fast, and since it integrates NNTP many people find it easier +to administer only one package. The package was publicly released on +August 20, 1992. For more information, see the paper ``InterNetNews: +Usenet Transport for Internet Sites'' published in the June 1992 +Usenix Technical Conference Proceedings. INN can be obtained +from many places, including the 4.4BSD tape; its official +archive site is ftp.uu.net in the directory +/networking/news/nntp/inn. Rich's last official +release was 1.4sec in Dec 1993. + + Our comments: The original paper by Rich Salz about + INN, where he proposed the design of an alternate Usenet server + software, is a must-read for readers interested in Usenet server + software. So is the paper by C News authors, cited before it. Most of + the issues that Rich Salz had with C News, as stated in his paper, + were very relevant at that time. Today, with the current version of + NNTPd and the incorporation of the message ID daemon and NOV, these + issues are no longer relevant, and the choice of C News+NNTPd versus + INN is now based more on the level of maintenance of source code, + familiarity and personal preferences than on core design factors. + + +In June 1995, David Barr began a series of unoffical releases +of INN based on 1.4sec, integrating various bug-fixes, enhancements +and security patches. His last release was 1.4unoff4, found in +ftp://ftp.math.psu.edu:/pub/INN. This site is also the +home of contributed software for INN and other news administration +tools. + +INN is now maintained by the Internet Software Consortium +(inn@isc.org). The official INN home is now +http://www.isc.org/isc/ and the latest version (1.7.2) +can be obtained from ftp://ftp.isc.org/isc/inn/. + + Our comments: The URL for the INN home page above + is probably incorrect. Try http://www.isc.org/products/INN/. + + +Towards the end of 1992, Geoff Collyer implemented NOV (News +Overview): a database that stores the important headers of all news +articles as they arrive. This is intended for use by the implementors +of news readers to provide fast article presentation by sorting and +``threading'' the article headers. (Before NOV, newsreaders like +trn, tin and nn +came with their own daemons and databases that used a nontrivial amount +of system resources). NOV is fully supported by C News, INN and NNTP-t5. +Most modern news readers use NOV to get information for their threading +and article menu presentation; use of NOV by a newsreader is fairly easy, +since NOV comes with sample client-side threading code. + + + ... + +Details on many other mail and news readers for MSDOS, Windows and +OS/2 systems can be found in the FAQ posted to +comp.os.msdos.mail-news. + + + + +]]> + + +
+ +
Newsfeed management software + +Gup, the Group Update Program is a Unix +mail-server program that lets a remote site change their newsgroups +subscription on their news feed without requiring the intervention of +the news administrator at the feed site. Gup operates with the INN +(and likely the C News) batching mechanisms. The news administrators +at the remote sites simply mail commands to gup to make changes to +their own site's subscription list. The mail/interface is password +protected. Gup checks the requests for valid newsgroup names, +patterns that have no effect and so on. Gup's authors are Mark +Delany (markd@mira.net.au) and Andrew Herbert +(andrew@mira.net.au). Its official FTP location +is ftp.mira.net.au:/unix/news/gup-0.4.tar.gz, +but since that's not as well connected as UUNET, people are strongly +advised to obtain it from a mirror site, e.g. +ftp.uu.net:/networking/news/misc/gup-0.4.tar.gz. + +dynafeed is +a package from Looking Glass Software Limited that maintains a +.newsrc for every remote site and generates the batches +for them. Remote sites can use UUCP or run a program to change their +.newsrc dynamically. It comes with a program that the +remote site can run to monitor readership in newsgroups and dynamically +update the feed list to match reader interest. The goal of this is +to get a feed that sends only exactly the groups currently being read. +dynafeed can be obtained from +ftp://ftp.clarinet.com/sources/dynafeed.tar.Z. + +
+ +
News processing software + +Software also exists to automatically archive Usenet newsgroups. +The package rkive, written by Kent Landfield +(kent@sterling.com) can be configured to archive +news automatically based on different headers -- Archive-Name, +Volume-Issue, Chronological, Subject and External-Command to +name a few. It can be run in batch mode from the command line or +from cron. It can also be installed in the sys +or newsfeeds file to process articles as they are +received. rkive supports local spool directories as +well as NNTP based access. rkive is available via +FTP from ftp://ftp.sterling.com/rkive. + +Newsclip is a programming language for writing news +filtering programs, from Looking Glass Software Limited, marketed +by ClariNet Communications Corp. It is C-like, and translates to +C, so a C compiler is required. It has data-types to represent +the kinds of things found in article headers and bodies. It can +maintain databases of users, message-ids, patterns, subjects, etc. +These can be used to decide whether to ignore or select an article. +Newsclip can either operate as a standalone program or as part +of rn. It is free for non-commercial use and is available from +ftp://ftp.clarinet.com/sources/nc.tar.Z. Contact +clari-info@clarinet.com with a subject line of +``newsclip'' for more info. + + +
+ +
Commercial software + +DNEWS is a commercial product from NetWin. DNEWS licenses +are provided free to educational institutions for non profit +use. With DNEWS, the news is stored in a database so as not to +overload the raw file system. DNEWS supports 'sucking' where only +groups which users read are pulled over from the feeder site. DNEWS +is currently known to run on VMS, Windows NT, Solaris, SunOS, +Unixware, HP/UX. DNEWS binaries are available by anonymous FTP from +ftp://ftp.std.com/ftp/vendors/netwin/dnews or from +http://world.std.com/~netwin/ DNEWS sources can be +obtained on request, see the file source.txt in the +FTP area for more information. + +Our comments: The information on DNEWS may be dated. We +have been seeing DNEWS on their own Website for quite a few years now. +Check www.netwinsite.com. Moreover, there are other +commercial Usenet server software systems available, including the one +bundled with the Internet Information Server of Microsoft Windows NT and +the ones from iPlanet. And for carrier class systems, there are many +commercial Usenet routers available. + + +
+ +
Special note on ``notes'' and old versions of news + + ... + +``B'' news software is currently considered obsolete. Unix sites +joining the Usenet should install C news or INN to ensure proper +behavior and good performance. Most old B news software had +compiled-in limits on the number of newsgroups and the number of +articles per newsgroup; the increasing volume of news means that B +news software cannot reliably cope with a moderately-full newsfeed. + +
+ +
+ +