Every time that an user establishes a connection with a server, the program which serves that service writes in a file several informations about that client and the actions performed by him. This file is called log file and in a UNIX system is usually handled through syslog, an external program which provides an unified logging mechanism for all daemons. These log files are needed by those who manage each service because they’re almost the only way to know how each program is working and to detect system failures or misconfigurations.
When a newsreader opens a connection with nntp.aioe.org than downloads some article, the program used to serve the clients (nnrpd) writes in a file (/var/log/news/news.notice) through syslog somewhat like this :
Dec 30 14:30:59 aioe nnrpd: X.X.X.X (X.X.X.X) connect Dec 30 14:31:22 aioe nnrpd: X.X.X.X group it.sport.calcio.estero 2 Dec 30 14:31:22 aioe nnrpd: X.X.X.X exit articles 2 groups 1 Dec 30 14:31:22 aioe nnrpd: X.X.X.X times user 0.092 system 0.008 idle 0.112 elapsed 22.838 Dec 30 14:31:22 aioe nnrpd: X.X.X.X artstats get 2 time 0 size 3570 Dec 30 14:31:22 aioe nnrpd: X.X.X.X time 22853 idle 22626(6) readart 0(2) nntpwrite 1(18)
These lines say that a client with X.X.X.X as IP address has established a connection at 14:30:59 (first line), has downloaded two articles from "it.sport.calcio.estero" than has closed the channel after 22.838 seconds and 3570 bytes downloaded. That’s all.
1167488126 <Message-ID> X.X.X.X esterni_news
Several lines are also logged through syslog (X.X.X.X represents the sender’s IP address) :
Dec 30 15:30:14 aioe nnrpd: X.X.X.X (X.X.X.X) connect Dec 30 15:30:33 aioe nnrpd: SERVER perl filtering enabled Dec 30 15:30:33 aioe postfilter: Message <email@example.com> accepted Dec 30 15:30:33 aioe nnrpd: X.X.X.X post ok <firstname.lastname@example.org> Dec 30 15:30:34 aioe nnrpd: X.X.X.X posts received 1 rejected 0 Dec 30 15:30:34 aioe nnrpd: X.X.X.X times user 0.152 system 0.028 idle 0.000 elapsed 20.286 Dec 30 15:30:34 aioe nnrpd: X.X.X.X time 20286 idle 19822(8) nntpwrite 1(6)
After all, a news server logs only a few privacy related informations. For each locally posted article it’s recorded only the sender’s ip address and the userid provided by the client if the access was authenticated. About the articles downloaded by each user, the server logs only which groups were read without recording the exact articles received by each client.
Aioe.org logs are composed by two files: the first one is the standard UNIX log file generated by syslog for the news user and it’s usually called news.notice. The second one is a custom log file created by Postfilter in order to quickly record the sender’s IP address of each locally posted article. We apply a different policy to these files. Syslog logs are deleted after a relatively short amount of time, usually a couple of weeks, if we don’t need to study them in order to track serious abuses. They’re always thrown away after 12 months. Custom log file - which includes only the sender’s IP address of each local article - is expired after 12 months so we are able to discover only the author of each article sent in the last 12 months through our hosts.
Aioe.org is an italian project but our servers are hosted in Germany and they're subjected to the german law. Only a german judge has got the legal authority to ask us for our logs but in this case we have to deliver them.
A few organizations - notably Autistici.org between 1999 and 2003 and ECN.org - claims to manage UNIX hosts without any kind of logging mechanism. Past experiences seem to reveal that usually those who assert this are lying. Without logs, the system administrators are almost blind because they can’t discover what each server is doing and who is using and abusing it. This behaviour can also favour some kind of attacks because those who manage that host can’t neither become aware of abnormal conditions nor react against them since the logs are missing. Most security related UNIX documents refer to some log file and several statistical tools need persistent data to compute. Without logs, administration becomes a spell and administrators aren’t sorcerers.
In many Countries, a national law imposes to keep all logs for at least an amount of time - months or years - which changes from State to State. Every site must follow those laws which are effettive in the place where each host is located.
There’re sites that state to keep the logs for the minimum amount of time needed for system administration. This method doesn’t protect the users’ privacy because a minimum amount is an undefined unit of measure. This means that many logs are deleted by the administrators on a discretionary base when they bring theirself to clear out their log directories. If a judge asks for a specific log and the administrators reply that it was already deleted by them but there isn’t a plain policy that explains when each log is expired, someone can suspect that the administrators are colluding with criminals.