Logging Policy

Table of contents

  1. What Aioe.org logs
  2. Log Data Retention
  3. Why does Aioe.org keep the logs ?


1. What Aioe.org logs

Every time that an user establishes a connection with a server, the program which serves that service writes in a file several informations about that client and the actions performed by him. This file is called log file and in a UNIX system is usually handled through syslog, an external program which provides an unified logging mechanism for all daemons. These log files are needed by those who manage each service because they’re almost the only way to know how each program is working and to detect system failures or misconfigurations.
When a newsreader opens a connection with nntp.aioe.org than downloads some article, the program used to serve the clients (nnrpd) writes in a file (/var/log/news/news.notice) through syslog somewhat like this :


Dec 30 14:30:59 aioe nnrpd[29501]: X.X.X.X (X.X.X.X) connect
Dec 30 14:31:22 aioe nnrpd[29501]: X.X.X.X group it.sport.calcio.estero 2
Dec 30 14:31:22 aioe nnrpd[29501]: X.X.X.X exit articles 2 groups 1
Dec 30 14:31:22 aioe nnrpd[29501]: X.X.X.X times user 0.092 system 0.008 idle 0.112 elapsed 22.838
Dec 30 14:31:22 aioe nnrpd[29501]: X.X.X.X artstats get 2 time 0 size 3570
Dec 30 14:31:22 aioe nnrpd[29501]: X.X.X.X time 22853 idle 22626(6) readart 0(2) nntpwrite 1(18)


These lines say that a client with X.X.X.X as IP address has established a connection at 14:30:59 (first line), has downloaded two articles from "it.sport.calcio.estero" than has closed the channel after 22.838 seconds and 3570 bytes downloaded. That’s all.

BY default INND doesn’t log neither which articles are downloaded by each user nor which newsreader is used by each client. Our news server is configured to follow the default INND log policy. If nnrpd is executed by an inetd server like xinetd, it’s possible to increase the log verbosity through the command line flag -t : in this way all commands sent from each client to the server are logged through syslog. This is an option useful for developers because it allows to exactly verify what an (experimental) program is sending to the server but it also represents a privacy hole if used on a production host. Aioe.org doesn’t use it.
When an user tries to post an article, more informations are recorded by the news server but this action is usually performed by an external script which checks whether each article sent by a local user has to be accepted or rejected. For this purpose Aioe.org uses postfilter, a perl filter for local messages executed by nnrpd each time that an users posts a message through this site.
When an user makes an attempt to post an article through our server, if this message is accepted, Postfilter records in a special file (which is called legal log) an entry with the current UNIX time, Message-ID, sender’s IP and userid as shown in the example below :


1167488126	<Message-ID>	X.X.X.X		esterni_news


Several lines are also logged through syslog (X.X.X.X represents the sender’s IP address) :


Dec 30 15:30:14 aioe nnrpd[9953]: X.X.X.X (X.X.X.X) connect
Dec 30 15:30:33 aioe nnrpd[9953]: SERVER perl filtering enabled
Dec 30 15:30:33 aioe postfilter[9953]: Message <en5t5n$9n1$1@aioe.info> accepted
Dec 30 15:30:33 aioe nnrpd[9953]: X.X.X.X post ok <en5t5n$9n1$1@aioe.info>
Dec 30 15:30:34 aioe nnrpd[9953]: X.X.X.X posts received 1 rejected 0
Dec 30 15:30:34 aioe nnrpd[9953]: X.X.X.X times user 0.152 system 0.028 idle 0.000 elapsed 20.286
Dec 30 15:30:34 aioe nnrpd[9953]: X.X.X.X time 20286 idle 19822(8) nntpwrite 1(6)


After all, a news server logs only a few privacy related informations. For each locally posted article it’s recorded only the sender’s ip address and the userid provided by the client if the access was authenticated. About the articles downloaded by each user, the server logs only which groups were read without recording the exact articles received by each client.


2. Log Data Retention

Aioe.org logs are composed by two files: the first one is the standard UNIX log file generated by syslog for the news user and it’s usually called news.notice. The second one is a custom log file created by Postfilter in order to quickly record the sender’s IP address of each locally posted article. We apply a different policy to these files. Syslog logs are deleted after a relatively short amount of time, usually a couple of weeks, if we don’t need to study them in order to track serious abuses. They’re always thrown away after 12 months. Custom log file - which includes only the sender’s IP address of each local article - is expired after 12 months so we are able to discover only the author of each article sent in the last 12 months through our hosts.
Aioe.org is an italian project but our servers are hosted in Germany and they're subjected to the german law.  Only a german judge has got the legal authority to ask us for our logs but in this case we have to deliver them.


3. Why does Aioe.org keep the logs ?

A few organizations - notably Autistici.org between 1999 and 2003 and ECN.org - claims to manage UNIX hosts without any kind of logging mechanism. Past experiences seem to reveal that usually those who assert this are lying. Without logs, the system administrators are almost blind because they can’t discover what each server is doing and who is using and abusing it. This behaviour can also favour some kind of attacks because those who manage that host can’t neither become aware of abnormal conditions nor react against them since the logs are missing. Most security related UNIX documents refer to some log file and several statistical tools need persistent data to compute. Without logs, administration becomes a spell and administrators aren’t sorcerers.
In many Countries, a national law imposes to keep all logs for at least an amount of time - months or years - which changes from State to State. Every site must follow those laws which are effettive in the place where each host is located.

There’re sites that state to keep the logs for the minimum amount of time needed for system administration. This method doesn’t protect the users’ privacy because a minimum amount is an undefined unit of measure. This means that many logs are deleted by the administrators on a discretionary base when they bring theirself to clear out their log directories. If a judge asks for a specific log and the administrators reply that it was already deleted by them but there isn’t a plain policy that explains when each log is expired, someone can suspect that the administrators are colluding with criminals.