I was on a client’s reporting server and noticed that an “ls” of their report logs took about 10 minutes. The directory had a log for every report run since June 2010, which is around 1.3 million files!

Here’s a transcript of the error:

[root@morpheus log]# pwd
/home/morpheus/tools/birt-runtime-2_0_1/Report Engine/log
You have new mail in /var/spool/mail/root
[root@morpheus log]# rm *
-bash: /bin/rm: Argument list too long

The solution is to use the find command to get the list of files, first checking the list with less. After the list is verified, the list is piped via stdin to xargs, which chops it up and runs the rm command with the argument list in manageable size chunks.

[root@morpheus log]# find . -name "*.log" | less
[root@morpheus log]# find . -name "*.log" | xargs /bin/rm

Files with unusual names and spaces may require variations of this command. See the Xargs reference for more details/examples.

http://en.wikipedia.org/wiki/Xargs

If the command takes too long to run, such that the terminal connection times out, running it as an argument to “nohup” may be necessary, so it will run without your terminal connection as a dependency.

[root@morpheus log]# nohup find . -name "*.log" | xargs /bin/rm

In this particular situation, clearing out the files freed up about 30% of the server’s disk space. I guess it’s time to set-up a maintenance script to periodically purge old files!