When a List of Files is Too Long for a Typical “rm” Command

I was on a client’s report­ing server and noticed that an “ls” of their report logs took about 10 min­utes. The direc­to­ry had a log for every report run since June 2010, which is around 1.3 mil­lion files!

Here’s a tran­script of the error:

[root@morpheus log]# pwd
/home/morpheus/tools/birt-runtime-2_0_1/Report Engine/log
You have new mail in /var/spool/mail/root
[root@morpheus log]# rm *
-bash: /bin/rm: Argument list too long


The solu­tion is to use the find com­mand to get the list of files, first check­ing the list with less. After the list is ver­i­fied, the list is piped via std­in to xargs, which chops it up and runs the rm com­mand with the argu­ment list in man­age­able size chunks.

[root@morpheus log]# find . -name "*.log" | less
[root@morpheus log]# find . -name "*.log" | xargs /bin/rm

Files with unusu­al names and spaces may require vari­a­tions of this com­mand. See the Xargs ref­er­ence for more details/examples.

http://en.wikipedia.org/wiki/Xargs

If the com­mand takes too long to run, such that the ter­mi­nal con­nec­tion times out, run­ning it as an argu­ment to “nohup” may be nec­es­sary, so it will run with­out your ter­mi­nal con­nec­tion as a depen­den­cy.

[root@morpheus log]# nohup find . -name "*.log" | xargs /bin/rm

In this par­tic­u­lar sit­u­a­tion, clear­ing out the files freed up about 30% of the server’s disk space. I guess it’s time to set-up a main­te­nance script to peri­od­i­cal­ly purge old files!

Leave a Reply

Your email address will not be published. Required fields are marked *