Reducing Output of ls -l Using awk

Introduction

In my day to day work, I use a PuTTY shell to log in to various LINUX servers and edit/view various files. I’ve recently encountered a directory that had a large number of files in it and I needed a command to limit the long ls listing to only show the most recent files. This is what this article is about.

How Many Files

In order to get an exact count of the number files in the directory, I used the following command:

ls -1 |wc -l

In the above command, “ls -1” does a long listing but only shows files but not directories. The command “wc -l” does a special word count of only counting lines. So the command pipes line listings into a word count of lines, and prints out the number. In my case, I had a total of “1139” files in the respective directory (which is a lot). I want to reduce the output listing to something less than 10 if possible.

AWK to the Rescue

AWK is a text processing and reporting tool. It can be used to assist in formatting output on the command line. So I needed to pipe the “ls -halt” output into AWK and then to format the output to only show files from “Aug” (August) the 15^th (2017) and newer (this is the date that I was working on this problem).

The long listing of the ls output is described in this link, and I needed to filter on columns 6 (Month) and 7 (Day) respectively. In AWK, you use “$” and the column number to reference a specific column. For example “$6” for the month value. The below is an example of what I see in the particular directory I was working on:

some-host [TEST] /admin/dgwspool$ ls -halt
drwxrwx---. 2 dwadmin dgw  36K Aug 16 10:25 ./
-rw-rw----. 1 dwadmin dgw  658 Aug 16 01:19 dap1636772A.act
-rw-rw----. 1 dwadmin dgw  12K Aug 16 01:19 dap1636772L.log
-rw-rw----. 1 dwadmin dgw 2.5K Aug 16 01:18 rad12job.log
-rw-rw----. 1 dwadmin dgw  36K Aug 16 01:18 rad30e35965L.log

Now if I wanted to filter on the Month and Day in AWK, where the month is in August and the Date is greater than 14, then I can use the following AWK command:

awk '$6 == "Aug" && $7 > 14'

The above command filters for column 6 being equal to “Aug” and column 7 being greater than the value 14.

So then, if we combine that with “ls -l”, we can come up with the following command to filter for all files in a directory that are in the Month August and with a Date greater than 14:

ls -halt |awk '$6 == "Aug" && $7 > 14'

I’ve added the ls options “h” for human readable sizes, “a” for all (includes starting with “.”) files, and “t” for sort by modification time. The result will look like the following output:

some-host [TEST] /admin/dgwspool$ ls -halt |awk '$6 == "Aug" && $7 > 14'
drwxrwx---. 2 dwadmin dgw  36K Aug 16 10:25 ./
-rw-rw----. 1 dwadmin dgw  658 Aug 16 01:19 dap1636772A.act
-rw-rw----. 1 dwadmin dgw  12K Aug 16 01:19 dap1636772L.log
-rw-rw----. 1 dwadmin dgw 2.5K Aug 16 01:18 rad12job.log
-rw-rw----. 1 dwadmin dgw  36K Aug 16 01:18 rad30e35965L.log
-rw-rw----. 1 dwadmin dgw  685 Aug 16 01:18 rad30e35965A.act
-rw-rw----. 1 dwadmin dgw 2.1K Aug 16 01:16 rad30e35902L.log
-rw-rw----. 1 dwadmin dgw  687 Aug 16 01:16 rad30e35902A.act
-rw-rw----. 1 dwadmin dgw 1.3M Aug 16 01:15 rad30e13200L.log
-rw-rw----. 1 dwadmin dgw 2.7K Aug 16 01:15 dap22b13200L.log
-rw-rw----. 1 dwadmin dgw  673 Aug 16 01:15 dap22b13200A.act
-rw-rw----. 1 dwadmin dgw 1.1K Aug 16 01:15 rad30e13200A.act
-rw-rw----. 1 dwadmin dgw  658 Aug 15 01:39 dap1635579A.act
-rw-rw----. 1 dwadmin dgw  12K Aug 15 01:39 dap1635579L.log
-rw-rw----. 1 dwadmin dgw  36K Aug 15 01:36 rad30e35053L.log
-rw-rw----. 1 dwadmin dgw  685 Aug 15 01:36 rad30e35053A.act
-rw-rw----. 1 dwadmin dgw 2.1K Aug 15 01:33 rad30e34963L.log
-rw-rw----. 1 dwadmin dgw  687 Aug 15 01:33 rad30e34963A.act
-rw-rw----. 1 dwadmin dgw 1.3M Aug 15 01:32 rad30e10695L.log
-rw-rw----. 1 dwadmin dgw 2.7K Aug 15 01:32 dap22b10695L.log
-rw-rw----. 1 dwadmin dgw  674 Aug 15 01:32 dap22b10695A.act
-rw-rw----. 1 dwadmin dgw 1.1K Aug 15 01:32 rad30e10695A.act

So you can see the above is sorted by timestamp and only shows the files on August 15^th and the 16^th, and list is much smaller than 1139 files which meets all my requirements.

I hope this can help someone out.

Tagged: awk, LINUX