Linux.com

Author Message
Joined: May 27, 2008
Posts: 2
Other Topics
Posted May 27, 2008 at 7:34:54 PM
Subject: Parsing a log file to count file download
Just as the title says. My original plan was to just run [code=xml] grep "GET /files/filename.lol" access.log | wc -l [/code] The problem is there are multiple access per IP per timestamp. What I'd like to do is group the lines by IP (which is the first entry in every line) and count those groups. Ideas? I'm relatively new to bash, but an experienced programmer.
Back to top Profile Email Website
linuxdynasty
Joined Aug 07, 2007
Posts: 47

Other Topics
Posted: May 29, 2008 2:46:23 AM
Subject: Parsing a log file to count file download
grep "GET /files/filename.lol" access.log | sort | uniq -c uniq -c will give you the count of each occurance... Hope this helps http://linuxdynasty.org Where IT pros come and share their knowledge..\
Back to top Profile Email Website
Novi
Joined May 27, 2008
Posts: 2

Other Topics
Posted: May 29, 2008 4:57:03 AM
Subject: Parsing a log file to count file download
[quote=linuxdynasty]grep "GET /files/filename.lol" access.log | sort | uniq -c uniq -c will give you the count of each occurance... Hope this helps [/quote] What I ended up doing was running [code=xml] grep "/files/filename.lol" access.log | awk -F' ' '{print $1}' > out.txt [/code] which gave me a list of the IPs, and I planned on writing a quick python script to find the unique IPs. But after reading your posting I combined it with sort and uniq -c and wc -l, and that gave me exactly what I wanted. Thank you very much friend, I can't express how grateful I am. You saved me a bit of time! Thanks friend [Modified by: Novi on May 29, 2008 04:58 AM]
Back to top Profile Email Website
linuxdynasty
Joined Aug 07, 2007
Posts: 47

Other Topics
Posted: May 29, 2008 12:08:14 PM
Subject: Parsing a log file to count file download
Hey no problem I was just glad I could help... I'm big into Python as well. Python is my language of choice... But as a good friend of mine told me... Use the right tool for the right situation!! http://linuxdynasty.org Where IT pros come and share their knowledge..\
Back to top Profile Email Website
Johannes Truschnigg

Joined Jun 15, 2008
Posts: 27

Other Topics
Posted: Jun 15, 2008 8:16:54 PM
Subject: Parsing a log file to count file download
awk has regex matching built-in natively, so [code=xml] grep "/files/filename.lol" access.log | awk -F' ' '{print $1}' > out.txt [/code] might more efficiently be written as [code=xml] awk -F' ' '/\/files\/filename\.lol/{print $1}' access.log > out.txt [/code] Of course, it's also possible to implement the functionality of the `sort` and `uniq` programs in awk, as it's a complete programming language, but that would not actually benefit performance and/or complexity. [Modified by: Johannes Truschnigg on June 15, 2008 09:17 PM]

Free software. Free society. Better lives.

Back to top Profile Email Website
Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya