Parsing a log file to count file download
|
Author |
Message |
|
|
Posted : Tue, 27 May 2008 19:34:54
Subject :
Parsing a log file to count file download
Just as the title says.
My original plan was to just run
[code=xml]
grep "GET /files/filename.lol" access.log | wc -l
[/code]
The problem is there are multiple access per IP per timestamp. What I'd like to do is group the lines by IP (which is the first entry in every line) and count those groups.
Ideas? I'm relatively new to bash, but an experienced programmer.
|
|
|
|
linuxdynasty
|
Posted : Thu, 29 May 2008 02:46:23
Subject :
Parsing a log file to count file download
grep "GET /files/filename.lol" access.log | sort | uniq -c
uniq -c will give you the count of each occurance... Hope this helps
http://linuxdynasty.org Where IT pros come and share their knowledge..\
|
|
Novi
|
Posted : Thu, 29 May 2008 04:57:03
Subject :
Re: Re: Parsing a log file to count file download
[quote=linuxdynasty]grep "GET /files/filename.lol" access.log | sort | uniq -c
uniq -c will give you the count of each occurance... Hope this helps
[/quote]
What I ended up doing was running
[code=xml]
grep "/files/filename.lol" access.log | awk -F' ' '{print $1}' > out.txt
[/code]
which gave me a list of the IPs, and I planned on writing a quick python script to find the unique IPs. But after reading your posting I combined it with sort and uniq -c and wc -l, and that gave me exactly what I wanted.
Thank you very much friend, I can't express how grateful I am. You saved me a bit of time!
Thanks friend
[Modified by: Novi on May 29, 2008 04:58 AM]
|
|
linuxdynasty
|
Posted : Thu, 29 May 2008 12:08:14
Subject :
Parsing a log file to count file download
Hey no problem I was just glad I could help... I'm big into Python as well. Python is my language of choice... But as a good friend of mine told me... Use the right tool for the right situation!!
http://linuxdynasty.org Where IT pros come and share their knowledge..\
|
|
Johannes Truschnigg
|
Posted : Sun, 15 June 2008 20:16:54
Subject :
Re: Re: Re: Re: Parsing a log file to count file download
awk has regex matching built-in natively, so [code=xml]
grep "/files/filename.lol" access.log | awk -F' ' '{print $1}' > out.txt
[/code]
might more efficiently be written as
[code=xml]
awk -F' ' '/\/files\/filename\.lol/{print $1}' access.log > out.txt
[/code]
Of course, it's also possible to implement the functionality of the `sort` and `uniq` programs in awk, as it's a complete programming language, but that would not actually benefit performance and/or complexity.
[Modified by: Johannes Truschnigg on June 15, 2008 09:17 PM]
|