Linux.com

Home Linux Community Community Blogs General Linux Download and search WikiLeaks

Download and search WikiLeaks

After all that mess about US Embassy Cables on WikiLeaks I think this is a good moment to explain to a political analyst/columnist why he or she should use Linux. I will not cover here how to download distro, cut a CD and install Linux, every distro is giving those instruction in detail. I will describe how it looks on Ubuntu, though it will look about the same on any other distro.

Get files

We point Firefox to http://wikileaks.ch/cablegate.html and towards the bottom of the page we locate “Click here to download full site in single archive.”. Since that is the link to torrent, Firefox will offer to open it with the default application Transmission. We accept that by clicking the OK button. Then Transmission will ask if we want to add torrent. We click Add button and give some time to Transmission to finish the download. We may wish to help the others by providing upload or we stop uploads using File->Pause All and quit Transmission. That is the procedure for handling any kind of torrent.

Do processing of files

We can find the result of a successful download in Downloads folder and that is 7z archive. Since Ubuntu doesn't support 7z out of the box we need to install it. We can do it like this: Applications->Accessories->Terminal and execute:



sudo apt-get install p7zip



It will prompt for password and install 7z. Now we can return to GUI, right click on archive, and from the menu select Extract Here. Back to terminal and we change the directory and do the search:



cd Downloads/cablegate-201012200724/cable/

find . -name "*.html" | xargs grep -l "UFO"

 

which will produce a list of files which contain UFO:

 

./2009/12/09STATE129362.html

./2009/09/09LISBON514.html

./2009/02/09STATE11937.html

./2009/04/09PRISTINA148.html

./2008/03/08PARIS461.html

./2008/09/08LAGOS368.html

./2008/08/08PARIS1501.html

./2008/08/08LISBON2300.html

./2008/02/08MADRID174.html

./2008/02/08ABUJA320.html

./2006/03/06MINSK311.html

./2006/09/06KINSHASA1410.html

./2006/06/06MINSK641.html

./2010/01/10PRISTINA44.html

./2010/02/10PRISTINA84.html

./2010/02/10ADDISABABA288.html

./2010/02/10BAMAKO52.html

./2007/03/07KINSHASA282.html

./2007/07/07ANKARA1842.html

./2007/07/07KINSHASA797.html

./2007/07/07HARARE638.html

 

Unfortunately, those are not real UFO's but some other acronims like EUFOR. To see what is in the file we can use Gedit or Firefox like this:

 

gedit ./2008/08/08PARIS1501.html

firefox ./2008/08/08PARIS1501.html

 

To do more complicated searches we can go mastering find and grep or switch to SWISH++. Again we need to install it:

 

sudo apt-get install swish++

 

then we create index file:

 

index++ -v3 -e "text:*.html" .

 

and we can do the search:

 

search++ Chapman

 

The result is:

 

# results: 4

100 ./2009/11/09MAPUTO1291.html 46552 09MAPUTO1291.html

89 ./2009/07/09MAPUTO713.html 51555 09MAPUTO713.html

87 ./2010/01/10MAPUTO86.html 52688 10MAPUTO86.html

87 ./2010/01/10MAPUTO80.html 49769 10MAPUTO80.html

 

What would Anna Chapman do in Maputo? So we do a better search:

 

search++ Anna near Chapman

 

where we get a disappointing 0 files. To learn more about using and, not, or and near we execute:

 

man search++

 

There are quite a few examples towards the end of man file.

During indexing we may want to save log file and see what words index++ will discard:

 

index++ -v3 -e "text:*.html" . > log

 

There are quite a few of them and those are frequent words like Moscow, Clinton and so on. In order to search on those words we can do this:

 

search++ Mosco*

 

It will skip Moscow in the origin section. Alternatively we can always fall back to find and grep.

 

 

Comments

Subscribe to Comments Feed

Upcoming Linux Foundation Courses

  1. LFD320 Linux Kernel Internals and Debugging
    03 Nov » 07 Nov - Virtual
    Details
  2. LFS416 Linux Security
    03 Nov » 06 Nov - Virtual
    Details
  3. LFS426 Linux Performance Tuning
    10 Nov » 13 Nov - Virtual
    Details

View All Upcoming Courses


Who we are ?

The Linux Foundation is a non-profit consortium dedicated to the growth of Linux.

More About the foundation...

Frequent Questions

Join / Linux Training / Board