Linux.com

Community Blogs



30 Cool Open Source Software I Discovered in 2013

These are full-featured open source software products, free as in beer and speech that I started to use recently. Vivek Gite picks his best open source software of 2013.

#1 Replicant - Fully free Android distribution

Replicant is entirely free and open source distributions of Android on several devices including both phones and tablets. I have installed it on an older Nexus S. You can install apps from F-Droid store a GPLv2 client app that comes configured with a repository hosting only free as in freedom applications.

Read more: 30 Cool Open Source Software I Discovered in 2013

 

See Behind That Shortened URL Using Python

I'm a big user of URL shortening, especially when sending links via email to family, friends, and/or coworkers. But there are occasions when I come across a shortened URL either on a website or from an unknown source.  So, with just a few lines of Python code, I managed to write a script that, given a shortened URL, will reveal the actual URL behind the shortened link.

Copy the following code into a file and make it executable:

#!/usr/bin/env python
#
# showURL.py
#
# -- simple python script to take a shortened url
# -- and return the true url that it's pointing too
#
import sys
import urllib2

# get the url from the command-line, bail
# if nothing was entered and show usage
if len(sys.argv) != 2:
print '[!] Usage: showURL.py <shortened url>'
sys.exit(1)

# store the given url
url = sys.argv[1]

# connect to the url and retrieve the real
# url
shortenedUrl = urllib2.urlopen(url)
realUrl = shortenedUrl.geturl()

# display result
print '[+] Shortened URL: ' + url
print '[+] Real URL: ' + realUrl

# done
sys.exit(0)

The execute the script like so:

./showURL.py http://tinyurl.com/x0vw
[+] Shortened URL: http://tinyurl.com/x0vw
[+]          Real URL: http://www.linux.com/

There you have it.  A quick and easy way to show what's behind those shortened URLs.

 

Linux: Keep An Eye On Your System With Glances Monitor

Glances is a free LGPL licensed cross-platform curses-based monitoring tool that can provide me a maximum of information about your cpu, disk I/O, network, nfsd, memory and more in a minimum of space in a terminal. It can also work in a client/server mode for remote monitoring. This utility is written in Python and uses the psutil library to fetch the statistical values from your server.

 

Read more: Linux: Keep An Eye On Your System With Glances Monitor

 

Setting up an ARM Based Cassandra Cluster with Beagle Bone Black

A great project to try on cheap ARM boards such as the BeagleBone cluster is to set up a database cluster. For a developer, this is useful both for storage for projects and to gain experience administrating NoSQL databases with replication. Using my existing three BeagleBone Black cluster I decided to try using the Cassandra database. Cassandra is easy to use for those already familiar with SQL databases and is freely available. All of these steps were done on an Ubuntu install and should work on any ARM board running an Ubuntu based OS.

To get started, you need the Cassandra binaries. I was unable to find a repository that had an arm version of Cassandra so I downloaded straight from the apache site and untared it. You can go to http://cassandra.apache.org/download/ and use wget to get the gzip file from one of the mirrors. The version I downloaded was apache-cassandra-2.0.2-bin.tar.gz. Once you have it on each machine, place it in a directory you want to use for its home, for example /app/cassandra and then unzip it:

tar -xvzf apache-cassandra-2.0.2-bin.tar.gz

Now you have everything you need to configure and run cassandra. To run in a cluster, we need to set up some basic properties on each machine so they know how to join the cluster. Start by navigating to the conf directory inside the folder you just extracted and open the cassandra.yaml file for editing. First find listen_address and set it to the ip or name of the current machine. For example for the first machine in my cluster:

listen_address: 192.168.1.51

Then do the same for rpc_address:

rpc_address: 192.168.1.51

Finally we list all of our ips in the seeds section. As I have three nodes in my cluster, the setting on each machine looked like this:

- seeds: "192.168.1.51,192.168.1.52,192.168.1.53"

Additionally if you want to give your cluster a specific name, you can set the cluster_name property. Once all three machines are set up as you like them, you can start up cassandra by going to the bin directory on each and running:

sudo ./cassandra

Using Cassandra

Once all three nodes are running, we can test on one of the nodes with cqlsh and create a database. Cqlsh is a command line program for using Cassandra's SQL like language called CQL. I connected to the utility from my first node like this:

./cqlsh 192.168.1.51

The first step is to create a keyspace which acts like a schema in a SQL database like Oracle. Keyspaces store columnsets which act like tables and store data with like columns:

>create keyspace testspace with replication = { 'class': 'SimpleStrategy', 'replication_factor': 3 };
>use testspace;

Now we can create our column set and add some rows to it. It looks just like creating a database table:

>create table machines (id int primary key, name text);
>insert into machines values (1, 'beaglebone1');
>insert into machines values (2, 'beaglebone2');
> insert into machines values (3, 'beaglebone3');
>select * from machines;

Now we have a simple set of columns with some rows of data. We can check that replication is working by logging in from a different node and performing the same selection:

./cqlsh 192.168.1.52
>use testspace;
>select * from machines;

So now we have a working arm based database cluster. There is a lot more you can do with Cassandra and some good documentation can be found here. The biggest issue with using the BeagleBone Black for this is the speed of the reads and writes. The SD card is definitely not ideal for real time applications or anything needing performance but this tutorial is certainly applicable for faster arm machines like the Cubieboard and of course desktop clusters as well.

 

How to Install ImageMagick on CentOS, RHEL and Ubuntu

ImageMagick is a software suite to create, edit, compose, or convert bitmap images. It can read and write images in a variety of formats like GIF, JPEG, PNG, Postscript, and TIFF. We can also use ImageMagick to resize, flip, mirror, rotate, distort, shear and transform images, adjust image colors, apply various special effects, or draw text, lines, polygons, ellipses and Bézier curves.

ImageMagick is typically used from command line. Also we can use it from any programming language by using its interface like Magick.NET (.Net),IMagick (PHP), PerlMagick (Perl) etc.

Read complete article at Install ImageMagick on CentOS, RHEL and Ubuntu

 

10 basic examples of Linux ps command

Linux ps command The ps command on linux is one of the most basic commands for viewing the processes running on the system. It provides a snapshot of the current processes along with detailed information like user id, cpu usage, memory usage, command name etc. It does not display data in real time like top or htop commands. But even though being simpler in features and output it is still an essential...
Read more... Comment (0)
 

Setup Apache 2.4 and Php FPM with mod proxy fcgi on Ubuntu 13.10

mod_proxy_fcgi The module mod_proxy_fcgi is a new one and it allows apache to connect to/forward requests to an external fastcgi process manager like php fpm. This allows for a complete separation between the running of php scripts and Apache. Earlier we had to use modules like mod_fcgid and mod_fastcgi which all had some limitations. Mod_fcgid for example did not properly utilise the process management capability of php-cgi whereas mod_fastcgi is a third party module. With the arrival of mod_proxy_fcgi Apache...
Read more... Comment (0)
 

Three Best Network Programming Debugging Tools

3 Best Network Programming Debugging Tools
==========================================

It is always time consuming if we don't use the right network debugging tools when do we socket programming or trying to run a client server program for the first time.

When we do network programming sometimes you want to know why send() from your client or
serverfailing, why I'm not re-start my server program, whether any other process is already using the port that
you are planning to use for your server etc.

There are many Tools available today in Linux. But we will see the most important 3 Tools in this article.

I.netstat
=========

Netstat command displays various network related information such as network connections, routing tables, interface statistics, masquerade connections, multicast memberships etc

1) Show the list of network interfaces

OpenSuse12.3#netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0 1500 0 0 0 0 0 0 0 0 0 BMU
lo 65536 0 45 0 0 0 45 0 0 0 LRU
wlan0 1500 0 25092 0 0 0 22958 0 0 0 BMRU

2) To list all Ports(both listening and non-listening, TCP, UDP, Unix)

OpenSuse12.3#netstat -at
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 localhost:smtp *:* LISTEN
tcp 0 0 192.168.1.5:6688 *:* LISTEN
tcp 0 0 192.168.1.5:49875 safebrowsing:www-http ESTABLISHED
tcp 0 1 192.168.1.5:60804 fls.doubleclic:www-http FIN_WAIT1
tcp 0 0 192.168.1.5:43589 safebrowsing.c:www-http ESTABLISHED
tcp 0 0 *:33532 *:* LISTEN
unix 2 [ ACC ] STREAM LISTENING 8645 /var/run/sdp
unix 2 [ ] DGRAM 12241

3) List only TCP Port

OpenSuse12.3#netstat -at
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 localhost:smtp *:* LISTEN
tcp 0 0 192.168.1.5:6688 *:* LISTEN
tcp 0 0 localhost:ipp *:* LISTEN
tcp 0 0 *:33532 *:* LISTEN

Similarly for UDP, "netstat -au"

3) List the Sockets which are in Listening state

OpenSuse12.3#netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 localhost:smtp *:* LISTEN
tcp 0 0 192.168.1.5:6688 *:* LISTEN
tcp 0 0 *:52980 *:* LISTEN
tcp 0 0 localhost:ipp *:* LISTEN
tcp 0 0 *:33532 *:* LISTEN
Active UNIX domain sockets (only servers)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ACC ] STREAM LISTENING 10227 private/scache
unix 2 [ ACC ] STREAM LISTENING 11714 @/tmp/dbus-NmyF9Qx2gH

List only listening TCP Ports using netstat -lt
List only listening UDP Ports using netstat -lu
List only the listening UNIX Ports using netstat -lx

4) Display PID and program names in netstat output using netstat -p

5) Print netstat information continuously
netstat -c

6) Find out on which port a program is running

OpenSuse12.3#netstat -ap | grep servermine
tcp 0 0 192.168.1.5:6688 *:* LISTEN 2135/servermine

II. tcpdump
===========
tcpdump allows us to capture all packets that are received and sent. This helps us to see what all tcp segments are
sent and received (like SYN, FIN, RST etc) and can understand the root cause of our issue.

1) Capture packets from a particular ethernet interface using tcpdump -i
tcpdump capture for a simple tcp client & server example starting from SYN to FIN/ACK with one data packet in between.

OpenSuse12.3#tcpdump -i lo
11:05:27.026304 IP 192.168.1.5.34289 > 192.168.1.5.6688: Flags [S], seq 1990318384, win 43690, options [mss 65495,sackOK,TS val 6116309 ecr 0,nop,wscale 7], length 0
11:05:27.026331 IP 192.168.1.5.6688 > 192.168.1.5.34289: Flags [S.], seq 3856734826, ack 1990318385, win 43690, options [mss 65495,sackOK,TS val 6116309 ecr 6116309,nop,wscale 7], length 0
11:05:27.026357 IP 192.168.1.5.34289 > 192.168.1.5.6688: Flags [.], ack 1, win 342, options [nop,nop,TS val 6116309 ecr 6116309], length 0
11:05:27.026689 IP 192.168.1.5.6688 > 192.168.1.5.34289: Flags [P.], seq 1:27, ack 1, win 342, options [nop,nop,TS val 6116310 ecr 6116309], length 26
11:05:27.026703 IP 192.168.1.5.34289 > 192.168.1.5.6688: Flags [.], ack 27, win 342, options [nop,nop,TS val 6116310 ecr 6116310], length 0
11:05:27.026839 IP 192.168.1.5.34289 > 192.168.1.5.6688: Flags [F.], seq 1, ack 27, win 342, options [nop,nop,TS val 6116310 ecr 6116310], length 0
11:05:27.027445 IP 192.168.1.5.6688 > 192.168.1.5.34289: Flags [.], ack 2, win 342, options [nop,nop,TS val 6116311 ecr 6116310], length 0
11:05:32.026898 IP 192.168.1.5.6688 > 192.168.1.5.34289: Flags [F.], seq 27, ack 2, win 342, options [nop,nop,TS val 6121310 ecr 6116310], length 0
11:05:32.026920 IP 192.168.1.5.34289 > 192.168.1.5.6688: Flags [.], ack 28, win 342, options [nop,nop,TS val 6121310 ecr 6121310], len

2) Capture only N number of packets using tcpdump -c
OpenSuse12.3#tcpdump -c 100 -i lo
capture only 100 packets

3) Capture the packets and write into a file using tcpdump -w
OpenSuse12.3# tcpdump -w myprogamdump.pcap -i lo
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
9 packets captured
18 packets received by filter
0 packets dropped by kernel

4) Reading/viewing the packets from a saved file using tcpdump -r
OpenSuse12.3#tcpdump -tttt -r myprogamdump.pcap
reading from file myprogamdump.pcap, link-type EN10MB (Ethernet)
2013-11-30 11:12:55.019872 IP 192.168.1.5.34290 > 192.168.1.5.6688: Flags [S], seq 2718665633, win 43690, options [mss 65495,sackOK,TS val 6564303 ecr 0,nop,wscale 7], length 0
2013-11-30 11:12:55.019899 IP 192.168.1.5.6688 > 192.168.1.5.34290: Flags [S.], seq 2448605009, ack 2718665634, win 43690, options [mss 65495,sackOK,TS val 6564303 ecr 6564303,nop,wscale 7], length 0
2013-11-30 11:12:55.019929 IP 192.168.1.5.34290 > 192.168.1.5.6688: Flags [.], ack 1, win 342, options [nop,nop,TS val 6564303 ecr 6564303], length 0
2013-11-30 11:12:55.020228 IP 192.168.1.5.6688 > 192.168.1.5.34290: Flags [P.], seq 1:27, ack 1, win 342, options [nop,nop,TS val 6564303 ecr 6564303], length 26
2013-11-30 11:12:55.020243 IP 192.168.1.5.34290 > 192.168.1.5.6688: Flags [.], ack 27, win 342, options [nop,nop,TS val 6564303 ecr 6564303], length 0
2013-11-30 11:12:55.020346 IP 192.168.1.5.34290 > 192.168.1.5.6688: Flags [F.], seq 1, ack 27, win 342, options [nop,nop,TS val 6564303 ecr 6564303], length 0
2013-11-30 11:12:55.020442 IP 192.168.1.5.6688 > 192.168.1.5.34290: Flags [.], ack 2, win 342, options [nop,nop,TS val 6564304 ecr 6564303], length 0
2013-11-30 11:13:00.020477 IP 192.168.1.5.6688 > 192.168.1.5.34290: Flags [F.], seq 27, ack 2, win 342, options [nop,nop,TS val 6569304 ecr 6564303], length 0
2013-11-30 11:13:00.020506 IP 192.168.1.5.34290 > 192.168.1.5.6688: Flags [.], ack 28, win 342, options [nop,nop,TS val 6569304 ecr 6569304], length 0

5) Receive only the packets of a specific protocol type like arp, tcp, udp, ip etc

OpenSuse12.3#tcpdump -i wlan0 ip
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wlan0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:18:04.193704 IP 132.213.238.6.http > 192.168.1.5.32991: Flags [.], seq 2723848246:2723849686, ack 3820601748, win 6432, options [nop,nop,TS val 786299612 ecr 6873162], length 1440
11:18:04.194241 IP 192.168.1.5.50414 > 192.168.1.1.domain: 36798+ PTR? 5.1.168.192.in-addr.arpa. (42)
11:18:04.196315 IP 132.213.238.6.http > 192.168.1.5.32991: Flags [P.], seq 1440:2880, ack 1, win 6432, options [nop,nop,TS val 786299612 ecr 6873162], length 1440

5) Receive packets flows on a particular port using tcpdump port
tcpdump -i eth0 port 4040

6) Capture packets for particular destination IP and Port
tcpdump -w mypackets.pcap -i eth0 dst 192.168.1.6 and port 22

III. lsof
=========
lsof meaning 'LiSt Open Files' is used to find out which files are open by which process. As we all know Linux/Unix considers everything as a files (pipes, sockets, directories, devices etc).
So by using lsof, you can get the information about any opened files. But here we primarily see options related to
network files.

1) List all network connections

OpenSuse12.3#lsof -i
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd 1 root 32u IPv6 6955 0t0 TCP *:ipp (LISTEN)
avahi-dae 475 avahi 11u IPv4 9245 0t0 UDP *:mdns
avahi-dae 475 avahi 14u IPv6 9248 0t0 UDP *:46627
master 766 root 12u IPv4 10100 0t0 TCP localhost:smtp (LISTEN)

2) List processes which are listening on a particular port

OpenSuse12.3#lsof -i :6688
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
servermine 3127 prince 3u IPv4 1256979 0t0 TCP 192.168.1.5:6688 (LISTEN)

3) List all TCP or UDP connections

OpenSuse12.3#lsof -i tcp; lsof -i udp
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd 1 root 32u IPv6 6955 0t0 TCP *:ipp (LISTEN)
master 766 root 12u IPv4 10100 0t0 TCP localhost:smtp (LISTEN)
master 766 root 13u IPv6 10102 0t0 TCP localhost:smtp (LISTEN)
gnome-ses 800 prince 13u IPv6 11789 0t0 TCP *:33532 (LISTEN)
gnome-ses 800 prince 14u IPv4 11790 0t0 TCP *:52980 (LISTEN)
cupsd 1029 root 4u IPv6 6955 0t0 TCP *:ipp (LISTEN)
cupsd 1029 root 10u IPv4 12739 0t0 TCP localhost:ipp (LISTEN)

4) List all IPv4 and IPv6 network files

OpenSuse12.3#lsof -i 4
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd 1 root 33u IPv4 6956 0t0 UDP *:ipp
avahi-dae 475 avahi 11u IPv4 9245 0t0 UDP *:mdns
avahi-dae 475 avahi 13u IPv4 9247 0t0 UDP *:37715
master 766 root 12u IPv4 10100 0t0 TCP localhost:smtp (LISTEN)
gnome-ses 800 prince 14u IPv4 11790 0t0 TCP *:52980 (LISTEN)
dhclient 926 root 6u IPv4 12038 0t0 UDP *:bootpc

OpenSuse12.3#lsof -i 6
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd 1 root 32u IPv6 6955 0t0 TCP *:ipp (LISTEN)
avahi-dae 475 avahi 12u IPv6 9246 0t0 UDP *:mdns
avahi-dae 475 avahi 14u IPv6 9248 0t0 UDP *:46627
master 766 root 13u IPv6 10102 0t0 TCP localhost:smtp (LISTEN)
gnome-ses 800 prince 13u IPv6 11789 0t0 TCP *:33532 (LISTEN)
dhclient 926 root 21u IPv6 12022 0t0 UDP *:55332
cupsd 1029 root 4u IPv6 6955 0t0 TCP *:ipp (LISTEN)

5) To list all the running process of open files of TCP Port ranges from 1-1024

OpenSuse12.3#lsof -i TCP:1-1024
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd 1 root 32u IPv6 6955 0t0 TCP *:ipp (LISTEN)
master 766 root 12u IPv4 10100 0t0 TCP localhost:smtp (LISTEN)
master 766 root 13u IPv6 10102 0t0 TCP localhost:smtp (LISTEN)
cupsd 1029 root 4u IPv6 6955 0t0 TCP *:ipp (LISTEN)
cupsd 1029 root 10u IPv4 12739 0t0 TCP localhost:ipp (LISTEN)

6) List all network files in use by a specific process
OpenSuse12.3#lsof -i -a -p 234

7) list of all open files belonging to all active processes

OpenSuse12.3#lsof

COMMAND PID TID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd 1 root cwd DIR 8,6 4096 2 /
systemd 1 root rtd DIR 8,6 4096 2 /
systemd 1 root mem REG 8,6 126480 131141 /lib64/libselinux.so.1
systemd 1 root mem REG 8,6 163493 131128 /lib64/ld-2.17.so
systemd 1 root 0u CHR 1,3 0t0 2595 /dev/null
systemd 1 root 6r DIR 0,18 0 3637 /sys/fs/cgroup/systemd/system
systemd 1 root 16u unix 0xffff88007c0ec100 0t0 3857 socket

 

How to setup Remi repository on CentOS 5/6 and Fedora 18/19/20

Remi repository The Remi (Les RPM de Remi) repository provides the latest version of various software packages related to php and mysql for redhat based linux distros like centos, fedora and RHEL. It provides php, mysql, pecl packages, pear packages any many other open source/free php applications and libraries and many other php related packages. So its designed to assist in setting up apache+php based web servers with various kinds of open source applications. The default centos/fedora distros do no have...
Read more... Comment (0)
 

Power of Linux Top Command with its Options

Linux top command is one of most powerful in built tool that all system adminstrator use to monitor the system health status everyday. Its also really important to under each parameters in the top command. There are lot of options available, that are really handy to understand system's behaviour. Refer this nice article witten by Raghu Sharma on linux top command and its options.

 

How to install FFmpeg on CentOS, RHEL and Ubuntu

FFmpeg is a cross-platform solution for streaming audio and video as well as recording and conversion.  This article will describe you to how to install FFmpeg on CentOS/RHEL 6/5 and Ubuntu 12.04/12.10 systems with easy steps. Also provides basic uses of ffmpeg.

 

Read complete artile at  How to install FFmpeg on CentOS, RHEL and Ubuntu

 
Page 6 of 125

Who we are ?

The Linux Foundation is a non-profit consortium dedicated to the growth of Linux.

More About the foundation...

Frequent Questions

Join / Linux Training / Board