Linux.com

Home Linux Community Community Blogs Business (or Enterprise)

Community Blogs



Easy backup with RSync, linux client configuration

After the basics and the server configuration here's a quick 'n' dirty example of my linux client configuration.

Each Linux client (but generically speaking a UNIX or OS/X client) only needs the rsync program installed, no additional dependencies, each linux distribution has it, just type:

~$ rsync --version
rsync version 3.0.6 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, no xattrs, iconv, symtimes

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.

To see if it's properly installed.

This example coming from real world is connected to an RSync server as detailed in my previous article,the script itself is not complex, it just feet my needs and I use it with cron each day, in the first section there is a configuration part for remote host setup, the second part is where the business logic resides, a little bit of logging is inserted as well

~/bin$ cat backup.rsync
#!/bin/bash
# Sync sensible data of this pc to a remote rsync host
#


# Configuration Section, change your parameters below

# Remote host name
REMOTE_HOSTNAME=myfavoritenas

# RSync connection on remote host (username as well)
REMOTE_SYNC_POINT=array1_backup

# Remote directory name where rsync copies will be created
REMOTE_DIR=linux_client_host

# Local directory name to sync (not the whole disk...)
LOCALE_DIR=$HOME

# RSync password for the connection (see rsyncd.secret)
RSYNC_PASSWORD=idonttellmypasswdtoyou

# Exclude FILES
EXCLUDE_FILES="
--exclude "$HOME/.Trash" --exclude "$HOME/.bittorrent" --exclude "$HOME/.dbus" --exclude "$HOME/.evolution" --exclude "$HOME/.fontconfig" --exclude "$HOME/.gnochm" --exclude "$HOME/.icons" --exclude "$HOME/.macromedia" --exclude "$HOME/.metacity" --exclude "$HOME/.mozilla" --exclude "$HOME/.mysqlgui" --exclude "$HOME/.nautilus" --exclude "$HOME/.nx" --exclude "$HOME/.python" --exclude "$HOME/.qt" "


# Business logic, don't change anything below this line
MY_RSYNC_OPTIONS="
--verbose --recursive --compress --perms --owner --group --specials --stats --devices --links --times --delete $EXCLUDE_FILES "

# Exec command
RSYNC_COMMAND="`which rsync`"
export RSYNC_PASSWORD
DATE_BEGIN="Begin : `date`"
if [ "$1" != "" ]; then
$RSYNC_COMMAND $MY_RSYNC_OPTIONS $LOCALE_DIR/ $REMOTE_SYNC_POINT@$REMOTE_HOSTNAME::$REMOTE_SYNC_POINT/$REMOTE_DIR
echo $DATE_BEGIN
echo "End : `date`"

else
$RSYNC_COMMAND $MY_RSYNC_OPTIONS $LOCALE_DIR/ $REMOTE_SYNC_POINT@$REMOTE_HOSTNAME::$REMOTE_SYNC_POINT/$REMOTE_DIR > /$LOCALE_DIR/backup.rsync.log 2>&1
echo $DATE_BEGIN >> $LOCALE_DIR/backup.rsync.log
echo "End : `date`" >> $LOCALE_DIR/backup.rsync.log
fi

As you can see from example you can invoke this script just by typing:

~/bin$ ./backup.rsync

If you provide an additional parameter on command line you can store rsync operations in a log file (backup.rsync.log)

~/bin$ ./backup.rsync log

 

Glad to improve my example if someone of you wants to contribute or add something
Next article covers windows client configuration, stay tuned

Next:
Windows RSync client side configuration

Steps:
Easy backup with RSync, introduction
RSync server side config on linux platform
Linux RSync client side configuration
Windows RSync client side configuration

 

Glad to see your comments
Andrea (Ben) Benini

 

Easy backup with RSync, server configuration

Now after this quick solution preview I'd like to share with you my current rsync server configuration, as I've already explained you rsync is quite easy to understand and configure, just install it in your favorite distro and configure these things:

  • /etc/rsyncd.secret file, this file contains rsync shared connection and password
  • /etc/rsyncd.conf file, this file contains rsync server configuration itself

Now let's have a look of rsycnd.secret file, it's something like:

root@myfavoritenas:/etc# cat rsyncd.secret
array1_backup:idonttellmypasswdtoyou
.... (and so on)....

this config file was taken from my current NAS, each line have two columns: share name and password, in the example above share name is "array1_backup" and password is "idonttellmypasswdtoyou", more lines may follow depending on your config. Due to security reasons you cannot read the file except if you're root, so file have chmod 0400

root@myfavoritenas:/etc# ls -la |grep rsyncd.secret
-rw------- 1 root root 368 Jul 10 2009 rsyncd.secret

Now the big part is inside rsyncd.conf, let's have a look:

root@myfavoritenas:/etc# cat rsyncd.conf
uid = root
gid = root
use chroot = yes

[array1_backup]
path = /mnt/array1/backup/.
read only = no
auth users = array1_backup
secrets file = /etc/rsyncd.secret

This config has been taken from a Buffalo TeraStation, I think you may have something similar in your favorite distro, I don't like so much buffalo configuration but this is really simple and easy to understand, I can even attach a config from your favorite distro if you like.

As you can see there's uid/gid for rsync process (don't like to see root run it), rsync chrooting, and you can see a section for each share you define (array1_backup in my example)

Next: Linux Client Configuration

Steps:
Easy backup in a Windows/Linux network with RSync
Easy backup with RSync, server configuration
Easy backup with RSync, linux client configuration
Easy backup with RSync, windows client configuration

 

Glad to see your comments

Andrea (Ben) Benini

 

Easy backup in a Windows/Linux network with RSync

This time I'd like to show you a really easy backup method using open source solutions as usual.

I mainly work as a network administrator in a company, keeping services running and having an efficient network is my primary task. One of my big troubles in the network is not related to servers or network physical backbone, troubles comes frequently from users and client desktop PCs, mainly Windows machines.

Problems are divided in two different categories: hardware and software failures; crappy PCs nowadays are frequent and cheap hardware is always a problem, software failures are mainly related to the operating system (mainly XP and Vista, no 7 in my net yet).

You cannot mess with hardware, you can try to buy better PCs or from well known hw manufacturers but if you're trying to recycle old PCs sometimes you run into troubles. Hard Disks are one of my biggest point of failuires. Operating system and mostly user misconfigurations also drives you to every kind of mess, in a regular network like mine corporate users don't have custom or strange apps, they're using Office Automation tools and few more; real problems comes from user custom data, not always backed up as it should be ("what are backups ?") and always locally stored on workstation hard disk even if they're not allowed to do it.
This solution covers data backup only, I don't care about applications setup, operating system setup or something more, just only data backup. In my case, in my network, operating system setup takes me just an hour and applications needed are only Office Automation tools as I told you before. One tool comes in mind to me to achieve this kind of backup: "rsync".

Here are main benefits:

  • You don't need to install additional software or tools in each client, rsync app compiled with cygwin comes with few dll dependencies, no install required
  • It comes for free, just deploy GPLv2 file with the license along with rsync files
  • no strange windows config on it, just need to schedule a task to run rsync periodically, windows scheduler is enough for running this kind of task
  • you don't need to rely on windows auth, permissions, samba, Active Directory or whatever
  • you only need a network and rsync server. RSYNC servers are really easy to setup, you can run them on linux (my choice) or windows or whatever you want
  • you've plenty of options for easy setup from client side as well as from server side

Next:
RSync server side config on linux platform

Steps:
Easy backup with RSync, introduction
RSync server side config on linux platform
Linux RSync client side configuration
Windows RSync client side configuration

Hope it helps

Andrea (Ben) Benini

 

SOS Open Source: automated qualification and online selection of Open Source Software

SOS Open Source is an automated methodology to qualify and select open source software that uses data from directories, forges and meta-forges and creates comprehensive information.

SOS Open Source returns a score for each open source candidates, based on strength (stable, mature and backed by a viable community), level of support (either by a community or vendors) and promise of evolution (have readable and maintainable code).

Here is the home page of SOS Open Source

 

Why Open Digital Standards Matter in Government

The full text of the essay I wrote for the O'Reilly Open Government book is finally online at Stop!Zona-M. In it I discuss what open file formats and other digital standards really mean for us, why they are often even more important than Free Software and why Government must lead the way when it comes to their adoption:
http://stop.zona-m.net/digiworld/why-open-digital-standards-matter-government

 

Orange Juice vs Coke

It happened few days back when I explained about Open source technology in a Food Service forum.  I was telling about my Open source Point of Sale for Restaurants named  Floreant POS ( http://freepos.us) and the way vendor of close source products  reacted was very interesting.  Here I am sharing the conversation between Robert  Lehman, owner of a POS software company and me.

Why promote Open Source? Do you programmers not want to be paid for your work? What do you only want part-time editors making changes as their hobby? It just doesn't make sense to me. Please
explain if I've got it wrong, but that won't change my opinion.
Robert Lehman
www.ViewGistics.com


Dear Robert, Open source is an established business model. Instead of selling
software license, open source model sells service (support,
customization, modification of reports etc). There are many successful
projects who left their source open and still making enough money to
sustain.
Traditional software business rely on hiding source code and they may
get surprised how a software company can survive giving away their hard
work.
Let me give answer your question with a layman's example. Think
-Source code is like Recipe for Cooking.
-Compiling can be compared to Cooking and
-Binary Executable are final cooked food. Now Closed source (Traditional) companies are those restaurants who hide
their recipe. They hide recipe because no other can produce same taste,
so they would make more profit and lock their guests. This model works
when cooks get sure their recipes are unique.
But we know there are already lots of Recipe books in the market and one
can cook delicious foods without special recipe. Most of the
restaurants in the world in fact runs with common recipes and people
eats there even though they could cook same thing at home. Interestingly all those generic recipe restaurants make business. How
can they survive? Reason is market is so big that one player cannot
capture the whole. Guests can find food in his area and whoever gives
better service makes better profit.
I told before that Close source works best when you have a special
formula which is hard to repeat by anybody(like Coca cola). In software highly researched mathematical program may be similar candidate, But
Restaurant business, Accounting App or ERP have almost similar business
process for many years. Literally there are nothing new and most of the
software we program reinvents the same wheel. In contrast to Coke, It
could be Orange juice where many companies can produce same orange juice
and make business.
In Assembly or machine language days all software companies were locking
business like Coke companies! There was a day when writing code
required huge time and compilers were expensive. Now paradigm has
changed. There are IDE, Wizard and tons of freely available snippets,
that reduced cost of software. So its proven that hiding source code now
give little advantage. In contrast if a programmer give away the source
code in public domain he may invite our competitors to work on same
code but market will expand dramatically. Such way it benefits a big community.
Not sure if you could get some idea from that. BTW there are tons of articles in internet and live example of projects like Apache, MySQL who are big companies based on open source technology.  You may be using Firefox - its open source too.

 

 

Help Get Coreboot Onto System76 Machines!

   Not too long ago, I had stumbled across a post on the Ubuntu Forums in the System76 support section. But first, perhaps I had better back up a bit.

 

  System76 is a computer hardware company located in Denver, Colorado. They are particularly unique because they offer Ubuntu Linux across their entire hardware line. All of their hardware is tested to be Linux-compatible, and there are numerous reviews from users that love their machines. To use a cliche here, It Just Works. For beginners and people that are new to the GNU/Linux world, this is the go-to for getting a machine that works out of the box with Linux. Heck, it's great for seasoned users as well!

 

   However, there is a small problem that plagues System76 machines. For everyday Average Joes, this isn't much of an issue. But, to Free Software enthusiasts, it's a glaring problem. Their BIOS are proprietary.

 

Well, that doesn't sound so bad, you say. But think about it. BIOS is the system software that runs on the motherboard. It is the system firmware on most desktop and laptop computers. For years, various middleware companies such as American Megatrends have peddled their own BIOS offerings onto machines. These BIOS often rarely get updates for years on end, and actually hamper capabilities of the machine itself. In this sense, BIOS can be restrictive.

 

However, a project exists called Coreboot. It was started in late 1999 to provide an alternative to proprietary BIOS, using a lightweight Linux kernel (or other boot extensions) to configure a system. It is capable now of running GNU/Linux, FreeBSD, and even Windows versions all the way up to Windows7 (No word on running MacOSX, of course).

 

The fact of the matter is, Coreboot is extremely flexible. It is also supported on a plethora of hardware currently, and herein lies the problem: although it supports quite a range of hardware, Coreboot is not officially supported on any laptops. A few netbooks support it, such as the Lemote Yeeloong. But there aren't really any laptops that ship exclusively with this Free Software BIOS alternative. I think System76 has a great chance to step up and support Free Software by adopting Coreboot.

 

And this is where we come in. I have started a petition to System76 to voice our opinions about Coreboot on their machines. I would love for everyone to take a moment to sign the petition and spread word throughout the Free Software community. This is a chance to have a hardware manufacturer truly support Free Software in North America with laptops, desktops, and servers for consumers. This is our chance to prove to the world that Free Software is profitable by voting with our money.

 

I have started the petition here, be sure to sign it and pass it along to anyone that might be interested!

 

VMWare Server 2.x thoughts

I've some servers with VMWare 2 installed on linux machines, my latest v2 was installed on a debian x64 (AMD64) host, as you can read from my previous posts I've done some mods and I'm quite happy about it, see:

HOWTO: VMWare Server 2, Disable Web Server Interface
Install VMWare Server 2 on a Debian Lenny AMD64
Access VMWare Server 2 remote virtual machine with VMWare Player

And so on...

I've used VMWare server for a while and I'm quite happy about this product and its wonderful "bare metal emulation", I only need bare metal emulation and this is the product I want. If you need to run a big business you obviously go with ESX (now vSphere) but if you'd like to run your private net without too much hassles or you're just using virtualization in a small company like mine ESX/vSphere is not so cheap.

Many of us are still using VMWare Server because is a reliable and stable solution, it was the first solution here and it's still here to stay, few thoughts recently make me angry about it, nothing strange, just the most common disappointments for a normal linux user:

  • Webserver, service and WebInterface sounds cool but it's not. If you like the Keep It Simple Stupid approach (KISS) and you only want services you really need in you machine (despite of its compute power) you're probably against this new "feature" installed with VMWare Server 2.0. Web interface relies on Tomcat webserver but it's ugly, slow and impossible to use for heavy duty machines
  • Virtual Interface Client (VIC) was the way to go to access your Linux VMWare Server installation, it's not so well documented and features for accessing VMServer are quite hidden but it was working fast and it's nice to have it but...
  • Again next trouble: VIC is available only on Windows platforms. It could be a problem if you've a linux only environment (or plan to have it), yeah you can run it with WINE on linux but I don't wanna mess with WINE libs on my machine and as many of you I prefer a REAL linux client. VMWare promised a lot of time ago a Linux version is "planned" but nobody has already seen it here. Time passed by and we're still waiting (grr...)
  • At the time if you don't like WINE libs in your linux workstation the only way is to have VIC installed on a Windows Machine (Apple "cousins" have the same problem)
  • New vSphere architecture sounds promising and a challenging world but OpenSource doesn't seems to be planned there... VMWare Server is nice but no new updates and neither the expected VIC client
  • Virtual Infrastructure Client (VIC) is now vSphere Client (the new VIC), guess what ? Windows only... and still worse... if you need it running these are its requirements: .NET 3.x framework needed and J# redistributable package. I Hope they're planning a totally new client even for linux, but it doesn't seem they're planning a new port from this product. They need to build a totally new one if they wanna get a rid of .NET framework and J# (this is not a portable project !)
  • Again... when installing new vSphere Client and trying to connect to a VMWare Server installation... it doesn't work ! if you read their official specs (here) it not even supported, this makes me really upset. After reading a lot of forums and threads (like this one from their community forum) I'm totally disappointed by their politics (but read the good news below)

By now I still continue to keep their product running in my opensource+vmware environment but I'm really watching around and making experiments with VirtualBox / KVM / XEN / ... and I'm really planning effects of moving away from them.

What I actually need is bare metal emulation and VMWare it's still the best but I'm looking other competitors as well


NOTE: At the end of this blog finally a good news. vSphere Client is "not supported" by VMWare Server but "not compatible" is wrong, in fact I've it running (still on a Windows machine grr...) and connected to VMWare Server
with a little trick check my next blog here:

Accessing VMWare Server with vSphere (the unsupported way)

 

 

Andrea (Ben) Benini

 

the first POWER7 machines are out

I've always been a big fan of the PowerPC architecture, and I was really bummed out when Apple decided to go with Intel. But new POWER7 is a MONSTER! It's got 4 , 6, or 8 cores, and each core can handle 4 thread simultaneously ... Mmmmm 32 simultaneous threads per processor. Only 1.2 Billion transistors in the 8 core chip, too. Beats the pants of Intel's Tukwilla. 

Now, if I only had roughly $34,000 so I could get a machine form IBM just to play with. 

Here's some more info:

http://arstechnica.com/hardware/news/2009/09/ibms-8-core-power7-twice-the-muscle-half-the-transistors.ars

and

http://www-03.ibm.com/systems/power/hardware/

 

 

Coming Home, Finally!

After spending most of my professional career taking care of Windows-biased systems (don't get me wrong, Windows has given me a good life financially and technically), but in the last part of my time before heading out, I finally get to see the introduction of mass UNIX -- in the form of Linux distros -- desktops to the once impregnable fortress of Windows.

Windows has many strengths as well a many weaknesses, but with its overwhelming popularity it was the system of choice in which to build and run a successful 20+ year consulting/training business. However, the progress of Linux distros with the likes of Ubuntu 9.10, freeBSD, openSUSE, etc into very usable desktop environments that do not require, remember these "the UNIX install parties," such hand  holding, give my next few years some very "coming home" type feelings.

I have several contracts where I am going to introduce a small set of Linux-biased desktops to clients that are fed-up with the Redmond mindset of control, proprietary bent, and endless patches -- not unlike the Macintosh world. Linux can free users from both those tyrannies.

Frequently as the experience pile up, I will come back and post the more interesting stories as my clients walk with me into this new, but not new, experiences of sans-Windows.

Back later on.

Nanouk

 

 

JumpBox- Open Source for non-Techies and task-oriented Business people

Open source is a lovely thing. Getting open source software running, configured, backed up and functioning can be considerably less lovely. I hear some of you saying, "If only there was a company that could just configure the software for me so that I could use the program instead of spending my time installing it and resolving dependencies, I'd do cartwheels in the parking lot." Well, start stretching, because it's cartwheel time. Jumpbox is a company that specializes in making pre-configured application-specific open source software virtual machines that can be easily deployed in Windows, Linux, or Macs using virtualization technologies. JumpBox was founded in 2006 by Kimbro Staken and Sean Tierney. For a quick explanation on how Jumpboxes work, there is a 90 second introductory video available on their website, as well as a tour of the basic structure of each Jumpbox.

Read more... Comment (0)
 
Page 4 of 7

Upcoming Linux Foundation Courses

  1. LFS426 Linux Performance Tuning
    08 Sep » 11 Sep - New York
    Details
  2. LFS520 OpenStack Cloud Architecture and Deployment
    08 Sep » 11 Sep - Virtual
    Details
  3. LFD320 Linux Kernel Internals and Debugging
    15 Sep » 19 Sep - Virtual
    Details

View All Upcoming Courses


Who we are ?

The Linux Foundation is a non-profit consortium dedicated to the growth of Linux.

More About the foundation...

Frequent Questions

Join / Linux Training / Board