There's a basic usage for netcat helping you to transfer data/files across hosts without ssh or ftp services
Here's what I usually do on server (example IP: 184.108.40.206)
nc -l -p 6666 | uncompress -c | tar xvfp -
Basically I open TCP port 6666 (or whatever you want) on server and listen for raw compressed data
On my client I need to issue a command like this
tar cpf - /my/cool/dir | compress -c | nc -w 3 220.127.116.11 6666
Where /my/cool/dir is what I want to transfer, 18.104.22.168 is the destination host (server) and 6666 is the previously opened TCP port; you can even set a timeout (in seconds) for connection, suggested if you're in a busy network
Data on this stream are compressed so faster downloads are possible, this kind of trick is useful when you need to transfer data on the fly or when you don't have ssh shell host, scp or ftp server; I use it heavily expecially on embedded devices or dumb machines
Hope it helps
Andrea Ben Benini
As many of you already know I'm experiencing some sort of hylafax/samba integration for a mixed Windows environment, this blog is for creating a PDF/PostScript printer easily, no matter of what you do with the printer; when you print on it you'll produce a Postscript/PDF document
A samba server installation up and running
Create a PostScript/PDF document from a spool so every Windows user in your net can generate documents without client side software installation (Exception for the windows printer driver)
If you want to create your PDF files and customize them each time (custom passwd protection, comments, tags) you'd probably use PDFCreator, it has a windows installation and a lot of funny options to deal with. If you're searching for a dumb massive pdf generation for average windows users (lazy users without knowledge and time) you'd better stay here to see my config.
I'm using it for a Windows network where users just need to use a document management and storage system, they simply want to archive files on storage array in PDF format, no matter for file input (DOC, XLS, EMails, ...). This is particularly useful in big environments and when input file may vary, here's a nice Windows spooler usage
After installing samba and integrating it in your network (Workgroup, Domain, Active Directory, ... need some help or need a new blog for it ?) just add a section to your /etc/samba/smb.conf (or wherever your samba config file is) like this one:
# SAMBA PDF SHARED PRINTER
# for windows clients
comment = my cool samba printer
public = yes
print command = ( ps2pdf "%s" "/my/storage/dir/`date +"%Y-%m-%d.%H-%M.pdf"`" > /dev/null; rm "%s" ) &
printable = yes
writable = no
path = /var/spool/samba
use client driver = yes
This config was taken from a running server, exception for the print command line, in my case I've a custom script making other things, in this sample when a file is received it will be converted from PS (input) to PDF (output) and temp PS file is then deleted.
PDF in the example will have this name /my/storage/dir/YYYY-MM-DD.HH-MM.pdf
change it to whatever you want
Now the big question is: what should I have to install on the windows side ?
Here's the easiest part of the trick.
Install a "Apple LaserWriter 12-640 PS" printer, common alternatives are:
"Apple LaserWriter 12-600 PS"
"HP Laserjet 4/4M PS"
I strongly suggest you to install Apple Laserwriter printer family, these are already supported and driver already available in Windows XP, You don't even need to supply Windows install CD
Vista users may choose HP Laserjet (no cd required, already installed), Laserwriter drivers are not there
These printers are PostScript printers so each file you send to them will be translated to postscript language (thanks to Windows Printer Spooler).
I use this system to store files in a common and unique format (PDF) in a mixed environments when different types of files exists and problems are raised when you only need to read a simple document
Hope it helps
Glad to read your comments
Andrea Ben Benini
Here's a quick command for a quite common problem:
Is there a way to find a text string inside a bunch of files in a dir ?
Sure it is, here's one :
grep 'MyCoolString' /my/preferred/directory/*
Searches "MyCoolString" inside the specified path and willcard, or if you prefer to ignore case take a look at:
grep -i 'mycoolstring' /my/preferred/directory/*
Easy isn't it ?
Obviously this is not the only method but it's my preferred one, easy syntax, easy to remember, glad to see your comments or alternative methods for this task
Hope it helps
This how-to describes the process for migrating a SLES Xen virtual machine from one physical host to another. In this scenario, the new VM will be hosted on shared storage mounted via NFS. This scenario may also work for Physical to Virtual (P2V) migration.
1. Shut down source vm on Dom0, mount the hard drives.
xm shutdown vm1
# list the drives to find which one is the root
fdisk -l /dev/sda
# Gain access to the old virtual hard drive: create the mapping under /dev/mapper
kpartx -a /dev/sda3
mount -o loop /dev/mapper/sda3p2 /mnt/temp
2. Create a new hard drive image file (60GB) on the new location using dd:
dd if=/dev/zero of=/mnt/vm2/disk0-new.img bs=1M count=1 seek=60000
# Make the file system
/sbin/mkreiserfs -q -f disk0-new.img
# Make the swapfile
dd if=/dev/zero of=/var/lib/xen/images/vm2/vm2-8g.swap bs=1k count=8392930
mkswap vm2-8g.swap # it is large because of oracle reqs
# Mount the new filesystem
mount -o loop /mnt/vm2/disk0-new.img /mnt/img
3. Create new virtual machine minimal install with SLES 10 SP2 on new Xen host, pointing to new hard drive files. Use Yast's "Create a Virtual Machine."
4. rsync the file system from /dev/mapper/sda3p2 to /mnt/img
$ rsync -rlpogt --progress --exclude-from="exclude_file.txt" /dev/mapper/sda3p2/ /mnt/img/
5. Correct entries in files in the newly replaced /etc for the new vm configuration: /etc/fstab (now /dev/xvda1 instead of hda2 for /, /dev/xvdb1 for swap instead of hda1), /etc/passwd, /etc/group, /etc/sysconfig/network/ifcfg_eth.. Kept the new vm's mac address and updated the IP for vm2 in the config file /etc/xen/vm/vm2.
6. Start up the new virtual machine. Don't forget to do an 'xm delete vm1' on the old system once you are sure it is no longer needed.
In these days I'm stressing hylafax a lot, I think it's a nice and powerful program, stable, complete and reliable. When my job will be completed I'll publish some thoughts about it.
Server part is so stable and secure, client part, expecially for Windows clients have some lacks, there're a lot of win client all around but every software I've tried has some lacks so as a lot of you I've decided to write my own.
Few of them have everything I need, except the license and price, I mean I think it's right to charge for your work but while talking about an high ranking opensource software I was thinking it has good opensource clients as well.
No matter I'm deploying a base installation with a lot of different clients so I think I'll buy some commercial clients suitable for my needs (I was really impressed about ifax.com HylaFSP client) but by the way I think I'll deploy even few clients maded by me, I'm in an Active Directory network and few considerations and limitations about existing open source clients made me take the decision of writing some clients on my own.
Now let's start with the basics, you've started reading this article for getting information about sending faxes from command line, don't you ?
While deploying my new client I've read about how to send something from command line, this solution will be integrated in my new client. First you need to install command line hylafax-client, check out your favorite distro, most of them have a package called "hylafax-client" (Gentoo, Debian based and others), if don't have it take a log at a command called sendfax, your distro should have it somewhere.
here's the command:
sendfax -f "
" -R -r "Fax Subject" -c "Coverpage comments" -x "Your recipient" -d "Recipient@1234567890" sendfax.example.document.ps
First you need to create your example document in PostScript format, if you've installed hylafax you'll have ps2xx utilities (ps2pdf, ps2ascii, ps2txt, pdf2ps, ...), use them to convert from your current format to Postscript if you don't have a PS ready document (dummy example: pdf2ps input.pdf output.ps)
then modify parameters according to your email address if you want to receive notification about the job status (failed, success), fax subject, coverpage comments if you've it and so on, obviously change 1234567890 with your destination fax number.
Pretty easy, isn't it ?
That's why I'm writing my own windows and linux wrapper, backend sendfax program is so easy to use so I just need few changes for adapting my AD integration
Let me know your thoughts
Here I'm, back again on SSH stuff, as you can see from my previous posts (search blogger name = "ben") OpenSSL and SSH stuff is very interesting and useful for me, so I wrote down a lot of notes on them, this time I'll show you how to connect to an SSH host without password input.
Yeah, I know, there're a lot of folks all around explaining you how to do that but I promise to make it easy 'n' dirty, without hassling you too much, just the basic steps for connecting to your remote host and make it working.
What would you do with this tutorial ? for example:
- you can ssh to your remote host without requiring a password, this is safe and secure (it uses SSH public/private keys) until you keep your private keys for yourself. A quite recurring task if you've a lot of machines to manage
- Copy files from an host to another, not only as utility but even for basic administration task, if you manage a network you know what I mean
- Grant someone access to certain hosts for his job (be careful ok ?)
- Use all the other SSL suite across hosts, this is not only for ssh or scp, all SSL suite is involved, look at my articles on SSH port forwarding for example, there are a lot of them (blogger: ben)
- Impress your boss or whatever you'd like
Ok, let's get started
Let's assume you've two hosts:
mylocal - the host from where you want to connect
myremote - the host where you want to connect to
1) From mylocal create an ssh rsa key pair for host validation, here's how:
mylocal:~# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
The key's randomart image is:
+--[ RSA 2048]----+
2) Now you need to copy the public key to your remote host, do NOT copy the private key, obviously use scp to do this
mylocal:~# scp ~/.ssh/id_rsa.pub myremote:~
id_rsa.pub 100% 391 0.4KB/s 00:00
so now you've your public key copied fine, let's connect to remote host now
3) Connect to your remote machine (myremote)
mylocal:~# ssh root@myremote (root or your remote username)
Last login: Wed May xx xx:xx:xx xxxx 2009 from mylocal on ssh
myremote ~ #
4) Check out .ssh stuff, if .ssh dir doesn't exist you need to create it
myremote ~ # ls -la ~/.ssh
ls: cannot access /root/.ssh: No such file or directory
If you get something like this you need to create the dir, so:
myremote ~ # mkdir .ssh
myremote ~ # chmod 700 .ssh
5) Now copy your ssh public key into authorized keys file and delete it when finished, so:
myremote ~ # cat ~/id_rsa.pub >> .ssh/authorized_keysNOTE
myremote ~ # chmod 600 .ssh/authorized_keys
myremote ~ # rm id_rsa.pub
: If you've a Debian remote host you MUST use this instead:
myremote ~ # cat ~/id_rsa.pub >> .ssh/authorized_keys2
myremote ~ # chmod 600 .ssh/authorized_keys2
myremote ~ # rm id_rsa.pub
First row is used for all major distros (Gentoo in my real example), Debian users must use the second one, check your ssh man page for details on your setup (first is the most common case)
6) FINAL TEST
Ok let's go back to our local host and try to make something to see what happens:
mylocal:~# scp example.file root@myremote:/tmp/
example.file 100% 169 0.2KB/s 00:00
mylocal:~# ssh root@myremote
Last login: Wed May xx xx:xx:xx xxxx 2009 from mylocal on ssh
myremote ~ # ls -la /tmp/example.file
-rwxr-xr-x 1 root root 169 May xx xx:xx example.file
Did you see it ? I'll hope so.
As you can see you can copy or connect to host without supplying passwds
Note (read)Sometimes additional configurations are requested on remote ssh daemon, this may vary from your distro setup and basic security configuration, if final test failed you'll probably have PublicAuthentication or RSA disabled.
In this case you need to change them, don't worry it doesn't affect or lower your current security, tipically this change is done by editing /etc/ssh/sshd_config file, you need root access for it.
sshd_config path may vary between different distro even it's the most common name
To get the correct configuration, see that the following attributes are set (not commented or set to "no") in your sshd_config file
If you change sshd_config file with these values you need to restart ssh daemon (something like: /etc/init.d/sshd restart)
Hope it helps someone
Let me know if you need help or further suggestions
Andrea Benini Ben
I still say: I love the Suse linux enterprise server software. It is easy to use and has a great integration with Wind##s server product. So till the moment where we can enjoy a 100% Linux enviroment an excellent choice.
Try hard as we might, it's impossible to get away from the OS from Redmond. All too often we need to be able to get a file off or onto a Windows system in a secure manor. Samba won't work over the Internet, NFS isn't an option either over the internet. Linux to the rescue, along with some FOSS for Windows.
FUSE or File sytem in USEr space, allows Linux to create file systems of multiple kinds that can be mounted and manipulated by an unpriviledged user, meaning no root privileges are needed at any time. SSHFS or SSH File System, is the type of file system we will use. Other Windows connectable file systems exist but since we are going over the open Internet and we want data integrity, and security I've chosen SSHFS for this. I've tested it on win2k XP and 2003 server so far without a problem.
Step one: go to http://sshwindows.sourceforge.net/download/ and grab the binary installation file for Windows. (Currently i386 version only) Install this on the Windows system you wish to mount from a remote location. The process on how to install an application on Windows and create Users is an exercise I'll leave to the reader. However to someone who is familiar at all with SSH running the install and answering it's questions should be straight forward. NOTE: If you have cygwin installed you should install the SSH server with cygwins package manager as it will be less likely to cause a conflict. At this point, from your Linux box ssh into the windows box and make sure it all works. Once it does. You are done with the windows work.
Step two: If you are running RHEL/CentOS 5.x the modules you need will already be installed in the running kernel, and the packages will be available in the normal repository. RHEL/CentOS 4.x will require that you use either DAG's repository or do the normal configure/make/make install routine. I highly recommend DAG's work as it's always rock solid, and his packages are correctly integrated into RH/CentOS system so as to not cause future conflicts. Ubuntu and Deb (sarge and Lenny) both have FUSE available in the apt repository and the dependency will be brought in when you install SSHFS.
Step three: Install SSHFS With RHEL/CentOS you will need DAG's repository setup. Then just yum install fuse-sshfs, or Ubuntu/Debian do apt-get sshfs. All dependency's will be pulled. The following should be taken care of before you begin. Edit /etc/group (I use vigr) and add all users who you want to be able to use SSHFS into the group fuse. Once this is done we are ready to test things out.
Now we get to mount our remote windows system. Create a dir in your home directory to mount the remote system and then prepare to do a simple test run.
usage: sshfs [user@]host:[dir] mountpoint [options]
Is the proper format for running the command, and remember that if you skip the [user@] portion, just as in Linux it will assume that your current logged in user is the username to use on your remote server as well.
#> sshfs myname@mywinbox: /myhome/mountpoint
This will grab the user home or C: (depending on System) mount it locally and give you access to read/write these files as if they were locally held. Also at the same time any user on the Windows box will also be able to read/write the same file (no file locking is provided. ) so it's best to co-ordinate with the user on the other end if needed.
At this point the movement of files will be at the same speed as a normal ssh, man sshfs can be read and used to fine tune the sshfs mount the most important from my viewpoint is the ability to set the UID and GID of the files so that the correct user has access to them. Again since this isn't a comprehensive tutorial, I'll leave it to the reader to learn from the man page.
I just setup a Xen DomU (Debian Lenny) and found that when I tried to live migrate it between hosts, any pings would just hang. I found my logs filled with clock warnings about going backwards in time. Here is how I resolved this.
These are instructions for creating an Ubuntu Jaunty DomU on Debian Lenny Dom0 using Xen 3.2.1 that comes with Lenny. I'm installing into an LVM rather than using a file image. See this HowtoForge article for background of my particular setup. This tutorial assumes you're working from a similar setup with an iscsi target and LVM-based VMs.
The reason I'm posting this is because I had such a hard time finding some combination of tools that worked correctly to do this. I tried a few options but had little success:
- debootstrap - Didn't seem to work with Jaunty at all. It would just crash
- vm-install - Doesn't support LVM-based VMs by default, but there is a procedure for converting the qcow image to an LVM
What I found is that I could find no way of creating the LVM-based VM in Lenny. I actually had to create a sid instance first and then install my open-iscsi and xen-utils into the sid VM, then use SID to populate the VM. Perform the following actions in your SID vm or workstation
Note: You'll need your /etc/xen-tools/xen-tools.conf setup like the instructions in the HowtoForge article (things like gateway, netmask, broadcast, passwd, fs type, install-method, etc).
Whenever I would try to build the vm with the default partitions, it would always fail. For some reason the resulting vm would have an fstab that shows something like sda1 and sda2 instead of xvda1 and xvda2. I had to create my own partition scheme before anything would work right. Here's mine:
$ sudo cat /etc/xen-tools/partitions.d/with-data
You will need to create a symlink for jaunty so that the option is recognized at the command-line.
$ cd /usr/lib/xen-tools
$ sudo ln -s edgy.d jaunty.d
Now, finally, you can create the image. This step may take some time
$ xen-create-image --hostname=myserver --dhcp --lvm=vg_xen --dist=jaunty --mirror=http://archive.ubuntu.com/ubuntu --size=4Gb --memory=512Mb --swap=512MB --arch=i386 --partitions=with-data
Since you're building the LVM on a different machine than it will eventually run on, you'll need to copy the resulting xen config to the correct server:
$ scp /etc/xen/myserver.cfg root@realhost:/etc/xen/
That should be all you need to get this working.