Linux.com

Community Blogs



Create a custom distro: Create your first image (USB)

I've just received two messages with the same question related to my previous article Create your first ISO image: "Is there a way to create an USB stick with it ?". Short answer is: "Yes, there's  script to do it" but if you're still reading you may want more information. Here's what you might do if you want to create a bootable USB stick from your virtual slax build machine.

I'm still referring to my previous article: "Create a custom distro: Create your first ISO image", path and directories are still the same

Take a look at the source !

The creation of a bootable USB stick is not a mess, it's quite simple, if you take a closer look to "bootinst.sh" script located into the /boot directory you might understand what you can do with it but if you're lazy or you want to have more automation you may use the script listed below, I've created it for personal purposes but it's basically a wrapper of bootinst.sh. 

Let's keep the same virtual machine, same paths and locations, just place a script in the same dir where you've placed the script for creating the ISO image for CDs (/root/target/).

#!/bin/bash
# This script builds an ISO file from the image created
BASEPATH=/mnt/sda1/target
TARGET=/dev/sdb1
MOUNTPOINT=/mnt/sdb

echo "Mounting and formatting the USB Stick"
mkdir -p $MOUNTPOINT
umount $TARGET 2>/dev/null
mkfs.ext2 $TARGET -m 0
# Device not found or not mounted
if [[ $? -ne 0 ]]; then
    echo -e "TARGET USB DEVICE NOT FOUND, INSTALLATION FAILED !!"
    exit
fi
mount $TARGET $MOUNTPOINT
echo -e "Generating `du -kLs image` size"
echo "Copying contents into the USB Stick..."
cp -Rd $BASEPATH/image/* $MOUNTPOINT/
$MOUNTPOINT/boot/liloinst.sh
echo -e "Target disk size"
df $TARGET
umount $TARGET
echo -e  "Installation Completed"

This script is very simple but just to be clear here's what it does:

  1. Create the mount point for your usb stick
  2. Format the usb stick (ext2 is sufficient but you may use something else)
  3. Mount your usb drive
  4. Copy the contents with your new image in it
  5. Add lilo to make it bootable

In my virtual environment /dev/sda is my primary virtual hard disk, take a look at previous article for path usage, /dev/sdb is what VMWare uses when I connect an USB stick to the virtual machine. Before running this script make some checks in your current system, as you can see it's a batch script to format and wipe sdb internal contents, so you've been warned !

If you're inside my virtual environment you don't need to worry about it

liloinst.sh script is smart enough to understand where it resides and it applies lilo in the MBR of the mounted drive.

There's a lot of space for improvements so if you modify it or automate the script a little bit more just drop me few notes or send me your version and I'll modify and republish it for the community

As usual feel free to contribute with your comments

 

Previous step:

Create a custom distro: Create your first ISO image (CD)

Next step:

Create a bare bone machine with minimal footprint and requirements

 

Create a custom distro: Create your first ISO image

We finally have our own build machine, SLAX fully loaded, system up and running (in a virtual environment) so we can make a lot of tests with it.

Now it's time to create the first image, it's just a matter of minutes.

 

Preparation

When the login screen comes around you may enter in your system with the root user, “toor” is the default password with SLAX unless you change it, /etc/issue is reporting a lot of useful information for newbies as well (root/toor, xconf, startx, …)

 

Now in less than 5 minutes you'll be able to create your new virtual machine, let's create a simple environment:

root@slax:~# mkdir target target/image target/iso
root@slax:~# cd target/image/

Now it's time to copy the contents of your build machine into your target host space:

root@slax:~/target/image# cp /mnt/sda1/boot . -R
root@slax:~/target/image# cp /mnt/sda1/slax . -R
root@slax:~/target/image# ls -la
total 16
drwxr-xr-x 4 root root 4096 May 30 17:35 ./
drwxr-xr-x 3 root root 4096 May 30 17:33 ../
dr-xr-xr-x 6 root root 4096 May 30 17:35 boot/
dr-xr-xr-x 7 root root 4096 May 30 17:35 slax/

Where /mnt/sda1/ is my root partition, if you've followed my previous article you may have the same device name, expecially if you've installed it in a virtual host (I'm using VMWare), if you're not under /mnt/sda1 you may discover your root partition name, it ain't that difficult:

root@slax:~/target/image# df
Filesystem 1K-blocks Used Available Use% Mounted on
aufs 7115576 631668 6122448 10% /
tmpfs 255440 0 255440 0% /dev/shm
/dev/sda1 7115576 631668 6122448 10% /mnt/sda1

 

Now in your target directory you've the essential structure of your new target device

 

root@slax:~/target/image# ls -la
total 16
drwxr-xr-x 4 root root 4096 May 30 17:35 ./
drwxr-xr-x 3 root root 4096 May 30 17:33 ../
dr-xr-xr-x 6 root root 4096 May 30 17:35 boot/
dr-xr-xr-x 7 root root 4096 May 30 17:35 slax/

This is just what you need to create your new target system

 

 

 

Creating a target ISO

to create a target ISO with an exact copy of your system you only need to use a ready made script file located under target/image/slax directory, invocation is straightforward:

root@slax:~/target# ./image/slax/make_iso.sh iso/mycoolimage.iso
2.01.01a53 (i686-pc-linux-gnu)
Scanning ../.
Scanning .././boot
Scanning .././boot/dos
Scanning .././boot/isolinux
Excluded by match: .././boot/isolinux/isolinux.boot
Scanning .././boot/pxelinux.cfg
Scanning .././boot/pxelinux.cfg/web
Scanning .././boot/pxelinux.cfg/web/conf
Scanning .././boot/syslinux
Scanning .././slax
Scanning .././slax/base
Scanning .././slax/modules
Scanning .././slax/optional
Scanning .././slax/rootcopy
Scanning .././slax/tools
Scanning .././slax/tools/WIN
Writing: Initial Padblock Start Block 0
Done with: Initial Padblock Block(s) 16
Writing: Primary Volume Descriptor Start Block 16
Done with: Primary Volume Descriptor Block(s) 1
Writing: Eltorito Volume Descriptor Start Block 17
Size of boot image is 4 sectors -> No emulation
Done with: Eltorito Volume Descriptor Block(s) 1
Writing: Joliet Volume Descriptor Start Block 18
Done with: Joliet Volume Descriptor Block(s) 1
Writing: End Volume Descriptor Start Block 19
Done with: End Volume Descriptor Block(s) 1
Writing: Version block Start Block 20
Done with: Version block Block(s) 1
Writing: Path table Start Block 21
Done with: Path table Block(s) 4
Writing: Joliet path table Start Block 25
Done with: Joliet path table Block(s) 4
Writing: Directory tree Start Block 29
Done with: Directory tree Block(s) 17
Writing: Joliet directory tree Start Block 46
Done with: Joliet directory tree Block(s) 15
Writing: Directory tree cleanup Start Block 61
Done with: Directory tree cleanup Block(s) 0
Writing: Extension record Start Block 61
Done with: Extension record Block(s) 1
Writing: The File(s) Start Block 62
4.89% done, estimate finish Mon May 30 17:44:43 2011
9.77% done, estimate finish Mon May 30 17:44:43 2011
14.65% done, estimate finish Mon May 30 17:44:43 2011
19.53% done, estimate finish Mon May 30 17:44:43 2011
24.42% done, estimate finish Mon May 30 17:44:43 2011
29.29% done, estimate finish Mon May 30 17:44:43 2011
34.18% done, estimate finish Mon May 30 17:44:45 2011
39.05% done, estimate finish Mon May 30 17:44:45 2011
43.94% done, estimate finish Mon May 30 17:44:45 2011
48.82% done, estimate finish Mon May 30 17:44:45 2011
53.71% done, estimate finish Mon May 30 17:44:44 2011
58.58% done, estimate finish Mon May 30 17:44:44 2011
63.47% done, estimate finish Mon May 30 17:44:44 2011
68.34% done, estimate finish Mon May 30 17:44:44 2011
73.22% done, estimate finish Mon May 30 17:44:44 2011
78.11% done, estimate finish Mon May 30 17:44:44 2011
82.99% done, estimate finish Mon May 30 17:44:44 2011
87.86% done, estimate finish Mon May 30 17:44:44 2011
92.75% done, estimate finish Mon May 30 17:44:44 2011
97.64% done, estimate finish Mon May 30 17:44:45 2011
Total translation table size: 2048
Total rockridge attributes bytes: 9444
Total directory bytes: 32768
Path table size(bytes): 208
Done with: The File(s) Block(s) 102222
Writing: Ending Padblock Start Block 102284
Done with: Ending Padblock Block(s) 150
Max brk space used 21000
102434 extents written (200 MB)

And that's it, you now have a file called mycoolimage.iso under the iso directory

 

root@slax:~/target# ls -la iso/
total 205084
drwxr-xr-x 2 root root 4096 May 30 17:44 ./
drwxr-xr-x 4 root root 4096 May 30 17:36 ../
-rw-r--r-- 1 root root 209784832 May 30 17:44 mycoolimage.iso

you may create a little script if you want to automate the whole process:

#!/bin/bash
# This script builds an ISO file from the image created
BASEPATH=/mnt/sda1/target
REMOTEHOST=192.168.84.1

mkdir -p $BASEPATH/iso
$BASEPATH/image/slax/make_iso.sh $BASEPATH/iso/image.iso
scp $BASEPATH/iso/image.iso ben@$REMOTEHOST:/iso/image.iso

p { margin-bottom: 0.08in; }

To make my own tests I've two virtual machines, one it's called “Slax Build Machine”, one it's called “Slax Test Machine”. “Slax Build Machine” is the machine we're now using and the same machine we've built in the previous chapter. “Slax Test Machine” it's an empty VMWare virtual machine, no hard drives, just a virtual cdrom device pointing to an ISO file located on the real host (in the directory /tmp/image.iso). When I'd like to make some tests I just need to create an ISO file (in the build machine with the previous script). The script copies the .ISO image in the physical host after the creation, then I start the Test Machine when needed, I don't need to waste CDs or reinstall/configure the virtual machine each time, it's really easy

 

In this chapter we've just created an identical copy of our build machine, in the next chapter we'll customize the test machine, we'll start with a bare bone system and we'll add modules/functionalities later on

I guess this phase is really easy but it's important to understand it well, from the next phase you'll understand why customizing and building a slax machine is really easy and powerful

 

Previous Chapters

Next Chapters

 

As usual let me know your thoughts and drop a note below if you want

 

Andrea (Ben) Benini

 

 

 

 

openSUSE Weekly News 176 is out!

We are pleased to announce our openSUSE Weekly News Issue 176.

In this Issue:

  • openSUSE Conference 2011 to be creative and open!
  • Manu Gupta: My GSoC Project – SaX3
  • Linux Tag 2011
  • Will Stephenson: Have you BURPed yet today?
  • and many more ...

You can download it there:

We hope you enjoy the reading :-)

If you want to help us collecting interesting articles for the openSUSE Weekly News, so you can all your stuff into our new ietherpad: http://os-news.ietherpad.com/2.

Found Bugs? Please place it in our Bugtracker: http://developer.berlios.de/bugs/?group_id=12095

Features, Ideas and Improvements can placed in our Featuretracker: http://developer.berlios.de/feature/?group_id=12095

Older content can be found there.

Flattr this

 

Create a custom distro: building the build machine (p.2)

This page follows from: Create a custom distro: building the build machine (p.1)

 

Installation Process

The first thing to do is to partition the virtual hard drive, fdisk is my favorite tool and you'll be able to create a boot disk quickly:

root@slax:~# fdisk /dev/sda[ENTER]

The number of cylinders for this disk is set to 1044.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): n[ENTER]
Command action
e extended
p primary partition (1-4)
p[ENTER]
Partition number (1-4): 1[ENTER]
First cylinder (1-1044, default 1): [ENTER]
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1044, default 1044): 900[ENTER]

Command (m for help): n[ENTER]
Command action
e extended
p primary partition (1-4)
p[ENTER]
Partition number (1-4): 2[ENTER]
First cylinder (901-1044, default 901): [ENTER]
Using default value 901
Last cylinder, +cylinders or +size{K,M,G} (901-1044, default 1044): [ENTER]
Using default value 1044

Command (m for help): t[ENTER]
Partition number (1-4): 2[ENTER]
Hex code (type L to list codes): 82[ENTER]
Changed system type of partition 2 to 82 (Linux swap)

Command (m for help): a[ENTER]
Partition number (1-4): 1[ENTER]

Command (m for help): p[ENTER]

Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x17159cab

Device Boot Start End Blocks Id System
/dev/sda1 * 1 900 7229218+ 83 Linux
/dev/sda2 901 1044 1156680 82 Linux swap

Command (m for help): w[ENTER]
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
root@slax:~#

 

 

Now you've a drive with two partitions (resize them according to your needs), let's format them:

 

root@slax:~# mkfs.ext2 /dev/sda1[ENTER]
mke2fs 1.41.3 (12-Oct-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
452480 inodes, 1807304 blocks
90365 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1853882368
56 block groups
32768 blocks per group, 32768 fragments per group
8080 inodes per group
Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Writing inode tables: done Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 27 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
root@slax:~# mkswap /dev/sda2[ENTER]
Setting up swapspace version 1, size = 1156676 KiB
no label, UUID=1bbc1448-eef5-4faf-874b-f6d9f873459d
root@slax:~# swapon /dev/sda2[ENTER]
root@slax:~# free[ENTER]
total used free shared buffers cached
Mem: 510880 272320 238560 0 38944 183456
-/+ buffers/cache: 49920 460960
Swap: 1156672 0 1156672
root@slax:~#

Now it's time mount the target drive into your system:

root@slax:~# mkdir /mnt/sda1[ENTER]
root@slax:~# mount /dev/sda1 /mnt/sda1[ENTER]
root@slax:~# df[ENTER]
Filesystem 1K-blocks Used Available Use% Mounted on
aufs 306528 916 305612 1% /
tmpfs 255440 0 255440 0% /dev/shm
/dev/hdc 204850 204850 0 100% /mnt/hdc
/dev/sda1 7115576 15900 6738216 1% /mnt/sda1

 

As you can see my virtual appliance mapped my virtual CD-ROM unit as /dev/hdc, please change your settings according to your virtualization software if needed because it's time to enter into the CD-ROM root and copy everything in the virtual hard disk, here's:

 

root@slax:~# cd /mnt/hdc/[ENTER]
root@slax:/mnt/hdc# ls -la[ENTER]
total 10
drwxr-xr-x 4 root root 2048 Aug 4 2009 ./
drwxr-xr-x 5 root root 100 May 20 12:59 ../
dr-xr-xr-x 6 apache apache 4096 May 20 10:31 boot/
dr-xr-xr-x 7 apache apache 4096 May 20 10:31 slax/
root@slax:/mnt/hdc# cp -R * ../sda1/[ENTER]

I've just copied the entire content of the Slax CD-ROM drive in the hard disk, now we only need to install LILO to have a bootable hard disk.

Enter into your hard drive location and issue the liloinst.sh command, it's fairly simple to understand what it does, by the way this script will explain you what it's for and asks you to read few messages to continue the installation

 

root@slax:/mnt/hdc# cd ../sda1/[ENTER]
root@slax:/mnt/sda1# ./boot/liloinst.sh[ENTER]

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Welcome to Slax boot installer -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

This installer will setup disk /dev/sda to boot only Slax from /dev/sda1.
Warning! Master boot record (MBR) of /dev/sda will be overwritten.
If you use /dev/sda to boot any existing operating system, it will not work
anymore. Only Slax will boot from this device. Be careful!

Press any key to continue, or Ctrl+C to abort...[ENTER]





Flushing filesystem buffers, this may take a while...
Updating MBR to setup boot record...
Warning: The initial RAM disk is too big to fit between the kernel and
the 15M-16M memory hole. It will be loaded in the highest memory as
though the configuration file specified "large-memory" and it will
be assumed that the BIOS supports memory moves above 16M.
Added Slax ? *
Disk /dev/sda should be bootable now. Installation finished.

Read the information above and then press any key to exit...[ENTER]

root@slax:/mnt/sda1#

 

 

We did it. Now your machine is fully bootable and ready to go, power off the system and remove your virtual CD-ROM from VMWare configuration menu, it's not needed anymore.

Next step will prepare your new build machine to host a “simple binary toolchain”, with it you'll be able to create appliances and understand Slax configuration as well

As normal just drop me a note if you'd like to have information about it or if you'd like to have more details on what I've done

 

Previous Chapters

Next Chapters

 

Andrea (Ben) Benini

 

Create a custom distro: building the build machine (p.1)

Introduction

Back again with a new episode of my favorite saga: Creating a custom live distro for a special appliance, if you've read all the introduction articles (this link goes to the summary) you may know that now it's time to go deep with some serious stuff. As I told you before in previous episodes I've taken SLAX as my base due to its marvelous features if you need to deal with custom distro like me.

What I'll do now it's to create “the building machine”, from there you'll create new distros and explore its powerful features: create new packages, install them, do some custom stuff and so on.

 

 

Download

 

The first thing to do is obviously download an ISO from their site (http://www.slax.org), you may add or remove your favorite software from the basic installation, if you visit the site for a while you'll notice how simple is to get something from them, for this example and this custom build machine it's not necessary to download additional software, just grab the ready made slax machine as it is, it's about 200Mb so it's not that big. In the download area get the ISO image, we'll use it to install our machine.

 

Create the Machine

Now to work in a clean environment I'll create a virtual build machine and I'll install Slax there, you don't need to worry about your host operating system, this build machine is a virtual appliance so you may move or use it wherever you want. I've used VMWare because that's what I use more often, obviously you may create your own machine with your favorite virtualization software. If you're lazy and you don't even want to create your machine, you may ask me a clean Slax build machine. I don't have a web space online for 200Mb, if you can supply me something I'll publish this machine for free in that area

Now this virtual machine has just basic components: CPU, memory (512Mb), CD-ROM, hard disk, nothing more is required for a basic startup.

  • Memory is not an huge problem, our Slax will run even on memory constrained systems, my choice is to use 512Mb of ram
  • CD-ROM needs to be mapped to your fresh ISO just downloaded from Slax site, you don't need to burn a CD for a virtual machine (even if is up to you to decide what to do), after your final installation you may remove the CD-ROM device, we'll install Slax in a virtual hard drive 
  • Hard Disk, just take a standard 8Gb virtual disk, even if Slax will never use it all

 

 

 

Here's a screenshot from my VMWare Player with my basic startup configuration

 

First Boot

Now power on your virtual machine, we'll install the exact image contained into the CD-ROM in our hard drive. After the boot you'll see Slax splash screen, just select “text only boot mode”, we don't need XWindow now

Wait for a while until you'll see your login and type “root” as username and password “toor” as reported in the welcome message; now you'll see the prompt and you're ready for your clean install.

 

 

In the next page there's the installation process

 

 

 

 

 

 

 

openSUSE Weekly News 175 is out!

We are pleased to announce the new openSUSE Weekly News Issue 175!

In this Issue:

  • openSUSE 11.2 has reached end of SUSE support - 11.2 Evergreen goes on!
  • Sebastian Kügler: What’s new in Plasma Active?
  • Thanasis Rous: Medical Report
  • ZDNet/Steven J. Vaughan-Nichols: Novell will continue to support LibreOffice
  • and many more ...

You can download it there:

We hope you enjoy the reading :-)

If you want to help us collecting interesting articles for the openSUSE Weekly News, so you can all your stuff into our new ietherpad: http://os-news.ietherpad.com/2.

Found Bugs? Please place it in our Bugtracker: http://developer.berlios.de/bugs/?group_id=12095

Features, Ideas and Improvements can placed in our Featuretracker: http://developer.berlios.de/feature/?group_id=12095

Older content can be found there.

Flattr this

 

My first blog about Gentoo

Gentoo is not easy to use, but once you get familiar with it, you will fall in love with it.

 

openSUSE Weekly News 174 is out!

We are pleased to announce our new openSUSE Weekly News Issue 174.

In this Issue:

  • Alex Eftimie: GSoC 2011: PackageKit and AppStream integration for Software Center
  • Sebastian Kügler: Plasma Active at Tokamak 5
  • openSUSE Ambassadors are rocking all over the world
  • h-online/Thorsten Leemhuis: Kernel Log: Coming in 2.6.39 (Part 3) - Architecture and infrastructure
  • and many more ...

You can download it there:

We hope you enjoy the reading :-)

If you want to help us collecting interesting articles for the openSUSE Weekly News, so you can all your stuff into our new ietherpad: http://os-news.ietherpad.com/2.

Found Bugs? Please place it in our Bugtracker: http://developer.berlios.de/bugs/?group_id=12095

Features, Ideas and Improvements can placed in our Featuretracker: http://developer.berlios.de/feature/?group_id=12095

Older content can be found there.

Flattr this

 

Create a custom distro: Choice of the right base (P.2)

previous page...

 

Debian

This is one of my favorite distribution for servers, when I don't have enough time for a fine tuned HW optimization (Gentoo) or when I don't have so much time to spend on the installation and configuration process (Arch) I usually adopt Debian. With Debian you can do some sort of “dumb/quick installation” (Debian is not dumb!), the installation process gives you a working machine in half an hour. That's what I'm definitively using it on production servers, rock-stable, optimized and well supported. ….but, I've tried to install it in an USB stick with space constraints, read-only root partition and some customizations, you can do it but someone else gives you a better result (read below). I've tried to apply Linux Live-CD scripts for my personal read-only installation, you can do it but I've had few issues, mostly on:

  • startup scripts optimization, I'd like to install packages and modify startup scripts later. If you'd like to have a “prebuilt-template” to apply to it you've to manually add everything later on

  • Debian is not aimed to a live USB/CD environment, it's made for something else (servers for example)

  • root partition is not compressed (knoppix style), you may do it externally. Disk space for a tipical installation doesn't fit under 100Mb. Now “minimal” Debian installation is not so minimal

  • You may do something on a typical installation and then reduce it later on but it's not an automated system and you need to reproduce each step manually every time you need to change something

Sadly this is one of my favorite choices but this is not the right kind of solution to my own problem

 

OpenSuSe

I've not against SuSe, it even has the best tool for building a tailored distro, everything on the web in the cloud, SuSe studio is an impressive tool but:

  • Like Debian their minimal installation is not so minimal, you cannot go under 200Mb

  • You may do customizations on it after the installation but what I'm trying to do is a read-only image

  • Cloud computing is fine for me but I prefer to have my own virtual machines in my own environment and waste my own disk as often as possible, what happens if I don't renew my account or if I don't log often in their website ? What happens if I'd like to work on my own machine and I don't have an Internet connection ?

  • I want to apply patches or modify init scripts for my installation, after package selection but before installation

I like it but as Debian I think it's not aimed to Live-CD/USB creation

 

if you're still reading this article...

Relax, I don't want to write a review about every possible Linux distribution available in our planet, I've really made these tests before choosing the right one (the one I'm most comfortable with in this particular situation). I'd just like to tell you a simple thing:

“every distribution was made for a specific task, you may do everything you want with it but there're easier approaches to your problem every time you need something different, an easy approach with a specialized distribution is always preferred if it doesn't compromise stability, speed and optimization“

I've really spent a lot of time to find out possible solutions with all these listed distros and I've finally decided to use a distro for their unique characteristics. My final choice for this particular problem is : SLAX. If you're still reading this article and you're still interested in my decision here are topics achieved with it:

  • I was using LiveCD Scripts (they're very popular) with LFS and Gentoo, I think this the best approach and the simple one

  • You don't need to deal with targeted/specialized packages (legacy distro packages: gentoo, ubuntu, fedora, ...) to build a live appliance, you've a live distro aimed built around live CD creation

  • you've a modular approach, this is tipical in a live cd environment, on slax there's just one required package: “The Core” (~50Mb), everything else is an option

  • You've AUFS (http://aufs.sourceforge.net) ready to use and it's already there

  • Every “module” (a package) is compressed and easily pluggable into your own personal system, you just need to copy it into the “modules” directory

  • You may create your own LiveCDs or LiveUSBs in just few minutes

  • Documentation is simple

  • You may add your own scripts or alter final file system structure just by adding files to the final root dir. This is the thing I love more, SLAX is using UnionFS approach to make live customizations. If you don't know UnionFS take a look at http://en.wikipedia.org/wiki/UnionFS, this is one of the biggest benefits when dealing with live distributions

  • It's based on Slackware, that's one of the reasons I've never cited it in the first part of the article. Slackware has a lot benefits for this kind of system but SLAX is really focused on my own system

  • A lot of cool different projects are using it for live cd tools (registry editors, demos, security, auditing, ...)

I was discussing in the forum section of this site about Live CD scripts a month ago with other folks, obviously Slackware addicted guys (:-) Matthew, Eric, … thanks :-) ) were more focused on it, I was already using it but I've always used it as the “Slackware little bro”. I've ran it few times (auditor, …) and used it for a while but I've always left it in a corner.

But after spending weeks with LiveCD scripts (http://www.linux-live.org/) and other distros (mostly LFS, Gentoo and Debian) I've decided to use them in their “original environment”: SLAX.

It was damn easy to use it, customize a current Slax installation, produce my own USB install, create and port new packages to it. In just half an hour (GCC installation time on Gentoo...) I've created my Slax build station and I was ready to create USB sticks. Everything available from the original Slax environment, easy customizations, minimal footprint (my first install: 80Mb!) and no external tools involved in my process creation. Just a file manager to handle/copy files and few minutes for everything ! That's what I need and this is definitively what I need to finish my appliance.

 

Where's the code ? Where's the solution ?

System requirements, hardware selection and distribution choice are mandatory sections (even the boring ones) for a project like that, in the next section I'll start with hands on descriptions and I'll start publishing some code involved in this particular project. I've decided to explain my specs with these chapters so I may start a discussion with you about the overall project, I could start my articles with sentences like: “I use Slax to do that” but maybe building a custom Linux distro it's not so appealing if you don't know the reasons and initial requirements

 

As usual feel free to add your comments or discuss about my choices.

 

Andrea (Ben) Benini

 

 

Previous chapters:
Choosing the hardware
Creating a live distro for target appliance

Next chapters:
Create a custom distro: building the build machine

Create a custom distro: building the first target machine

Building a minimal image

 

 

Create a custom distro: Choice of the right base (P.1)

Intro

I'm still following base directives for creating a custom live distro for a special appliance, you've certainly read my previous considerations related to hardware and initial planning requirements. For this particular home server I've decided to stay with a common x86 device, power consumption could be a big issue if compared to low power CPU boards (MIPS and ARM devices for example) but at least I've an huge choice and many different Linux distributions.

I've decided to use an Atom based motherboard with an Intel NM10 chipset, an Atom N450 CPU is big enough (even too big...) for my designed tasks but I've decided to install a tailored Linux distro instead of installing a full blown stock distro. If you've read my previous considerations and constraints I'd like to achieve these tasks:

  • No further modifications are needed on this appliance after initial installation

  • I've already decided basic services this appliance should provide to my home network

  • It's not a desktop, there's no mouse, keyboard, screen or whatever, it has to work on a LAN, no direct user interaction

  • It must run unmanaged or without an IT specialist around for a long period, I often spend a lot of time away from where this machine is, I don't have a skilled user and I don't even need it to run this device.

  • It's just a stupid appliance, no frills

  • To avoid HW failures I'd like to reduce breaking parts, no fans, no hard drives, no SDD. This board needs to be attached to a common (and cheaper and easily replaceable) ATX PSU. Again: no moving parts or fan around. I've few power supply units with low specs and low power consumptions, I've just used one of them

  • Hard drives may fail, to replace them easily I've decided to run everything without an hard drive, I've decided to use a cheap and common USB Key as primary root disk. To avoid troubles with cheap USB keys I've decided to use one of them and have at least a backup of it elsewhere, if I need to replace my “main USB root disk” I only need to throw away the bad one and use a new one

  • To avoid troubles with USB sticks and possible breakage in the long term I've decided that my root disk has to be read only, USB sticks are excellent devices when you only need to read them, you may corrupt them easily in the long term if you write data on it (it's still a flash..., it has limited write cycles)

  • It has to be a thin distro so I can use cheap USB sticks, I don't need a 4Gb USB stick just for running an OS with few basic services. In my case I've decided to stay under 128Mb, there are plenty of sticks around with 128/256/512Mb and they're so cheap...

  • I don't want to keep basic distro behavior and I obviously need some customizations on services, networking and some special personal needs

  • I want to build my personal toolchain to be able to reproduce the same image and do more customizations on it

If you've carefully read my specs and you tend to keep a small footprint on tailored machines like me you've probably understood why I haven't even considered to install mainstream distributions like Ubuntu/Fedora Core/Mint/... I'm not against them but these are for a common desktop user and full of unused utilities (even bloatware sometimes...) and not really optimized for my special task so I've started to consider other distros I already use, here are my thoughts:

 

LFS (Linux From Scratch)

This is definitively my first choice and the approach I prefer with my personal computers, it's not even a distro (it's just a series of books). THIS IS LINUX AND GNU, nothing more. You've the GNU operating system and a Linux kernel, this is the approach (not even a distro) for the definitive geek. You can do everything you want with it: you may customize scripts, dependencies, packages makefiles, your toolchain and build scripts as well. It takes a while (!!!) to compile everything but at the end you have your own Linux system with LSB compliance as well. This is what we all need to use all the time... but.... in this case this is not what I've used, that's why:

  • x86 support for LFS is excellent and the base book is really something to read at least once in a lifetime. I've my own build script (a set of bash scripts and makefiles) to automate the whole process, even with this magical script my installation procedure after some changes takes at least: 6-8 hours for building the base toolchain, 4-6 hours to build my basic system. When you test it or when you make some trials to adjust your internal requirements you may spend a week on it just to make some tests, that's definitively what I don't want to do

  • An LFS installation process gives you a full blown GNU OS with a Linux kernel, installation takes across 300-400 Mb (for a bare bone system) or more probably around 500-600 Mb (samba, httpd server, php, DLNA, torrent, …). You may tweak it with a busybox but it requires extra steps, even binary stripping is required but this is not so easy, you may even cut unnecessary man pages as well as other things (you already have manuals somewhere...). More extra steps, more time to realize the project

  • It's not aimed to be a read only system, you obviously have the Live CD and some scripts but if you tend to recreate your system frequently you need extra time (again), live cd scripts (www.linux-live.org) are impressive but LFS needs a lot of care

  • There's no automatic HW recognition (it's not so important BTW)

  • Your live image it's not compressed (like knoppix or stuff like that), you may do it because you've “the father of every Linux distro” but again: it requires a lot of extra time

  • UnionFS, and aufs may be attached to it to support some sort of changes in a read-only environment but it's a slow process if you need to implement them

Conclusions:

This is my best choice and my first attempt but unfortunately I'll be able to build my own system in a month or maybe more and there are even things I don't even know how to realize. I really appreciate LFS books and it's still one of my favorite “distros” (even if it's not a distro !) but I cannot use it in this moment

 

Gentoo

I've started with Gentoo in 2000-2001 when it was a young distribution, I was looking for a distribution with these requirements:

  • It has to be really close to LFS approach, in the beginning I've started with it because it was something like “LFS with steroids”: an LFS install with an automated build script

  • I'd like to have a “super-build-and-do-everything-for-me” script to create a running installation without worrying about copying and pasting instructions from the book. I don't want to read the same paragraph 100 times to discover after few hours what kind of instruction I've missed before

  • I've been an happy BSD user and I really love it, one of the best things about BSD is their makefile approach. When you need a program you just need to enter in their package structure, locate your favorite program (just use “find” utility), and install it with make (make clean, make, make install). Even better: you want to upgrade your whole system to current stable BSD ? Just issue “make world” and your machine will upgrade itself and install everything needed (well, it's not so simple but this is the approach). I respect monthly scheduled released distro (6 month for SuSe, Ubuntu, Fedora, …) but I'd like to have “the latest one” just with a simple upgrade command.

  • Gentoo and Arch are rolling update distributions, there's no version vX.xx, there's “the distribution you're using” and “current distribution”, with a simple command you switch from “your old one” to “current”. I really love rolling updates distros

Even more: do you want to make your favorite customizations to every package because you want or don't want that particular feature (doc, man pages, Xorg, Lang, ...) ? Are you looking for a special patch or an optimization for your particular CPU ? Gentoo is the way to go. This is my first distribution and what I use more at work (my main desktop) and at home, it was obvious for me to test it and adapt it to my appliance. But even if I still prefer Gentoo I've discarded it, that's why:

  • If you need to create a system with a limited space (my USB stick) you need to drop portage directory structure and create a system without it. There's a way to do it (read excellent Gentoo docs) but it's not so easy if you want to make frequent changes to it or do minor upgrades

  • You need a “build machine” with a full Gentoo installation and a special portage structure for creating your target machine, cross compilation it's one of the Gentoo specialties but it's not aimed to machine creation

  • There are tools like Catalyst or tools/docs for creating “stage 4” steps but these are a little bit cryptic and sometimes aimed to create other Gentoo tipical installations, most of the times you've excellent results when you need to deal with a LiveCD but it takes a lot of time. I've used catalyst a while but it's not so easy to create a live image under 200Mb ! It requires lots more

  • There are Live-USB methods, something with AUFS or UnionFS but it requires a lot to reproduce them all and there's no root disk compression (like Knoppix for example). You obviously may do it but it's like LFS, it takes a lot of time to do it

  • If you upgrade your “build station” and “toolchain” frequently you always need to make few adjustments to the build system if you've modified startup scripts for your target machine

Sadly I've discarded it due to some problems: available time, overall image size, readonly unavailability and live cd/usb compression. It's still my favorite choice for a full blown desktop or server but this time I've used something else for my installation

 

Arch

I really like it because it has the same rolling update approach like Gentoo, there are no “major versions” there's just the version you're using and the current stable, current stable changes every hour (depending on project activities...). Even more: it's definitively binary based so you don't have to wait ages to get something compiled (emerge, emerge, emerge, …), you may loose 0,00001% optimizations from your particular HW but someone still lives without it. But my poor Arch knowledge drove me to nothing, I'm an happy Arch user, I use it for building tailored servers for example and it's nice but it seems it's not really oriented to LiveUSB/CD creation like others (read below). You may use Arch for Servers or Desktop installations but in my particular case I've chosen to use something else

 

continues on the next page...

 

 

Previous chapters:
Creating a live distro for target appliance
Choosing the hardware

 

LDR: The birth of a Linux distro - Part 1

So I've always been a huge fan of Arch Linux. For many years it has been my weapon of choice for pretty much anything from web servers to development environments. It's incredibly lightweight, fast, stable yet up-to-date, and highly customizable.

Sure it doesn't have all the bells and whistles of other distros, nor does it take the responsibility of configuring complex areas such as Sound support or Network hot-swapping; but I think this is quite healthy... I spent years swearing at Microsoft products because they feel they have to do everything for you (no matter how complex) and then things just didn't work. Ironically if they have left things lite and transparent and left the responsibility of getting it working to best suit my requirements to me, then I wouldn't have held them responsible when something wouldn't work.

Thankfully I am out of that world now, and am taking what I have learnt from it with me.

My biggest issue with Arch was that it would take an entire evening to get everything I need installed and configured the way I like it. So I eventually decided to invest some time into creating my own derivative.

This derivative included my own custom repository, so I can install my packages I have acquired from various places (or written myself) from a single command.

It also contains configuration changes to the core platform, so it behaves the way I want it to (and sports my own branding!)

Hunting around the Arch forums, looking for people who have done something similar, I stumbled upon a set of script files called archiso. Unfortunately I couldn't get archiso working straight away, as there was some problems with the Makefile included on the Arch wiki. I began to edit it, adding my own parts so it could do extra things like branding.

To be able to create a derivative, I first needed to brush up on how a Live CD works. The best way to do this is to break the boot sequence into logical chunks:

The Boot Loader

The boot loader is the first component to be loaded. Arch CDs use the ISO version of SYSLINUX (ISOLINUX) which lives in the /boot folder. It loads up and reads the config file /boot/syslinux/syslinux.cfg which gives it a list of boot configurations (eg. which partition, initrd, kernel, kernel boot options etc..) that can be loaded. Once one of these is chosen, the kernel is executed using the chosen options.

Initial Ram Disk

Also known as "early user space", the initial ram disk is an image which contains the kernel and a super lite root filesystem which is just enough to prepare the system for the real root system to be mounted. This involves running kernel hooks and hardware detection.

You can create an initrd by using mkinitcpio with a config file which contains a list of the kernel hooks that need to be run during this early boot stage.

One of the hooks for Arch's Live CDs reads a file on the root of the CD called isomounts. This is an fstab file for the CD. It mounts the root file system and the Init scripts gets run.

Init scripts

The init scripts begin with inittab. This is a file which determines which runlevel should be ran (these tend to be different for each distribution, but 3 generally means a multi-user environment with networking, which is the default for arch). 5 means the same, but with a desktop manager (eg. GNOME, KDE, OpenBox, FLuxbox, wmii, dwm etc..). inittab is also the script which is responsible for respawning the login manager if you kill Xorg.

Arch also has a load more init scripts for various purposes, you can read more about them here. rc.sysinit runs on boot; it does more hardware configuration, logs to dmesg, starts the daemons etc.. Followed by rc.multi and rc.local which contains any finish up code before the login is displayed.

LDR Linux

So I now have a working Arch derivative with all these steps configured. You can check it out at: http://www.tommed.co.uk/ldr. Obviously, as Arch is GPL based, I have open sourced by build scripts and packages here. I hope it is of good use to people who want to do the same, or use as a learning tool!

I released LDR recently as an ISO image, but didn't quite realise the impact of doing so... I was added to Distrowatch and quickly started receiving thousands or hits per day, which quickly destroyed my little home web server! :(

I managed to get the ISO mirrored (BIG thanks to Neil Bright of Georgia Tech) which deferred the bandwidth load from my home server, which runs quite happily again now! 

I didn't released LDR Linux planning on turning it into the next big distro, it was more about being able to quickly install Arch with all my custom trimmings in less than 1hr and also  sharing my code and my work to inspire others, but from the feedback I have received, it's plain to see that others want LDR on their machines + they want it supported too!

I am happy to do this, but could use a bit of help from anyone in any capacity (even if it's just spell checking on the wiki, or general testing) any help or donations would be greatly appreciated!

Google Chrome, wmii, OSDBattery (run battery-status or Mod4+B), Adobe Flash, and PulseAudio all work out-of-the-box when testing in VirtualBox!

LDR Linux 1.06

 

 
Page 4 of 17

Upcoming Linux Foundation Courses

  1. LFS426 Linux Performance Tuning
    21 Apr » 24 Apr - Virtual
    Details
  2. LFS520 OpenStack Cloud Architecture and Deployment
    05 May » 09 May - Virtual
    Details
  3. LFD320 Linux Kernel Internals and Debugging
    12 May » 16 May - Virtual
    Details

View All Upcoming Courses


Who we are ?

The Linux Foundation is a non-profit consortium dedicated to the growth of Linux.

More About the foundation...

Frequent Questions

Join / Linux Training / Board