Basics of (Linux) remastering
----------------------------------

This small text file may grow into a book one day, or maybe not.
You should still search the internet for info, this document cannot suffice everything.

>> Also, no use reading this if you're unfamiliar with the shell, and Linux usage
covered in the Newbie BASH docs. <<

If you want to create your own Linux distro, Anubis-Linux is at your disposal.
It offers you all the tools you need right out-of-the-box.
You should also check out the "Linux From Scratch" project,
 http://www.linuxfromscratch.org/ or the LFS Folder.

Right then.

A Linux system is made of 2 major things:
(1) the kernel
(2) everything else

-> (1)
The kernel (the source of which you'll find somewhere in the /ANUBIS/ folder) is the heart and brain
of a Linux OS. It controls everything and important functionality lies within the kernel.
For instance, if your kernel doesn't understand the B-tree filesystem (Btrfs), then
you simply can't make use of a Btrfs-formatted partition, and so on.
The kernel is a compressed executable of around 4 MB, saved as /boot/vmlinuz-version.

The kernel is written in good ole C language. Which will be 40 years old soon (now it's 2010).
C is a general-purpose programming language - but works great as a systems programming language.
So you'll need to learn a bit of C programming, then system programming in it. Assembly
knowledge not needed, but it can be used to impress the ladies.

Read the kernel's ReadMe for info on how to compile it.
Basically, 300 MB of kernel source code will turn into a 4 MB kernel executable, and
60+ MB of kernel modules - components that are "integrated" when needed or at boot,
and finally even more hundred MBs of temporary junk.
Mostly everything is automatic, so if you're copying something by hand - you're doing it wrong.

If you don't have at least 800 MB of free space, don't even think of decompressing the kernel source,
stop reading this file and move on to something more fun.

Let's assume we're in the kernel's source directory and /kernel is the folder for the
temporary files i.e. garbage. Comments are preceded by #, don't type them in the shell.

make O=/kernel/ menuconfig
# "make" is a program that looks for a file named "Makefile"
# we specify /kernel/ as the output directory
# "menuconfig" is a routine in the given Makefile, which will display a nice kernel
#  configuration menu
make O=/kernel/
# we now ran this - compilation started and it'll last for an hour or so. not kidding.
make O=/kernel/ install modules_install
# like "menuconfig", install and modules_install are routines in the Makefile
#  they install the kernel in /boot and respectively the modules in /lib/modules

The Makefile is pure text, feel free to edit it. make can also be told to use a makefile
other than Makefile by the "-f filename" switch.
In the end, make simply runs gcc, g++, ld, shell commands, and other compilers or tools
as specified in the makefile. This makes work a bit easier.

We should also create an "initial RAM disk" for the kernel. We so do by:
 update-initramfs -c -k your_kernel_version
It'll be put in /boot as initrd.img-your_kernel_version.
An initramfs is useful for faster booting, and paramount for booting with an encrypted
root partition.

For building Debian packages of the kernel, please use make-kpkg.

-> (2)
Everything else. Well, these are the installed programs, and the window manager (WM) or
desktop environment (DE). The difference between WM and DE lies in what they offer to
the user. If besides a nice way of showing and manipulating things in a GUI, they also
offer their own suite of applications, maybe their own file format standards, and
take part in configuring your system - then it's a DE. If they just help you work in a GUI
then it's a WM. Example: KDE, XFCE and Gnome are DE's. Fluxbox, Blackbox, JWM are WM's.

Let's focus our attention on more technical stuff. The LiveCD - offering a Linux OS
that runs without ever touching the hard drive. It's also only 700 MB, whilst the full
system will eat up over 2 GB of your hard drive. Why? Because the LiveCD is compressed.
For Anubis-Linux, I use:
 mksquashfs uncompressed_filesystem/ anubis.lz -lzmadic 1M
This will take the contents of that directory and compress them into anubis.lz,
and using 1 MB dictionary size for when LZMA compression is used.
(mksquashfs uses both old Zlib and LZMA compression, always checking which is better
for the given data chunk.)
So that anubis.lz file is a Sqlzma filesystem (old SquashFS 3.4 enhanced with Lzma.)

But how can you "extract" the SquashFS/Sqlzma filesystem, from say Anubis-Linux?

(1) The classic way is to mount, then copy it:
 mount -o loop -t squashfs ./anubis.lz /tmpfs
 cp -a /tmpfs/* ./destination/

(2) The modern way is by using unsquashfs:
 unsquashfs -d ./destination/ ./anubis.lz

The choice is yours.

PROTIP: installed "unsquashfs" tool cannot extract modern Squashfs 4.0 filesystems.
 You will have to classically mount by adding "-t squashfs4" and then copy.
 Nobody stops you from installing the Squashfs4 tools but the current ones wield better compression.
 Again, the choice is yours.

To make changes to your filesystem, you'll have to change root to it (by using "chroot").
Then you can use apt-get and dpkg to add or remove software, configure stuff and so on.
Of course you can change it as described only *before* you compress it with mksquash, duh. ;)
Also - you can't compile a kernel for 32-bit machines if you're currently running a 64-bit OS.
Not even by chroot'ing into a 32-bit filesystem. This may change in the future, and maybe not.

After chroot'ing you can install packages with dpkg, apt-get and maybe configure them with
dpkg-reconfigure. This is true for as long as you stay in the Debian family - other distros
have other package management systems.

Sometimes the software you are installing will want to configure itself. For this to happen,
you may have to mount /proc and /sys. As you can see, these aren't true devices. They're a way
for the kernel to show you what's going on in the memory - on the filesystem! [APPLAUSE]
So this will work even if you chroot'ed because you're still running the same kernel, etc.
Anyway, commands are:

mount -t proc /proc /proc
mount -t sysfs /sys /sys

You may have some trouble unmounting though. Do not try to squash the filesystem with either
mounted or there will be errors. I prefer to just restart the OS, but I guess that's my
personal cowardly way of dealing with the situation. ;)

If you take a look at the CD contents, you'll notice a build_iso.sh script.
Check it out. It creates a bootable ISO filesystem from the files in the current directory.
Grub 0.97 is used to boot, but many other distros use Syslinux/ISOLINUX instead.
Syslinux is included in Anubis, and it can come in handy if you decide not to use Grub.
They have the same goal, but are different and having their own strong and weak points.
For instance you may have noticed that you can quite conveniently boot DBAN from the
Anubis LiveCD. Grub can boot it - but only because of a Syslinux component, "memdisk" which
can load bootable floppy disk images. Grub isn't able to do this on its own.

Also, the initrd.gz in the CD's /boot is not the same as the standard initrd.img! The
latter is a GZipped CPIO archive, whilst the former is a GZipped Ext2 filesystem-in-a-file.
Check it out, unzip then mount it. Be careful of the modifications you make, they'll
be automatically saved. The linuxrc script is of paramount importance.
And therefore a LiveCD shows the importance of the initial RAM disk. initrd.gz
is used by the kernel to open up anubis/anubis.lz then continue with booting it.

As a final note - don't forget to clean up the filesystem before you compress it.

PROTUT:
	Don't want to waste CDs and/or hate virtual machines but must test your distro?
	Put it on a flash drive! (This applies for Grub 0.97.)
	(1) copy the files (usually the squashed filesystem and /boot folder are enough)
	(2) make sure that the format files are available in {flashdrive}/boot/grub/.
		if your flash drive is FAT16 or FAT32 -formatted then fat_stage_1_5, stage1
		and stage2 should be enough. rely on your sense of logic if your flash drive is say,
		Ext2 formatted. (you'll usually find everything you need in /boot/grub/.)
	(3) touch a unique file on your flash drive
	(4) must be root; type "grub" (alternatively, you're free to try this with grub-install,
		but I recommend against that)
	(5) type "find {yourfile}", and let's suppose your flash drive was identified as (hd1,0)
	(6) type "root (hd1,0)" then "setup (hd1)" then "quit" - and it's done, phew.
	This "find" method is lame indeed - but the "grub shell" currently doesn't support
	autocompletion in AL and I didn't bother researching. I also don't trust grub-install.

PROTIP: Grub starts numbering from 0. So if you insert a flashdrive it will be considered
 a second hard disk, and designated hd1. Running "root (hd1,0)" will also tell you what kind
 of partition it is (0 being first partition, 1 second partition and so on).

Well that's the end for now.
I hope you enjoyed reading and hopefully this was useful to you.

----------------------------------------------------
Special Bonus Section for Anubis-Linux 2 Nine!
----------------------------------------------------

In case you were aching to know, this is how I work:
although I hate the 32-bit version, I must maintain it for my (currently very silent) users.
Plus the 64-bit version is not currently capable of compiling 32-bit kernels.
Anyway, this means that I'm forking my projects directory into 32/ and 64/.
Both directories have these directories: 1iso/ 2squash/ 3temp/ 4orig/.
I use Krusader Root because I'm a power user - aren't you? So naturally, one
panel will open the 32/ folder, the other, 64/. Updating files is pretty easy because all
you need to do is push the equal button then change 64 with 32 or viceversa and
you're there. Or you could be smarter than I, and actually use Krusader's "sync-browsing".
And I use scripts for automated filesystem cleanup and compression.
I test my work on a flash drive, because I don't like virtual machines although Qemulator
would do probably the trick. Then I use Krusader to upload my masterpiece to the FTP server.

See, it's easy - you can do it too. Why not create your own LiveDVD? Completely customised
as you like it - and if something bad happens to the OS you reinstall it all in one go,
and it's pretty much as it was last time it booted!

I know some people would like OpenOffice.org, or Eclipse, or Javathis, or Mplayerthat or whatever in AL,
but that's not going to happen. AL must fit on a CD, that is (still) one major goal of the project.

Have fun. And don't forget to stop by at our forums.

-RF