A simple Syndgen use case: A tutorial to set an home server running on an old machine
We will consider here an old Pentium III based computer. It is tempting to use it as a small server instead of trashing it. As a server, we do not need to have any built tools (gcc and so on...), we just want sshd, our preferred irc client, a small web server like cheetah. Nothing more. To compile the whole things we prefer using a desktop which is much more powerful. To do that we will use a small bash script dedicated to create chrooted environment: chenvr.
Building the basis
We first create the chrooted environment with chenvr:
chenvr -t my_server -c i686 -d funtoo
We do also some automations in my_server/.chenvr/custom.sh and my_server/.chenvr/prepare.sh but anyway, let's proceed.
Now we enter the chroot:
~ # chenvr -t my_server Mounting necessary directories...done. Architecture: 32 bits. >>> Regenerating /etc/ld.so.cache... / #
We install and configure all the softwares we need, plus the basics (hostname, ...) etc... We will not describe since it is trivial (Gentoo's documentation is done for that). We also create a dedicated kernel (Let's say hardened-sources with grsec common gentoo configuration, no module support and only the needed hardware support)
Now we are ready to install it on on the real server.
Syndgen's job
We install syndgen via portage overlay, still inside the chroot of course, and it needs to be configured. (we use version 0.2). First thing to do is to generate the ssh keys:
/ # syndgen --keygen Generating syndgen sshd keys (no password) Generating syndgen root ssh client keys ... Authorizing this root key only to login. Sshd keys created/configured
Generating a small helper script is now required:
syndgen --make-deployer
Now, in /root, we have a script called "deploy.sh". It is used to synchronize the server with the actual installation. It will in fact do an rsync over ssh, so this deploy.sh embeds the ssh key file to authenticate.
Since syndgen uses rsync to synchronize distantly the content of installation, it needs to know whether it should exclude some dir/files or not. That is what we are going to configure in 2 locations. First common.exclude file in /etc/syndgen/deploy sets the basic things which are not needed on the real server:
# Syndgen basic exclude list for rsync /.chenvr /etc/syndgen /etc/make.conf /etc/make.profile /sys/* /dev/* /proc/* /tmp/* /var/db /var/lib/portage /var/tmp/* /var/run/rsyncd.pid /usr/include /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/include /usr/portage /usr/locale/portage /usr/src/linux* *.la *.a
Now and then, we will select certain package to exclude, that could be done in /etc/syndgen/deploy/package_exclude.list where we put:
app-admin/syndgen app-portage/gentoolkit app-portage/portage-utils sys-apps/portage sys-devel/gcc sys-devel/gcc-config sys-devel/binutils sys-devel/binutils-config sys-devel/make sys-devel/automake-wrapper sys-devel/automake sys-kernel/genkernel
As you can see, it excludes almost everything that makes a Gentoo working as a "Gentoo" with its package management and so on. (See, we removed /var/db previously and more)
Now we need, from that list, to create the real rsync exclusion file, so:
syndgen --exclude
Which generates /etc/syndgen/deploy/rsyncd.exclude, it is a combination of common.exclude and the result of exclusion list form package_exclude.list: (note: It will not check against runtime dependencies so be sure that you are not excluding a critical package)
# Syndgen basic exclude list for rsync /.chenvr /etc/syndgen /etc/make.conf /etc/make.profile /sys/* /dev/* /proc/* /tmp/* /var/db /var/lib/portage /var/tmp/* /var/run/rsyncd.pid /usr/include /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/include /usr/portage /usr/locale/portage /usr/src/linux* *.la *.a ## app-admin/syndgen ## /etc/syndgen/deploy /etc/syndgen/syndgen.conf /etc/syndgen/ssh/sshd_config /usr/lib/python2.4/site-packages/syndgen /usr/bin/syndgen ## app-portage/gentoolkit ## /etc/env.d/99gentoolkit-env /etc/eclean /etc/revdep-rebuild /usr/lib/gentoolkit /usr/share/doc/gentoolkit-0.2.3 /usr/share/man/man1/equery.1.bz2 /usr/share/man/man1/glsa-check.1.bz2 /usr/share/man/man1/eread.1.bz2 /usr/share/man/man1/eclean.1.bz2 /usr/share/man/man1/euse.1.bz2 /usr/share/man/man1/revdep-rebuild.1.bz2 /usr/bin/glsa-check /usr/bin/eread /usr/bin/eclean /usr/bin/eclean-dist /usr/bin/eclean-pkg /usr/bin/euse /usr/bin/revdep-rebuild /usr/bin/equery ## app-portage/portage-utils ## ## sys-apps/portage ## /usr/lib/portage /usr/bin/tbz2tool /usr/bin/repoman /usr/bin/xpak /usr/bin/ebuild /usr/bin/portageq /usr/bin/emerge /usr/sbin/emerge-webrsync /usr/sbin/etc-update /usr/sbin/regenworld /usr/sbin/fixpackages /usr/sbin/update-etc /usr/sbin/emaint /usr/sbin/quickpkg /usr/sbin/update-env /usr/sbin/dispatch-conf /usr/sbin/archive-conf /usr/sbin/env-update /usr/share/man/man5/ebuild.5.bz2 /usr/share/man/man5/make.conf.5.bz2 /usr/share/man/man5/color.map.5.bz2 /usr/share/man/man5/portage.5.bz2 /usr/share/man/man1/emerge.1.bz2 /usr/share/man/man1/quickpkg.1.bz2 /usr/share/man/man1/etc-update.1.bz2 /usr/share/man/man1/repoman.1.bz2 /usr/share/man/man1/emaint.1.bz2 /usr/share/man/man1/dispatch-conf.1.bz2 /usr/share/man/man1/ebuild.1.bz2 /usr/share/man/man1/env-update.1.bz2 /usr/share/doc/portage-2.1.4.4 /etc/make.conf.example /etc/portage/.keep_sys-apps_portage-0 /etc/logrotate.d/elog-save-summary /etc/make.globals /etc/dispatch-conf.conf /etc/env.d/05portage.envd /etc/etc-update.conf ## sys-devel/gcc ## /usr/lib/gcc /usr/bin/i686-pc-linux-gnu-c++-4.1.2 /usr/bin/i686-pc-linux-gnu-gcc-4.1.2 /usr/bin/c++-4.1.2 /usr/bin/cpp-4.1.2 /usr/bin/i686-pc-linux-gnu-g++-4.1.2 /usr/bin/i686-pc-linux-gnu-cpp-4.1.2 /usr/bin/g++-4.1.2 /usr/bin/gcc-4.1.2 /usr/bin/i686-pc-linux-gnu-gfortran-4.1.2 /usr/bin/gfortran-4.1.2 /usr/i686-pc-linux-gnu/gcc-bin /usr/libexec /usr/share/gcc-data /etc/env.d/gcc/i686-pc-linux-gnu-4.1.2 /etc/eselect ## sys-devel/gcc-config ## /usr/lib/misc/gcc-config /usr/bin/gcc-config ## sys-devel/binutils ## /usr/lib/binutils /usr/i686-pc-linux-gnu/binutils-bin /usr/share/binutils-data /usr/share/doc/binutils-2.18-r1 /etc/env.d/binutils/i686-pc-linux-gnu-2.18 ## sys-devel/binutils-config ## /usr/bin/binutils-config /usr/share/man/man8/binutils-config.8.bz2 ## sys-devel/make ## /usr/bin/make /usr/bin/gmake /usr/share/man/man1/gmake.1.bz2 /usr/share/man/man1/make.1.bz2 /usr/share/locale/he/LC_MESSAGES/make.mo /usr/share/locale/hr/LC_MESSAGES/make.mo /usr/share/locale/fi/LC_MESSAGES/make.mo /usr/share/locale/uk/LC_MESSAGES/make.mo /usr/share/locale/zh_CN/LC_MESSAGES/make.mo /usr/share/locale/ko/LC_MESSAGES/make.mo /usr/share/locale/es/LC_MESSAGES/make.mo /usr/share/locale/ja/LC_MESSAGES/make.mo /usr/share/locale/pl/LC_MESSAGES/make.mo /usr/share/locale/vi/LC_MESSAGES/make.mo /usr/share/locale/fr/LC_MESSAGES/make.mo /usr/share/locale/gl/LC_MESSAGES/make.mo /usr/share/locale/rw/LC_MESSAGES/make.mo /usr/share/locale/pt_BR/LC_MESSAGES/make.mo /usr/share/locale/sv/LC_MESSAGES/make.mo /usr/share/locale/ru/LC_MESSAGES/make.mo /usr/share/locale/da/LC_MESSAGES/make.mo /usr/share/locale/tr/LC_MESSAGES/make.mo /usr/share/locale/be/LC_MESSAGES/make.mo /usr/share/locale/nl/LC_MESSAGES/make.mo /usr/share/locale/ga/LC_MESSAGES/make.mo /usr/share/locale/de/LC_MESSAGES/make.mo /usr/share/locale/id/LC_MESSAGES/make.mo /usr/share/info/make.info-1.bz2 /usr/share/info/make.info-2.bz2 /usr/share/info/make.info.bz2 /usr/share/doc/make-3.81 ## sys-devel/automake-wrapper ## /usr/lib/misc/am-wrapper.sh /usr/bin/aclocal /usr/bin/automake /usr/share/aclocal/.keep_sys-devel_automake-wrapper-0 ## sys-devel/automake ## /usr/share/automake-1.10 /usr/share/info/automake1.10.info-2.bz2 /usr/share/info/automake1.10.info.bz2 /usr/share/info/automake1.10.info-1.bz2 /usr/share/aclocal-1.10 /usr/share/doc/automake-1.10.1 /usr/share/man/man1/aclocal-1.10.1.bz2 /usr/share/man/man1/automake-1.10.1.bz2 /usr/bin/automake-1.10 /usr/bin/aclocal-1.10 ## sys-kernel/genkernel ## /etc/genkernel.conf /usr/share/doc/genkernel-3.4.9 /usr/share/bash-completion/genkernel /usr/share/man/man8/genkernel.8.bz2 /usr/share/genkernel /usr/bin/genkernel
Yes it is quite much but hopefully it is shorter than excluding file per file of a package and it also ensures that it will not exclude a directory containing files from an other package.
Anyway, everything is ready, let's start the syndgen's ssh/rsync daemons (note: they are totally different than the installation ones and own their configuration in /etc/syndgen so do not start those via /etc/init.d/):
syndgen --rsyncd-run --sshd-run
We are now ready to "sync" the real server. So we boot it with a minimal cd, configure the basic things (partitioning, network...) and we copy the so-called "deploy.sh" generated previously. We run it like:
./deploy.sh 192.168.0.5 2222 /mnt/gentoo/
Now, synchronization is on-going for a few minutes. When it's ready we just have to chroot into this fresh install, running grub so it will install it on MBR. We can reboot and server is up and running!
We stop syndgen's ssh/rsync daemons:
syndgen --rsyncd-stop --sshd-stop
Exiting the chroot (chenvr made)...
That's it!
==Updating the real server==
After a while, the server have been widely used. Some files have been created in /home and so on. We do not want next ssyndgen's synchronization removes those (deploy.sh uses --delete option of rsync which means: something which is not on syndgen source but in syndgen destination will be removed. So here our files in /home) It can be override via /root/syndgen.exclude, create the file and add inside (You might wont to protect /root as well for synchronisation)
/root /home
On chenvr's server environment (syndgen source), we just apply GLSA via
~# syndgen --glsa
Indeed in syndgen configuration file we have set this:
[Glsa] ## check (default), fix mode=fix
So the previous command will check for GLSA but will aply fixes if needed.
Then we start syndgen's ssh/rsync server as we did the first time.
On the real server, we run again the deploy.sh script:
./deploy.sh 192.168.0.5 2222 /mnt/gentoo/
Old binaries and/or libraries that were obsolete have been removed and so on... but /home have not been affected by the synchronisation.
==Bi-directionnal synchronization==
In our case we needed to have server without gcc, portage etc... But it might happen that you would like to have *exactly* the same environment as the syndgen source.
In such particular situation the idea is to put no exclusion at all in /etc/syndgen/deploy files. When you will deploy the syndgen source to the destination: both will own exactly same files.
If destination happens to change (new softwares installed) and you would like the source synchronized over the destination... Well let's revert the rules with syndgen. Start the Ssh/Rsync server on destitaniton environment, and run the script deploy.sh on the source environment. This should work.
