Quick setup own distcc farm

When I migrated to Gentoo, I have realised how easy can be compiling from the sources. We have faster computers, couple of cores in one processor. Seems to compiling everything from scratch shouldn’t be difficult. But how we can speed up this process. If you poses some unused PCs, laptops and you could use them for the moment, you can create computer farm for the tool called distccd. Ok but you can say there is a Windows or no system. It doesn’t matter. If there is a network adapter and computer is bootable from pendrive/cdrom, that’s everything what you need. The knowledge about clouds, matrix or clustering is unnecessary. I will try to explain how to easy create own distcc-livecd image.

Distcc

As we can read on official website, distcc is a program to distribute builds of C, C++, Objective C or Objective C++ code across several machines on a network. Distcc should always generate the same results as a local build, is simple to install and use, and is usually faster than a local compile. Configuration is also written well on that website and better in the Gentoo context we may find on Gentoo Distcc Documentation

Calculate Linux

Calculate linux is one of the distributions created on top of Gentoo. We can use portage and the same tools. This distro can be downloaded in few different versions:

  • Calculate Linux Scratch (CLS) – a special distribution, designed to build a livecd for any task.
  • Calculate Linux Desktop (CLD) is a multifunctional desktop that can provide all tasks of your business.
  • Calculate Directory Server (CDS) is an advanced, LDAP-based, authentication server designed to be a domain controller for business networks.

First one seems to be ideal for our purposes.

Own distro

For creating own LiveCD we can use this nice tutorial. It is very easy. I want to spot what I added and why. I have at home 2 more laptops. I prefer rather WLAN then LAN. However I don’t want to install unnecessary X window managers in order to save memory for compilation. For this reason I decided to use wicd as default network daemon. It is easy configurable using ncurses dialog – wicd-curses. Of course by default CLS doesn’t provide on CD tools like gcc or distccd. We must install them.

OK but what about installing new software which I want to have on my CD. Normally it would follow this way:

# emerge-sync; emerge gcc distcc wicd other_software…

Everything fine but during creating livecd. Those operations would eat our operating memory. The easier way is using packages instead of sources. What you need to do is adding to /etc/make.conf :

PORTAGE_BINHOST=”http://tinderbox.dev.gentoo.org/default-linux/x86″

for installing we add:

# emerge –usepkg –getbinpkg gcc distccd

Of course we can’t have impact on USE flags so much as in normal way, but it depends on what you really need from you livecd.

Finally you can find image of my distcc-distro which is available on DropBox* under this link. After downloading you can boot it in the Builder Mode and change further for your needs.

How start with my distro

After booting in Normal Mode we should see root prompt. Next step is creating network connection by use wicd-curses. If you need you have to add your host to distccd configuration. By default distccd allow connection from 192.168.0.0/16 10.0.0.0/16 and 172.21.0.0/16. It is fully insecure but I assume that we use local home network secured by NAT and firewall otherwise is strongly recommended to change it asap (/etc/conf.d/distccd). Once your interface is up wicd execute script /etc/wicd/scripts/postconnect/services_start.sh:

/etc/init.d/distccd start

Last thing you must do is adding to /etc/distcc/hosts your servers.

(*) If you like DropBox and you are going to try, please use this link for creating account. Both of us will get extra 250GB of the storage space.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s