DAVROMANIAK
Le site de Cyril "Davromaniak" Lavier, sysadmin ascendant geek
If you're watching this, it means I'm dead now.

Keyword - planet.ubuntu.com

Fil des billets

mardi 12 juillet 2011

Pbuilder and tmpfs can be friends

A way to speed up Debian/Ubuntu package builds using pbuilder.

Lire la suite...

mardi 21 juin 2011

Some news

Hi.

After few months of inactivity, there's some news.

So you might see it now, I wrote a new template for this website, less colored, more readable.

Then, the Debian/Ubuntu repositories I'm maintaining (which are now in the main menu)

The OWFS repository is still here, and updated regularly, with the latest release (2.8p9 released at the end of May). I am peacefully preparing the add of Ubuntu Oneiric, which will occur in the next weeks.

The newcomer, DDB (for Davromaniak's Debian Backports) is a repository containing all the backports I'm working on. The goal is to work on proposing them for inclusion in the official backport repository. It's a stable repository, but as it contains backports, it may be risky to use it.

For now, backports for nginx 1.0.4, chromium 11.0.696.71 and audacious 2.4.4 are available, all along their build dependencies and binary dependencies.

Also, nginx 1.0.4 was recently accepted for inclusion in the official backports, we just need to wait until it's included. So the package available in the DDB repository is the same as the official one, only the version changes (I use "davrobpo60+1" instead of "bpo60+1" to mark the difference between official packages and DDB's packages).

For chromium, it was a hard to perform backport (between 20 to 30 hours of work to make it work), but it's not so clean, as I prefered to backport the binutils package in order to have a backported package as close as the official package.

For audacious, it works, but it's still a work in progress, mostly on the amount of dependencies to backport.

I try to make my packages available for the majority of hardware architectures, but it all depends on the package size. For now, only OWFS and nginx are available under 5 hardware architectures (amd64, armel, i386, mips and powerpc), because they don't take too long to compile. For audaicous, the amount of dependencies make it difficult to compile under other architectures than amd64 and i386. For chromium, it's only compilable in amd64, i386 and armel, but it's compilation lasts for 25 minutes on my building machine (8 cores, 16GB of RAM), my armel machine being the complete opposite in term of power, building could last for days.

That's all folks.

lundi 7 mars 2011

OWFS : Debian and Ubuntu packages

Hello.

OWFS is a free software suite, licensed under GPL. It's an easy and reliable way to use the 1-wire system from Maxim, which controls a widespread range of devices (sensors for temperature, moisture, luminosity, door openning/closing, and many more, even LCD screens).

Today, I sent a mail to the OWFS mailing list, which makes the release official, here is the mail :

Hi everybody.

In the company which I'm working for, we use 1-wire for more than 18 months.

We use it in our server rooms, to monitor the temperature in it (and send alerts when it's too hot) and we plan to monitor temperatures in the offices.

As we use Debian on our servers, I worked to make packages for making the installation easier.

At the beginning, packages where a little rough, but we decided to give back what the owfs community gave us, and to help this project by publicly releasing these packages.

So I cleaned the packages, and now, by this mail, I'm releasing them.

Packages are available the following versions of Debian and Ubuntu :
- Debian 5.0 "Lenny" (oldstable)
- Debian 6.0 "Squeeze" (stable)
- Debian 7.0 "Wheezy" (testing)
- Debian Sid (unstable)

- Ubuntu 8.04 "Hardy" (LTS)
- Ubuntu 9.04 "Jaunty"
- Ubuntu 9.10 "Karmic"
- Ubuntu 10.04 "Lucid" (LTS)
- Ubuntu 10.10 "Maverick"
- Ubuntu 11.04 "Natty" (testing)

So now, some explanation about the packages.

For the moment, I do not package owperl, owmon, owtap, owphp, owcapi, owpython, owtcl , owftpd and owhttpd.

However, packages for owftpd and owhttpd might come later, maybe with a 2.8p8 depending on the time I can take on packaging.

For the other packages, they will come when I will have enough time to do a clean packaging.

Here is the packages list (available in versions 2.8p6 and 2.8p7 if not indicated) :
- owfs-common : contains the /etc/owfs.conf file, and the manpage
- owfs-server : contains the /usr/sbin/owserver binary (originally in /usr/bin, but I moved it in the package, because of the Debian Policy requirements), the manpage and the init.d script
- owfs (only 2.8p7) : contains the /usr/sbin/owfs binary, to use with fuse (originally in /usr/bin, but I moved it in the package, because of the Debian Policy requirements), the manpage and the init.d script
- owfs-client : contains the client tools (owdir, owpresent, owread, owwrite and owread), with the manpages associated.
- libow-2.8-7 (only 2.8p7) : contains the libow for version 2.8p7
- libow-2.8-6 (only 2.8p6) : contains the libow for version 2.8p6
- owfs-dev : contains the file /usr/include/owfs_config.h
- owfs-doc : contains all the remaining manpages
- owfs-dbg : contains the debugging symbols

So if you need the server and the clients, install owfs-server and owfs-client (it will install owfs-common and libow-2.8-6 or libow-2.8-7 depending on which version you are installing). If you need the fuse filesystem, only install owfs (it will also install the dependencies needed).

All information you need is on the repository website (updated after every package update) : http://owfs.davromaniak.eu/

I would like to thank Paul Alfille for his work on OWFS, and all the other contributors for their respective work.

Also, I had to patch the sources for having full compliance to the Debian Policy, Paul, I will mail you later today or tomorrow to share these patches.

If you detect any problem, feel free to mail me, or mail the mailing list.

Thanks. 

The repository is available here : http://owfs.davromaniak.eu/

By the way, I may work on including these packages to Debian/Ubuntu official repositories. I will keep you informed on the evolution.

Bye.

jeudi 3 mars 2011

The ultimate Debian/Ubuntu package building system

Here is a "small" article which crosses the border between personal stuff and work but may be useful for many people.

In my current job, making Debian and Ubuntu packages takes me a lot of time.

At the beginning, I used to work on 2 virtual machine with a small pbuilderrc file, but I had the opportunity to work on a more powerful server (4 cores and 4GB of RAM), and I thought it would be useful to merge both machines and finaly have only one building machine.

My building system runs on a Debian Sid, even if I was a long time Ubuntu user, now I use Debian (including on my workstation).

Why choosing Debian Sid ?. Only to be able to install the latest versions of build tools, especially for debootstrap.

It might also work on Debian stable, and Ubuntu (but some points are different)

So let's begin.

All the commands preceded by a "#" need to be run with the root user, commands preceded by a "$" need to be run with a normal user, without privileges.

Notice : my server is running 64 bits OS, so I can compile 32 and 64 bits packages.

First, we need to install sudo, because as time goes by, I learned the hard way that it's dangerous to work on packages with the user root because it's not necessary and also, after some hours of keyboard typing and with tiredness by, we can act like asses and transforming into a "Rage Guy" for the next 5 minutes (or hours).

# aptitude install sudo

Then, we need to change the settings of sudo depending of what we will do with pbuilder.

# visudo

Here is my sudo configuration file :

#
# This file MUST be edited with the 'visudo' command as root.
#
# Please consider adding local content in /etc/sudoers.d/ instead of
# directly modifying this file.
#
# See the man page for details on how to write a sudoers file.
#
Defaults	env_reset,env_keep="DIST ARCH CONCURRENCY_LEVEL"

# Host alias specification

# User alias specification

# Cmnd alias specification

# User privilege specification
root	ALL=(ALL:ALL) ALL

# Allow members of group sudo to execute any command
%sudo	ALL=(ALL:ALL) ALL

#includedir /etc/sudoers.d

# Members of the admin group may gain root privileges
%adm ALL=(ALL:ALL) ALL

Note the "Defaults" line, which contains instructions for keeping the values of variables DIST, ARCH and CONCURRENCY_LEVEL. Keep those variable name in head, we will use them later.

Now, we will work on the real part of compilation, we will install the tools needed for compilation.

# aptitude install debhelper build-essential dpkg-dev pbuilder devscripts debootstrap

Among those packages, the most remarkable are "pbuilder" and "debootstrap"

  • pbuilder allows us to build our packages in a chrooted environment which contains the linux distribution of our choice.
  • debootstrap is called by pbuilder to create this chrooted environment

The lintian package is also useful, but optional, it tests the Debian Policy conformity of packages.

Now all these tools are installed, we need to configure the pbuilder, with the pbuilderrc file. Since I don't like to edit the default configuration file (/etc/pbuilderrc), I choose to create the /root/.pbuilderrc file, which here is mine.

# Codenames for Debian suites according to their alias. Update these when
# needed.
UNSTABLE_CODENAME="sid"
TESTING_CODENAME="wheezy"
STABLE_CODENAME="squeeze"
STABLE_BACKPORTS_SUITE="$STABLE_CODENAME-backports"
OLD_STABLE_CODENAME="lenny"
OLD_STABLE_BACKPORTS_SUITE="$OLD_STABLE_CODENAME-backports"

# List of Debian suites.
DEBIAN_SUITES=($UNSTABLE_CODENAME $TESTING_CODENAME $STABLE_CODENAME $OLD_STABLE_CODENAME "unstable" "testing" "stable" "experimental")

# List of Ubuntu suites. Update these when needed.
UBUNTU_SUITES=("natty" "maverick" "lucid" "karmic" "jaunty" "intrepid" "hardy" "gutsy")

# Mirrors to use. Update these to your preferred mirror.
DEBIAN_MIRROR="ftp.fr.debian.org/"
UBUNTU_MIRROR="fr.archive.ubuntu.com"

# Use old-releases mirrors for EOL versions
if [ "${DIST}" == "gutsy" ]; then
	UBUNTU_MIRROR="old-releases.ubuntu.com"
fi
if [ "${DIST}" == "intrepid" ]; then
	UBUNTU_MIRROR="old-releases.ubuntu.com"
fi
# Optionally use the changelog of a package to determine the suite to use if
# none set.
if [ -z "${DIST}" ] && [ -r "debian/changelog" ]; then
    DIST=$(dpkg-parsechangelog | awk '/^Distribution: / {print $2}')
    # Use the unstable suite for Debian experimental packages.
    if [ "${DIST}" == "experimental" ]; then
        DIST="unstable"
    fi
fi

# Optionally set a default distribution if none is used. Note that you can set
# your own default (i.e. ${DIST:="unstable"}).
: ${DIST:="$(lsb_release --short --codename)"}

# Optionally set the architecture to the host architecture if none set. Note
# that you can set your own default (i.e. ${ARCH:="i386"}).
: ${ARCH:="$(dpkg --print-architecture)"}

NAME="$DIST"
if [ -n "${ARCH}" ]; then
    NAME="$NAME-$ARCH"
    DEBOOTSTRAPOPTS=("--arch" "$ARCH" "${DEBOOTSTRAPOPTS[@]}")
fi
BASETGZ="/var/cache/pbuilder/$NAME-base.tgz"
DISTRIBUTION="$DIST"
BUILDRESULT="/var/cache/pbuilder/$NAME/result/"
APTCACHE="/var/cache/pbuilder/$NAME/aptcache/"
BUILDPLACE="/var/cache/pbuilder/build/"

if $(echo ${DEBIAN_SUITES[@]} | grep -q $DIST); then
    # Debian configuration
    MIRRORSITE="http://$DEBIAN_MIRROR/debian/"
    COMPONENTS="main contrib non-free"
    if $(echo "$STABLE_CODENAME stable" | grep -q $DIST); then
        EXTRAPACKAGES="$EXTRAPACKAGES debian-backports-keyring"
        OTHERMIRROR="$OTHERMIRROR | deb http://backports.debian.org/debian-backports $STABLE_BACKPORTS_SUITE $COMPONENTS"
    elif $(echo "$OLD_STABLE_CODENAME stable" | grep -q $DIST); then
        EXTRAPACKAGES="$EXTRAPACKAGES debian-backports-keyring"
        OTHERMIRROR="$OTHERMIRROR | deb http://backports.debian.org/debian-backports $OLD_STABLE_BACKPORTS_SUITE $COMPONENTS"
    elif $(echo "unstable" | grep -q $DIST); then
	DIST="$UNSTABLE_CODENAME"
	OTHERMIRROR="$OTHERMIRROR | deb http://ftp.fr.debian.org/debian/ experimental main"
    fi
elif $(echo ${UBUNTU_SUITES[@]} | grep -q $DIST); then
    # Ubuntu configuration
    MIRRORSITE="http://$UBUNTU_MIRROR/ubuntu/"
    COMPONENTS="main restricted universe multiverse"
     v=0
     n=0
     for i in ${DEBOOTSTRAPOPTS[@]}; do
	if [ $v -ne 0 ]; then
		DEBOOTSTRAPOPTS[$n]="/usr/share/keyrings/ubuntu-archive-keyring.gpg"
	fi
	if [ $i == "--keyring" ]; then
		v=1;
	fi
	n=$((n+1))
     done
else
    echo "Unknown distribution: $DIST"
    exit 1
fi

To make this installation easier, you can download it from here

However, before jumping in the pbuilder creation of Ubuntu versions, there's a little thing to do if you are using Debian.

Debootstrap packages from Debian and Ubuntu are different. 1 line differs in the file /usr/share/debootstrap/scripts/gutsy. This different can cause trouble, especially under Ubuntu 11.04 "Natty".

At the line 72 of the file /usr/share/debootstrap/scripts/gutsy, add the folllowing line :

ln -nsf . "$TARGET/var/lib/dpkg/info/$ARCH"

Now, we are finally ready.

With this handsome .pbuilderrc, you will just have to type the following ligne to create a 64 bits Debian Squeeze pbuilder :

$ DIST=squeeze ARCH=amd64 sudo pbuilder create

Or a 32 bits Ubuntu Natty pbuilder :

$ DIST=natty ARCH=i386 sudo pbuilder create

But now, you are wondering "Why did he talked about the CONCURRENCY_LEVEL variable ?".

Here is the answer.

The variable CONCURRENCY_LEVEL is used to run several processes for only one compilation, in other words, for person who use and know make, this variable acts like the "-j" option.

To compile a source package with 4 processes under a 64 bits Debian Squeeze :

$ CONCURRENCY_LEVEL=4 DIST=squeeze ARCH=amd64 sudo pbuilder build paquet.dsc

To compile a source package with 2 processes under a 32 bits Ubuntu Natty 32 bits :

$ CONCURRENCY_LEVEL=2 DIST=natty ARCH=i386 sudo pbuilder build paquet.dsc

Here we are, the build system is up and running.

But knowing that I like to have my own useful functions which help me to automate some tasks, I create a .scripts folder in the home directory of my unprivileged user, and a script calledpbuilder_utils which here is the content.

function update_pbuilder() {
	nprocs=$(grep -cE "^processor" /proc/cpuinfo)
	dists=$@
	for i in $dists; do
		CONCURRENCY_LEVEL=$nprocs DIST=$i ARCH=i386 sudo pbuilder update
		CONCURRENCY_LEVEL=$nprocs DIST=$i ARCH=amd64 sudo pbuilder update
	done
}
function create_pbuilder() {
	nprocs=$(grep -cE "^processor" /proc/cpuinfo)
	dists=$@
	for i in $dists; do
		CONCURRENCY_LEVEL=$nprocs DIST=$i ARCH=i386 sudo pbuilder create
		CONCURRENCY_LEVEL=$nprocs DIST=$i ARCH=amd64 sudo pbuilder create
	done
}
function clean_pbuilder() {
	nprocs=$(grep -cE "^processor" /proc/cpuinfo)
	dists=$@
	for i in $dists; do
		CONCURRENCY_LEVEL=$nprocs DIST=$i ARCH=i386 sudo pbuilder --clean
		CONCURRENCY_LEVEL=$nprocs DIST=$i ARCH=amd64 sudo pbuilder --clean
	done
}
function build_pbuilder() {
	nprocs=$(grep -cE "^processor" /proc/cpuinfo)
	dsc=$1
	shift
	dists=$@
	for i in $dists; do
		CONCURRENCY_LEVEL=$nprocs DIST=$i ARCH=i386 sudo pbuilder build $dsc
		CONCURRENCY_LEVEL=$nprocs DIST=$i ARCH=amd64 sudo pbuilder build $dsc
	done
}

In the .bashrc file I added the following line : source ~/.scripts/pbuilder_utils.

From here, it might be a bit raw, but here are explanations :

  • update_pbuilder updates given pbuilders in both architectures, example : update_pbuilder squeeze natty
  • create_pbuilder creates given pbuilders in both architectures, example : create_pbuilder squeeze natty
  • clean_pbuilder cleans the given pbuilders in both architectures, example : clean_pbuilder squeeze natty
  • build_pbuilder compiles a source package for distributions given in both architectures, example : build_pbuilder paquet.dsc natty lenny squeeze

That's nearly all.

For the pbuilderrc file given here, I took as base the one given at the following address : https://wiki.ubuntu.com/PbuilderHowto#Multiple%20pbuilders

For the remaining things, it's hair pulling, some hacking and multiples missed tries.

I hope you enjoy reading.

mercredi 24 mars 2010

SFS Update 0.3pre : a sneak peek of the 0.3

Hi.

Few weeks ago, SFS Update 0.2 was released, with some promises for the 0.3.

In order to contain your impatience and to show what the 0.3 will look like, I decided to release a preliminary version of the 0.3, which supports a part of the forum engines listed on the last article.dernier billet.

Here is a list of the forum engines which are supported by the 0.3pre version :

  • PHPBB2
  • PHPBB3
  • Fluxbb 1.2 (experimental)
  • Beehive 0.9 (experimental)
  • FUDForum 3 (experimental)
  • IceBB (experimental)
  • MyBB (experimental)

I had to dismiss the idea of supporting these forum engines :

  • bbPress : No ban mangement included
  • kusaba : Project not developed anymore
  • MercuryBoard : PProject not developed anymore
  • MiniBB support : Ban management only contains IP ban, no username or email address ban management at this time

SFS Update 0.3 will support the following forum engines (depending on the feasibility) :

  • NextBBS
  • NinkoBB
  • Quicksilver Forums
  • UseBB
  • Vanilla
  • XMB

0.3 release will also mark the opening of a public GIT for this project, so that anybody will be able to improve and contribute to SFS Update

Here are the links for download : sfs_update-0.3pre.tar.bz2
sfs_update-0.3pre.tar.gz

If you notice any bug or defect, feel free to contact me.

Thanks.

samedi 6 mars 2010

SFS Update comes back with a 0.2 version

And the 0.3 is announced.

Lire la suite...

lundi 22 février 2010

SFS Update 0.1.4

A new version with plenty of new developments.

Lire la suite...

lundi 15 février 2010

SFS Update 0.1.3

Two new features

Lire la suite...

lundi 10 août 2009

SFS Update 0.1.1

A little upgrade

Lire la suite...

samedi 8 août 2009

SFS Update

A tool for phpbb forum administrators

Lire la suite...

- page 2 de 3 -