DAVROMANIAK
Le site de Cyril "Davromaniak" Lavier, sysadmin ascendant geek
Well I'm mad for crack!

Keyword - planet.ubuntu.com

Fil des billets - Fil des commentaires

samedi 28 décembre 2013

Zimbra mailbox backup script

Hello.

Over the last 2 weeks, I tried a Zimbra Collaboration Suite (Open Source Edition). And before going into production mode, I wanted to perform some mailbox backup/restore tests.

After some searches, I found this page on the Zimbra wiki which contains some backup/restore script. I downloaded the script written by Richardson Lima called "zimbraBackupAllAccounts.sh", then I started adapting the script to my needs.

As I finished my switch to Zimbra, I added this script in the prebackup script on my BackupPC server.

And I thought I could share this modified script, so I created a github repository and gave him a nice name. It's now called zimbashckup (for Zimbra Bash Backup).

It operates in a non-interactive way (to be launched in the cron task or by a backup system) and can perform per folder backups. Which is useful when we only want to restore a folder instead of the whole mailbox.

This script can backup the full mailbox (Briefcase contents, address book, calendar) and even the filters exported to the sieve format (thanks to Hackman61 for the idea).

However, when an object is share, only the original owner version is backed up (it's a zmmailbox limit).

For the remaining features, I think the README file is readable enough :).

The GIT repository is here : https://github.com/davromaniak/zimbashckup

Thanks !!

mardi 27 août 2013

Imapsync

Hello.

A quick article about imapsync, a great tool when we need to move emails from server to another.

This tool has switched license and it costs 50€ now. The developer asked the Debian project to remove imapsync package from its repository. Imapsync is not under a personal license (created by the developer) which permit to do whatever you want with the source code only if you bought it.

So somebody created a github repository which contains the last version.

To install it under Debian Wheezy, you need git and some perl libraries. This aptitude command will help you installing all the prerequisites (you need to launch it as root)

  • aptitude install makepasswd libmail-imapclient-perl libterm-readkey-perl git

Then, the installation :

  • As a non-privileged user :
    • git clone https://github.com/imapsync/imapsync.git
    • cd imapsync
    • mkdir dist
    • > ./dist/path_1.558.txt
  • As root or with sudo :
    • make install

The use is simple : : imapsync --host1 oldmx.domain.tld --user1 jojothefrite@domaine.tld --password1 verysecurepasswordwithalotofdifferentchartypes --host2 newmx.domaine.tld --user2 jojothefrite@domaine.tld --password2 verysecurepasswordwithalotofdifferentchartypes

On my servers, it ran at about 5 mail by second, and 8 hours were need to transfer 125000 emails. And during this operation, you can receive and read new mails on your new mailbox.

Thanks.

vendredi 16 août 2013

Use XZ with logrotate

Hello.

XZ is a compression format which I like to use, mostly for its good results when compressing text files. And I was seeking to use it with logrotate.

And it's done now with these 3 lines added in /etc/logrotate.conf :

compresscmd /usr/bin/xz
compressext .xz
uncompresscmd /usr/bin/unxz

The default compress argument is "-9", which is working with xz, so no need to edit.

In my servers, logrotate only rotates logfiles in /var/log, so this find command uncompressed GZIP files and compressed them in XZ format :

find /var/log -name "*.gz" | while read filename; do gunzip $filename && xz -vz9 ${filename%.gz}; done

On Debian, the XZ format tools are available in the xz-utils package.

That's all folks !

vendredi 21 septembre 2012

Incorporate a file in another with AWK

Hello.

Recently, I had to replace a string of characters by a text file content.

Example : I want to replace "##__youpi_tralala__##" by the content of the text file called "youpi_tralala".

I tried with sed, but it wasn't reliable, as sed tried to execute the text file content.

So I worked with my best friend AWK.

Here is the result, the script called include_file.awk :

{
	str=$0
	where=match(str,regexp)
	if (where) {
		while (("cat "file) | getline tstr > 0) {
			if (newstr == "") {
				newstr=tstr
			} else {
				newstr=newstr"\n"tstr
			}
		}
		close (tstr)
		sub(regexp,newstr,str)
	}
	print str
}

It's a beautiful AWK script, which can be launched like this : awk -v regexp="awesome regexp" -v file=/path/to/the/included/file -f include_file.awk text_file_with_the_regex

I use it in LaTeX, to debug some includes, to see if the file content or the include which is the origin of the issue.

Here is an example :

awk -v regexp="^%%__part1__%%$" -v file=includes/part1.tex -f include_file.awk document.tex | pdflatex -jobname=document

Here, the string placed in the document.tex file is "%%__part1__%%".

That's all folks :D.

dimanche 1 juillet 2012

Debian Wheezy has been frozen

And it's the first time I actually live a Debian freeze.

Lire la suite...

lundi 2 avril 2012

DDB lenny-backports discontinued

Hello everybody.

As Debian discontinued the support of the 5.0 "Lenny" release, I decided to do so in the DDB.

I will continue to maintain the squeeze-backports, and lenny-backports will be moved into another repository at http://archives.ddb.davromaniak.eu which will be created in the next hours or days.

Thanks.

dimanche 25 mars 2012

My pbuilderrc using qemu-debootstrap

Hello everybody.

First, I'm sorry for this late translation. I had a quite bad and busy period of time, so I was unable to take time to perform this translation.

As you are already aware of, Debian packaging takes a lot of my time, as well on the practical side as on the theoretical part of it.

Recently, I became co-maintainer of audacious and audacious-plugins Debian packages. Also, in few weeks, I will become co-maintainer of the NGINX package in Debian (it happened on March 16th).

I'm also working on making the NGINX maintaining team work as an actual team. I try to achieve this by scheduling milestones and managing Wheezy goals for the packaging. I also try to make all co-maintainers speaking when an important decision need to be taken (new upstream release to upload, module adding/removal, etc...). By far, I think it's working and we are finally working as a team.

But without any building/development environment, Debian packaging would be like playing Russian-roulette with a fully loaded machine-gun.

Why ?

It's simple.

Before any source package upload in Debian, building a package in a clean environment is highly recommended. You need to be sure of the build-deps and binary dependencies. Building a package on your own PC doesn't ensure you the well working of a package. Because like every user, you have packages already installed on machine, which some of them may be undeclared build-deps. So the package built well on your machine, but won't build in a clean environment.

There are a lot of ways to have a clean environment, here is a list :

  • Virtualized machines, which will be reinstalled after every build try (perfect for patient people with a lot of spare time).
  • The same as above, but with physical machines (sponsored by the electrical company because of the power consumption and useful during winter, because you can shutdown you house heating system).
  • Find a intern for reinstalling machines while you are working on source packages (expensive and may cause antidepressant drugs abuse by the intern).
  • Handmade chroots with debootstrap (We barely have a long-lasting solution, because we need to recreate chroots after every build. Hmmm, not so handy).
  • Use Debian powered tools, which create clean chroots (compressed in TGZ) and unpack them to build packages in it (pbuilder and its friend debootstrap are doing this very well).

As you guessed, the following will be about the pbuilder tool.

A pbuilderrc file can be used to configure pbuilder. Often placed in $HOME/.pbuilderrc, this i a shell script which is launched by pbuilder at the beginning of its work.

Like I already explained in the posts The ultimate Debian/Ubuntu package building system and Pbuilder and tmpfs can be friends, the .pbuilderrc file helps setting all aspects of these chroots creation and use.

After a lot of work on this configuration, I wrote a quite complete pbuilderrc now, supporting the use of qemu-debootstrap, which is an evolved debootstrap. It uses QEMU to virtualize the chroot hardware architecture. So now we can build ARM packages on a I386 or AMD64 machine without using a cross-compilation.

Before using this pbuilderrc, you need to install the qemu-debootstrap package. Available on squeeze-backports, wheezy and sid for Debian, and starting natty for Ubuntu.

I also recommend the reading of the post The ultimate Debian/Ubuntu package building system to have all needed tools to use pbuilder with this kind of configuration.

The pbuilderrc is available on my GIT repository named "scriptomaniak" available on github.

Download it in your /root/ folder, under the .pbuilderrc name.

Don't hesitate to comment this work and submit patches/pull requests.

That's all folks !

vendredi 30 décembre 2011

Thruk + NGINX + authentication

Sometimes, when one line is missing, the whole system seems broken.

Lire la suite...

mardi 29 novembre 2011

Debian/Ubuntu package building using virtual machines

Hi.

As you know, I often work on Debian package building.

For some months now, the DDB provides packages for others architecures (powerpc and mips).

For the basic architectures (amd64, i386 and armel), I have physical machines for building, but for powerpc and mips, I use QEMU virtual machines.

As it took some time to create them, I decided to distribute them.

Every virtual machine is pre-configured using this article posted on March 2011, and pbuilders are created for Debian Lenny, Squeeze, Wheezy, Sid and Ubuntu Hardy, Karmic, Lucid, Maverick, Natty, Oneiric, Precise. Also, a README file containing all information for launching machines.

So here they are :

here are the MD5 sums, to check if the download went well :

  • AMD64 : 3e55dd81c20c2a6f67a43a97d7eab986
  • I386 : 75c10975e65b1f816bc0a0711e9d12f3
  • MIPS : 9192d16ec83d31b3de0b381179e6eeef
  • MIPSEL : e01b1b1cdae64786d56b278c97e90c65
  • POWERPC : 63d0d2ceac849de0b0c1ad51eea4ee44

To conclude, I thank Aurélien Jarno (aurel32), because he distributes QEMU virtual machines for most architectures, which served as base for these building machines.

Thanks.

samedi 6 août 2011

Backports and some moments of joy

Good evening everybody.

A month after proposing the backport to Debian Squeeze of audacious 2.4.4, when I was starting to think it won't be accepted in the backports, I received 3 mails from Kilian Krause, who wants to sponsor the packages (audacious, audacious-plugins and libmowgli).

I was very surprised and so happy because this backport is very different from the nginx backport I made some months ago (and I'm working with the maintainers to keep the backport up to date, and to synchronise work).

As some people know, I use nginx as a daily base, it's installed on the server which serves this website, I used to make custom packages for my work (with adding some third party modules), so backporting nginx and contributing to it was barely normal.

But for audacious, it's completely different. Mid June, on the backport mailing list, somebody (rent0n) requested a backport of audacious, I simply answered to the mail saying I've added the maintainers in CC and that I think we might wait until the 2.5.1 gets in unstable, to make the backport. Few days later, no answer from the maitainers, so I wanted to try to make the backport, in case it's not possible. So it was possible, but, as a first try, I didn't wanted to change the needed versions of the build dependencies, so it was a raw backport, but I had to recompile barely the whole world (it was about 10-15 packages). 2 days later, I cleaned the package, so I just needed to backport the libmowgli. Then, the requester tested it, was happy, and for me, that was all, I just asked for anybody to review it and upload. Actually, I was not thinking about official inclusion, and so I didn't upload the package to mentors.debian.net. So then, I leaved for 10 days, to go on holidays, and I forgot about the package, but not rent0n, which was very enthousiast, and sent me a mail about upload the packages to mentors, so I did it, because I thought maybe somebody will be interested about this package.

One month later, the package was sponsored, and now it's waiting the ultimate validation.

So it's the second time I get a backport sponsored and uploaded. Strangely, it feels the same as if it was the first time. Some excitation about the next steps, then I will regularily check the buildd status, and I will be divised into 2 feelings. The first being the relief, this work is done, so all the moments passed working on it are just memories. The second being the joy, as people think my work is good enough to be included in the official backport repository.

Also, for every package I backport, I subscribe to the qa page, in order to receive notifications about bugs, updates, and everything related to the package.

Now, some news about the DDB.

Few days ago, I decided to remove all references to the chromium backport, as I was unable to maintain it, and it was not clean at all. Because of this, I may create a "dev" component for the DDB, but for the moment, it's not decided.

Also, I updated the nginx backport to the version 1.1.0, it's also uploaded to mentors, but will have to wait at least a week, to have it uploaded.

That's all folks !

- page 1 de 3