Wenn man mit der Deutschen Bahn fährt, hat ein Zug ja schonmal ein wenig Verspätung. Ich jammere da normalerweise relativ wenig drüber. Auch dieses Mal gibt es nichts zu jammern.

Heute gibt's eine epische Ansage vom Zugführer:

Guten Tag, sehr geehrte Fahrgäste. Sie befinden sich im RE11 nach Mönchengladbach. Unser Zug hat eine Verspätung von vier Minuten...

In dem Moment holt er noch einmal tief Luft und haut folgendes raus:

Grund dafür ist ein blonder Jüngling, der den Zug doch tatsächlich mit einem Taxi verwechselt hat.

Err. Wut? :-)

(Ja, heute ist erster Mai.)

...erm. "Benjamin Franklin?"

I have to admit, that I was fairly late to realise how awesome xkcd really is. And it is. That means that I don't know every last clever strip that Randall did over the years.

Given my profession, this one is spectacular:

Hug your friendly neighbourhood electrical engineer every once in a while! :-D

Es gibt ja jede Menge Programmiersprachen auf der JavaVM. Die beiden Dinge (also die VM und die Sprache) gleichzusetzen ist so ziemlich das Unzutreffenste, das man annehmen kann.

Es gibt Portierungen von Sprachen wie Ruby, Python, Tcl (JRuby, Jython, Jacl) und vielen anderen. Aber es gibt auch eine Reihe von Sprachen, die speziell für die JVM ausgelegt worden sind.

Als erstes fällt einem dabei vielleicht Groovy ein. Wenn man typische Skriptaufgaben auf der JVM lösen möchte, ist das vielleicht sogar das Mittel der Wahl. Besonders, wenn man Erfahrung mit Java hat, denn Java-Code ist in den allermeisten Fällen syntaktisch korrekter Groovy-Code.

Dann ist da Scala. Und Scala ist im Moment "everybody's darling", so scheint es mir. Es handelt sich dabei um eine Sprache, die objektorientierte und funktionale Ansätze zu vereinen versucht.

Zu guter Letzt ist da Clojure, was im Kern ein Lisp ist. Das heißt, die Sprache hat eine extrem einfache Syntax. Clojure hat zudem hohe Interoperabilität mit Java (wenn man möchte - oder muß). Die Sprache implementiert Software Transactional Memory als Teil der Sprache, was Clojure zum Einsatz in Programmen mit simultanen Abläufen prädestiniert.

Als echtes Lisp hat Clojure natürlich eine REPL (in emacs steht sogar slime als Interaktionsmodus zur Verfügung - Common-Lisper werden nun frohlocken...). Und natürlich hat die Sprache ein Makrosystem. Keine "hygenic macros", wie in Scheme; eher klassische Lisp Makros, wie in Common Lisp. Allerdings hat Clojures System ein paar Tricks eingebaut, die unbeabsichtigte Namenskonflikte umschiffen sollten. (Ein Makrosystem ist kein zu unterschätzendes Argument für eine Sprache. Wenn man nur die Makros aus C kennt, hat man vielleicht Angst vor dem Konzept. Richtige Makrosysteme, wie sie die verschiedenen Lisps implementieren sind dagegen unglaublich mächtige Werkzeuge in einer Sprache. Sie machen Lisps - und damit Clojure - zu programmierbaren Programmiersprachen. Wer die Möglichkeiten davon noch nicht kennt verpaßt etwas. Wirklich.)

Scala und Clojure stehen beide schon lange auf der Liste der Sprachen, die ich mir genauer ansehen wollte. Und ich denke, man kann schon heraushören, welche der beiden Sprachen mich mehr beeindruckt bisher. Steve Yegge geht im Vorwort des Buches "The Joy of Clojure" sogar so weit zu sagen: "[...]Clojure feels like a minor miracle. It's an astoundingly high-quality language, sure - in fact, I'm beginning to think it's the best I've ever seen[...]".

Der Entwickler von Clojure (Rich Hickey) hat vor einiger Zeit längere Einführungsvorträge in die Sprache gehalten. Einmal für Leute die aus der Java Ecke kommen und ein anderes Mal für Leute mit Lisp Hintergrund. Hier sind Links:

• Clojure für Javaprogrammierer (Teil 1, Teil 2)
• Clojure für Lispprogrammierer (Teil 1, Teil 2)

Lisper können sich die Java-Vorträge sparen. Java-Leute möchten sich vielleicht erst die Java-Version geben und danach zur Vertiefung noch einmal die Lisp-Version ansehen, da hier naturgemäß noch etwas tiefer auf den Lisp-Charakter von Clojure eingegangen wird.

Die Vorträge sind ziemlich lang, also viel Spaß beim Schauen!

I've recently answered a question about how the whole zsh loadable function business works. That's a useful addition to my last post about zsh's function based completion system, so if you're interested you should probably read both. ;)

Introduction

Zsh's completion system (compsys) is one of the most praised and one of the most complex parts of the shell. That's true for users, as well as developers.

People asking for gentle introductions hit the `#zsh' IRC channel on freenode every once in a while; often enough that we added a question about it to the wikifaq (question 15 at the time of writing). The complaint is usually, that `zshcompsys(1)' and `zshcomwid(1)' are dense and dry to read and that they lack a general overview of how the system works.

This is an attempt at such a general overview.

It's a set of functions is what it is

First of all, compsys is made up of functions. Shell functions, like this:

hello() {
    printf 'Hello world.\n'
}

That would create a function called `hello', that can be called just like any other command and it would print "Hello world." to your terminal. The completion system's functions are shell-functions, too. They get called automatically when you hit the tab key and they use a set of special commands, to interact with the shell's line editor, which will in turn present possible completions back to you.

But I've seen parts of the completion system, and that was just code in files, with a funny looking first line...

...I hear you scream.

You're missing the concept of zsh's function path (the `$fpath' array). It is a list of directories, that contain files which contain code for functions that are named just like the file the code came from. If you type "print -l $fpath" at the shell prompt, you should see a list of those directories. For ordinary functions, those directories are not scanned by default, you'll have to tell zsh which files to load code from and that is done by using the `autoload' utility. We could drop a file called `hello' into one of those directories with just one line:

printf 'Hello world.\n'

To have a function called `hello', as soon as we tell zsh to try to load the code from a file in `$fpath' as soon as it is referenced for the first time:

autoload -Uz hello

"-Uz"? Yes, that is "the right thing"[tm] almost always, so I won't discuss it here. See the manual for details.

Minor loadable functions wizardry

Sometimes, when completions get more complex, you need to define additional functions within such a function file. And you can just do that.

However, if you do, it makes sense to also explicitly define a function named like the file too and call that at the very end of the file.

What?

Here is an example (say this file is named "_foo"):

_bar() {
    echo "This is bar()"
}

_foo() {
    _bar
    echo "This is foo()"
}

_foo "$@"

So, there's a function `_foo' defined in a file that would create it while autoloading it. This is useful, to create helper functions within a function file when the function is called for the first time. You can also do initialisations you may need to do only once.

Among others, the _tmux and _git completions do that. The _tmux completion is an example that also does initialisations upon the first call.

The last line is important, it calls the newly defined function with the same arguments as the original function. That makes sure the actual functionality is called when the function runs for the first time, too.

But I don't have to autoload any of the completion functions manually, what gives?

Almost true again. You are calling one function from compsys. And that is `compinit'. It initialises the completion system. In particular, it looks at files in `$fpath', that start with an underscore ("_" - by convention, those are functions that provide completion code for something) and it finds out in which situation to use the code. This is where the weird-looking first line of a command's completion comes into play.

Let's take a look at the completion for the curses-based mail user agent `elm':

#compdef elm

_arguments -s \
  '::recipient:_email_addresses' \
  '-a[use the arrow pointer regardless]' \
  '-A+[attach file]:file attachment:_files' \
  '-c[check the given aliases only]:*:alias' \
  '-d+[set debug level]:debug level' \
  '-f+[specify mailbox to load]:mailbox: _mailboxes' \
  '-h[display help]' \
  '-i+[specify file to include in message]:include file:_files' \
  '-m[turn off menu, using more of the screen]' \
  '-s+[specify a subject]:subject:' \
  "-t[don't use termcap/terminfo ti/te entries]" \
  '-V[enable sendmail voyeur mode]' \
  '-v[display elm version]' \
  '-w[write .elm/elmrc]' \
  '-z[start only if new messages]'

The "#compdef elm" instructs compsys to use the code from that file when the word in command position for the current cursor position is "elm".

The rest of the code is just one call to `_arguments' (a powerful and complex helper function, that is able to deal with many MANY situations where standard getopt(3) as well as GNU-style long-options option handling is involved).

So what happens from the shell's startup to the point where you're doing this:

% elm -A <TAB>

Here is what:

• The shell starts and somewhere in your `.zshrc' file you are calling "autoload -Uz compinit; compinit". Now, compsys is online.
• While compinit is running it will find the `_elm' file in one directory of `$fpath'.
• Compinit will read the first like of that file, find out that it's a completion for a command called "elm" and make a note of that in a mapping for later.
• Then compinit will call `autoload' for the "_elm" file, so its code is loaded from the file when it is referenced for the first time.
• At the prompt, you typed "elm -A " and pressed the tab key, which will set the completion system in motion.
• Compsys recognises, that for the current cursor position the word in command position is "elm".
• It'll look up which completion function is in charge for that command in the mapping it made during startup. It'll find that that's "_elm".
• That function gets called and (when it is run for the first time) zsh automatically loads its code from the "_elm" file in `$fpath'.
• The `_arguments' function analyses the situation and figures out, that it needs to handle an argument to the "-A" option of the command, which it delegates to the `_files' function - as specified in the option's optspec.

The result is that you are being presented with a list of files, which is useful for elm's "-A" option.

And that is also all that's happening. Compsys is just a set of functions of which the code is contained in files, that by convention start with an underscore, in directories, that are listed in the shell's function search path and which are marked for automatic loading during the `compinit' run in your zshrc file.

What about zstyle then. I thought that was the completion system.

Wrong. Common misconception, though. Actually, `zstyle' is not even part of compsys.

It is a system for expressing and storing context-sensitive configuration information. That may sound scary, but it's really not. `zstyle' is just a configuration system.

Compsys happens to use it (but other sub-systems of the shell do, too - like `vcs_info'). That makes sense, because compsys is very context-sensitive. (Legend has it, that the zstyle configuration system was introduced to have a way to express the level of context sensitivity within a configuration system that is demanded by compsys. But that still does not make it part of compsys.)

Systems that use zstyle keep track of the currently active context and describe that context in a string (for example: ":completion::complete:ls::").

`zstyle' gets called like this:

% zstyle "context-pattern" style value

The "context-pattern" is matched against the active context string when the system looks up the value a style. For example:

% zstyle ':completion:*' verbose yes

The completion system always uses ":completion:" as a prefix when looking up styles (the context for compsys can become pretty complex, but that's beyond the scope of this article, see the manual for details), so ":completion:*" will always match when a style is looked up while compsys is running. And so, with the above command anytime the `verbose' style is looked up in compsys, zstyle will yield the value `yes'. Unless there is another setting for the `verbose' style that is more specific, than ":completion:*", like:

% zstyle ':completion:*:complete:ls:*:*' verbose no

The rule of thumb is: The longer the pattern, the more specific it is.

And that's all again.

A style setting with a very general context pattern can be used to set default values. And style settings with more specific patterns can be used to override that default where ever it is needed.

Here are a few example styles, that will give you a reasonable experience from a z-shell instance without any other completion setup:

# Load compsys and one of its fancy modules
zmodload zsh/complist
autoload -Uz compinit
compinit

# And set some styles...
zstyle ':completion:*' completer _complete _approximate
zstyle ':completion:*:descriptions' format "- %d -"
zstyle ':completion:*:corrections' format "- %d - (errors %e})"
zstyle ':completion:*:default' list-prompt '%S%M matches%s'
zstyle ':completion:*' group-name ''
zstyle ':completion:*:manuals' separate-sections true
zstyle ':completion:*:manuals.(^1*)' insert-sections true
zstyle ':completion:*' menu select
zstyle ':completion:*' verbose yes
zstyle ':completion:*' rehash yes
zstyle -e ':completion:*:approximate:*' max-errors \
          'reply=( $(( ($#PREFIX + $#SUFFIX) / 3 )) )'

For practise, find out what the last part of the context pattern in the examples is about (the "descriptions" and "corrections" part). And find the documentation for the styles used in the example (that would be "format", "list-prompt", "menu" etc). And find out what the `-e' option of `zstyle' does.

Putting it all together

I didn't give much hands-on code in this article. And that wasn't the point. The point was to give a very high level idea of how the completion system works. To give you an idea where to look and where to put things.

Say, you want to write a new completion function, for your own awesome program "foobar". You'd pick the completion function name "_foobar" (unless its already taken) and put the following into the first line:

#compdef foobar

And then you can do whatever gets the job done. There is always more than one way to skin a cat when it comes to completion code.

To get on the right track, you should think of a command that already has a completion and resembles the way your command takes its options (if you didn't invent an entirely new way to handle command line arguments, there should be plenty of example code).

There are very short completion functions (like _elm above - that's actually all its code); and there are massively large completion functions, like _git, _perforce or _tmux that jump through every conceivable hoop in order to provide the user with the most accurate and helpful completion candidates possible.

This marks the end of this short introduction. Compsys has much MUCH more to offer. So much more, that you could easily fill an entire book with discussions about its features and best practises. Way too much for a cute little blog article.

Further reading

Here is a list of suggested reading, if you really want get into zsh's compsys:

• If the zsh's loadable function system still confuses you, I've gone into more detail on that over at stackexchange. That should clear things up, I think.
• A chapter about compsys in "From Bash to Z-Shell". Two out of three authors are z-shell developers. And the chapter is the most gentle introduction into compsys that I know of. The book is not terribly expensive and contains a lot of interesting information about zsh. I'm pretty sure, that you'll find something useful in that book even if you're a seasoned zsh user. And the introductory compsys chapter really is helpful.
• The compsys chapter in the "user-friendly user guide" by pws.
• "man zshcompsys": This is the main reference manual for compsys. It's of interest for both users and developers.
• "man zshcompwid": Reference manual for many completion internals. Like documentation for the most basic compsys builtins and variables that are available in completion functions. It's only interesting for developers.
• Finally, read existing code. This cannot be overstated. If you're a beginner, you may want to read existing completions that handle completion for commands that handle command line arguments as similar as possible as the target command.

Okay, I think I'm done for now. Have fun and don't give up.

Festhalten: Merkel lobt "Transparenz" des Bundespräsidenten

Arename version 4.0 will hit Debian's sid mirrors soon (as 4.0-2).

Maximilian Gaß stepped down as arename's Debian maintainer. So, let me take the time to thank him for getting the program into Debian and maintaining the package for as long as he did. Thank you.

Seems like Debian's Perl Group has adopted the package and gregor herrmann uploaded an updated package earlier tonight. So it'll be available in the mirrors tomorrow-ish. Thanks, gregor.

So, das Jahr läuft ja wieder. Kurzer Rückblick auf den Kongress: Was mich interessiert habe ich im Wesentlichen gesehen. Besonders gefallen haben mir "Packets in Packets" von Travis Goodspeed, "The Science of Insecurity" von Meredith Patterson und "Cellular protocol stacks for Internet" von Harald Welte.

Unterhaltsam (und sehr zu empfehlen, wenn man von dem Thema noch nichts im Detail gehört hat) ist der Vortrag zum Staatstrojaner.

Etwas, das mir bisher durch die Lappen gegangen war, ist das Nick Farr Bullshit Bingo das kurz vor dem "Security Nightmares" talk ablief. Spassig.

Die Videos sind praktisch vollständig auf youtube, sowie in höherer Videoqualität auf den üblichen Quellen beim CCC, zu sehen.

Kurz erwähnt sei noch, daß das FEM dieses Jahr einen super Job bei den Streams gemacht hat. Daumen hoch, für alle die da mitgearbeitet haben. Auf die Leistung darf man auch ruhig mal stolz sein.

Heute noch schnell "Stephanus steinigen" und schon geht morgen der diesjährige Chaoscongress los.

Wie immer interessiert einen natürlich nicht alles. "A Brief History of Plutocracy" juckt mich zum Beispiel jedenfalls nicht besonders. Auf andere Veranstaltungen habe ich dagegen jede Menge Lust:

Tag 1 (27.12.2011)

• The Atari 2600 Video Computer System: The Ultimate Talk (12:45, Saal 3) Um. Yay! So eine Kiste war das erste Computerspielsystem für mich.
• Der Staatstrojaner (16:00, Saal 1)
• 802.11 Packets in Packets (18:30, Saal 1) Physical-Layer Attacken für den geneigten Nachrichtentechniker
• Black Ops of TCP/IP 2011 (23:00, Saal 1) Bin nicht so richtig sicher, was das hier werden soll. Aber den Vortrag gab's wohl schon auf der Defcon und der BlackHat; und Dan Kaminsky ist eigentlich imm für einen zumindest sehr unterhaltsamen Vortrag gut...

Tag 2 (28.12.2011)

• Reverse Engineering USB Devices (14:00, Saal 2) Wieder ein Hardwarevortrag für den Nachrichtentechniker in mir.
• The Science of Insecurity (16:00, Saal 1) Bei dem Ding bin ich mir wieder mal nicht sicher was das geben soll, aber die Beschreibung klingt interessant genug, daß ich hier definitiv genauer hinsehen werde.
• Hacking MFPs, Part 2 - Postscript: Um, you've been hacked (17:15, Saal 1) Allein die versprochene Einführung in die Sprache wird mich diesen Vortrag schauen lassen.
• Reverse Engineering a Qualcomm baseband (20:30, Saal 3) Wieder was Hardware-nahes. (Vielleicht sollte ich mir vorher nochmal den "All your Baseband are belong to us" Vortrag vom letzten Jahr ansehen...)
• Apple vs. Google Platforms (How you end up being the victim) (21:45, Saal 1)

Tag 3 (29.12.2011)

• TRESOR: Festplatten sicher verschlüsseln (14:30, Saal 2) Benutzt ein Debugging Register vom x86-64 um den Schlüssel abzulegen? Okay.
• Cellular protocol stacks for Internet (17:15, Saal 1) ...dieses Jahr ist eine Menge dabei, was in die Nachrichtentechnik fällt. I approve.
• Behind the scenes of a C64 demo (21:45, Saal 3) Demo-scene-lastiger Talk aufm C64. (Fabian, ließt du das hier? *g*)

Tag 4 (30.12.2011)

• bup: Git for backups (13:15, Saal 2) Eigentlich bin ich ja der Meinung, daß git dafür nicht direkt perfekt geeignet ist. Aber vielleicht werde ich ja eines Besseren Belehrt.

Tja, der letzte Tag ist etwas dünn besetzt für meinen Geschmack. Aber egal. Neben dem Genannten gibt's natürlich die Alle-Jahre-Wieder Nummern wie "Jahresrückblick", "FNORD Newsshow", "Hacker Jeopardy", "Security Nightmares" etc., die sich normalerweise auch lohnen.

Werde wohl nicht alles live schauen können, aber es gibt sicher wieder Aufnahmen von fast allen Veranstaltungen, die man sich dann auf langeren Zugfahrten geben kann.

Mein Kumpel Fabian weist mich auf etwas bizarres hin. Ich wette, daß ist nicht das erste Mal, daß das jemandem auffällt, aber es ist zu schön um es nicht zu verbloggen...

Wenn man nicht aus dem schönen Deutschland am Internet teilnimmt, dann kennt man das Problem vielleicht gar nicht. Schon bei den lieben Nachbarn in Östereich greifen die Maßnahmen, die hier zu Hause so vielen auf die Nerven gehen, nämlich nicht.

Die Situation ist immer die gleiche: Man hört von einem Künstler und will mal bei youtube reinhören, ob man in Geld in ein neues, oder auch altes, Werk investieren soll. Bei uns bekommt man da aber immer häufiger folgendes:

Das, nebenbei, auch in offiziellen Band- oder Labelkanälen.

Offenbar ist es schmerzhaft genug sich mit der Gesellschaft auseinander zu setzen, daß die Labels hingehen und sich ihr eigenes youtube zurecht schrauben. Was!? Nein, wirklich. Das Video, das mir youtube verwehren will darf ich bei universalmusic.com schauen so oft ich will:

• Godsmack - Cryin like a Bitch

Wer den Laden der für die lustigen youtube Vorenthalte zuständig ist noch nicht so richtig kennt, der überfliege mal den entsprechenden Wikipedia Artikel. Insbesondere den "Kritik" Abschnitt.

Und aus gegebenen Anlaß schaue ich mir nun eine Nummer der Band "Eure Mütter" an (die Mütter sind eh gut - man klicke ruhig mal weiter!). In diesem Sinne. Rinnjehaun.

Post Scriptum: Nein, ich kaufe keinen Godsmack Tonträger. Keine Sorge.

Ich mag den Original-Film ja sehr. Und diese Version ist auch großartig:

Evil Dead, in 60 Sekunden. Mit... Knete. - Oh yeah.

I just released version 4.0 of arename, your friendly neighbourhood audio file batch renaming application.

The highlights of this release are a new audio file handling backend, which allows for many more supported file formats, a new template parser which implements a much more powerful template syntax and an rewritten output system with clearer semantics and support for colourful output (which can be deactivated if so desired). In other words, the three most integral parts of the program have been entirely redone. This could not be done, without breaking some backward compatibility. This is why this release took so long and is a major version update.

I've gone into some detail about those before, so I won't do it again.

I've let arename v4.0 run across my 170GiB of audio files and it behaved exactly as expected. If you should run into any problems, then you'll be happy to hear that this is also the first release with an enabled bug tracker.

Here are a number of direct links to help you along:

• Version 4.0 release tarball
• UPGRADING document - required reading
• CHANGES, the user's changelog
• The git repository at github
• the bug tracker at github

If Maximilian finds the time, then - who knows - maybe this release will hit Debian and Ubuntu servers soon. They're currently stuck with version `3.1'.

[-Epilogue-]

I have already merged the latest work (the `extensions' branch) into the `master' branch of the repository. It implements extensions as Perl modules and is supposed to streamline the idea of minimising the changes to the core module `ARename.pm'. So, if you're interested in that, try the master branch.

It'll let you do the following in your `hooks' file to enable replacing spaces with underscores, for example:

use ARename::RemoveSpaces;
ARename::RemoveSpaces::register();