Useful font is useful

Posted Fri 11 Apr 2014 11:34:36 CEST Tags:

Previously I was using matplotlib. Things went fairly smoothly. The only awkward places were those where I tried to customise the looks of things a bit. Having to do that for each and every plot seems cumbersome. And I guess I'm not alone with that feeling, because matplotlib let's you provide a configuration file, that sets up things the way you'd like. You name the thing “matplotlibrc” in your current working directory to have different setups in different places. So I created a file that contains the following:

lines.linewidth  : 1.5
axes.grid        : True
axes.color_cycle : b, r, g, c, m, y, k
xtick.labelsize  : 8
ytick.labelsize  : 8
legend.fancybox  : True
legend.fontsize  : small

Then the plotting script reduces the Python code to only pieces that are pretty self-explanatory:

import numpy as np
import matplotlib.pyplot as pp

data = map(lambda x: [ x[0],
                       complex(x[1], x[2]),
                       complex(x[3], x[4]),
                       complex(x[5], x[6]),
                       complex(x[7], x[8]) ],
           np.loadtxt("wifi-antennas.s2p", skiprows=5))

def column(data, i):
    return [ row[i] for row in data ]

def lin2db20(data):
    return 20 * np.log10(np.abs(data))

xx = np.multiply(1/1.0e9, column(data, 0))
s11 = lin2db20(column(data, 1))
pp.plot(xx, s11, label = '2.4GHz antenna')
pp.xticks(list(pp.xticks()[0]) + [ 2.4 ])
pp.xlim(0, xx.max())
pp.ylim(np.min([s11.min(), s22.min()]) - 1, 1)
pp.xlabel("frequency [GHz]")
pp.ylabel("S11 [dB]")

pp.savefig("wifi-antennas.pdf", bbox_inches = 'tight')

Better.

Posted Fri 21 Mar 2014 15:24:38 CET Tags:

I'm a fan of high-quality representation of data. Be it typesetting, plotting, technical drawings or circuit schematics. And when it comes to plots, the output most applications deliver are nothing short of abysmal. If your plots are not available in a vector format, they are useless. If you decide to plot light-green lines on white background, that is utterly useless. If your axis labels read “db(mag(scatter(port1,port1)))” without the option of changing them: useless.

Luckily, most applications recognise that they suck at producing high quality plots and provide ways to get the numeric data out of them to use a dedicated utility to generate publication quality plots. The upside with that is, that all your plots will also look uniformly - which is important if you are working on something like a paper or thesis, where you'll surely want a consistent high-quality look to what you are preparing.

One such program is my good old friend “gnuplot”. I've written a fair number of small tools that turn data from simulators and measurement equipment into gnuplot scripts or into gnuplot-digestable formats. Gnuplot produces high-quality scalable plots and is immensely powerful, to satisfy most if not all of your plotting needs. However this story is not about gnuplot.

Because sometimes, I feel like I either hit the limit of gnuplot's capabilities or the language you control it by seems to get in the way. Sometimes I just want to create a sort of plot, that I think should be trivial to produce, and when I try it's just very very hard, while other things that looks incredibly hard seem trivial to create with it.

When you look for alternatives, your attention will quickly turn to “matplotlib”. That's a Python library for producing all sorts of plots and all of them in publication quality. Actually, together with NumPy and SciPy it forms a set of tools to tackle scientific and numeric problems. But let's not get distracted by that.

In the lab, we got a fairly big number of high-end measurement equipment, like high-bandwidth digital scopes, spectrum-, logic-, network-analyzers and the like. And the plots these things produce vary from mildly irritating to complete and utter crap. But that's okay, since there are usually ways to that the actual data out of them and into our high quality plotting engines (be it gnuplot or matplotlib).

Today I copied a dataset from a Rhode&Schwarz network analyzer measuring the Scattering Parameters of a setup with two wifi-antennas (one for 2.4GHz and the other in the 5GHz band) from 9kHz to 8.5GHz, so I had something to throw matplotlib against in the evening.

What S11 roughly represents (for those of you who are too lazy to read the wikipedia entry on scattering parameters) with respect to the antenna is this: The analyzer blows sine waves of all frequencies in the measured range into the antenna and looks at how much of the wave came back to it. For frequencies at which an antenna works, that value is typically quite small since most of the wave is supposed to be send off into the air. For frequencies at which the antenna doesn't work, a lot of the wave is reflected back to the analyzer. If everything of a wave is reflected and not otherwise lost on the way, incident and reflected amplitudes are the same and the magnitude of S11 at that frequency is 1 - the largest possible value in this kind of setting.

With two ports (like here with the each of the two antennas connected to one port) you get more scattering parameters: S22 is like S11 but for the second port. S21 describes how much of a wave from port 1 arrives at port two and S12 is the same just in the other direction.

Scattering parameters are usually plotted in logarithmic scale (S11log = 20·log10(|S11|)): A value of 1 would correspond to 0dB; 0.1 would be -20dB. So with this 2.4GHz antenna, you'd expect S11 to be small (like -10dB) around 2.4GHz and not-so-small everywhere else.

The network analyzer exports its datasets in touchstone format. A "s1p" file in that format has three columns. Since scattering parameters are complex numbers, the lines look either like this:

frequency   real(S11)   imag(S11)

Or like this:

frequency   magnitude(S11)   angle(S11)

"s2p" files contain the scattering parameters of a two port system (i.e. S11, S22, S21 and S12) in very much the same way. Either:

frequency real(S11) imag(S11) real(S21) imag(S21) ...etc...

Or:

frequency magnitude(S11) angle(S11) magnitude(S21) angle(S21) ...etc...

The Rhode&Schwarz analyzer uses the former version. The touchstone files also contain header lines (that define how the lines and columns are formatted, if you don't know the format) and comments at the top.

So. I got a "s2p" file that contains the measurement data and I want to plot the S11 values (representing the reflection data at port one, where the 2.4GHz antenna was connected). This is pretty easy to do in gnuplot, so matplotlib better doesn't make it too hard or I'll be demotivated massively. :)

Some constraints: The Y-axis should be labeled "S11 [dB]" and the X-axis should be labled "frequency [GHz]". It should be a plot, that you can use in a regular document: That means no title, since there will usually be a caption below or above the plot's figure in the document. And it also means that there should be minimal bounding box around the plot, so that no space is wasted. Also, it should obviously be a scalable format. I'd also like an extra tick at 2.4GHz on the frequency axis.

So postscript, pdf or svg. Proper inclusion in LaTeX documents is possible with postscript and pdf. And since there are more standalone pdf viewers than postscript viewers, let's settle for pdf.

Most of the work here is to read the dataset from the touchstone file. Luckily, the NumPy package provides a useful function to slurp in text-based datasets like this: loadtxt. So first things first:

import matplotlib.pyplot as pp
import numpy as np

Now we have access to ‘loadtxt’ as well as matplotlib's ‘pyplot’ API. The thing is, that we need to turn the real/imaginary pairs into actual complex numbers. But that's not so hard, if you know the ‘map’ function:

data = map(lambda x: [ x[0],
                       complex(x[1], x[2]),
                       complex(x[3], x[4]),
                       complex(x[5], x[6]),
                       complex(x[7], x[8]) ],
           np.loadtxt("wiki-antennas.s2p", skiprows=5))

Like I said, ‘loadtxt’ is the work-horse here. The ‘skiprows’ parameter just tells it to ignore the first five lines of the input file (that's the touchstone file's header and comments). The first argument to ‘map’ here is an anonymous function, that reorganises the data sucked in by ‘loadtxt’: It leaves the first column alone (that's the frequency entry) and then it takes the next columns in pairs and turns them into complex numbers. That does the trick.

You have to realise that my Python-fu is rather weak. I didn't know how to cleverly access an array of arrays column-wise. But the internet came the the recue:

def column(data, i):
    return [ row[i] for row in data ]

Now getting the data is trivial:

xx = np.multiply(1/1.0e9, column(data, 0))
yy = 20 * np.log10(np.abs(column(data, 1)))

Then prepare the plot:

# The font-size of the tick marks render a little big
# in pdf output by default. This needs to be tuned early:
fig = pp.figure()
plot = fig.add_subplot(111)
plot.tick_params(axis='both', which='major', labelsize=8)
plot.tick_params(axis='both', which='minor', labelsize=6)

# Prepare the plot, and take into account, that I like
# the plot's lines a tad ticker than the default.
pp.plot(xx, yy, linewidth = 1.5)

Now it's time to tune the plot to our liking:

# Extra tick at 2.4GHz:
pp.xticks(list(pp.xticks()[0]) + [ 2.4 ])
# Data range for X and Y axes:
pp.xlim(xx.min(), xx.max())
pp.ylim(yy.min() - 1, yy.max() + 1)
# A grid helps with orientation on the canvas:
pp.grid(True)
# Axes labels like specified:
pp.xlabel("frequency [GHz]")
pp.ylabel("S11 [dB]")

Finally, write the bugger to a file:

pp.savefig("mpl-antenna.pdf",
           # Remember? Minimal bounding box!
           bbox_inches = 'tight')
# Also, a PNG for the blog post as a thumbnail
# for the PDF:
pp.savefig("mpl-antenna.png",
           bbox_inches = 'tight')
# Alternatively, if uncommented the following
# shows the data in a plotting window:
#pp.show()

And here's the result (the image links to a scalable PDF version):

matplotlib plotting S11 of a 2.4GHz antenna

I can work with that. :)

Posted Thu 20 Mar 2014 21:53:40 CET Tags:

At work, I am (reluctantly) using a windows machine for most tasks. It's a 64-bit windows 7 installation. And to be honest, using the system is not half bad. It is probably the best user experience I ever had with any OS from Redmond (I got cygwin and a tiny virtual machine, that runs a linux system on top of windows, whenever I feel limited by the environment). In particular, the system is pretty damn stable. Sure, the odd third party program would crash every once in a while. But that's hardly Microsoft's fault. …I had almost forgotten what a bluescreen looked like.

Until now. :)

My system has a dual-monitor setup. And I usually have a field-simulator, or EDA software running on one and communication software (email etc.) on the other monitor. When one day, the following transpired: Suddenly, the mouse pointer refuses to react to movements of the pointer device, that usually controls it. Then the left monitor turns black. Then, a second or two later, the right monitor turns black. Then, after a few seconds, both monitors display the same bluescreen, with the OS dumping its memory to disk. After that, the system dies and boots again.

Unfortunately, the camera on my phone didn't start up quickly enough for me to take a screenshot. Bummer, right? But don't dispair, crashes always come in pairs, as the ancient Aztec saying tells us… Two days later, the system crashed again and this time I recognised the early symptoms and got my camera running quickly enough:

Window 7 BlueScreen of Death

When the machine came back up, it greeted me with the following dialog box:

Window 7 Crash Dialog

Cute, right? “Unerwartetes Herunterfahren” or “Unexpected Shutdown” (Reason: BlueScreen). No Shit, Sherlock! That was indeed unexpected as fuck, thankyouverymuch. The simulation software that you forced to the ground alongside yourself also asked me to convey its sincerest gratitude, since the problem it was solving for me was boring anyway.

Let's hope that this was the entirety of my share of BlueScreen crashes for this decade.

Posted Mon 17 Mar 2014 14:26:21 CET Tags:

Sometimes you're forced to use device drivers that upload binary-only firmware blobs to the actual hardware when they boot up (they all suck for various reasons: If there's a bug in it, you can't tackle it; you also got to trust the company to be one of the good guys™ - but hey, who'd suspect companies of being dicks towards their customers these days - Right? Right.). But sometimes, you're forced into using them anyway (unless you got intact principles, I guess…).

Like this one time, at band camp, when I was trying to use Texas Instrument's MSP-FET430UIF. That's an USB connected flash emulation tool (read: a JTAG-thingy) for the MSP430 micro-controller series. Plugging it into my Debian laptop yielded this:

usb 4-1: new full-speed USB device number 2 using uhci_hcd
usb 4-1: New USB device found, idVendor=0451, idProduct=f430
usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 4-1: Product: MSP-FET430UIF JTAG Tool
usb 4-1: Manufacturer: Texas Instruments
usb 4-1: SerialNumber: TUSB3410334C17506C42FFD2
ti_usb_3410_5052 4-1:1.0: TI USB 3410 1 port adapter converter detected
usb 4-1: firmware: agent aborted loading ti_usb-v0451-pf430.fw (not found?)
usb 4-1: firmware: agent aborted loading ti_3410.fw (not found?)
ti_usb_3410_5052: probe of 4-1:1.0 failed with error -5

So the system sees the thing just fine, but it can't locate the blob to upload into the device to ultimately make it work. Now you got three choices: a) Go and cry for a while and pretend this never happened. b) Build yourself a GoodFET; or c) Bite the bullet and find the god-forsaken blob. Solution "a" is not viable, since we're engineers. So that don't fly. Solution "b" is actually quite appealing, but takes time that we might not have. Let's face it, chances are, we'll want to use the blob solution behind door "c".

The device that requires the driver is actually not the FET itself but the TUSB3410 chip (that's a device that turns a USB link into a serial port as far as the operating system and its applications are concerned; which is nice with a great many applications, since a serial link is so much easier to handle than a raw USB connection) that's built into it. And its firmware blob is missing.

I couldn't find a package for debian wheezy containing the blob (yeah, I got non-free in my sources.list, but I'm weary of putting third-party sources in there). My google-fu didn't turn up much useful, which is why we're here in this public service announcement… If you're looking for firmware blobs to use with linux device drivers, here's the git repository that'll help you:

% git clone git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git

There's a ‘ti_3410.fw’ file in there, that'll work with the device driver. You'll have to copy it somewhere the kernel will look for firmware blobs (on my debian system that's “/lib/firmware”). Hope that helps you.

Finally, if you're looking for a USB⇔Serial converter for an upcoming project, there are devices available that don't require blobs in the system (one such chip would be the FT232R, the price of which is in the same ballpark as the TI device).

Posted Thu 27 Feb 2014 21:55:15 CET Tags:

An einigen Nummern kommt man einfach nicht vorbei…

In Baden-Württemberg will man Schülern im Unterricht in Zukunft ja erzählen, daß noch andere mögliche Orientierungen neben der Heterosexualität gibt, und daß diese auch eine völlig normale Sache sind. So weit, so unspektakulär. Normalerweise würde ich erwarten, daß das kein großer Aufreger ist.

Nun habe ich beim Spiegel eine Überschrift gesehen: Kirchen lehnen "sexuelle Vielfalt" im Unterricht ab. Gut möglich, daß da Captain Obvious gefragt worden ist, ob er nicht noch eine Überschrift für den Internetarm des Nachrichtenmagazins hat.

Oder um es mit Louis de Funès zu halten: - Nein. - Doch! - Oh! …eigentlich immer noch kein Grund, sich zu der Debatte zu äußern.

Was mich nicht an der Geschichte vorbei kommen läßt ist dagegen ein Satz im Untertitel des Artikels: „Die Kirchen lehnen die geplante Leitlinie ab. Begründung: Schüler würden indoktriniert.” Das muß man sich auf der Zunge zergehen lassen: Die Kirchen lehnen eine Leitlinie, die ihnen nicht paßt mit der Begrüngung ab, daß Kinder indoktriniert würden.

Ich nehme ja an, daß man sich da einfach nur auf die Füsse getreten fühlt, da die Indoktrination von Kindern doch eine der wenigen Kernkompetenzen der Kirche ist…

Posted Sat 11 Jan 2014 10:47:47 CET Tags:

If you happen to use emacs and vim, your muscle memory might from time to time use the shortcuts from one application in the other. At least that's what happens to me.

Personally, I use evil to give emacs modal editing capabilities. Its user interface is modeled after vi(m). So everything is fine on the emacs front.

I do prefer vi's interface to the default emacs interface, so naturally I don't miss much when using vim. However, there is one short-cut, that I keep hitting while I'm in vim. And that's emacs' "<C-x><C-s>" for saving the currently focused buffer's file. And if the binding would just be missing, that wouldn't be so bad. But what's worse is, that vim does not disable the terminal's flow control while it's active. I can't understand why any full screen terminal application would leave it enabled (and sadly, vim is not alone in choosing to behave this way...). Because of this, the terminal vim is running in freezes as soon as you hit "<C-s>" (you can obviously un-freeze it, by hitting "<C-q>", but ... come on). It's annoying.

So. The shell to rescue: Let's create a wrapper function ‘vim’, that takes care of disabling flow control for us (handling flow control is a non-issue in gvim):

vim () {
    # ‘local’ is not a POSIX sh feature, but if you don't use something
    # like zsh, mksh, bash or ksh93 as your interactive shell, I can't
    # help you anyway. ;)
    local terminal="$(stty -g)"
    command stty -ixon
    command vim "$@"
    command stty "$terminal"
}

Now that that's out of the way, to get the emacs short-cut working in both insert and normal mode, use this:

inoremap <c-x><c-s> <c-o>:w<cr>
noremap <c-x><c-s> :w<cr>

And that's it.

[Update]

This obviously only works if you start vim from your shell. If an application starts the editor, it doesn't care much about the shell function that you defined. To cover these cases as well, you can create a wrapper script, that does what the function does. But the real question is, why vim doesn't disable flow control itself - if not by default, then why is it not an option. (And no, I do not want to disable flow control altogether, because it might come in handy with non-full-screen terminal applications, that generate a lot of output.)

Posted Thu 19 Dec 2013 13:49:31 CET Tags:

So, seems like GNU Octave has a graphical user interface now, if you run a sufficiently recent version of it:

GUI Octave

Posted Mon 09 Dec 2013 00:25:56 CET Tags:

Did you ever take a look at the information from CDDB servers before you rip your CD collection to your hard drive? People are fucking insane. Well, they are not. But they do have different tastes. And what you end up with in your audio files, is an inconsistent mess. And consistency is good. Inconsistency is not.

In order to make things more consistent again, you have to change the meta data contained within your audio files. You know, ID3 tags and all that. Different file types obviously use different kinds of meta data. And some file types may use more than one, because some designers are high on crack (“Hello id3v1 development team!”): For example, in mp3 files, you're likely to find id3v1 tags, id3v2 tags (of differering sub-versions) and apetag tags (as far as I know, this is basically because the guys writing the foobar2000 audio player for windows were fed up with the id3* mess, and thought adding one slightly less confusing system would be the least painful solution).

There are command line utilities for all those formats. And they all work differently. That's to be expected, but it's inconvenient to write higher level tools with them to handle many file types transparently.

You know the obvious reaction of anyone who is a little too full of himself: “Man... How hard can it be?“ - That feeling takes as long as it takes you to weed through the first few pages of implementation details for the first tag implementation for the first format you want to support.

It's certainly not easy. Luckily, there are people who are crazy enough to write libraries for this sort of thing. The library that stuck out the most in this field (at least to my untrained eye) is TagLib. It support a large list of audio file types along with all tag implementations you would ever care about.

It's a C++ library, and since I'm not much of a C++ programmer, I was happy to see that TagLib has bindings for C programmers, too. So I wrote a little program called ‘taggit’, that uses those bindings. Unfortunately, the library's C bindings are vastly incomplete. They only cover the most basic tag information like “artist”, “album” and such. To support more, I starting writing a bit of glue code to interface the C++ part of the library from C. And boy, is that an experience I don't want to relive ever again.

The real solution would be to rewrite the program using C++ to properly leverage TagLib's native API. But then there's me and C++. We never got quite along. This was in 2009, so the new and shiny C++11 standard wasn't quite done yet. Still, I started the effort, but soon got bored with it. And so, ‘taggit’ bit-rotted for the better part of four years.

Then recently, my brother hinted me at a bunch of talks, that show-cased some of the things that got way nicer in C++11 and I was motivated to give my little project another go. And it got along fairly quickly.

Then I found out, that the name ‘taggit’ was already taken by another project. So I had to come up with another name (that process - by the way - is the stuff that large posts in the ‘rant’ category are made of; so I'll skip that). The name I settled for is ‘amded’, which is short for: Audio-Meta-Data-EDitor

At the moment it supports mp3 files, ogg-vorbis files and ogg-flac files. In mp3 files, all three major tag implementations are supported (id3v1, id3v2 and apetag). All thanks to TagLib. Adding other file types, supported by TagLib should not be too much of a problem, since all the infrastructure is already there. The same goes for tags, as long as TagLib puts them into their PropertyMap abstraction.

‘amded’ is not really meant to be used directly by a user. It's more of a backend tool for higher-level utilities to use. For that, it features things like machine-readable output when listing the values of tags in a file.

If you just want something to edit tag, take a look at ‘easytag’ instead (although that one doesn't support apetags in mp3 files).

It's also not too broadly tested. I still need to move my front-end code from using separate tools (‘id3v2’, ‘vorbiscomment’ and ‘metaflac’) to ‘amded’.

Long story short:

Needs a fairly recent version of TagLib and a C++11 compiler for building the executable and txt2tags for building the manual page.

Posted Wed 06 Nov 2013 15:34:46 CET Tags:
Posted Wed 03 Jul 2013 00:47:28 CEST Tags:

Der Spiegel (jedenfalls der Online Arm) jammert ja seit ein paar Tagen rum, man solle mal seinen Adblocker ausschalten. Wegen der Kinder, oder so. Das unerwartete Ergebnis: Es gab offenbar noch jede Menge Nutzer unter den SPON Lesern die noch garnicht wußten, was ein Adblocker ist... Dementsprechend positiv war der Effekt bei den Leuten von adblockplus.org:

AdblockPlus Twitter Dings

Von daher hat SPON da doch ganz gute Aufklärungsarbeit geleistet... ;)

Die Titanic hat natürlich ihre eigene amüsante Sicht auf die Dinge:

Titanic: SPON Zeug

Dpa-Flatrate, *schenkelklopf*. Dem habe ich nichts hinzuzufügen. :-)

(Jaja, die Onlineangebote von anderen Zeitungsverlagen jammern gerade ebenfalls rum. Nicht nur der Spiegel. Aber irgendwie klingt die Meldung die der Spiegel einblendet am drastischsten - und soweit ich weiß waren sie auch die ersten die damit angefangen haben.)

Posted Wed 15 May 2013 15:23:50 CEST Tags:

    If you talk to him...

Posted Sat 27 Apr 2013 14:13:26 CEST Tags: