banner

For a full list of BASHing data blog posts, see the index page.     RSS

The magic of BASH string expansion

A tab character is invisible in a terminal, but you can "visualise" it by using cat -T to replace the tab with "^I":

magic1

A neater method is to replace the tab with a couple of Unicode characters and a couple of spaces. Together these make a "tab-like" impression:

magic2

The font in my Xfce4-terminal is Google's NotoMono-Regular.ttf. Like other modern monospace fonts, Noto Mono supports a very large set of Unicode characters.

Unfortunately AWK (GNU AWK 4.1.4) is confused by Unicode character codes in replacement commands, and so is sed (GNU sed 4.4):

magic3

In the past, there were complicated workarounds for dealing with Unicode character codes. For AWK, one was to find the hex value of the Unicode character and use that in the replacement command. Another way was to define the output of BASH printf as an AWK variable, and use the variable in the command:

magic4

The Unicode-to-hex trick also worked with sed:

magic5

However, if you're running a recent BASH (mine is version 4.4), the solution is magically simple: just put a "$" in front of the single-quoted AWK or sed command string:

magic6

The $'...' construction is "quoted string expansion". BASH understands Unicode character codes, and when it expands the command string it replaces those codes with the printing characters represented by the codes. These characters can then be used by AWK and sed. In the screenshot I use echo to show what BASH is doing to the single-quoted string, and I've added cat -T to visualise the tab in the expanded string as "^I".

magic7

Here's another example. To get a one-character-wide filled block in a terminal you can use the Unicode "full block" character, U+2588:

magic8

An alternative is to colour the background of a small character like "." with ANSI colour codes. In the screenshot below, I'm using the background "light yellow" colour (#ffff55 from the ANSI 256-colour set) as a good match to my terminal's text colour (#fce94f):

magic9

There are lots of sources on the Web for ANSI colour codes; my favourite is https://misc.flogisoft.com/bash/tip_colors_and_formatting.

AWK understands ANSI codes but sed doesn't, unless you add the magic initial "$" to expand the command string:

magic10

An old-school alternative for sed is to put the escape character (it's the character assigned to the ESC key) into a shell variable, and use that variable in a double-quoted command:

magic11

Last update: 2019-05-19
The blog posts on this website are licensed under a
Creative Commons Attribution-NonCommercial 4.0 International License