logo

On this page:
    Progressive split on regex
    Choose a command template
    Hex values in a text string
    Top and bottom of a text file

On the Extras 3 page:
    A better uniq count
    Reformat tab-separated items
    Visualise single spaces
    A special date check
    A special cross-file check

On the Extras 1 page:
    A data auditing procedure
    Reporting results

On the Extras 2 page:
    Single-record viewing
    Single-record paging (YAD)
    Bulk replacements (YAD)
    Field-aware search


TSV This marker means that the recipe only works with tab-separated data tables.


Progressive split on regex

This recipe splits a table into two new tables: one with a target string or regex in a particular field, and one without. It renames the "without" table as the original, unsplit table, so to keep that original, unsplit table, make a copy first! The recipe can be re-run to find another target in the "without" table, so each search grows the "with" table and shrinks the "without" one.

Progressively split a table on a string or regex into "with" and "without" tables.    TSV

awk -F"\t" '{if ($[selected field] ~ /regex/) print >> "with"; else print >> "tmp"}' table && mv tmp table


Choose a command template without launching it

You can store command templates in a text file, then use the script below to choose one of the templates and put it at a prompt without launching it. You can then edit the command to suit the file, fields and other features before launching.

I keep my templates in a text file called "sniplist" in my ~/scripts folder. Each template occupies 2 lines in "sniplist": a numbered line with a description of the command, and a following line with the template. In the screenshot below, the script is called "snips" and I've chosen template 3.

#!/bin/bash
 
echo
cat ~/scripts/sniplist
echo
read -p "Select a command number: "
echo
foo=$(awk -v line="$REPLY" 'NR==(line*2)' < ~/scripts/sniplist)
stty -echo &&
xdotool type --delay 1 "$foo" &&
stty echo
exit

snips

Hex values in a text string

The "hexcha" function will show individual characters in a text string and their corresponding hexadecimal values. If the text string is more than one word (if it contains spaces), the string should be enclosed in single quotes.

hexcha() { while read -rn1 char; do echo -ne "$char\t" && hexdump -e '/1 "%02x" " "' <<< "$char" | sed 's/ 0a //'; echo; done <<<"$1" | sed '$d'; }

hexcha

Top and bottom of a text file

The "toptail" function prints the first and last 10 lines in a text file. It's especially useful for viewing the results of a long tally.

toptail() { sed -u '10q'; echo "---"; tail; }

toptail