banner

For a full list of BASHing data blog posts see the index page.     RSS

How to move selected lines within a file

If you're working with a GUI text editor and want to move a particular line from one place to another, you might use cut and paste. Cut out the line, delete the gap that's left behind, insert a blank line at the destination place and paste into the blank the line from the clipboard.

If you're working with a text file on the command line (not in an editor like emacs or vim) and don't want to use a clipboard, line-moving can be done with a single AWK command.

For example, suppose I have the following file, called "demo":

SaleID|Class|Item|Count|UnitP|TotalP
146|fish|BG fillets|3|15.00|45.00
2785|fruit|banana|1|0.45|0.45
0039|fruit|banana|1|0.45|0.45
119|meat|liver paste|1|2.10|2.10
6253|veg|carrot bunch|2|4.90|9.80
8847|fish|tin tuna|4|1.50|6.00
3776|veg|pak choy|2|2.50|5.00
295|fruit|apple|6|0.94|5.64
534|fish|tin tuna|1|1.50|1.50
1221|meat|pork slices|8|4.20|33.60

and I want the "apple" line to follow the second "banana" line without a change in the order of the other lines. The following AWK command will do the job by processing "demo" twice: once to store the "apple" line in a variable, and again to do the insertion and deletion.

awk -F"|" 'FNR==NR {if ($1=="295") {x=$0; next} else next} \
$1=="0039" {$0=$0 RS x} $1!="295" {print}' demo demo

shift1

(I've used red arrows above to highlight the "apple" line before and after moving. For another way to highlight the line, see the last section of this blog post.)
 
The FNR==NR method of processing two files separately is explained here. AWK processes "demo" the first time and looks for the line in which "295" is the entry in the first pipe-delimited field ($1=="295"). When the line is found, the whole line is stored in the variable "x"; instead of printing the line (AWK's default), AWK follows the next instruction and moves to the next line. At any other line, the instruction is just next: don't print, just move to the next line.
 
When AWK moves to "demo" for the second time (FNR no longer equals NR), it looks for a line in which the field 1 entry is "0039". When that line is found, it's replaced by the same line, a newline (the default record separator, or RS) and the line stored in the variable "x". AWK moves to its final instruction, which is to print any line where the field 1 entry isn't "295", thus deleting the "295" line. The printing will include the modified line beginning with "0039", which is now actually two lines.

Above I've identified lines by the contents of the first field. Lines can also be identified for AWK by line number, but note that in the second pass the current line number is FNR, not NR:

awk 'FNR==NR {if (NR==9) {x=$0; next} else next} \
> FNR==4 {$0=$0 RS x} FNR!=9 {print}' demo demo

shift2

The line-number command for moving a single line can be written as a function to save a lot of typing. I call my function "shiftline":

shiftline() { awk -v target="$2" -v putafter="$3" 'FNR==NR {if (NR==target) {x=$0; next} else next} FNR==putafter {$0=$0 RS x} FNR!=target {print}' "$1" "$1"; }
 
shiftline [filename] [line number to be moved] [line number preceding the new position]

shift3

The same command structure can be used to move multiple lines. Suppose in "demo" I want to gather up the two "meat" lines and put them just under the header. This command will do it:

awk -F"|" 'FNR==NR {if ($1=="119") {x=$0; next} \
else if ($1=="1221") {y=$0; next} else next} \
FNR==1 {$0=$0 RS x RS y} $1!="119" && $1!="1221" {print}' demo demo

shift4

As a special case, there's a much easier way to move lines if their new position is to be either at the start of a file or the end. Here's the CSV "4firstlast":

aaa,111,mmm
bbb,222,nnn
ccc,333,ooo
ddd,444,ppp
eee,555,qqq

To move the "bbb" and "ddd" lines to either the start or the finish I can use cat and grep:

cat <(grep -E "bbb|ddd" 4firstlast) <(grep -vE "bbb|ddd" 4firstlast)
cat <(grep -vE "bbb|ddd" 4firstlast) <(grep -E "bbb|ddd" 4firstlast)

shift5

On my system, grep versions are aliased to grep --color=auto. To highlight the "apple" line and also show the rest of the "demo" file, I can use either of these two commands:

grep -E "^295\|.*$|$" demo
grep -E "^295\|.*$|^" demo

In each case grep is looking for either a line that starts with "295|" (^295\|) and finishes with zero or more characters (.*$), or an empty string. "^" is the empty string at the start of every line, and "$" is the empty string at the end of every line. grep will color what it finds, but with empty strings there's nothing to color.

shift6

Last update: 2020-05-13
The blog posts on this website are licensed under a
Creative Commons Attribution-NonCommercial 4.0 International License