unix sort for 2 fields numeric order

Question

I need to sort some data with unix sort but I can't figure exactly the right syntax, the data looks like

3.9.1 Step 10:
3.9.1 Step 20:
3.8.10 Step 20:
3.10.2 Step 10:
3.8.4 Step 90:
3.8.4 Step 100:
3.8.4 Step 10:

I want to sort it using first the major number, then the step number, e.g. the data sorted above would look like.

3.8.4 Step 10:
3.8.4 Step 90:
3.8.4 Step 100:
3.8.10 Step 20:
3.9.1 Step 10:
3.9.1 Step 20:
3.10.2 Step 10:

I have found the way to sort by first number on this site:

sort -t. -k 1,1n -k 2,2n -k 3,3n

but I am struggling to now sort by the 3rd column Step number without disturbing the first sort

Answer 1

How about transforming the Step and : on the way into sort, and then transforming back afterwards? I believe this gets the results you're looking for:

cat your-file.txt \
    | sed -e 's/ Step \(.*\):$/.\1/g' \
    | sort -t. -k1,1n -k2,2n -k3,3n -k4,4n \
    | sed -e 's/\(.*\)\.\(.*\)$/\1 Step \2:/g'

(Just using cat here for expository purposes. If it's just a regular file, then it could be passed to the first sed.)

Answer 2

There's a fascinating article on re-engineering the Unix sort ('Theory and Practice in the Construction of a Working Sort Routine', J P Linderman, AT&T Bell Labs Tech Journal, Oct 1984) which is not, unfortunately, available on the internet, AFAICT (I looked a year or so ago and did not find it; I looked again just now, and can find references to it, but not the article itself). Amongst other things, the article demonstrated that for Unix sort, the comparison time far outweighs the cost of moving data (not very surprising when you consider that the comparison has to compare fields determined per row, but moving 'data' is simply a question of switching pointers around). One upshot of that was that they recommend doing what danfuzz suggests; mapping keys to make comparisons easy. They showed that even a simple scripted solution could save time compared with making sort work really hard.

So, you could think in terms of using a character that's unlikely to appear in the data file naturally (such as Control-A) as the key field separator.

sed 's/^\([^.]*\)[.]\([^.]*\)[.]\([^ ]*\) Step \([0-9]*\):.*/\1^A\2^A\3^A\4^A&/' file |
sort -t'^A' -k1,1n -k2,2n -k3,3n -k4,4n |
sed 's/^.*^A//'

The first command is the hard one. It identifies the 4 numeric fields, and outputs them separated by the chosen character (written ^A above, typed as Control-A), and then outputs a copy of the original line. The sort then works on the first four fields numerically, and the final sed commands strips off the front of each line up to and including the last Control-A, giving you the original line back again.

Answer 3

This might work for you:

 sort -k3,3n file | sort -nst. -k1,1 -k2,2 -k3,3

or a very iffy:

 sort -nt. -k1,1 -k2,2 -k3,3 -k3.7 file

The first uses two sorts:

1. sort -k3,3n sorts by steps
2. sort -nst. -k1,1 -k2,2 -k3,3 sorts by major numbers but keeps the step order

The second works but only if the 3rd major number remains below 100.

or perhaps:

sed 's/ /./2' file | sort -nt. -k1,1 -k2,2 -k3,3 -k4,4 | sed 's/\./ /3'

Answer 4

UPDATED:

This will generate the output you specified:

sed 's/Step /Step./' data|sort -t. -n -k1,1 -k2,2 -k3,3 -k4|sed 's/Step./Step /'

result:

3.8.4 Step 10:
3.8.4 Step 90:
3.8.4 Step 100:
3.8.10 Step 20:
3.9.1 Step 10:
3.9.1 Step 20:
3.10.2 Step 10:

The challenge with this sort is that the sorting fields are defined by both '.' (for the version numbers) and the default whitespace (for the Step numbers). You can't specify several/different field separators for the same sort command. Combining several sorts with different field separators did not yield the right output.

This solution works by replacing the blank space after the Step field temporarily with a '.' so that all sorting fields can be separated with the same character ('.'). After the sort is done, the '.' is replaced with a blank again.