Top

A Linux User Reference

Search tips
  • search ignores words that are less than 4 characters in length
  • searches are case insensitve
  • if a search does not return anything try it in Boolean mode then Query expansion mode by checking the appropriate radio button e.g. searching for 'cron' in just the Administration category returns nothing - presumably because the 50% threshold is reached. Boolean mode ignores this threshold so a search for 'cron' returns several hits
  • in Boolean mode preceding a word with a '+' means the result must include that word, a '-' means it must not
  • in Boolean mode '+crontab -anacron' means match articles about crontab that DO NOT mention anacron
  • to match a phrase e.g. 'manage system' check the Boolean mode radio button and enclose the phrase in quotes "some phrase ..."
  • in Query expansion mode the search context is expanded beyond the keywords you entered - relevancy of hits may well be degraded

FILES AND DIRECTORIES

Text processing

  • Cut selected columns, fields from file(s)
    /usr/bin/cut

    Command usage

    cut option ... [file] ...
    
    Common options:
     -c | --characters<int list>   Print out character(s)
     -d | --delimiter<IFS>         Define field delimiter
     -f | --field<int list>        Select only these fields
    

    Set : as field separator, print fields 1 and 3-4

    # cut -d: -f1,3-4 /etc/passwd | head -3**
    root:0:0
    daemon:1:1
    bin:2:2
    

    Print characters in columns 1-3 and 5

    # cut -c 1-3,5 /etc/passwd | head -3
    roo:
    daeo
    binx
    
  • Convert spaces to tabs and vice versa
    /usr/bin/expand, unexpand
    • 'expand' converts tabs to spaces
    • 'unexpand' converts spaces to tabs
    expand   [options] [file] ...
    unexpand [options] [file] ...
    
    Common options:
     -i | --initial             Do not covert tabs after non-blanks, first tab
                                if line starts with a tab
     -t | --tabs<int>           Set tabs int characters apart (not 8 - default)
    

    Sample file

    $ cat /dev/null > file2;  echo -e '\t1\t2\t3\t4' > file2
    $ cat file2
       1   2   3   4
    

    Set tabs at 4, 10, 20 and 24 characters from start of line

    $ expand -t 4,10,20,24 file2
       1     2         3   4
    

    Set tabs 4 characters apart

    $ expand -t 4 file2
       1   2   3   4
    

    Set the first tab only, one character from start of line

    $ expand -i -t 1 file2
     1   2   3   4
    
  • Format paragraphs in files
    /usr/bin/fmt

    Command usage

    fmt [-digits] [options] [file]...
    
    Common options:
     -u | --uniform-spacing         One space between words, two after sentences
     -w | --width=WIDTH             Maximum line width (default of 75 columns)
    

    Default format, in this case the original file

    $ fmt file2
    Paragraph number one.  An interesting one ........
    Paragraph number two.  Another  interesting one ........
    

    Format with uniform spacing

    $ fmt -u file2
    Paragraph number one.  An interesting one ........
    Paragraph number two.  Another interesting one ........
    

    Format to a 30 character wide page

    $ fmt -w 30 file2
    Paragraph number one.  An
    interesting one ........
    Paragraph number two.  Another
    interesting one ........
    

    Same as previous command

    $ fmt -30 file2
    Paragraph number one.  An
    interesting one ........
    Paragraph number two.  Another
    interesting one ........
    
  • Output first and last lines
    /usr/bin/head,/tail
    • head outputs the first lines of a file or files
    • tail outputs the last lines of a file or files
    head [options] [file] ...
    tail [options] [file] ...
    
    Common options:
     -<int> | -n <int>      Print <int> number of lines from beginning or end of file(s)
     -c <int>               Print <int> number of bytes from beginning or end of file(s)
     -f                     Continuously display a file(s)
     -s <int>               Sleep <int> seconds
    

    Show first 5 lines of file1 and file2

    $ head -5 file1 file2
    

    Show last 5 lines of file1 and file2

    $ tail -5 file1 file2
    

    Continuously display file1 and file2, sleeping every 10 seconds

    $ tail -f -s 10 file1 file2
    
  • Join together pairs of lines
    /usr/bin/join

    Join pairs of lines from file1 and file2 that share the same join field.

    join [options] [file] ...
    
    Common options:
     -j FIELD              Equivalent to '-1 FIELD -2 FIELD'
     -t CHAR               Use CHAR as input and output field separator
     -1 FIELD              Join on this FIELD of file 1
     -2 FIELD              Join on this FIELD of file 2
    

    Set up a couple of test files

    #file1
    1 Title:
    2 Location:
    3 Country:
    
    #file2
    1 John Smith
    2 Liverpool
    3 UK
    

    Join each line using the the first field as the join field

    $ join -j 1 file1 file2
    1 Title: John Smith
    2 Location: Liverpool
    3 Country: UK
    

    Note: if file2 had it's last line as, for example, '4 UK' then only the first two lines of each file will be joined since the join fields would not be the same for the third line in each file.

  • Number each line
    /usr/bin/nl

    Command usage

    nl [options] [file] ...
    
    Common options:
     -b | --body-numbering=STYLE          Use STYLE for numbering body lines
     -f | --footer-numbering=STYLE        Use STYLE for numbering footer lines
     -h | --header-numbering=STYLE        Use STYLE for numbering header lines
    
    Styles:
      a        Number all lines
      t        Number only non-empty lines
      n        Number no lines
      pBRE     Number only lines that contain a match for the basic regular 
               expression (BRE)
    

    Number each line in body of file(s) that match BRE i.e. lines that start with 2

    $ nl -bp^2 file1 file2
           1 Title:
         1   2 Location:
           3 Country:
           1 John Smith
         2   2 Liverpool
           3 UK
    

    New sample file1

    $ cat file1
    \:\:\:
    Header
    \:\:
    Body
    body
    \:
    Footer
    

    Number none-empty lines in header and footer of file(s)

    $ nl -ht -bn -ft file1
         1 Header
          Body
          body
         2 Footer
    
  • Dump files in several formats
    /usr/bin/od

    Command usage

    od [options] [file] ...
    
    Common options:
     -c | -t c             Ascii
     -x | -t x             Hex
     -o | -t o             Octal
     -a | -t a             Named character
     -u | -t u             Unsigned decimal
     -d | -t d             Signed decimal
     -f | -t f             Floating point
    

    Dump file2 in named character format

    $ od -t a file2
    0000000   1  sp   J   o   h   n  sp   S   m   i   t   h  nl   2  sp   L
    0000020   i   v   e   r   p   o   o   l  nl   4  sp   U   K  nl
    0000036
    

    Dump file2 in hex format

    $ od -t x file2
    0000000 6f4a2031 53206e68 6874696d 4c20320a
    0000020 72657669 6c6f6f70 5520340a 00000a4b
    0000036
    
  • Paste corresponding lines into columns
    /usr/bin/paste

    Command usage

    paste [options] [file] ...
    
    Common options:
     -d | --delimiters=list     Use characters from list as delimiter(s) instead of tabs
     -s | --serial              Paste one file at a time instead of in parallel
    

    Two files

    $ cat file1 file2
    Title:
    Location:
    John Smith
    Liverpool
    

    Paste together columns from file1 and file2 using space as delimiter

    $ paste -d' ' file1 file2
    Title: John Smith
    Location: Liverpool
    

    Paste file1's columns first then file2's

    $ paste -s file1 file2
    Title:   Location:
    John Smith   Liverpool
    
  • Convert text into columned pages
    /usr/bin/pr

    Command usage

    Plenty more page formatting options.

    pr [options] [file] ...
    
    Common options:
     -h <header>            Use <header> instead of filename
     -l <int>               Page length
     -o                     Indent
    

    Use 'Details-Header' as header, 11 lines/page and indent each line 3 characters

    $ pr -h Details-Header -l 11 -o 3 file1
    2009-03-28 13:35   Details-Header      Page 1    (Auto-generated header using provided string)
       Title:                                        (Line 1 of file1, the next 10 lines are blank)
       .....                                        
    2009-03-28 13:35   Details-Header      Page 2
       Location:                                     (Line 12 of file1, now line 1 of page 2)
    

    Blank lines omitted and spacing modified for brevity.

  • Sort file(s)
    /usr/bin/sort

    Loads of variations - just the basics here.

    sort [options] [file] ...
    
    Common options:
     -t                  Field separator
     -k                  Keypos1[,keypos2]
     -n                  Numeric sort
     -r                  Reverse sort
     -f | --ignore-case
    

    Sample file

    $ cat file3
    1: Bert: Sachs: 34
    2: Samantha: Trump: 47
    3: Dave: Andrews: 41
    

    Sort file3 on the third field using : as the field separator

    $ sort -t: -k 3 file3
    3: Dave: Andrews: 41
    1: Bert: Sachs: 34
    2: Samantha: Trump: 47
    

    Numeric sort of file3 using : as the field separator and fourth field as sort key

    $ sort -t: -k 4 -n file3
    1: Bert: Sachs: 34
    3: Dave: Andrews: 41
    2: Samantha: Trump: 47
    

    Reverse numeric sort of file3 using : as the field separator and fourth field as sort key

    $ sort -t: -k 4 -n -r file3
    2: Samantha: Trump: 47
    3: Dave: Andrews: 41
    1: Bert: Sachs: 34
    
  • Split a file into chunks of lines
    /usr/bin/split
    • output directed to a succession of files
    • split a large file into several smaller ones
    split [options] [input] [prefix]
    
    Common options:
     -l | --lines=NUMBER        Put NUMBER lines per output file
     PREFIX                     Prefix to use for output files
    

    Split file3 into as many files, prefixed with files3_, as there are lines in file

    $ split -1 file3 file3_
    $ more file3_*
    ::::::::::::::
    file3_aa
    ::::::::::::::
    1: Bert: Sachs: 34
    ::::::::::::::
    file3_ab
    ::::::::::::::
    2: Samantha: Trump: 47
    ::::::::::::::
    file3_ac
    ::::::::::::::
    3: Dave: Andrews: 41
    
  • Reverse 'cat'
    /usr/bin/tac

    Command usage

    tac [options] [file] ...
    
    Common options:
     -b |--before               Attach the separator before instead of after
     -r | --regex               Interpret the separator as a regular expression
    

    Display file3, last line first

    $ tac file3
    3: Dave: Andrews: 41
    2: Samantha: Trump: 47
    1: Bert: Sachs: 34
    
  • Translate characters from Set1 to Set2
    /usr/bin/tr

    Command usage

    tr [option] Set1 [Set2]
    
    Common options:
     -c | -C | --complement           First complement Set1.
     -d | --delete                    Delete characters in SET1, do not translate.
     -s | --squeeze-repeats           Replace each input sequence of a repeated 
                                      character that is listed in SET1 with a 
                                      single occurrence of that character.
    
    Common Sets:
    \\ backslash              \b backspace   
    \f form feed              \n newline   
    \r carriage return        \t horiz. tab  
    \v vert. tab              \nnn 1-3 octal digits
    
    [:alnum:] alphanumeric                     [:alpha:] letters      
    [:cntrl:] ctrl characters                  [:xdigit:] hex
    [:lower:] lower case                       [:upper:] upper case   
    [:blank:] horiz. whitespace                [:digit:] digits
    [:space:] horiz. and vert. whitespace      [:punct:] punctuation chars.
    [:graph:] printable chars. not whitespace
    

    Covert lower case to upper

    $ cat file | tr a-z A-Z
    

    Same as previous command

    $ cat file | tr '[:lower:]' '[:upper:]'
    

    Remove all non-printable characters except newline

    $ cat file | tr -dc '[:print:]\n'
    

    Replace sequences of more than one \n with just one \n

    $ cat file | tr -s '\n'
    
  • Discard any successive identical lines
    /usr/bin/uniq

    Command usage

    uniq [options] [input] [output]
    
    Common options:
     -c | --count               Prefix lines by the number of occurrences
     -d | --repeated            Only print duplicate lines
     -i | --ignore-case         Ignore differences in case when comparing
     -s | --skip-chars=N        Avoid comparing the first N characters
     -u | --unique              Only print unique lines
    

    Sample file

    $ cat file4
    this is a line
    this is a line
    this is another
    this is a line
    

    Discard any successive identical lines

    $ uniq file4
    this is a line        (Line 2 has been discarded)
    this is another
    this is a line        (Identical to earlier lines, but not successive to any of them)
    

    Print non-unique lines

    $ uniq -d file4
    this is a line
    

    Count the unique and successive identical lines

    $ uniq -c file4
      2 this is a line     (2 successive, identical lines)
      1 this is another
      1 this is a line
    
  • Print newline, word and byte counts
    /usr/bin/wc

    Command usage

    wc [options] [file] ...
    
    Common options:
     -l                    Newline
     -w                    Word count
     -c                    Byte count
     -m                    Character count
     -L                    Max line length
    

    Count the number of lines, words, bytes, characters and max line length for each file - give a total

    $ wc -l -w -c -m -L file3 file4
      3  12  63  63  22 file3
      4  15  61  61  15 file4
      7  27 124 124  22 total