Top

A Linux User Reference

Search tips
  • search ignores words that are less than 4 characters in length
  • searches are case insensitve
  • if a search does not return anything try it in Boolean mode then Query expansion mode by checking the appropriate radio button e.g. searching for 'cron' in just the Administration category returns nothing - presumably because the 50% threshold is reached. Boolean mode ignores this threshold so a search for 'cron' returns several hits
  • in Boolean mode preceding a word with a '+' means the result must include that word, a '-' means it must not
  • in Boolean mode '+crontab -anacron' means match articles about crontab that DO NOT mention anacron
  • to match a phrase e.g. 'manage system' check the Boolean mode radio button and enclose the phrase in quotes "some phrase ..."
  • in Query expansion mode the search context is expanded beyond the keywords you entered - relevancy of hits may well be degraded

THE SHELL

Scripting

  • Executing, running a shell script

    To be able to execute a script file or a file containing shell commands the file needs to be either

    • sourced, in which case it will run in the current shell or
    • made executable, in which case it will run in it's own shell.
    The shebang (#!)

    As you can't always guarantee which shell a user might be using you should try to ensure that any script you write is portable to all shell environments. This can prove to be quite difficult.

    A solution to this is to instruct whatever shell the script is to be run to run the script in specific shell - this is done by using the shebang #!.

    This is the first line of a script.

    #!/bin/bash
    

    Tells whatever shell a user is using to run the script in/under the bash shell.

    Script SUID rights

    An executable file with either suid or sgid permissions set will run in a shell with effective permissions of the file's owner.

    The program will be able to do anything that the owner or group could do, according to which permission bit is set.

    Source a file

    $ source ./script_name
                             (or)
    $ . ./script_name
    

    Make a file executable

    $ chmod +x <script_name>             (Makes the file executable to all)
    
    $ chmod o-x <script_name>            (Remove world, other execute permission)
    

    Make executable just for owner and group

    $ chmod ug+x <script_name>
    
  • Debugging scripts
    • Double quotes cause 'bash' to expand variables before executing a command.
    • Single quotes protects the variable from bash so it can be expanded by a command.

    Switch on the shell's debugger and run the script to trace commands and their arguments as the shell executes them.

    Enable/disable shell debugging

    # set -x                             (Enable)
    # set +x                             (Disable)
    

    Print variables to stout, or to log files

    echo "Value of var = "$var >> /tmp/mylog
    
    $ export FRED=33
    $ printf "\$FRED = "$FRED"\n"
    $FRED = 33
    
    $ echo "Value of \$FRED = "$FRED
    Value of $FRED = 33
    
  • Enable, disable shell options
    shopt
    • Bash builtin, turns shell options on or off.
    • When used in a script, the shell option(s) set by the script will apply to the script's own non-interactive shell.
    shopt -[pqsu] <shell option>
    
    Options:
     -p                    Print current value, returns 0 if option is set
     -q                    Don't print anything, returns 0 if option is set
     -s                    Set the option
     -u                    Unset the option
    

    Print current value of nocasematch option

    $ shopt -p nocasematch
    shopt -u nocasematch                  (Unset)
    
    $ shopt -q nocasematch; echo $?
    1                                     (Unset)
    

    Set an option

    $ shopt -s nocasematch
    
    $ shopt -p nocasematch
    shopt -s nocasematch                  (Set)
    

    Unset an option

    $ shopt -u nocasematch
    
    $ shopt -p nocasematch
    shopt -u nocasematch                  (Unset)
    

    Example script - shopt_ex.sh

    #!/bin/bash
    # shopt_ex.sh
    # case_script
    echo "Formats: dvd cd tape"
    
    # The DEFAULT for 'read' is no prompt and store reply in 'REPLY' variable.
    # read from input, use a prompt string and store reply in variable 'answer'
    read -p "select format type: " answer
    
    # if $answer is zero length then set answer=none
    [[ -z $answer ]] && answer="none"
    
    # use bash builtin 'shopt' to determine if 'nocasematch' is on or off,
    # save the current setting to variable 'restorecase'
    if shopt -q nocasematch
    then
       restorecase="-s"
    else
       restorecase="-u"
       shopt -s nocasematch
    fi
    case "$answer" in
       "dvd")    echo "you chose dvd ... "$answer
                 ret=0 ;;
       "tape")   echo "you chose tape ... "$answer
                 ret=0 ;;
        "cd")    echo "you chose cd ... "$answer
                 ret=0 ;;
        *)       echo "invalid selection"
                 ret=1 ;;
    esac
    
    # restore shell option 'nocasematch' to whatever it was before script ran
    shopt $restorecase nocasematch
    exit 0
    

    Run script to test case insensitivity

    $ ./test.sh
    Formats: dvd cd tape
    select format type: dvd
    you chose dvd ... dvd
    
    $ ./test.sh
    Formats: dvd cd tape
    select format type: TApE
    you chose tape ... TApE
    
  • Read a line from stdin
    read

    Bash builtin command. Read a line from stdin or file.

    read [-ers] [-u fd] 
         [-t timeout] [-a aname]
         [-p prompt] [-n nchars] 
         [-d delim] [name ...]
    
    Options:
     -a aname            Read input into array 'aname'.  Indices are sequential, word
                         boundaries defined by IFS.
     -d delim            Use first char of 'delim' to terminate input string instead 
                         of '\n'.
     -e                  If stdin is coming from a terminal, readline is used to 
                         obtain the line.
     -n nchars           Return after reading 'n' characters
     -p prompt           Display a prompt.
     -r                  Treat backslash as a literal.
     -s                  Silent mode.  If input is coming from a terminal, chars are
                         not echoed.
     -t timeout          Return error if read not completed in 'timeout'
     -u fd               Read input from file descriptor fd.
    

    If no name(s) are supplied, the line read is assigned to the variable REPLY.

    Some examples

    $ read -p "Enter some stuff>"
    Enter some stuff>fdgdjgdghj
    $ echo $REPLY
    fdgdjgdghj
    
    $ echo "232645362" > test.txt
    $ exec 3<test.txt
    $ read -u 3
    $ echo $REPLY
    232645362
    
    $ read
    123456\789
    $ echo $REPLY
    123456789
    
    $ read -r -p "Enter some stuff>"
    Enter some stuff>123456\789
    $ echo $REPLY
    123456\789
    
    $ read -d 'a'
    12345a$
    
    $ echo $REPLY
    12345
    
    $ read -n 4 answer
    1234$ echo $answer
    1234
    
  • Functions
    declaration
    • A function can be defined in one of three ways.
    • Maybe useful to use keyword function so that scripts can be easily searched for functions i.e. grep "^function".
    function fnname ( ) {      (By the book ...)
        commands
        .......
    }
    
    function fnname {          (If keyword 'function' is present then '()' optional)
        commands
        .......
    }
    
    fnname () {                (If no keyword 'function' then '()' is mandatory)
        commands
        .......
    }
    
  • Function
    parameters
    • Double quotes cause 'bash' to expand variables before executing a command.
    • Single quotes protects the variable from bash so it can be expanded by a command.
    Parameter Description
    $0 Refers to the name of the program that started bash or the shell script the function is running within
    $1-9 Positional parameters 1= 1st. argument, 2=2nd. and so on ...
    $* Positional parameters starting from 1
    "$*" Expands to a single word bounded by IFS (default is blank, tab and newline)
    $@ All positional parameters starting from 1 as a single string
    "$@" Each parameter becomes a single word, so "$@" is equivalent to "$1" "$2" ....
    $# Number of parameters, not including parameter 0

    Access parameters when there are more than 9 of them

    • the first process is to save the first parameter ($1)
    • then use the shift command to drop parameter 1 and move all remaining parameters down 1, so that $10 becomes $9 and so on.
    • The value of $# will be updated to reflect the remaining number of parameters.

    Effect of " with function parameters

    $ function test { echo "$# parameters";echo $@;echo $*; echo "$@";echo "$*"; }
    
    $ test a b c
    3 parameters
    a b c                 ($@   - a single string of all positional parameters)
    a b c                 ($*   - each parameter)
    a b c                 ("$@" - each parameter represented as a single word)
    a b c                 ("$*" - each parameter represented as a single word 
                           bounded by IFS)
    
    $ test "a b" c
    2 parameters
    a b c
    a b c
    a b c
    a b c
    
    $ export IFS='"'
    $ test a b c
    3 parameters
    a b c
    a b c
    a b c
    a"b"c                 (IFS = ")
    
    $ test "a b" c
    2 parameters
    a b c
    a b c
    a b c
    a b"c
    
  • Function
    returning values

    To get a return value 'echo' or 'printf' the variable from within the function.

    $ result=$(test a b c)
    $ echo $result
    3 parameters a b c a b c a b c a b c
    
  • Conditional clauses
    if statements

    Format

    if test; then; else; fi
    

    Example for each of the four test formats

    $ ls -al > /dev/null
    $ if (( $? ));then echo "failure"; else echo "exit status="$?" success"; fi
    exit status=1 success
    
    $ ls -al > /dev/null
    $ if test $? ;then echo "exit status="$?" success"; else echo "failure";fi
    exit status=0 success
    
    $ ls -al > /dev/null
    $ if [ $? ];then echo "exit status="$?" success"; else echo "failure";fi
    exit status=0 success
    
    $ ls -al > /dev/null
    $ if [[ $? ]];then echo "exit status="$?" success"; else echo "failure";fi
    exit status=0 success
    

    Format

    if list; then list; [elif list; then list;] ... [else list;] fi
    

    A basic cmd-line arg checking script

    #!/bin/bash
    # test.sh
    
    if [[ $# == 0 ]]
    then
       echo "need one argument"; exit 1
    elif [[ $# > 1 ]]
    then
       echo "need only one argument"; exit 2
    else
       echo "correct - argument = "$1;  exit 0
    fi
    # script end
    

    Running the script

    $ ./test.sh
    need one argument
    $ echo $?
    1
    
    $ ./test.sh 1234
    correct - argument = 1234
    $ echo $?
    0
    
    $ ./test.sh 1234 and 5678
    need only one argument
    $ echo $?
    2
    
  • Lists

    A list is a sequence of one or more pipelines separated by one of the operators ;, &, && or || and optionally terminated by one of ;, & or '\n'.

    Precedence

    1. && (AND) and || (OR) have equal precedence, followed by
    2. ; (command string termination) and & (run command in the background) which have equal precedence
    3. command1 && command2 command2 is executed if, and only if, command1 returns an exit status of zero.
    4. command1 || command2 command2 is executed if and only if command1 returns a non-zero exit status.

    The return status of AND and OR lists is the exit status of the last command executed in the list.

  • Exit from a loop
    break

    Exits from a loop immediately, can specify a number of levels to break out of if nested loops e.g. break 2.

    break [level]
    
  • Skip remaining statements in a loop
    continue
    • Allows you to bypass remaining statements in the command list and go immediately to the next iteration of the loop.
    • If 'n' is specified, resume at the nth. enclosing loop. 'n' must be >= 1.
    continue [n]
    
  • Continuously execute a loop
    while

    Format

    while list; do list; done
    

    Continuously executes the do list as long as the last command in list returns an exit status of zero.

    Starting from 0, while 'x' is less do something

    $ export x=0
    $ while [[ $x < 4 ]]; do echo "loop :"$x; x=$((x+1)); done
    loop :0
    loop :1
    loop :2
    loop :3
    
  • For loops

    Format - 1

    for name [ in word ] ; do list ; done
    

    For each file returned by the 'ls' command print with a newline

    $ for word in $(ls ); do printf "%s\n" "$word"; done
    bert
    blueprint-placeholders-full.zip
    c5docs
    c5home_files
    c5home.html
    concrete5
    .....
    

    Format - 2

    for (( expr1 ; expr2 ; expr3 )) ; do list ; done
    

    Loop 4 times

    $ for (( x=1; x<5; x++)); do echo "loop_"$x; done
    loop_1
    loop_2
    loop_3
    loop_4
    
  • Until loop

    Format

    until list; do list; done
    

    Examples

    $ export X=$((X = 0))             (Setting variable X as an integer = 0)
    
    $ until (( $X == 4 )); do         (Loop until X = 4)
    > X=$((X + 1))                    (Increment X)
    > echo $X
    > done
    1
    2
    3
    4
    
    $ export let y=0                  (Can also use 'let' for arithmetic expressions)
    
    $ until (( $y == 4 )); do
    > let y=$y+1
    > echo $y
    > done
    ....
    
  • Bash select loop
    select

    Format

    select name [ in word ] ; do list ; done
    

    Example

    $ select choice in a b c d e q; do echo $REPLY" - chosen = "$choice; break; done
    1) a
    2) b
    3) c
    4) d
    5) e
    6) q
    #? 4
    4 - chosen = d
    
  • Sequences
    /usr/bin/seq

    Print a sequence of numbers.

    seq [option] last
    seq [option] first last
    seq [option] first increment last
    
    Common options:
     -f | --format=FORMAT               Use printf style floating-point FORMAT
     -s | --separator=STRING            Use STRING to separate numbers (default: \n)
     -w | --equal-width                 Equalise width by padding with leading zeroes
    

    Print 3 to 9 as floating point, 2 decimal places, pad result to 6 characters long

    $ seq -f %6.2f 3 9
      3.00
      4.00
      5.00
      6.00
      7.00
      8.00
      9.00
    

    Print 3 to 9 as floating point, at intervals of 3, 2 decimal places

    $ seq -f %.2f 3 3 9
    3.00
    6.00
    9.00
    

    Print the sequence to 3

    $ seq 3
    1
    2
    3
    

    Use specified separator between numbers 1 - 15 in steps of 3

    $ seq -s "-----" 1 3 15
    1-----4-----7-----10-----13
    
  • Test a condition
    test
    • A shell built-in command.
    • Returns 0 (True) or 1 (False) depending on the evaluation of an expression expr.
    • Can examine the return value by displaying $?.

    Various forms:

    test expr

    Using the test command.

    [ expr ]

    Somewhat unwieldy with its requirement for escaping and the difference between string and arithmetic comparisons

    (( expr ))

    Evaluates an arithmetic expression, returns 1 if expr = 0, or 0 if expression = non-zero. Do not need to escape operators between (( and ))

    [[ expr ]]

    Can do arithmetic tests as well however if < and > operators are not in a (( compound )) they will compare the operands as strings

    List of test operands

    -b file                    true if file is a block device
    -c file                    true if file is a character device
    -d file                    true if file is a directory
    -e file                    true if file exists
    -f file                    true if file is a regular file
    -g file                    true if file exists and has setgid bit set
    -G file                    true if file is owned by the effective group id
    -h | -L file               true if file is a symbolic link
    -k file                    true if file has sticky bit set
    -n string                  true if not null
    -N file                    true if modified since last read
    -O file                    true if file is owned by the effective user id
    -p file                    true if file is a named pipe
    -r file                    true if file is readable
    -s file                    true if file is not empty
    -S file                    true if file is a socket
    -t N                       file descriptor N points to a terminal
    -u file                    true if file has its setuid set
    -w file                    true if file is writeable
    -x file                    true if file is executable or a Dir and is searchable
    -z string                  true if length is zero
    
    stringA == stringB         equals
    stringA = stringB          posix equals
    stringA != stringB         does not match
    stringA =~ regexp          matches regexp
    exprA -a exprB             A and B is true
    stringA &amp; stringB          sorts before B lexicographically
    stringA > stringB          sorts after
    exprA -eq exprB            arithmetic equals
    exprA -ne exprB            arithmetic not equals
    exprA -lt exprB            less than
    exprA -le exprB            less than or equals
    exprA -ge exprB            greater than or equals
    exprA -o exprB             A or B is true
    exprA -gt exprB            greater than
    

    Is a directory and is writable

    $ [[ ( -d "$HOME" ) && ( -w "$HOME" ) ]] && echo "home is a writable directory"
    home is a writable directory
    

    Does string match patteren

    $ [[ "abc def .d,x--" == a[abc]*\ ?d* ]]
    $ echo $?
    0                            (True)
    

    'a[abc]*' matches an 'a' followed by an 'a' or 'b' or 'c' then any number of characters. '\' terminates the first pattern, next ' ?d*' matches a space followed by a single character then 'd' followed by any number of characters.

    $ [[ "acz defghhhhg cd,x-- jki" == a[abc]*\ ?d* ]];echo $?
    0                            (True)
    

    Matches "ac any-number-of-chars space single-character d any-number-of-chars".

    $ [[ "abc def d,x" == a[abc]*\ ?d* || (( 3 > 2 )) ]];echo $?
    0                            (True)
    

    '||' is a logical OR, arithmetic expression (( 3 > 2 )) always true. String expression is false as no space followed by a single character before either 'd'.

    $ [[ "abc def d,x" == a[abc]*\ ?d* && 3 -gt 2 ]];echo $?
    1                            (False)
    

    && logical AND. Arithmetic expression is true but String expression is false.

    If test is true echo true else echo false

    $ test 3 -gt 4 && echo True || echo false
    false
    

    Be sure about the return - all statemensts are TRUE

    $ test "abc"  != "def";echo $?
    0
    
    $ [ "abc" != "def" ];echo $?
    0
    
    $ [[ "abc" != "def" ]];echo $?
    0
    
    $ (( "abc" != "def" ));echo $?
    1
    

    (( .. )) returns one if an arithmetic expression = 0, because the expression is true it evaluates to 0.

    An arithmetic example

    $ let x=2 y=2**3 z=y*3                    (sets x=2, y=8, z=24)
    
    $ (( w=(y/x) + ( (~ ++x) & 0x0f ) ))      (sets x=3, y=8, w=16)      
    

    Explained:

    (y/x)              = 4     (8/2)
                     +
    (~ ++x) & 0x0f )   = 12
                         --
                         16
                         --
    
    (~ ++x) = 'C's bitwise 1's complement of x once x has been incremented
            = 2 + 1 = 3 
            = 0011  (binary for 3)
              1111  1's complement
              ----
            = 1100
    0x0f    = 1111  (binary for 15)
              ----
              1100  & is the bitwise AND operator
              ----
            = 12
    

    Bitwise AND, any bit that is 0 in either operand causes the corresponding bit to become 0.