A little of Bash Shell

Tables are inspired at tables you can find in:
O’Really, Learning the bash Shell, 3rd Edition By 

Utilities

The most wanted utilities in unix should surely be the following ones:

Utility

Purpose

cat

grep

sort

cut

sed

rt

awk

Copy input to output

Search for string in the input

Sort lines in the input

Extract columns from input

Perform editing operation on input

Translate characters in the input to other characters

Full-featured text processing language with a syntax reminiscent of C

For example the grep command is very useful to fing text inside files like shown below:

grep –Hnr “Fabio Bombardi” .

This command search for the string “Fabio Bombardi” in all files, recursively (-r) , starting from the current directory (.), showing the line number (-n) where this string is located in the found file.

I/O Redirectors and pipelines

Redirector

Purpose

<

>

Redirect the right side argument as input of the left side argument

Redirect the output of the left side argument to the left right side argument

For example if your cp command is broken you can use the cat command in the following way:

cat < file1 > file2

This behaves like the command

cp file1 file2

Redirector

Purpose

cmd1 | cmd2

Redirect the output of the left side command cmd1 to the input of the right side command cmd2

For example you can use the command

ls -l | more

to send the list the file in the current directory as the input of the more command.

It’s possible to use the I/O redirection in conjunction with pipelines like in the following example:

cut –d: -f1 < /etc/passwd | sort

this command extract the first field (f1) in the file /etc/passwd, where fields are separated by colons (-d:) and sort that first field showing it on the output.

The above redirectors are the most used. But you may found a plethora of other useful redirectors, above all for a system programmer, like those listed in the table below.

Redirector

Purpose

> file

< file

>> file

>| file

n>| file

<> file

n <> file

<< label

n> file

n< file

n>> file

n>&

n<&

n>&m

n<&m

&> file

<&-

>&-

n>&-

n<&-

Direct standard output to file

Take standard input from file

Direct standard output to file; append to file if it already exists

Force standard output to file even if noclobber is set

Force output to file from file descriptor n even if noclobber is set

Use file as both input and output for file descriptor n

Use file as both input and output for file descriptor n

Here-document; see text

Direct file descriptor n to file

Take file descriptor n from file

Direct file descriptor n to file; append to file if it already exists

Duplicate standard output to file descriptor n

Duplicate standard input from file descriptor n

File descriptor n is made to be a copy of the output file descriptor m

File descriptor n is made to be a copy of the input file descriptor m

Directs standard output and standard error to file

Close the standard input

Close the standard output

Close the output from file descriptor n

Close the input from file descriptor ni

For example if you want to redirect both the standard error and the standard output of the ls command to a logfile you could type what follows:

ls > logfile 2>&1

This will redirect the standard error (2) in the same place where the standard output (1) is directed. Since standard output is redirected to logfile hence also the standard error will be redirected there too.

If you want you could get the same result also with the following commnd:

ls &> logfile

Special characters and quoting

Special characters

Character

Purpose

~

#

$

&

*

(

)

\

|

[

]

{

}

;

<

>

/

?

!

Home directory

Comment

Variable expression

Background job

String wildcar

Start subshell

End subshell

Quote next character

Pipe

Start character-set wildcard

End Character-set wildcard

Smart command block

End command block

Shell command separator

Strong quote

Weak quote

Input redirect

Ouput redirect

Pathname directory separator

Single-character wildcard

Pipeline logical NOT

Quoting

When you want to use special character without their special meaning you have to use quoting. If you surround a string of characters with single quotation marks (or quotes), you strip all characters within the quotes of any special meaning they might have.

For example if you want to print in standard output the string “2*3 > 5 is an invalid inequality” you have to type the following command:

echo ‘2 * 3 > 5 is an invalid inequality’

Otherwise you will get a file named 5 containing the list of 2, all the files in the current directory, and the string “3 is an invalid inequality”!

Another way to change the meaning of a character is to precede it with a backslash (\). This is called backslash-escaping the character.

Quoting and find

The most useful use of the quoting is along with the command find.

If you want to find all the character with .c extension you have to act like below:

find . -name ‘*.c’

Control key setting

To know your control-key setting you can type

stty all

And you will get something like that:

erase kill werase rprnt flush lnext susp intr quit stop eof

^? ^U ^W ^R ^O ^V ^Z/^Y ^C ^\ ^S/^Q ^D

Or type

stty -a

If your Unix version derives from System III or System V (this include also Linux) to obtain a similar list of control-key.

History Expansion

In order to see the list of the history c-shell command you can type the following command:

fc -l

Command

Purpose

!

!!

!n

!-n

!string

!?string?

^string1^string2

Start a history substitution

Refers to the last command

Refers to command line n

Refers to current command line minus n

Refers to the most recent command starting with string

Refers to the most recent command containing string. The ending ? is optional.

Repeat the last command, replacing string1 with string2

Tabella 1 – event designator

For example, if your last command was the whoami command, you are able to retype it only by typing

!!

It’s also passible to refer to certain words in a previous command by the use of a word designator

Designator

Purpose

0

n

^

$

%

x-y

*

x*
x-

The zeroth (first) word in a line

The nth word in a line

The first argument (the second word)

The last argument in a line

The word matched by the most recent ?string search

A range of words from x to y. –y is synonymous with 0-y.

All words but the zeroth (fist). Synonymous with 1-$. If there is only one word on the line, an empty string is returned.

Synonymous with x-$

The words from x to the second last word

Tabella 2 – word designator

The word designator follows the event designator, separated by a colon. It’s possible, for example, repeat the previous command without arguments by typing

!!:0

Or it’s possible to repeat the previous command with different arguments

!!:0 arg0 arg1 arg2

For example if you type

!!:0 --version

(in this case the same like !! –-version)

Lat’s say that your last typed command is whoami, in this case, typing the above command it’s like you would have typed

whoami --version

Event designator may also be followed by modifiers. The modifiers follow the word designator, if there is one.

Modifier

Purpose

h

r

e

t

p

q

x

s/old/new

Remove a trailing pathname component, leaving the head

Removes a trailing suffix of the form .xxx

Removes all but the trailing suffix

Removes all leading pathname components, leaving the tail

Prints the resulting command but doesn’t execute it

Quote the substituted word, escaping further substitutions

Quote the substituted words, breaking them into words at blanks and newlines

Substitutes new for old

Tabella 3 – modifiers

For example if you have just type the following command

grep -Hnr “Fabio Bombardi” .

And you want to retype the above command but to search for “Elena Bombardi” you could type:

!!:s/Fabio/Elena

And you get the following command:

grep -Hnr “Elena Bombardi” .

Setting the X server keyboard map type

If you want to set a different map, let’s say Italian, to your X sever, you could type the following command:

setxkbmap -layout it

Patterns and Pattern Matching

Patterns are strings that can contain wildcard characters.

Operator

Meaning

${variable#pattern}

${variable##pattern}

${variable%pattern}

${variable%%pattern}

${variable/pattern/string}

${variable//pattern/string}

If the pattern matches the beginning of the variable’s value, delete the shortest part that matches and return the rest.

If the pattern matches the beginning of the variable’s value, delete the longest part that matches and return the rest.

If the pattern matches the end of the variable’s value, delete the shortest part that matches and return the rest.

If the pattern matches the end of the variable’s value, delete the longest part that matches and return the rest.

The longest match to pattern in variable is replaced by string. In the first from, only the first match is replaced. In the second form, all the matches are replaced. If the pattern begins with a #, it must match at the start of the variable. If it begins with a %, it must match with the end of the variable. If string is null, the matches are deleted. If variable is @ or *, the operation is applied to each positional parameter in turn and the expansion is the result list.

Tabella 4 – Pattern-Matching Operators

If you want some exaples here you are:

$path /home/shadowsheep/testpatterns/hello.shadowsheep.test

$path##/*/ hello.shadowsheep.test

$path#/*/ shadowsheep/testpatterns/hello.shadowsheep.test

$path%.* /home/shadowsheep/testpatterns/hello.shadowsheep

$path%%.* /home/shadowsheep/testpatterns/hello

If you want to substitute all the occurrence in a string you may want to type:

echo ${PATH//:/’\n’}

/home/usr/bin\n/usr/local/bin\n/bin\n

If you want the echo command to interpret the backslashes character ( like \n) so you may want to type:

echo –e ${PATH//:/’\n’}

/home/usr/bin

/usr/local/bin

/bin

Lessons

Lesson 1

If you want to edit a text file without open a text editor you can use the echo command in the following way:

fbombardi@linux:~/workspace/shell_commands> echo -e "a\n\

a\n\

a\n\

mount -n -t nfs -o nolock,rsize=1024,wsize=1024 172.27.30.179:/home/nfs /NetShared\n\

b\n\

b\n\

end" > test.log

In this way you create a text file named test.log whose content is the following:

a

a

a

mount -n -t nfs -o nolock,rsize=1024,wsize=1024 172.27.30.179:/home/nfs /NetShared

b

b

end

Now if you want to type the command

mount -n -t nfs -o nolock,rsize=1024,wsize=1024 172.27.30.179:/mnt/nfs /NetShared

You can use the following shortcut:

fbombardi@linux:~/workspace/shell_commands> more test.log | grep "mount -n" | sed -e s/home/mnt/

Or even better you can use this shortcut:

fbombardi@linux:~/workspace/shell_commands> grep "mount -n" test.log | sed -e s/home/mnt/

I think that this should be useful when you have a command in a very full written file, otherwise I think that you could be faster in typing the command from scratch =).

Lesson 2

Here we are again. This time we’ll go deeper in the world of the I/O redirection. Let’s say we want to get rid of all the boring messages that the command make show us during the compiling and linking time. What we have to do? The answer is simple and is located in the line below:

make > /dev/null 2>&1

In this way all the boring messages will be redirected in the “black-hole” /dev/null.

Let’s see now that we want to decide which kind of messages to display. I’ll give you a gift showing you on of my favorite shell script:

#

# My Make

# Use this program to customize the make output messages

#

# @author Fabio Bombardi

# @location Datasensor

# @version 2.1.0

# @last-updated 2k6.09.24

#

if [ -n $2 ] && [ "$2" = "-silent" ]; then

break;

else

echo -e "\n"

echo -e \#

echo -e \# My Make

echo -e \# Use this program to customize the make output messages

echo -e \#

echo -e \# @author "\t\t" Fabio Bombardi

echo -e \# "\t\t\t\t" fbombardi@datasensor.com

echo -e \# @location "\t\t" Datasensor

echo -e \# @version "\t\t" 2.1.0

echo -e \# @last-updated "\t" 2k6.09.24

echo -e \#

echo -e "\n"

fi

if [ -z $1 ]; then

echo -e "usage: mymake <type-of-massage>\n\n\t<type-of-massage>=\n\t\twarning\n\t\terror\n\t\tall\n\t\tclean\n"

exit -1

fi

case $1 in

warning )

make 2>&1 | grep "warning" | cat ;;

error )

make 2>&1 | grep "Error" | cat ;;

all )

make 2>&1 | grep "*" | cat ;;

clean)

make clean ;;

* )

echo -e "\ndefault selection: warning\n"

$0 wanrning

esac

In the above shell script the line in bold shows us only the messages containing the word “warning” and hiding all the other messages.

Lesson 3

Let’s say now that we want not only to choose between the make output messages but also to give different colors to this messages (e.g. yellow for warnings and red for error).

In this case we may want to make the following changes in the case statement:

case $1 in

warning )

GREP_COLOR="1;33"

make 2>&1 | grep --color=always "warning" | cat ;;

error )

GREP_COLOR="0;31"

make 2>&1 | grep --color=always "Error" | cat ;;

all )

make 2>&1 | grep "*" | cat ;;

clean)

make clean ;;

* )

echo -e "\ndefault selection: warning\n"

$0 warning ;;

esac

The grep command has the option flag –color that along with the variable GREP_COLOR highlights the pattern word with the color set in the variable GREP_COLOR.

With the above changes we should get something like that:

img_processing.cpp:148: warning: right-hand operand of comma has no effect

img_processing.cpp:367: warning: converting to `unsigned char’ from `float’

img_processing.cpp:477: warning: converting to `int’ from `float’

img_processing.cpp:478: warning: converting to `int’ from `float’

Colors

We could choose, of course, among a plethora of color:

Dark gray: 1;30

Blue: 0;34

Light Blue: 1;34

Green: 0;32

Light green: 1;32

Cyan: 0;36

Light cyan: 1;36

Red: 0;31

Light red: 1;31

Purple: 0;35

Light purple: 1;35

Brown: 0;33

Yellow: 1;33

Light gray: 0;37

White: 1;37

Lesson 4

Let’s say now that we want not only to highlight the keywords (warning, error) but the entire lines!

What follows is what we could do to be successful:

case $1 in

warning )

echo -e "\033[1;33m"

make 2>&1 | grep "warning" | cat

echo -e "\033[1;37m" ;;

error )

echo -e "\033[1;31m"

make 2>&1 | grep --color=always "Error" | cat

echo -e "\033[1;37m" ;;

all )

make 2>&1 | grep "*" | cat ;;

clean)

echo -e "\033[1;32m"

make clean

echo -e "\033[1;37m\n" ;;

* )

echo -e "\033[1;35m"

echo -e "\ndefault selection: warning\n"

echo -e "\033[1;37m\n"

$0 warning ;;

esac

And we’ll get the following output:

img_processing.cpp:148: warning: right-hand operand of comma has no effect

img_processing.cpp:367: warning: converting to `unsigned char’ from `float’

img_processing.cpp:477: warning: converting to `int’ from `float’

img_processing.cpp:478: warning: converting to `int’ from `float’

A little of Bash Shell