Shell Scripting

28 Notes
+ Change string to uppercase or lowercase (March 18, 2024, 5 p.m.) x="HELLO" echo $x # HELLO y=${x,,} echo $y # hello z=${y^^} echo $z # HELLO

+ Command - wait (Jan. 27, 2023, 1:40 p.m.)

wait: Without any parameters, the wait command waits for all background processes to finish before continuing the script. --------------------------------------------------------------------------------- wait <process or job ID> An added PID or job ID waits for a specific process to end before continuing the script. --------------------------------------------------------------------------------- wait -n Waits for only the following background process to complete and returns an exit status. --------------------------------------------------------------------------------- wait -f Terminating a program first waits for the background task to finish before quitting. --------------------------------------------------------------------------------- Example: # Create a simple background process: sleep 10 & # Confirm the job is running in the background with: jobs -l # Use the wait command without any parameters to pause until process completion: wait # The terminal waits for the background process to finish. # After 10 seconds (due to sleep 10), the console prints a Done message. --------------------------------------------------------------------------------- wait %1 Pauses for process 1 to complete. ---------------------------------------------------------------------------------

+ Chaining Operators (Jan. 27, 2023, 12:54 p.m.) ---------------------------------------------------------------------------- Ampersand Operator (&) The function of ‘&‘ is to make the command run in the background. Just type the command followed by a white space and ‘&‘. You can execute more than one command in the background, in a single go. Run one command in the background: tecmint@localhost:~$ ping ­c5 & Run two commands in the background, simultaneously: root@localhost:/home/tecmint# apt-get update & apt-get upgrade & ---------------------------------------------------------------------------- semi-colon Operator (;) The semi-colon operator makes it possible to run, several commands in a single go and the execution of the command occurs sequentially. root@localhost:/home/tecmint# apt-get update ; apt-get upgrade ; mkdir test The above command combination will first execute the update instruction, then the upgrade instruction, and finally will create a ‘test‘ directory under the current working directory. ---------------------------------------------------------------------------- AND Operator (&&) The AND Operator (&&) would execute the second command only if the execution of the first command SUCCEEDS, i.e., the exit status of the first command is 0. This command is very useful in checking the execution status of the last command. For example, I want to visit the website using the links command, in the terminal but before that, I need to check if the host is live or not. root@localhost:/home/tecmint# ping -c3 && links ---------------------------------------------------------------------------- OR Operator (||) The OR Operator (||) is much like an ‘else‘ statement in programming. The above operator allows you to execute the second command only if the execution of the first command fails, i.e., the exit status of the first command is ‘1‘. For example, I want to execute "apt-get update" from a non-root account and if the first command fails, then the second ‘links‘ command will execute. tecmint@localhost:~$ apt-get update || links In the above command, since the user was not allowed to update the system, it means that the exit status of the first command is ‘1’, and hence the last command ‘links‘ gets executed. What if the first command is executed successfully, with an exit status ‘0‘? Obviously! The second command won’t execute. tecmint@localhost:~$ mkdir test || links Here, the user creates a folder ‘test‘ in his home directory, for which the user is permitted. The command executed successfully giving an exit status ‘0‘ and hence the last part of the command is not executed. ---------------------------------------------------------------------------- NOT Operator (!) The NOT Operator (!) is much like an ‘except‘ statement. This command will execute all except the condition provided. To understand this, create a directory ‘tecmint‘ in your home directory and ‘cd‘ to it. tecmint@localhost:~$ mkdir tecmint tecmint@localhost:~$ cd tecmint Next, create several types of files in the folder ‘tecmint‘. touch a.doc b.doc a.pdf b.pdf a.xml b.xml a.html b.html Now delete all the files except ‘html‘ file all at once, in a smart way. rm -r !(*.html) ---------------------------------------------------------------------------- AND – OR operator (&& – ||) The above operator is actually a combination of the ‘AND‘ and ‘OR‘ operators. It is much like an ‘if-else‘ statement. For example, let’s do a ping to If success echo ‘Verified‘ else echo ‘Host Down‘. ping -c3 && echo "Verified" || echo "Host Down" ---------------------------------------------------------------------------- PIPE Operator (|) This PIPE operator is very useful where the output of the first command acts as an input to the second command. For example, pipeline the output of ‘ls -l‘ to ‘less‘ and see the output of the command. tecmint@localhost:~$ ls -l | less ---------------------------------------------------------------------------- Command Combination Operator {} Combine two or more commands, the second command depends upon the execution of the first command. For example, check if a directory ‘bin‘ is available or not, and output the corresponding output. tecmint@localhost:~$ [ -d bin ] || { echo Directory does not exist, creating directory now.; mkdir bin; } && echo Directory exists. ---------------------------------------------------------------------------- Precedence Operator () The Operator makes it possible to execute commands in precedence order. Command_x1 && Command_x2 || Command_x3 && Command_x4. In the above pseudo command, what if the Command_x1 fails? Neither of the Command_x2, Command_x3, or Command_x4 would be executed, for this, we use Precedence Operator, as: (Command_x1 && Command_x2) || (Command_x3 && Command_x4) In the above pseudo command, if Command_x1 fails, Command_x2 also fails but still, Command_x3 and Command_x4 execute depending upon the exit status of Command_x3. ---------------------------------------------------------------------------- Concatenation Operator (\) The Concatenation Operator (\) as the name specifies, is used to concatenate large commands over several lines in the shell. For example, The below command will open a text file test(1).txt. tecmint@localhost:~/Downloads$ nano test\(1\).txt ----------------------------------------------------------------------------

+ Shell piping (Jan. 27, 2023, 12:52 p.m.)

In Linux, a pipeline is a mechanism that allows two or more processes to be combined or executed concurrently. That means the process output will be handled as an input for the next one, and so on. It's not called a pipeline for anything: It refers to the concept of a process flow being channeled through a pipe from a source to a destination. command1 | command2 | command3

+ File Descriptors 2> 2>&1 &>> (Jan. 27, 2023, 12:30 p.m.)

What are file descriptors? File Descriptors are integers (numbers) that act as unique identifiers for an open file (or other I/O resources) in a Linux system. In Unix-like systems, everything is a file descriptor or a process. Everything can have a file descriptor. It is important and useful to understand how the so-called three standard file descriptors or standard streams work because all processes use these channels for input and output operations. User interactions with the system are input through standard input (stdin), which is channel/stream 0, usually by using a keyboard. Then, any command executed through an interactive shell connects to a text terminal on which the shell is running and sends the output through either standard output (stdout), which is channel/stream 1. if it is OK, or through standard error (stderr), which is channel/stream 2 if it is not OK. The stdout is usually the terminal displayed by the monitor. There are other channels and streams (3 and up) that any process can use and don't have a default input or output. ----------------------------------------------------------------------------------------------------- Shell I/O redirection: You can manipulate and change the default behavior of these three basic file descriptors by leveraging redirection and pipelines. For example, you can change your input from a keyboard to a file. Instead of getting messages in your terminal, you can redirect them to a file or even discard error messages instead of seeing them on your monitor. You can also redirect your output to the terminal and a file simultaneously. You may even process a command output as an input to another command. ----------------------------------------------------------------------------------------------------- There are three redirectors to work with: >, >>, and < ----------------------------------------------------------------------------------------------------- Redirection with > command > file: Sends standard output to <file> command 2> file: Sends error output to <file> command 2>&1: Sends error output to standard output command > file 2>&1: Sends standard output and the error output to a file command &> file: Sends standard output and the error output to a file command 2>&1 > file: Sends error output to standard input and the standard input to a file Append with >> command >> file: Appends standard output to a file command 2>> file: Appends error output to a file command >> file 2>&1: Appends standard output and error output to a file command &>> file: Appends standard output and error output to a file command 2>&1 >> file: Sends error output to standard input and appends standard input to a file Redirect with < command < input: Feeds a command input from <input> command << input: Feeds a command or interactive program with a list defined by a delimiter; this is known as a here-document (heredoc) command <<< input: Feeds a command with <input>; this is known as a here-string ----------------------------------------------------------------------------------------------------- Examples: # Redirect the standard output for a given command to a file: echo "Enable Sysadmin" > myfile cat myfile Enable Sysadmin # Redirect error output for a given command to a file: ls /root 2> myfile cat myfile ls: cannot open directory '/root': permission denied # Redirect error output for a given command to the standard output, the terminal: ls /root 2>&1 ls: cannot open directory 'root/': Permission denied # Redirect both standard output and error output for a given command to a file: find /usr -name ls > myfile 2>&1 find /usr -name ls &> myfile -----------------------------------------------------------------------------------------------------

+ Command "read" (Jan. 27, 2023, 11:18 a.m.)

Bash has no built-in function to take the user’s input from the terminal. The "read" command of Bash is used to take the user’s input from the terminal. This command has different options to take input from the user in different ways. Multiple inputs can be taken using the single read command. ------------------------------------------------------------------------------------- read [options] [var1, var2, var3…] ------------------------------------------------------------------------------------- -d <delimiter> It is used to take the input until the delimiter value is provided. -n <number> It is used to take the input of a particular number of characters from the terminal and stop taking the input earlier based on the delimiter. -N <number> It is used to take the input of a particular number of characters from the terminal, ignoring the delimiter. -p <prompt> It is used to print the output of the prompt message before taking the input. -s It is used to take the input without an echo. This option is mainly used to take the input for the password input. -a It is used to take the input for the indexed array. -t <time> It is used to set a time limit for taking the input. -u <file descriptor> It is used to take the input from the file. -r It is used to disable the backslashes. ------------------------------------------------------------------------------------- Using the "read" Command without Any Option and variable: If no variable is used with the read command, the input value is stored in the $REPLY variable echo "Enter your favorite color: " read echo "Your favorite color is $REPLY" ------------------------------------------------------------------------------------- Using Read Command with a Variable: echo "Enter the product name: " read item echo "Enter the color variations of the product: " read color1 color2 color3 echo "The product name is $item." echo "Available colors are $color1, $color2, and $color3." ------------------------------------------------------------------------------------- Using Read Command with -p Option: read -p "Enter the book name: " book echo "Book name: $book" ------------------------------------------------------------------------------------- Using Read Command with -s Option: read -sp "Enter your password: " password ------------------------------------------------------------------------------------- Using Read Command with -a Option: echo "Enter the country names: " read -a countries # country1, country2, country3, country4 echo "Country names are:" for country in ${countries[@]} do echo $country done ------------------------------------------------------------------------------------- Using Read Command with -n Option: echo "Enter the product code: " # Take the input of five characters read -n 5 code echo "" # To add a new line echo "The product code is $code" ------------------------------------------------------------------------------------- Using Read Command with -t Option: echo -n "Write the result of 10-6: " # Take the input of five characters read -t 3 answer # The prompt will wait for 3 for seconds until the user enters a value -------------------------------------------------------------------------------------

+ Use background and foreground color together (Jan. 10, 2021, 3:19 p.m.)

To set both the foreground and background colors at once, use the form: echo -e "\e[S;FG;BGm" For example: echo -e "\e[1;97;41m" (bold white foreground on red background)

+ Colors (Jan. 10, 2021, 3:08 p.m.)

# Reset Color_Off='\033[0m' # Text Reset # Regular Colors Black='\033[0;30m' # Black Red='\033[0;31m' # Red Green='\033[0;32m' # Green Yellow='\033[0;33m' # Yellow Blue='\033[0;34m' # Blue Purple='\033[0;35m' # Purple Cyan='\033[0;36m' # Cyan White='\033[0;37m' # White # Bold BBlack='\033[1;30m' # Black BRed='\033[1;31m' # Red BGreen='\033[1;32m' # Green BYellow='\033[1;33m' # Yellow BBlue='\033[1;34m' # Blue BPurple='\033[1;35m' # Purple BCyan='\033[1;36m' # Cyan BWhite='\033[1;37m' # White # Underline UBlack='\033[4;30m' # Black URed='\033[4;31m' # Red UGreen='\033[4;32m' # Green UYellow='\033[4;33m' # Yellow UBlue='\033[4;34m' # Blue UPurple='\033[4;35m' # Purple UCyan='\033[4;36m' # Cyan UWhite='\033[4;37m' # White # Background On_Black='\033[40m' # Black On_Red='\033[41m' # Red On_Green='\033[42m' # Green On_Yellow='\033[43m' # Yellow On_Blue='\033[44m' # Blue On_Purple='\033[45m' # Purple On_Cyan='\033[46m' # Cyan On_White='\033[47m' # White # High Intensity IBlack='\033[0;90m' # Black IRed='\033[0;91m' # Red IGreen='\033[0;92m' # Green IYellow='\033[0;93m' # Yellow IBlue='\033[0;94m' # Blue IPurple='\033[0;95m' # Purple ICyan='\033[0;96m' # Cyan IWhite='\033[0;97m' # White # Bold High Intensity BIBlack='\033[1;90m' # Black BIRed='\033[1;91m' # Red BIGreen='\033[1;92m' # Green BIYellow='\033[1;93m' # Yellow BIBlue='\033[1;94m' # Blue BIPurple='\033[1;95m' # Purple BICyan='\033[1;96m' # Cyan BIWhite='\033[1;97m' # White # High Intensity backgrounds On_IBlack='\033[0;100m' # Black On_IRed='\033[0;101m' # Red On_IGreen='\033[0;102m' # Green On_IYellow='\033[0;103m' # Yellow On_IBlue='\033[0;104m' # Blue On_IPurple='\033[0;105m' # Purple On_ICyan='\033[0;106m' # Cyan On_IWhite='\033[0;107m' # White

+ Environment Variables (Oct. 30, 2020, 3:52 p.m.)

echo ${USER} echo $USER --------------------------------------------------------- echo $(hostname) ---------------------------------------------------------

+ Remove / Replace characters from end of line (Feb. 28, 2020, 12:26 p.m.)

| tr ',' ' '

+ awk (Jan. 17, 2020, 10:08 p.m.)

Insert spaces in print statement: awk ‘{print $1 ” ” $3 ” ” $5}’ FILE -----------------------------------------------------------------

+ Check if Postgres database already exist (Jan. 11, 2020, 1:23 p.m.)

if [[ "$(psql -U postgres -lqt | cut -d \| -f 1 | grep -w mohsendb | xargs)" == 'mohsendb' ]]; then psql -U postgres -c "drop database mohsendb" fi Description: psql -l outputs database details in a tabular list. The -t flag removes headers and footers. cut -d \| -f 1 splits the output by the vertical pipe | character (escaped from the shell with a backslash) and selects field 1. grep -w matches whole words xargs trims the whitespaces around the database name

+ Date (April 28, 2018, 11:06 a.m.)

# Today date +%Y-%m-%d ----------------------------------------------------- # 3 days ago date --date="-3 days" +%Y-%m-%d ----------------------------------------------------- # Tomorrow date --date="1 days" +%Y-%m-%d ----------------------------------------------------- # Next Friday date --date="next Friday" +%Y-%m-%d ----------------------------------------------------- date --date='tomorrow' date --date='1 day' date --date='10 day' date --date='10 week' date --date='10 month' date --date='10 year' ----------------------------------------------------- date --date='yesterday' date --date='1 day ago' date --date='10 day ago' date --date='10 week ago' date --date='10 month ago' date --date='10 year ago' ----------------------------------------------------- date --date='fortnight' date --date='5 fortnight' date --date='fortnight ago' date --date='5 fortnight ago' date --date='2 hour' date --date='2 hour ago' date --date='20 minute' date --date='20 minute ago' ----------------------------------------------------- To print the date of this Friday, enter: date --date='this Friday' ## OR ## date --date='next Friday' ----------------------------------------------------- Days of the week may be spelled out in full: Sunday, Monday, Tuesday, Wednesday, Thursday, Friday or Saturday. Days may be abbreviated to their first three letters, optionally followed by a period date --date='this Fri' ## OR ## date --date='next Fri.' ----------------------------------------------------- You can also move forward supplementary weeks as follow: date --date='2 Fri' ## OR ## date --date='second Fri.' ## OR ## date --date='Second Friday' #### #### last DAY or next DAY move one week before or after the day that DAY by itself #### date --date='last Friday' date --date='next Friday' -----------------------------------------------------

+ String Comparison (May 15, 2018, 3:18 p.m.)

if ! [[ ${DATABASE} =~ ^(template0|template1|postgres)$ ]] ------------------------------------------------------ if ! [[ ${DATABASE} =~ ^(template0|template1|postgres)$ ]] ------------------------------------------------------

+ ANSI escape codes (March 18, 2018, 1:52 a.m.)

Black 0;30 Dark Gray 1;30 Red 0;31 Light Red 1;31 Green 0;32 Light Green 1;32 Brown/Orange 0;33 Yellow 1;33 Blue 0;34 Light Blue 1;34 Purple 0;35 Light Purple 1;35 Cyan 0;36 Light Cyan 1;36 Light Gray 0;37 White 1;37 Examples: RED='\033[31m' GREEN='\033[32m' BROWN_ORANGE='\033[33m' Usage: echo -e "${BROWN_ORANGE}No space!"

+ Understanding find -exec option (curly braces & plus sign) (March 15, 2018, 10:12 p.m.)

The curly braces will be replaced by the results of the find command, and the chmod will be run on each of them. The + makes find attempt to run as few commands as possible (so, chmod 775 file1 file2 file3 as opposed to chmod 755 file1,chmod 755 file2,chmod 755 file3). Without them, the command just gives an error. This is all explained in man find: -exec command ; Execute command; true if 0 status is returned. All following arguments to find are taken to be arguments to the command until an argument consisting of `;' is encountered. The string `{}' is replaced by the current file name being processed everywhere it occurs in the arguments to the command, not just in arguments where it is alone, as in some versions of find. -exec command {} + This variant of the -exec action runs the specified command on the selected files, but the command line is built by appending each selected file name at the end; the total number of invocations of the command will be much less than the number of matched files. ------------------------------------------------------------- “Obviously” -exec … must be terminated with either a semicolon (;) or a plus sign (+). Semicolon is a special character in the shell (or, at least, every shell I’ve ever used), so, if it is to be used as part of the find command, it must be escaped or quoted (\;, ";", or ';').

+ Total Physical Memory RAM (Aug. 7, 2017, 12:57 p.m.)

awk '/MemTotal/ {print $2}' /proc/meminfo

+ How to grep in dmesg (Aug. 6, 2017, 3:43 p.m.)

dmesg | grep "the BIOS has corrupted hw-PMU resources"

+ Is OS running on a virtual machine (Aug. 6, 2017, 3:24 p.m.)

egrep "hypervisor" /proc/cpuinfo if [ $? -eq 0 ]; then echo "This OS is running on a virtual machine."; else echo "This OS is NOT running on a virtual machine."; fi

+ Check if CPU supports Virtualization (Aug. 5, 2017, 3:41 p.m.)

egrep '(vmx|svm)' /proc/cpuinfo if [ $? -eq 0 ]; then echo "supported"; else echo "not supported"; fi

+ Close a process by its saved PID in file (Aug. 4, 2017, 11:39 a.m.)

if [ -f /tmp/ ] then /bin/kill $(cat /tmp/ fi

+ Get & Save running application PID in a file (Aug. 4, 2017, 11:38 a.m.)

/usr/bin/transmission-gtk > /dev/null & echo $! > /tmp/ exit

+ For Loop (July 4, 2017, 11:44 p.m.)

for i in {06..25}; do mkdir The\ Simpsons\ -\ Season\ $i; done ---------------------------------------------------------- for i in 1 2 3 4 5 do echo "Welcome $i times" done ---------------------------------------------------------- for i in {1..5} do echo "Welcome $i times" done ---------------------------------------------------------- echo "Bash version ${BASH_VERSION}..." for i in {0..10..2} do echo "Welcome $i times" done ----------------------------------------------------------

+ Infinite Loop (July 4, 2017, 11:46 p.m.)

for (( ; ; )) do echo "infinite loops [ hit CTRL+C to stop]" done

+ Copy files and take some part of the names (July 2, 2017, 10:38 a.m.)

for f in *.sample; do cp "$f" "${f/.sample/}"; done

+ Remove spaces from file names (April 22, 2017, 10:38 a.m.)

rename "s/ //g" *

+ Batch rename files (Dec. 4, 2014, 12:38 p.m.)

for i in *.mpa; do mv "$i" "${i/.mpa}".mp3; done ------------------------------------------------------------ Remove the character (comma , ) from filenames: for i in *; do mv "$i" "${i//,/ -}"; done; The substitution function has two slashes (//) to replace every occurrence. ----------------------------------------------------------------------------- Remove the "abc" from mp3 filenames: for file in *.mp3; do mv "$file" "${file/abc/}" done ----------------------------------------------------------------------------- Replace capital "MP3" to small "mp3" letters: for file in *.MP3; do mv "$file" "${file%.MP3}.mp3"; done; ----------------------------------------------------------------------------- Replace "html" with "txt": for file in *.html do mv "$file" "${file%.html}.txt" done ----------------------------------------------------------------------------- Add the "mp3" extension to the end of the files: for file in * do mv "$file" "$file.mp3" done ----------------------------------------------------------------------------- sudo apt install rename Remove the characters "a bc" from all file: rename s/a\ bc// * Remove the characters "ab c" from mp3 files: rename s/ab\ c// *.mp3 -----------------------------------------------------------------------------

+ Tutorial (Aug. 22, 2014, 8:33 a.m.)

The shell maintains a list of directories where executable files (programs) are kept, and just searches the directories in that list. If it does not find the program after searching each directory in the list, it will issue the famous command not found error message. This list of directories is called your path. You can view the list of directories with the following command: echo $PATH This will return a colon separated list of directories that will be searched if a specific path name is not given when a command is attempted. ------------------------------------------------------------------------------------- You can add directories to your path with the following command, where directory is the name of the directory you want to add: export PATH=$PATH:directory A better way would be to edit your .bash_profile file to include the above command. That way, it would be done automatically every time you log in. Most modern Linux distributions encourage a practice in which each user has a specific directory for the programs he/she personally uses. This directory is called bin and is a subdirectory of your home directory. ------------------------------------------------------------------------------------- .bashrc Though placing your aliases and shell functions in your .bash_profile will work, it is not considered good form. There is a separate file named .bashrc that is intended to be used for your custom scripts. You may notice a piece of code near the beginning of your .bash_profile that looks something like this: if [ -f ~/.bashrc ]; then . ~/.bashrc fi This script fragment checks to see if there is a .bashrc file in your home directory. If one is found, then the script will read its contents. If this code is in your .bash_profile, you should edit the .bashrc file and put your aliases and shell functions there. ------------------------------------------------------------------------------------- Here Scripts A here script (also sometimes called a here document) is an additional form of I/O redirection. It provides a way to include content that will be given to the standard input of a command. command << token content to be used as command's standard input token token can be any string of characters. I use "_EOF_" (EOF is short for "End Of File") because it is traditional, but you can use anything, as long as it does not conflict with a bash reserved word. The token that ends the here script must exactly match the one that starts it, or else the remainder of your script will be interpreted as more standard input to the command. cat <<- _EOF_ Changing the the "<<" to "<<-" causes bash to ignore the leading tabs (but not spaces) in the here script. The output from the cat command will not contain any of the leading tab characters. ------------------------------------------------------------------------------------- Environment Variables When you start your shell session, some variables are already ready for your use. They are defined in scripts that run each time a user logs in. To see all the variables that are in your environment, use the printenv command. ------------------------------------------------------------------------------------- Like constants, environment variables are given uppercase names by convention. ------------------------------------------------------------------------------------- echo "My host name is \"$HOSTNAME\"." My host name is "linuxbox".