Bash Scripting Review

## Bash Scripting Basics - **Bash scripting**: Writing a sequence of Bash commands in a file, which you can then execute with Bash to run the contents of the file. - **Shebang**: The commented line at the beginning of a script (e.g., `#!/bin/bash`) that indicates what interpreter should be used for the script. ```bash #!/bin/bash ``` - **Variable assignment**: Instantiate variables using the syntax `variable_name=value`. ```bash servers=("prod" "dev") ``` - **Variable creation rules**: Create variables with `VARIABLE_NAME=VALUE` syntax. No spaces are allowed around the equal sign (`=`). Use double quotes if the value contains spaces. ```bash NAME=John MESSAGE="Hello World" COUNT=5 TEXT="The next number is, " ``` - **Variable usage**: Access variable values by placing `$` in front of the variable name. ```bash echo $NAME echo "The message is: $MESSAGE" ``` - **Variable interpolation**: Use `$variable_name` to access the value of a variable within strings and commands. ```bash TEXT="The next number is, " NUMBER=42 echo $TEXT B:$NUMBER echo $TEXT I:$NUMBER echo "Pulling $server" rsync --archive --verbose $server:/etc/nginx/conf.d/server.conf configs/$server.conf ``` - **Variable scope**: Shell scripts run from top to bottom, so variables can only be used below where they are created. ```bash NAME="Alice" echo $NAME ``` - **User input**: Use `read` to accept input from users and store it in a variable. ```bash read USERNAME echo "Hello $USERNAME" ``` - **Comments**: Add comments to your scripts using `#` followed by your comment text. - Single-line comments start with `#` and continue to the end of the line - Comments are ignored by the shell and don't affect script execution ```bash # This is a single-line comment NAME="John" # Comment at end of line ``` - **Multi-line comments**: Comment out blocks of code using colon and quotes. ```bash : ' This is a multi-line comment Everything between the quotes is ignored Useful for debugging or documentation ' ``` - **Built-in commands and help**: - Use `help` to see a list of built-in bash commands - Use `help <command>` to get information about specific built-in commands - Some commands (like `if`) are built-ins and don't have man pages - Built-in commands are executed directly by the shell rather than as external programs - Use `help function` to see information about creating functions ```bash help help if help function ``` - **Finding command locations**: Use `which` to locate where executables are installed. - Shows the full path to executable files - Useful for finding interpreter locations (like bash) - Helps verify which version of a command will be executed ```bash which bash which python which ls ``` - **Manual pages**: Use `man` to access detailed documentation for commands. - Provides comprehensive information about command usage - Shows all available options and examples - Use <kbd>arrow keys</kbd> to navigate, <kbd>q</kbd> to quit - Not all commands have manual pages (built-ins use `help` instead) ```bash man echo man ls man bash ``` - **Help flags**: Many commands support `--help` for quick help information. - Alternative to manual pages for quick reference - Shows command syntax and common options - Not all commands support this flag (some may show error) ```bash ls --help chmod --help mv --help ``` - **Echo command options**: The `echo` command supports various options: - `-e` option enables interpretation of backslash escapes - `\n` creates a new line - Empty lines are only printed when values are enclosed in quotes - Useful for creating formatted output and program titles ```bash echo -e "Line 1\nLine 2" echo "" echo -e "\n~~ Program Title ~~\n" echo "Line 1\nLine 2" ``` - **Script arguments**: Programs can accept arguments that are accessible using `$` variables. - `$*` prints all arguments passed to the script - `$@` prints all arguments passed to the script as separate quoted strings - `$<number>` accesses specific arguments by position (e.g., `$1`, `$2`, `$3`) ```bash echo $* echo $@ echo $1 echo $2 ``` ## Double Bracket Expressions `[[ ]]` - **Double bracket syntax**: Use `[[ ]]` for conditional testing and pattern matching. - Must have spaces inside the brackets and around operators - Returns exit status 0 (true) or 1 (false) based on the test result ```bash [[ $variable == "value" ]] [[ $number -gt 10 ]] [[ -f filename.txt ]] ``` - **String comparison operators**: Compare strings using various operators within `[[ ]]`. - `==` (equal): Tests if two strings are identical - `!=` (not equal): Tests if two strings are different - `<` (lexicographically less): String comparison in alphabetical order - `>` (lexicographically greater): String comparison in alphabetical order ```bash [[ "apple" == "apple" ]] [[ "apple" != "orange" ]] [[ "apple" < "banana" ]] [[ "zebra" > "apple" ]] ``` - **Numeric comparison operators**: Compare numbers using specific numeric operators. - `-eq` (equal): Numeric equality comparison - `-ne` (not equal): Numeric inequality comparison - `-lt` (less than): Numeric less than comparison - `-le` (less than or equal): Numeric less than or equal comparison - `-gt` (greater than): Numeric greater than comparison - `-ge` (greater than or equal): Numeric greater than or equal comparison ```bash [[ $number -eq 5 ]] [[ $count -ne 0 ]] [[ $age -ge 18 ]] [[ $score -lt 100 ]] ``` - **Logical operators**: Combine multiple conditions using logical operators. - `&&` (and): Both conditions must be true - `||` (or): At least one condition must be true - `!` (not): Negates the condition (makes true false, false true) ```bash [[ $age -ge 18 && $age -le 65 ]] [[ $name == "John" || $name == "Jane" ]] [[ ! -f missing_file.txt ]] ``` - **File test operators**: Test file properties and existence. - `-e file`: True if file exists - `-f file`: True if file exists and is a regular file - `-d file`: True if file exists and is a directory - `-r file`: True if file exists and is readable - `-w file`: True if file exists and is writable - `-x file`: True if file exists and is executable - `-s file`: True if file exists and has size greater than zero ```bash [[ -e /path/to/file ]] [[ -f script.sh ]] [[ -d /home/user ]] [[ -x program ]] ``` - **Pattern matching with `=~`**: Use regular expressions for advanced pattern matching. - `=~` operator enables regex pattern matching - Pattern should not be quoted when using regex metacharacters - Supports full regular expression syntax - Case-sensitive by default ```bash [[ "hello123" =~ [0-9]+ ]] [[ "email@domain.com" =~ ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$ ]] [[ "filename.txt" =~ \.txt$ ]] ``` - **Variable existence testing**: Check if variables are set or empty. - Test if variable is empty: `[[ ! $variable ]]` ```bash [[ ! $undefined_var ]] ``` ## Double Parentheses Expressions `(( ))` - **Arithmetic evaluation**: Use `(( ))` for mathematical calculations and numeric comparisons. - Evaluates arithmetic expressions using C-style syntax - Variables don't need `$` prefix inside double parentheses - Returns exit status 0 if result is non-zero, 1 if result is zero - Supports all standard arithmetic operators ```bash (( result = 10 + 5 )) (( count++ )) (( total += value )) ``` - **Arithmetic operators**: Mathematical operators available in `(( ))`. - `+` (addition): Add two numbers - `-` (subtraction): Subtract second number from first - `*` (multiplication): Multiply two numbers - `/` (division): Divide first number by second (integer division) - `%` (modulo): Remainder after division - `**` (exponentiation): Raise first number to power of second ```bash (( sum = a + b )) (( diff = x - y )) (( product = width * height )) (( remainder = num % 10 )) (( power = base ** exponent )) ``` - **Assignment operators**: Modify variables using arithmetic assignment operators. - `=` (assignment): Assign value to variable - `+=` (add and assign): Add value to variable - `-=` (subtract and assign): Subtract value from variable - `*=` (multiply and assign): Multiply variable by value - `/=` (divide and assign): Divide variable by value - `%=` (modulo and assign): Set variable to remainder ```bash (( counter = 0 )) (( counter += 5 )) (( total -= cost )) (( area *= 2 )) (( value /= 3 )) ``` - **Increment and decrement operators**: Modify variables by one. - `++variable` (pre-increment): Increment before use - `variable++` (post-increment): Increment after use - `--variable` (pre-decrement): Decrement before use - `variable--` (post-decrement): Decrement after use ```bash (( ++counter )) (( index++ )) (( --remaining )) (( attempts-- )) ``` - **Comparison operators**: Compare numbers using arithmetic comparison. - `==` (equal): Numbers are equal - `!=` (not equal): Numbers are not equal - `<` (less than): First number is less than second - `<=` (less than or equal): First number is less than or equal to second - `>` (greater than): First number is greater than second - `>=` (greater than or equal): First number is greater than or equal to second ```bash (( age >= 18 )) (( score < 100 )) (( count == 0 )) (( temperature > freezing )) ``` - **Logical operators**: Combine arithmetic conditions. - `&&` (and): Both conditions must be true - `||` (or): At least one condition must be true - `!` (not): Negates the condition ```bash (( age >= 18 && age <= 65 )) (( score >= 90 || extra_credit > 0 )) (( !(count == 0) )) ``` - **Bitwise operators**: Perform bit-level operations on integers. - `&` (bitwise AND): AND operation on each bit - `|` (bitwise OR): OR operation on each bit - `^` (bitwise XOR): XOR operation on each bit - `~` (bitwise NOT): Invert all bits - `<<` (left shift): Shift bits to the left - `>>` (right shift): Shift bits to the right ```bash (( result = a & b )) (( flags |= new_flag )) (( shifted = value << 2 )) ``` - **Conditional (ternary) operator**: Use `condition ? true_value : false_value` syntax. - Provides a concise way to assign values based on conditions - Similar to the ternary operator in C-style languages - Evaluates condition and returns one of two values ```bash (( result = (score >= 60) ? 1 : 0 )) (( max = (a > b) ? a : b )) (( sign = (num >= 0) ? 1 : -1 )) ``` - **Command substitution with arithmetic**: Use `$(( ))` to capture arithmetic results. - Returns the result of the arithmetic expression as a string - Can be used in assignments or command arguments - Useful for calculations that need to be used elsewhere ```bash result=$(( 10 + 5 )) echo "The answer is $(( a * b ))" array_index=$(( RANDOM % array_length )) ``` ## Control Flow and Conditionals - **Conditional statements**: Use `if` statements to execute code based on conditions. - Basic syntax: `if [[ CONDITION ]] then STATEMENTS fi` - Full syntax: `if [[ CONDITION ]] then STATEMENTS elif [[ CONDITION ]] then STATEMENTS else STATEMENTS fi` - Can use both `[[ ]]` and `(( ))` expressions for different types of conditions - **elif (else if)**: Optional, can be repeated multiple times to test additional conditions in sequence - **else**: Optional, executes when all previous conditions are false - Can mix double parentheses `(( ... ))` and double brackets `[[ ... ]]` in same conditional chain ```bash if (( NUMBER <= 15 )) then echo "B:$NUMBER" elif [[ $NUMBER -le 30 ]] then echo "I:$NUMBER" elif (( NUMBER < 46 )) then echo "N:$NUMBER" elif [[ $NUMBER -lt 61 ]] then echo "G:$NUMBER" else echo "O:$NUMBER" fi ``` ## Command Execution and Process Control - **Command separation**: Use semicolon (`;`) to run multiple commands on a single line. - Commands execute sequentially from left to right - Each command's exit status can be checked individually ```bash [[ 4 -ge 5 ]]; echo $? ls -l; echo "Done" ``` - **Exit status**: Every command has an exit status that indicates success or failure. - Access exit status of the last command with `$?` - Exit status `0` means success (true/no errors) - Any non-zero exit status means failure (false/errors occurred) - Common error codes: `127` (command not found), `1` (general error) ```bash echo $? [[ 4 -le 5 ]]; echo $? ls; echo $? bad_command; echo $? ``` - **Subshells and command substitution**: Different uses of parentheses for execution contexts. - Single parentheses `( ... )` create a subshell - `$( ... )` performs command substitution - Subshells run in separate environments and don't affect parent shell variables ```bash ( cd /tmp; echo "Current dir: $(pwd)" ) current_date=$(date) file_count=$(ls | wc -l) echo "Today is $current_date" echo "Found $file_count files" ``` - **Sleep command**: Pause script execution for a specified number of seconds. - Useful for creating delays in scripts - Can be used with decimal values for subsecond delays ```bash sleep 3 sleep 0.5 sleep 1 ``` ## Loops - **While loops**: Execute code repeatedly while a condition is true. - Syntax: `while [[ CONDITION ]] do STATEMENTS done` ```bash I=5 while [[ $I -ge 0 ]] do echo $I (( I-- )) sleep 1 done ``` - **Until loops**: Execute code repeatedly until a condition becomes true. - Syntax: `until [[ CONDITION ]] do STATEMENTS done` ```bash until [[ $QUESTION =~ \?$ ]] do echo "Please enter a question ending with ?" read QUESTION done until [[ $QUESTION =~ \?$ ]] do GET_FORTUNE again done ``` - **For loops**: Iterate through arrays or lists using `for` loops with `do` and `done` to define the loop's logical block. ```bash for server in "${servers[@]}" do echo "Processing $server" done for (( i = 1; i <= 5; i++ )) do echo "Number: $i" done for (( i = 5; i >= 1; i-- )) do echo "Countdown: $i" done for i in {1..5} do echo "Count: $i" done ``` ## Arrays - **Arrays**: Store multiple values in a single variable. - Create arrays with parentheses: `ARRAY=("value1" "value2" "value3")` - Access elements by index: `${ARRAY[0]}`, `${ARRAY[1]}` - Access all elements: `${ARRAY[@]}` or `${ARRAY[*]}` - Array indexing starts at 0 ```bash RESPONSES=("Yes" "No" "Maybe" "Ask again later") echo ${RESPONSES[0]} # Yes echo ${RESPONSES[1]} # No echo ${RESPONSES[5]} # Index 5 doesn't exist; empty string echo ${RESPONSES[@]} # Yes No Maybe Ask again later echo ${RESPONSES[*]} # Yes No Maybe Ask again later ``` - **Array inspection with declare**: Use `declare -p` to view array details. - Shows the array type with `-a` flag - Displays all array elements and their structure ```bash ARR=("a" "b" "c") declare -p ARR # ARR=([0]="a" [1]="b" [2]="c") ``` - **Array expansion**: Use `"${array_name[@]}"` syntax to expand an array into individual elements. ```bash for server in "${servers[@]}" ``` ## Functions - **Functions**: Create reusable blocks of code. - Define with `FUNCTION_NAME() { STATEMENTS }` - Call by using the function name - Can accept arguments accessible as `$1`, `$2`, etc. ```bash GET_FORTUNE() { echo "Ask a question:" read QUESTION } GET_FORTUNE ``` - **Function arguments**: Functions can accept arguments just like scripts. - Arguments are passed when calling the function - Access arguments inside function using `$1`, `$2`, etc. - Use conditional logic to handle different arguments ```bash GET_FORTUNE() { if [[ ! $1 ]] then echo "Ask a yes or no question:" else echo "Try again. Make sure it ends with a question mark:" fi read QUESTION } GET_FORTUNE GET_FORTUNE again ``` ## Random Numbers and Mathematical Operations - **Random numbers**: Generate random values using the `$RANDOM` variable. - `$RANDOM` generates numbers between 0 and 32767 - Use modulo operator to limit range: `$RANDOM % 75` - Add 1 to avoid zero: `$(( RANDOM % 75 + 1 ))` - Must use `$(( ... ))` syntax for calculations with `$RANDOM` ```bash NUMBER=$(( RANDOM % 6 )) DICE=$(( RANDOM % 6 + 1 )) BINGO=$(( RANDOM % 75 + 1 )) echo $(( RANDOM % 10 )) ``` - **Random array access**: Use random numbers to access array elements randomly. - Generate random index within array bounds - Use random index to access array elements - Useful for random selections from predefined options ```bash RESPONSES=("Yes" "No" "Maybe" "Outlook good" "Don't count on it" "Ask again later") N=$(( RANDOM % 6 )) echo ${RESPONSES[$N]} ``` - **Modulo operator**: Use `%` to get the remainder of division operations. - Essential for limiting random number ranges - Works with `$RANDOM` to create bounded random values - `RANDOM % n` gives numbers from 0 to n-1 ```bash echo $(( 15 % 4 )) echo $(( RANDOM % 100 )) echo $(( RANDOM % 10 + 1 )) ``` ## Environment and System Information - **Environment variables**: Predefined variables available in the shell environment. - `$RANDOM`: Generates random numbers between 0 and 32767 - `$LANG`: System language setting - `$HOME`: User's home directory path - `$PATH`: Directories searched for executable commands - View all with `printenv` or `declare -p` ```bash echo $RANDOM echo $HOME echo $LANG printenv ``` - **Variable inspection**: Use `declare` to view and work with variables. - `declare -p`: Print all variables and their values - `declare -p VARIABLE`: Print specific variable details - Shows variable type (string, array, etc.) and attributes ```bash declare -p declare -p RANDOM declare -p MY_ARRAY ``` - **Command types**: Different categories of commands available in bash. - **Built-in commands**: Executed directly by the shell (e.g., `echo`, `read`, `if`) - **External commands**: Binary files in system directories (e.g., `ls`, `sleep`, `bash`) - **Shell keywords**: Language constructs (e.g., `then`, `do`, `done`) - Use `type <command>` to see what type a command is ```bash type echo type ls type if type ./script.sh ``` ## File Creation and Management - **File creation**: Use `touch` to create new empty files. - Creates a new file if it doesn't exist - Updates the timestamp if the file already exists - Commonly used to create script files before editing ```bash touch script.sh touch bingo.sh touch filename.txt ``` ## Creating and Running Bash Scripts - **Script execution methods**: Multiple ways to run bash scripts: - **`sh scriptname.sh`**: Run with the sh shell interpreter. - **`bash scriptname.sh`**: Run with the bash shell interpreter. - **`./scriptname.sh`**: Execute directly (requires executable permissions). ```bash sh questionnaire.sh bash questionnaire.sh ./questionnaire.sh ``` ## File Permissions and Script Execution - **Permission denied error**: When using `./scriptname.sh`, you may get "permission denied" if the file lacks executable permissions. - **Checking permissions**: Use `ls -l` to view file permissions. ```bash ls -l questionnaire.sh ``` - **Permission format**: The output shows permissions as `-rw-r--r--` where: - First character (`-`): File type (- for regular file, d for directory) - Next 9 characters: Permissions for owner, group, and others - `r` = read, `w` = write, `x` = execute - **Adding executable permissions**: Use `chmod +x` to give executable permissions to everyone. ```bash chmod +x questionnaire.sh ``` - **Script organization**: Best practices for structuring bash scripts. - Start with shebang (`#!/bin/bash`) - Add descriptive comments about script purpose - Define variables at the top - Group related functions together - Main script logic at the bottom ```bash #!/bin/bash NAME="value" ARRAY=("item1" "item2") my_function() { echo "Function code here" } my_function echo "Script complete" ``` - **Sequential script execution**: Create master scripts that run multiple programs in sequence. - Useful for automating workflows that involve multiple scripts - Each script runs to completion before the next one starts - Can combine different programs into a single execution flow - Arguments can be passed to individual scripts as needed - Can include different types of programs (interactive, automated, etc.) ```bash #!/bin/bash ./setup.sh ./interactive.sh ./processing.sh ./cleanup.sh ```