Bash Best Practices and Coding Standards

---
description: This rule enforces best practices and coding standards for Bash scripting to improve code quality, maintainability, and security. It covers naming conventions, formatting, error handling, security, and performance considerations.
globs: **/*.sh
---

# Bash Best Practices and Coding Standards

This document outlines the best practices and coding standards for writing Bash scripts. Following these guidelines will lead to more maintainable, readable, and robust code.

## 1. Code Organization and Structure

### 1.1. Directory Structure Best Practices

For large Bash projects, a well-defined directory structure is crucial. Consider the following structure:


project_root/
├── bin/          # Executable scripts (main entry points)
├── lib/          # Library scripts (reusable functions and modules)
├── config/       # Configuration files (e.g., settings, environment variables)
├── data/         # Data files (e.g., CSV, JSON)
├── log/          # Log files generated by scripts
├── tmp/          # Temporary files used during script execution
├── tests/        # Unit and integration tests
├── docs/         # Documentation (e.g., README, design documents)
└── README.md     # Project README


### 1.2. File Naming Conventions

*   **Executable Scripts:** Use descriptive names without extensions (e.g., `process_data`, `backup_system`).
*   **Library Scripts:** Use `.sh` extension (e.g., `string_utils.sh`, `database_functions.sh`). For bash-specific code use `.bash`.
*   **Configuration Files:** Use `.conf` or `.cfg` extensions (e.g., `app.conf`, `settings.cfg`).
*   **Test Files:** Use `_test.sh` suffix (e.g., `string_utils_test.sh`).

### 1.3. Module Organization

*   **Separate Concerns:** Divide your code into logical modules, each responsible for a specific task (e.g., database interaction, string manipulation, system monitoring).
*   **Reusable Functions:** Place reusable functions in library scripts (`lib/`) and source them in your main scripts using `source lib/string_utils.sh` or `. lib/string_utils.sh`.
*   **Modular Design:** Aim for a modular design where each module can be easily tested and reused in other projects.

### 1.4. Component Architecture

*   **Functions as Components:** Treat functions as components with well-defined inputs (parameters) and outputs (return values or side effects).
*   **Configuration Management:** Use configuration files to store settings that can be easily modified without changing the script's code.
*   **Logging:** Implement a consistent logging mechanism to track script execution and errors.

### 1.5. Code Splitting Strategies

*   **Function Decomposition:** Break down complex tasks into smaller, more manageable functions.
*   **File Inclusion:** Use `source` or `.` to include reusable code from other files.
*   **External Commands:** Delegate tasks to external commands when appropriate (e.g., `grep`, `sed`, `awk`).
*   **Consider a 'main' function:** Wrap the primary logic of the script into a `main` function to improve readability and maintainability. All scripts that are long enough to contain at least one other function should have a `main` function.

## 2. Common Patterns and Anti-patterns

### 2.1. Design Patterns

*   **Facade:** Create a simplified interface to a complex set of functions or modules.
*   **Strategy:** Define a family of algorithms and encapsulate each one in a separate function, making them interchangeable.
*   **Template Method:** Define the skeleton of an algorithm in a function, deferring some steps to subfunctions.
*   **Chain of Responsibility:** Pass a request along a chain of functions until one of them handles it.
*   **Singleton (Carefully):**  Use sparingly for global configuration, initialize only once.

### 2.2. Recommended Approaches for Common Tasks

*   **String Manipulation:** Use Bash's built-in string manipulation features (e.g., parameter expansion, substring extraction) instead of external commands like `sed` or `awk` when possible.
*   **File Handling:** Use Bash's built-in file handling commands (e.g., `read`, `write`, `mkdir`, `rm`) for basic operations.  For more complex operations, consider `find` with `-exec` or `xargs`.
*   **Looping:** Use `for` loops for iterating over lists of items and `while` loops for conditional execution.
*   **Conditional Statements:** Use `if`, `elif`, and `else` statements for branching logic. Prefer `[[ ]]` over `[ ]` for string comparisons.
*   **Exit Early:** Use `return` or `exit` to exit the script as soon as an error is detected.

### 2.3. Anti-patterns and Code Smells

*   **Hardcoded Values:** Avoid hardcoding values directly in the script. Use constants or configuration files instead.
*   **Unquoted Variables:** Always quote variables to prevent word splitting and globbing issues.
*   **Eval:** Avoid using `eval` as it can introduce security vulnerabilities and make the code difficult to understand.
*   **Backticks:** Use `$(command)` instead of backticks for command substitution.
*   **Relying on Globals:** Minimize the use of global variables to avoid naming conflicts and unexpected side effects. Use `local` within functions.
*   **Ignoring Errors:** Always check the return values of commands to handle errors gracefully. Use `set -e` to exit on errors automatically.
*   **Over-commenting Obvious Code:** Comments should explain *why* the code is doing something, not *what* the code is doing.
*   **Excessive Piping:** While powerful, long pipelines can become unreadable.  Consider breaking down complex operations into smaller, more manageable steps with intermediate variables.

### 2.4. State Management Best Practices

*   **Environment Variables:** Use environment variables to store global configuration settings that can be accessed by all scripts and functions.
*   **Temporary Files:** Use temporary files to store intermediate data during script execution. Use `mktemp` to create unique temporary file names and remove them when finished. They should be created under `/tmp` or another location if needed.
*   **Persistent Storage:** For persistent state, consider using a simple database (e.g., SQLite) or a key-value store (e.g., Redis).
*   **Configuration Files:** Store persistent configuration options in configuration files.

### 2.5. Error Handling Patterns

*   **`set -e`:** Exit immediately if a command exits with a non-zero status.
*   **Check Return Values:** Use `$?` to check the return value of a command and handle errors accordingly.
*   **Error Functions:** Create functions to handle common error scenarios (e.g., logging errors, displaying error messages, exiting the script).
*   **Signal Handling:** Use `trap` to handle signals (e.g., `SIGINT`, `SIGTERM`) and perform cleanup actions.
*   **Informative Error Messages:**  Provide clear and informative error messages to help users diagnose problems. The error messages should explain the cause and possibly the recommended solution.

## 3. Performance Considerations

### 3.1. Optimization Techniques

*   **Built-in Commands:** Prefer built-in commands over external commands whenever possible, as they are generally faster.
*   **String Operations:** Use Bash's built-in string manipulation features instead of external commands like `sed` or `awk` for simple operations.
*   **Avoid Loops:** Minimize the use of loops, as they can be slow in Bash. Consider using `find` with `-exec` or `xargs` for file processing instead.
*   **Command Substitution:** Avoid unnecessary command substitution, as it can be expensive.
*   **Arrays:** Use arrays to store lists of items instead of strings, as they are more efficient for iteration and manipulation.
*   **Parameter Expansion:** Use parameter expansion features like `${var:-default}` to provide default values for variables without using conditional statements.
*   **`set -f`:** Disable globbing when not needed to prevent unintended file expansion.

### 3.2. Memory Management

*   **Limit Data Size:** Avoid loading large files or data structures into memory.
*   **Stream Processing:** Use stream processing techniques to process data incrementally instead of loading it all into memory at once.
*   **Remove Variables:** Unset or reassign variables to release memory when they are no longer needed.

### 3.3. Rendering Optimization (If Applicable)

*   **Minimize Output:** Reduce the amount of output generated by the script, as it can slow down execution.
*   **Use `printf`:** Use `printf` for formatted output instead of `echo`, as it is generally faster and more portable.

### 3.4. Bundle Size Optimization (If Applicable)

*   **N/A:** Bash scripts do not typically have bundle sizes in the same way that web applications do.

### 3.5. Lazy Loading

*   **Source on Demand:** Only source library scripts when they are needed.
*   **Conditional Execution:** Use conditional statements to execute code blocks only when certain conditions are met.

## 4. Security Best Practices

### 4.1. Common Vulnerabilities and Prevention

*   **Command Injection:** Avoid using `eval` and carefully sanitize any user input used in commands.
*   **Path Injection:**  Sanitize path names to prevent malicious users from manipulating file operations.
*   **Denial of Service (DoS):** Implement resource limits to prevent scripts from consuming excessive CPU, memory, or disk space.
*   **Information Disclosure:** Protect sensitive information (e.g., passwords, API keys) by storing them in environment variables or configuration files with appropriate permissions.
*   **Shellshock:** Ensure your Bash version is patched against the Shellshock vulnerability.

### 4.2. Input Validation

*   **Sanitize User Input:** Carefully sanitize any user input to prevent command injection and other vulnerabilities.
*   **Regular Expressions:** Use regular expressions to validate input formats (e.g., email addresses, phone numbers).
*   **Whitelist Validation:** Validate input against a whitelist of allowed values.
*   **Length Limits:** Enforce length limits on input fields to prevent buffer overflows.

### 4.3. Authentication and Authorization

*   **Avoid Storing Credentials:** Avoid storing credentials directly in scripts or configuration files. Use environment variables or a secure credential store instead.
*   **Principle of Least Privilege:** Run scripts with the minimum necessary privileges.
*   **User Authentication:** Implement user authentication using `sudo` or other authentication mechanisms.
*   **Role-Based Access Control (RBAC):** Implement RBAC to restrict access to sensitive resources based on user roles.

### 4.4. Data Protection

*   **Encryption:** Use encryption to protect sensitive data at rest and in transit.
*   **Data Masking:** Mask sensitive data in log files and other output.
*   **Access Control:** Restrict access to sensitive data to authorized users only.

### 4.5. Secure API Communication

*   **HTTPS:** Use HTTPS for all API communication to encrypt data in transit.
*   **API Keys:** Use API keys to authenticate requests to external APIs.
*   **Rate Limiting:** Implement rate limiting to prevent abuse of your API.
*   **Input Validation:** Validate all input from external APIs to prevent injection attacks.

## 5. Testing Approaches

### 5.1. Unit Testing

*   **Test Individual Functions:** Write unit tests to verify the behavior of individual functions.
*   **Mock Dependencies:** Use mocking techniques to isolate functions from their dependencies.
*   **Assert Statements:** Use assert statements to verify that the functions return the expected values.
*   **Testing Frameworks:** Consider using a bash testing framework like Bats (Bash Automated Testing System).

### 5.2. Integration Testing

*   **Test Interactions:** Write integration tests to verify the interactions between different modules or components.
*   **Real Environments:** Run integration tests in a real or simulated environment.
*   **End-to-End Testing:** Write end-to-end tests to verify the entire script from start to finish.

### 5.3. End-to-End Testing

*   **Real Use Cases:** Emulate real-world use cases to test the script's overall functionality.
*   **System-Level Testing:** Verify that the script interacts correctly with the underlying system.

### 5.4. Test Organization

*   **Separate Test Directory:** Create a separate `tests/` directory to store test files.
*   **Naming Convention:** Use a consistent naming convention for test files (e.g., `module_name_test.sh`).
*   **Test Suites:** Organize tests into suites based on functionality or module.

### 5.5. Mocking and Stubbing

*   **Mock External Commands:** Create mock versions of external commands to isolate functions from their dependencies.
*   **Stub Functions:** Create stub versions of functions to control their behavior during testing.
*   **Environment Variable Mocking:** Mock environment variables to test different configurations.

## 6. Common Pitfalls and Gotchas

### 6.1. Frequent Mistakes

*   **Forgetting `set -e`:** Not exiting on errors can lead to unexpected behavior.
*   **Unquoted Variables:** Forgetting to quote variables can lead to word splitting and globbing issues.
*   **Incorrect Variable Scope:** Using global variables when local variables are needed can lead to naming conflicts.
*   **Not Checking Return Values:** Not checking return values can lead to errors being ignored.
*   **Over-reliance on Globals** Using global variables when local variables are preferable can lead to unintended side effects.
*   **Not using Functions** Code becomes difficult to maintain if its not broken into manageable functions.

### 6.2. Edge Cases

*   **Empty Variables:** Handle empty variables gracefully.
*   **Special Characters:** Handle special characters in file names and other input.
*   **Large Files:** Handle large files efficiently to prevent memory issues.
*   **Race Conditions:** Avoid race conditions when multiple scripts are running concurrently.

### 6.3. Version-Specific Issues

*   **Bash 3 vs. Bash 4:** Be aware of compatibility issues between different versions of Bash.
*   **POSIX Compliance:** If portability is a concern, stick to POSIX-compliant syntax.

### 6.4. Compatibility Concerns

*   **Operating Systems:** Ensure your scripts are compatible with the target operating systems.
*   **External Tools:** Be aware of the dependencies on external tools and ensure they are available on the target systems.

### 6.5. Debugging Strategies

*   **`set -x`:** Enable tracing to see the commands being executed.
*   **`echo` Statements:** Use `echo` statements to print variable values and debug messages.
*   **Shellcheck:** Use Shellcheck to identify common errors and warnings.
*   **Interactive Debugging:** Use an interactive debugger like `bashdb` to step through the code.

## 7. Tooling and Environment

### 7.1. Recommended Development Tools

*   **Text Editor:** Use a text editor with syntax highlighting and code completion for Bash (e.g., VS Code, Sublime Text, Vim).
*   **Shellcheck:** Use Shellcheck to lint your Bash code.
*   **Bats:** Use Bats for unit testing.
*   **Bashdb:** Use Bashdb for interactive debugging.
*   **Git:** Use Git for version control.

### 7.2. Build Configuration

*   **Makefile:** Use a Makefile to automate build and test processes.
*   **Dependency Management:** Use a dependency management tool like `dep` to manage external dependencies.

### 7.3. Linting and Formatting

*   **Shellcheck:** Use Shellcheck to lint your Bash code and enforce coding standards.
*   **shfmt:** Use shfmt for automatic code formatting.

### 7.4. Deployment Best Practices

*   **Package Scripts:** Package your scripts and dependencies into a deployable package (e.g., Debian package, RPM package).
*   **Configuration Management:** Use configuration management tools (e.g., Ansible, Puppet, Chef) to deploy and configure your scripts on target systems.
*   **Automated Deployment:** Use automated deployment pipelines to deploy your scripts quickly and reliably.
*   **Idempotent Scripts:** Ensure your deployment scripts are idempotent, meaning they can be run multiple times without causing unintended side effects.

### 7.5. CI/CD Integration

*   **Continuous Integration (CI):** Integrate your Bash scripts into a CI pipeline to automatically run tests and linting on every commit.
*   **Continuous Deployment (CD):** Integrate your Bash scripts into a CD pipeline to automatically deploy your scripts to production.

## 8. Naming Conventions

Follow these naming conventions:

*   **Function Names:** Lowercase, with underscores to separate words (e.g., `process_data`, `calculate_total`).
*   **Variable Names:** Lowercase, with underscores to separate words (e.g., `input_file`, `output_directory`). Loop variables should be similarly named for the variable you're looping through.
*   **Constant Names:** All caps, separated with underscores (e.g., `MAX_RETRIES`, `DEFAULT_TIMEOUT`). Declared at the top of the file.
*   **File Names:** Lowercase, with underscores to separate words (e.g., `data_processing.sh`, `configuration_settings.conf`).

## 9. Style and Formatting

*   **Indentation:** Use 2 spaces for indentation. No tabs.
*   **Line Length:** Limit lines to 80 characters. Use line continuation characters (`\`) to break long lines.
*   **Spacing:** Use spaces around operators and after commas.
*   **Comments:** Write clear and concise comments to explain the purpose and logic of your code. Any function that is not both obvious and short must be commented. Any function in a library must be commented regardless of length or complexity. Use `TODO` comments to mark code that needs to be improved or completed.
*   **Quoting:** Always quote variables unless careful unquoted expansion is required.
*   **Braces:** Use `${variable}` instead of `$variable` for clarity.
*   **Pipelines:** Format pipelines for maximum clarity. Break long pipelines into multiple lines with one pipeline element per line.

## 10. Exit Codes
*   Ensure exit codes are consistent throughout the scripts.
*   Use `exit 0` for successful completion.
*   Use non-zero exit codes (e.g., 1, 2, 127, etc.) to indicate failure.
*   Standardize exit codes for specific error conditions across your project.
*   Include custom error messages with meaningful context.

Following these best practices and coding standards will help you write more maintainable, readable, secure, and efficient Bash scripts.