Managing configurations and sensitive information effectively is crucial for the success and security of your projects. Environment variables and dotfiles are two essential tools that can help you achieve this. In this blog post, I’ll explore the importance of using environment variables and how dotfiles can streamline your development workflow.

What are environment variables?

Environment variables are dynamic values that can influence the behaviour of running processes on an operating system. They are used to store configuration settings, such as:

• API keys and secrets: Sensitive data needed to interact with external services.
• Database connection strings: Information required to connect to databases.
• Configuration settings: Various settings that control the behaviour of your applications.
• Environment-specific values: Values that differ between development, testing, and production environments.

Why use environment variables?

1. Security: Storing sensitive information, such as API keys and database credentials, in environment variables keeps them out of your source code. This reduces the risk of exposing sensitive data when sharing your code or committing it to a version control system such as Git.
2. Flexibility: Environment variables allow you to easily switch configurations without modifying your code. This is particularly useful when deploying the same application across multiple environments (e.g., development, staging, and production).
3. Configuration Management: Using environment variables helps centralize configuration management. Instead of hardcoding settings in your codebase, you can manage them externally, making it easier to update and maintain configurations.
4. Consistency: Environment variables provide a consistent way to manage settings across different platforms and deployment environments, ensuring your application behaves as expected regardless of where it is run.

Running a script with environment variables

To demonstrate how to run a script with environment variables, let’s consider a basic Python script that requires certain environment variables to function correctly. Suppose we have a script named main.py that connects to a database and uses an API key for some functionality.

Step-by-step guide

1. Set environment variables: You can set environment variables in your shell before running the script. For example, in a Unix-based system, you can use the export command:

   export DATABASE_URL="postgres://user:password@localhost/dbname"
   export SECRET_KEY="supersecretkey"
   export API_KEY="your_api_key"
   

   On Windows, you would use the set command:

   set DATABASE_URL="postgres://user:password@localhost/dbname"
   set SECRET_KEY="supersecretkey"
   set API_KEY="your_api_key"
   

2. Run the script: After setting the environment variables, you can run your Python script as usual:

   python main.py
   

   The script will now have access to the environment variables you’ve set.

Example Python script (main.py)

Here’s a simple example of what main.py might look like:

import os

def main() -> None:
    database_url = os.getenv('DATABASE_URL')
    secret_key = os.getenv('SECRET_KEY')
    api_key = os.getenv('API_KEY')

    if not all([database_url, secret_key, api_key]):
        raise ValueError("Missing one or more environment variables: DATABASE_URL, SECRET_KEY, API_KEY")

    print(f"Connecting to database at {database_url}")
    print(f"Using secret key: {secret_key}")
    print(f"API key: {api_key}")
    # Add your application logic here

if __name__ == "__main__":
    main()

Using a .env file with python-dotenv

For a more streamlined approach, especially during development, you can use a .env file with python-dotenv to manage environment variables. This allows you to store environment variables in a file and load them into your application.

1. Create a .env file: Create a .env file in the root directory of your project with the following content:

   DATABASE_URL=postgres://user:password@localhost/dbname
   SECRET_KEY=supersecretkey
   API_KEY=your_api_key
   

2. Load environment variables in main.py: Modify your script to use python-dotenv to load the environment variables from the .env file:

   import os
   from dotenv import load_dotenv
   
   def main():
       # Load environment variables from .env file
       load_dotenv()
   
       database_url = os.getenv('DATABASE_URL')
       secret_key = os.getenv('SECRET_KEY')
       api_key = os.getenv('API_KEY')
   
       if not all([database_url, secret_key, api_key]):
           raise ValueError("Missing one or more environment variables: DATABASE_URL, SECRET_KEY, API_KEY")
   
       print(f"Connecting to database at {database_url}")
       print(f"Using secret key: {secret_key}")
       print(f"API key: {api_key}")
       # Add your application logic here
   
   if __name__ == "__main__":
       main()
   

By following these steps, you can ensure that your application securely and efficiently handles configurations and sensitive information through environment variables and dotfiles.

Best practices

• Do not write secrets in your code. It will end up being as public as the code, which can, and often does, end up resulting in leaks.
• Do not commit your .env file to version control. Add .env to your .gitignore file to ensure it is not included in your version control system.
• Use separate .env files for different environments. For example, .env.development, .env.testing, and .env.production. Load the appropriate file based on the environment.

Final thoughts

Environment variables are a powerful tool for managing configuration settings securely and flexibly. By using python-dotenv, you can easily manage these variables using dotfiles, making your code more secure, flexible, and maintainable. Start incorporating environment variables into your projects today to improve your configuration management practices.