Introduction

The purpose of this document is to help DevOps teams associated with the University of Illinois fulfill their responsibility to comply with Illinois Cybersecurity standards, including IT05, IT07, and IT08.

It is best to prevent secrets from entering the repository and commit history. See Preventing Accidental Secret Commits below.

Additional protection can be gained by setting up detect-secrets to also run centrally in a GitHub action. See Detecting Secrets below.

Accidental Secret Commit

If a secret is leaked, the secret must be rotated.

Report leaks to security@illinois.edu.

GitHub repositories are not approved for storing secrets. Use GitHub Encrypted Secrets, or another approved secret store.

GitHub provides documentation regarding steps you can take to assist with Removing sensitive data from a repository:

Warning: Once you have pushed a commit to GitHub, you should consider any data it contains to be compromised. If you committed a password, change it! If you committed a key, generate a new one.

This article tells you how to make commits with sensitive data unreachable from any branches or tags in your GitHub repository. However, it's important to note that those commits may still be accessible in any clones or forks of your repository, directly via their SHA-1 hashes in cached views on GitHub, and through any pull requests that reference them. You can't do anything about existing clones or forks of your repository, but you can permanently remove cached views and references to the sensitive data in pull requests on GitHub by contacting GitHub Support or GitHub Premium Support.

Secret Rotation

In general, secrets should be rotated at least every 90 days.

If the secrets are rotated using an automated system (e.g. AWS Secret Manager), then secrets should be rotated at least every 45 days so they are revoked every 90 days.

Preventing Accidental Secret Commits

Before you commit changes, check to ensure secrets are not being added to the history.

Yelp's Detect Secrets is a tool recommended to catch secrets prior to a commit.

A development team member can follow the command line recipe below to create a pre-commit hook in the repository. After the steps below are completed, the rest of the development team need only complete the shorter recipe below.

Add detect-secrets to new repo (first time only)

The current recommendation is to use detect-secrets v1.0.5.

We recommend using a Linux or Mac system to generate the .secrets.baseline file. This is due to file encoding issues encountered on Windows (see issue Yelp/detect-secrets#272). If you are using a Windows system, Yelp recommends saving the .secrets.baseline with UTF-8 with BOM.

# Install software  
$ pip install detect-secrets==v1.5.0 pre-commit

# Configure pre-commit framework  
# Create a file named .pre-commit-config.yaml and add the following:  
repos:
-   repo: https://github.com/Yelp/detect-secrets
    rev: v1.5.0
    hooks:
    -   id: detect-secrets
        args: ['--baseline', '.secrets.baseline']

# Install git hooks  
$ pre-commit install

# Initial audit  
$ detect-secrets scan > .secrets.baseline
$ detect-secrets audit .secrets.baseline 

# Check in  
$ git add .secrets.baseline .pre-commit-config.yaml  
$ git commit

Setup repo with pre-commit (per developer configuration)

These steps should be completed by each DevOps team member after the one-time setup above is completed.

pre-commit is a framework for managing and maintaining multi-language pre-commit hooks. It is used here to ensure that each developer is using the same version of Yelp's detect-secrets.

Note: This works well on Windows, Mac, and Linux systems.

# Install software  
$ git pull  
# Pull in pre-commit configuration & baseline  
$ pip install pre-commit  
$ pre-commit install

Example usage with a failed check-in

Example output when the pre-commit hook finds something it believes is a secret:

$ git commit  Detect secrets...........................................................Failed  - hook id: detect-secrets - exit code: 1 
 Potential secrets about to be committed to git repo! Please rectify or explicitly ignore with an inline `pragma: allowlist secret` comment. 
 Secret Type: Secret Keyword  Location:    secrets.ini:1 
 Possible mitigations:   
 - For information about putting your secrets in a safer place,     please ask in #security  
 - Mark false positives with an inline `pragma: allowlist secret`     comment   
 - Commit with `--no-verify` if this is a one-time false positive 
 If a secret has already been committed, visit https://help.github.com/articles/removing-sensitive-data-from-a-repository

Detecting Secrets

If a secret is committed, a tool such as GitHub's Secret Scanning can notify you that a secret has been leaked after the fact.

Secret scanning is automatically enabled for public GitHub repositories.

For private repositories, see Enabling secret scanning for private repositories.

Note: We consider GitHub's secret scanning to be permissive. For a more thorough check, we prefer to have Yelp's detect-secrets also check for secrets.

To add detect-secrets to a GitHub workflow, you can use this example .github/workflows/detect-secrets.yaml:

name: Detect Secrets

on: pull_request

jobs:
  detect-secrets:
    runs-on: ubuntu-22.04
    container: python:latest

    steps:
    - uses: actions/checkout@v3

    - run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
    
    - name: Install Yelp's detect secrets
      run: |
        apt-get update && apt-get install -y jq
        pip install yq
        pip install detect-secrets==$(yq -r .repos[0].rev .pre-commit-config.yaml)
  
    - name: Detect potential secrets
      run: git ls-files -z | xargs -0 detect-secrets-hook --baseline .secrets.baseline