Use git-secret to Encrypt Secrets in the Repository [Tutorial Part 6]

PascalLandau

Rate me:

0.00/5 (No votes)

1 May 2022MIT13 min read

5.4K

How to setup git-secret to store secrets directly in the repository

In the sixth part of this tutorial series on developing PHP on Docker, we will setup git-secret to store secrets directly in the repository. Everything will be handled through Docker and added as make targets for a convenient workflow.

FYI: This tutorial is a precursor to the next part, Create a CI pipeline for dockerized PHP Apps because dealing with secrets is an important aspect when setting up a CI system (and later when deploying to production) - but I feel it's complex enough to warrant its own article.

All code samples are publicly available in my Docker PHP Tutorial repository on Github.
You find the branch with the final result of this tutorial at part-6-git-secret-encrypt-repository-docker.

Published Parts of the Docker PHP Tutorial

Setting up PHP, PHP-FPM and NGINX for local development on Docker (2018-07-08)
Setting up PhpStorm with Xdebug for local development on Docker (2018-08-06)
Structuring the Docker setup for PHP Projects (2019-05-20)
Docker from scratch for PHP 8.1 Applications in 2022 (2022-03-21)
PhpStorm, Docker and Xdebug 3 on PHP 8.1 in 2022 (2022-03-22)
Run Laravel 9 on Docker in 2022 (2022-03-23)
Set up PHP QA tools and control them via make (2022-04-25)
Use git-secret to encrypt secrets in the repository (2022-04-25)
Create a CI pipeline for dockerized PHP Apps (2022-04-25)

If you want to follow along, please subscribe to the RSS feed or via email to get automatic notifications when the next part comes out. :)

Introduction

Dealing with secrets (passwords, tokens, key files, etc.) is close to "naming things" when it comes to hard problems in software engineering. Some things to consider:

security is paramount - but high security often goes hand in hand with high inconvenience
- and if things get too complicated, people look for shortcuts...
in a team, sharing certain secret values is often mandatory
- so now, we need to think about secure ways to distribute and update secrets across multiple people
concrete secret values often depend on the environment
- inherently tricky to "test" or even "review", because those values are "by definition" different on "your machine" than on "production"

In fact, entire products have been built around dealing with secrets, e.g., HashiCorp Vault, AWS Secrets Manager or the GCP Secret Manager. Introducing those in a project comes with a certain overhead as it's yet another service that needs to be integrated and maintained. Maybe it is the exact right decision for your use-case - maybe it's overkill. By the end of this article, you'll at least be aware of an alternative with a lower barrier to entry. See also the Pros and cons section in the end for an overview.

Even though it's generally not advised to store secrets in a repository, I'll propose exactly that in this tutorial:

identify files that contain secret values
make sure they are added to .gitignore
encrypt them via git-secret
commit the encrypted files to the repository

In the end, we will be able to call:

make secret-decrypt

to reveal secrets in the codebase, make modifications to them if necessary and then run:

make secret-encrypt

to encrypt them again so that they can be committed (and pushed to the remote repository). To see it in action, check out branch part-6-git-secret-encrypt-repository-docker and run the following commands:

# checkout the branch
git checkout part-6-git-secret-encrypt-repository-docker

# build and start the docker setup
make make-init
make docker-build
make docker-up

# "create" the secret key - the file "secret.gpg.example" would usually NOT live in the repo!
cp secret.gpg.example secret.gpg

# initialize gpg
make gpg-init

# ensure that the decrypted secret file does not exist
ls passwords.txt

# decrypt the secret file
make secret-decrypt

# show the content of the secret file
cat passwords.txt

Tooling

We will set up gpg and git-secret in the php base image, so that the tools become available in all other containers. Please refer to Docker from scratch for PHP 8.1 Applications in 2022 for an in-depth explanation of the docker images.

Caution

All following commands are executed in the application container.

Tip
See Easy container access via din .bashrc helper for a convenient shortcut to log into docker containers.

Please note that there is a caveat when using git-secret in a folder that is shared between the host system and a docker container. I'll explain that in more detail (including a workaround) in section The git-secret directory and the gpg-agent socket.

gpg

gpg is short for The GNU Privacy Guard and is an open source implementation of the OpenPGP standard. In short, it allows us to create a personal key file pair (similar to SSH keys) with a private secret key and a public key that can be shared with other parties whose messages you want to decrypt.

gpg installation

To install it, we can simply run apk add gnupg and thus update .docker/images/php/base/Dockerfile accordingly:

# File: .docker/images/php/base/Dockerfile

RUN apk add --update --no-cache \
        bash \
        gnupg \
        make \
#...

gpg usage

I'll only cover the strictly necessary gpg commands here. Please refer to the "Using GPG" section in the git-secret docu and/or How to generate PGP keys with GPG for further information.

Create GPG Key Pair

We need gpg to create the gpg key pair via:

name="Pascal Landau"
email="pascal.landau@example.com"
gpg --batch --gen-key <<EOF
Key-Type: 1
Key-Length: 2048
Subkey-Type: 1
Subkey-Length: 2048
Name-Real: $name
Name-Email: $email
Expire-Date: 0
%no-protection
EOF

The %no-protection will create a key without password, see also this gist to "Creating gpg keys non-interactively".

Output:

$ name="Pascal Landau"
$ email="pascal.landau@example.com"
$ gpg --batch --gen-key <<EOF
> Key-Type: 1
> Key-Length: 2048
> Subkey-Type: 1
> Subkey-Length: 2048
> Name-Real: $name
> Name-Email: $email
> Expire-Date: 0
> %no-protection
> EOF
gpg: key E1E734E00B611C26 marked as ultimately trusted
gpg: revocation certificate stored as '/root/.gnupg/opengpg-revocs.d/74082D81525723F5BF5B2099E1E734E00B611C26.rev'

You could also run gpg --gen-key without the --batch flag to be guided interactively through the process.

Export, List and Import Private GPG Keys

The private key can be exported via:

email="pascal.landau@example.com"
path="secret.gpg"
gpg --output "$path" --armor --export-secret-key "$email"

This secret key must never be shared!

It looks like this:

All secret keys can be listed via:

gpg --list-secret-keys

Output:

$ gpg --list-secret-keys
/root/.gnupg/pubring.kbx
------------------------
sec   rsa2048 2022-03-27 [SCEA]
      74082D81525723F5BF5B2099E1E734E00B611C26
uid           [ultimate] Pascal Landau <pascal.landau@example.com>
ssb   rsa2048 2022-03-27 [SEA]

You can import the private key via:

path="secret.gpg"
gpg --import "$path"

and get the following output:

$ path="secret.gpg"
$ gpg --import "$path"
gpg: key E1E734E00B611C26: "Pascal Landau <pascal.landau@example.com>" not changed
gpg: key E1E734E00B611C26: secret key imported
gpg: Total number processed: 1
gpg:              unchanged: 1
gpg:       secret keys read: 1
gpg:  secret keys unchanged: 1

Caution: If the secret key requires a password, you would now be prompted for it. We can circumvent the prompt by using --batch --yes --pinentry-mode loopback:

path="secret.gpg"
gpg --import --batch --yes --pinentry-mode loopback "$path"

See also Using Command-Line Passphrase Input for GPG. In doing so, we don't need to provide the password just yet - but we must pass it later when we attempt to decrypt files.

Export, List and Import Public GPG Keys

The public key can be exported to public.gpg via:

email="pascal.landau@example.com"
path="public.gpg"
gpg --armor --export "$email" > "$path"

It looks like this:

List all public keys via:

gpg --list-keys

Output:

$ gpg --list-keys
/root/.gnupg/pubring.kbx
------------------------
pub   rsa2048 2022-03-27 [SCEA]
      74082D81525723F5BF5B2099E1E734E00B611C26
uid           [ultimate] Pascal Landau <pascal.landau@example.com>
sub   rsa2048 2022-03-27 [SEA]

The public key can be imported in the same way as private keys via:

path="public.gpg"
gpg --import "$path"

Example:

$ gpg --import /var/www/app/public.gpg
gpg: key E1E734E00B611C26: "Pascal Landau <pascal.landau@example.com>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1

git-secret

The official website of git-secret is already doing a great job of introducing the tool. In short, it allows us to declare certain files as "secrets" and encrypt them via gpg - using the keys of all trusted parties. The encrypted file can then by stored safely directly in the git repository and decrypted if required.

In this tutorial, I'm using git-secret v0.4.0:

$ git secret --version
0.4.0

git-secret installation

The installation instructions for Alpine reads as follows:

sh -c "echo 'https://gitsecret.jfrog.io/artifactory/git-secret-apk/all/main'" >> 
/etc/apk/repositories
wget -O /etc/apk/keys/git-secret-apk.rsa.pub 
'https://gitsecret.jfrog.io/artifactory/api/security/keypair/public/repositories/git-secret-apk'
apk add --update --no-cache git-secret

We update the .docker/images/php/base/Dockerfile accordingly:

# File: .docker/images/php/base/Dockerfile

# install git-secret
# @see https://git-secret.io/installation#alpine
ADD https://gitsecret.jfrog.io/artifactory/api/security/keypair/public/repositories/git-secret-apk /etc/apk/keys/git-secret-apk.rsa.pub

RUN echo "https://gitsecret.jfrog.io/artifactory/git-secret-apk/all/main" >> /etc/apk/repositories  && \
    apk add --update --no-cache \
        bash \
        git-secret \
        gnupg \
        make \
#...

git-secret usage

Initialize git-secret

git-secret is initialized via the following command run in the root of the git repository:

git secret init

$ git secret init
git-secret: init created: '/var/www/app/.gitsecret/'

We only need to do this once, because we'll commit the folder to git later. It contains the following files:

$ git status | grep ".gitsecret"
        new file:   .gitsecret/keys/pubring.kbx
        new file:   .gitsecret/keys/pubring.kbx~
        new file:   .gitsecret/keys/trustdb.gpg
        new file:   .gitsecret/paths/mapping.cfg

The pubring.kbx~ file (with the trailing tilde ~) is only a temporary file and can safely be git-ignored. See also Can't find any docs about keyring.kbx~ file.

The git-secret Directory and the gpg-agent Socket

To use git-secret in a directory that is shared between the host system and docker, we need to also run the following commands:

tee .gitsecret/keys/S.gpg-agent <<EOF
%Assuan%
socket=/tmp/S.gpg-agent
EOF

tee .gitsecret/keys/S.gpg-agent.ssh <<EOF
%Assuan%
socket=/tmp/S.gpg-agent.ssh
EOF

tee .gitsecret/keys/gpg-agent.conf <<EOF
extra-socket /tmp/S.gpg-agent.extra
browser-socket /tmp/S.gpg-agent.browser
EOF

This is necessary because there is an issue when git-secret is used in a setup where the codebase is shared between the host system and a docker container. I've explained the details in the Github issue "gpg: can't connect to the agent: IPC connect call failed" error in docker alpine on shared volume.

In short:

gpg uses a gpg-agent to perform its tasks and the two tools communicate through sockets that are created in the --home-directory of the gpg-agent
The agent is started implicitly through a gpg command used by git-secret, using the .gitsecret/keys directories as a --home-directory
Because the location of the --home-directory is shared with the host system, the socket creation fails (potentially only an issue for Docker Desktop, see the related discussion in Github issue Support for sharing unix sockets)

The corresponding error messages are:

gpg: can't connect to the agent: IPC connect call failed

gpg-agent: error binding socket to '/var/www/app/.gitsecret/keys/S.gpg-agent': I/O error

The workaround for this problem can be found in this thread: Configure gpg to use different locations for the sockets by placing additional gpg configuration files in the .gitsecret/keys directory:

S.gpg-agent

%Assuan%
socket=/tmp/S.gpg-agent

S.gpg-agent.ssh

%Assuan%
socket=/tmp/S.gpg-agent.ssh

gpg-agent.conf

extra-socket /tmp/S.gpg-agent.extra
browser-socket /tmp/S.gpg-agent.browser

Adding, Listing and Removing Users

To add a new user, you must first import its public gpg key. Then run:

email="pascal.landau@example.com"
git secret tell "$email"

In this case, the user pascal.landau@example.com will now be able to decrypt the secrets.

To show the users, run:

git secret whoknows

$ git secret whoknows
pascal.landau@example.com

To remove a user, run:

email="pascal.landau@example.com"
git secret killperson "$email"

FYI: This command was renamed to removeperson in git-secret >= 0.5.0.

$ git secret killperson pascal.landau@example.com
git-secret: removed keys.
git-secret: now [pascal.landau@example.com] do not have an access to the repository.
git-secret: make sure to hide the existing secrets again.

User pascal.landau@example.com will no longer be able to decrypt the secrets.

Caution: The secrets need to be re-encrypted after removing a user!

Reminder: Rotate the encrypted secrets

Please be aware that not only your secrets are stored in git, but who had access as well, i.e., even if you remove a user and re-encrypt the secrets, that user would still be able to decrypt the secrets of a previous commit (when the user was still added). In consequence, you need to rotate the encrypted secrets themselves as well after removing a user.

But isn't that a great flaw in the system, making it a bad idea to use git-secret in general?

In my opinion: No.

If the removed user had access to the secrets at any point in time (no matter where they have been stored), he could very well have just created a local copy or simply "written them down". In terms of security, there is really no "added downside" due to git-secret. It just makes it very clear that you must rotate the secrets ¯\_(ツ)_/¯

Adding, Listing and Removing Files for Encryption

Run git secret add [filenames...] for files you want to encrypt. Example:

git secret add .env

If .env is not added in .gitignore, git-secret will display a warning and add it automatically.

git-secret: these files are not in .gitignore: .env
git-secret: auto adding them to .env
git-secret: 1 item(s) added.

Otherwise, the file is added with no warning.

$ git secret add .env
git-secret: 1 item(s) added.

You only need to add files once. They are then stored in .gitsecret/paths/mapping.cfg:

$ cat .gitsecret/paths/mapping.cfg
.env:505070fc20233cb426eac6a3414399d0f466710c993198b1088e897fdfbbb2d5

You can also show the added files via:

git secret list

$ git secret list
.env

Caution: The files are not yet encrypted!

If you want to remove a file from being encrypted, run:

git secret remove .env

Output:

$ git secret remove .env
git-secret: removed from index.
git-secret: ensure that files: [.env] are now not ignored.

Encrypt files

To actually encrypt the files, run:

git secret hide

Output:

$ git secret hide
git-secret: done. 1 of 1 files are hidden.

The encrypted (binary) file is stored at $filename.secret, i.e. .env.secret in this case:

$ cat .env.secret
�☺♀♥�H~�B�Ӯ☺�"��▼♂F�►���l�Cs��S�@MHWs��e������{♣♫↕↓�L� ↕s�1�J$◄♥�;���ǆ֕�Za�����\u�ٲ& ¶��V�► ���6��
;<�d:��}ҨD%.�;��&��G����vWW�]>���߶��▲;D�+Rs�S→�Y!&J��۪8���ٔF��→f����*��$♠���&RC�8▼♂�☻z h��Z0M�T>

The encrypted files are de-cryptable for all users that have been added via git secret tell. That also means that you need to run this command again whenever a new user is added.

Decrypting Files

You can decrypt files via:

git secret reveal

Output:

$ git secret reveal
File '/var/www/app/.env' exists. Overwrite? (y/N) y
git-secret: done. 1 of 1 files are revealed.

The files are decrypted and will overwrite the current, unencrypted files (if they already exist)
- use the -f option to force the overwrite and run non-interactively
If you only want to check the content of an encrypted file, you can use git secret cat $filename (e.g., git secret cat .env)

In case the secret gpg key is password protected, you must pass the password via the -p option. E.g. for password 123456:

git secret reveal -p 123456

Show Changes Between Encrypted and Decrypted Files

One problem that comes with encrypted files: You can't review them during a code review in a remote tool. So in order to understand what changes have been made, it is helpful to show the changes between the encrypted and the decrypted files. This can be done via:

git secret changes

Output:

$ echo "foo" >> .env
$ git secret changes
git-secret: changes in /var/www/app/.env:
--- /dev/fd/63
+++ /var/www/app/.env
@@ -34,3 +34,4 @@
 MAIL_ENCRYPTION=null
 MAIL_FROM_ADDRESS=null
 MAIL_FROM_NAME="${APP_NAME}"
+foo

Note the +foo at the bottom of the output. It was added in the first line via echo "foo"> >> .env.

Makefile Adjustments

Since I won't be able to remember all the commands for git-secret and gpg, I've added them to the Makefile at .make/01-00-application-setup.mk:

# File: .make/01-00-application-setup.mk

#...

# gpg

DEFAULT_SECRET_GPG_KEY?=secret.gpg
DEFAULT_PUBLIC_GPG_KEYS?=.dev/gpg-keys/*

.PHONY: gpg
gpg: ## Run gpg commands. Specify the command e.g. via ARGS="--list-keys"
    $(EXECUTE_IN_APPLICATION_CONTAINER) gpg $(ARGS)

.PHONY: gpg-export-public-key
gpg-export-public-key: ## Export a gpg public key 
e.g. via EMAIL="john.doe@example.com" PATH=".dev/gpg-keys/john-public.gpg"
    @$(if $(PATH),,$(error PATH is undefined))
    @$(if $(EMAIL),,$(error EMAIL is undefined))
    "$(MAKE)" -s gpg ARGS="gpg --armor --export $(EMAIL) > $(PATH)"

.PHONY: gpg-export-private-key
gpg-export-private-key: ## Export a gpg private key 
e.g. via EMAIL="john.doe@example.com" PATH="secret.gpg"
    @$(if $(PATH),,$(error PATH is undefined))
    @$(if $(EMAIL),,$(error EMAIL is undefined))
    "$(MAKE)" -s gpg ARGS="--output $(PATH) --armor --export-secret-key $(EMAIL)"

.PHONY: gpg-import
gpg-import: ## Import a gpg key file 
e.g. via GPG_KEY_FILES="/path/to/file /path/to/file2"
    @$(if $(GPG_KEY_FILES),,$(error GPG_KEY_FILES is undefined))
    "$(MAKE)" -s gpg ARGS="--import --batch --yes 
    --pinentry-mode loopback $(GPG_KEY_FILES)"

.PHONY: gpg-import-default-secret-key
gpg-import-default-secret-key: ## Import the default secret key
    "$(MAKE)" -s gpg-import GPG_KEY_FILES="$(DEFAULT_SECRET_GPG_KEY)"

.PHONY: gpg-import-default-public-keys
gpg-import-default-public-keys: ## Import the default public keys
    "$(MAKE)" -s gpg-import GPG_KEY_FILES="$(DEFAULT_PUBLIC_GPG_KEYS)" 

.PHONY: gpg-init
gpg-init: gpg-import-default-secret-key 
gpg-import-default-public-keys ## Initialize gpg in the container, 
                               ## i.e. import all public and private keys

# git-secret

.PHONY: git-secret
git-secret: ## Run git-secret commands. Specify the command e.g. via ARGS="hide"
    $(EXECUTE_IN_APPLICATION_CONTAINER) git-secret $(ARGS)

.PHONY: secret-init
secret-init: ## Initialize git-secret in the repository via `git-secret init`
    "$(MAKE)" -s git-secret ARGS="init"

.PHONY: secret-init-gpg-socket-config
secret-init-gpg-socket-config: ## Initialize the config files to change the 
                               ## gpg socket locations
    echo "%Assuan%" > .gitsecret/keys/S.gpg-agent
    echo "socket=/tmp/S.gpg-agent" >> .gitsecret/keys/S.gpg-agent
    echo "%Assuan%" > .gitsecret/keys/S.gpg-agent.ssh
    echo "socket=/tmp/S.gpg-agent.ssh" >> .gitsecret/keys/S.gpg-agent.ssh
    echo "extra-socket /tmp/S.gpg-agent.extra" > .gitsecret/keys/gpg-agent.conf
    echo "browser-socket /tmp/S.gpg-agent.browser" >> .gitsecret/keys/gpg-agent.conf

.PHONY: secret-encrypt
secret-encrypt: ## Decrypt secret files via `git-secret hide`
    "$(MAKE)" -s git-secret ARGS="hide"

.PHONY: secret-decrypt
secret-decrypt: ## Decrypt secret files via `git-secret reveal -f`
    "$(MAKE)" -s git-secret ARGS="reveal -f" 

.PHONY: secret-decrypt-with-password
secret-decrypt-with-password: ## Decrypt secret files using a password 
                              ## for gpg via `git-secret reveal -f -p $(GPG_PASSWORD)`
    @$(if $(GPG_PASSWORD),,$(error GPG_PASSWORD is undefined))
    "$(MAKE)" -s git-secret ARGS="reveal -f -p $(GPG_PASSWORD)" 

.PHONY: secret-add
secret-add: ## Add a file to git secret via `git-secret add $FILE`
    @$(if $(FILE),,$(error FILE is undefined))
    "$(MAKE)" -s git-secret ARGS="add $(FILE)"

.PHONY: secret-cat
secret-cat: ## Show the contents of file to git secret via `git-secret cat $FILE`
    @$(if $(FILE),,$(error FILE is undefined))
    "$(MAKE)" -s git-secret ARGS="cat $(FILE)"

.PHONY: secret-list
secret-list: ## List all files added to git secret `git-secret list`
    "$(MAKE)" -s git-secret ARGS="list"

.PHONY: secret-remove
secret-remove: ## Remove a file from git secret via `git-secret remove $FILE`
    @$(if $(FILE),,$(error FILE is undefined))
    "$(MAKE)" -s git-secret ARGS="remove $(FILE)"

.PHONY: secret-add-user
secret-add-user: ## Remove a user from git secret via `git-secret tell $EMAIL`
    @$(if $(EMAIL),,$(error EMAIL is undefined))
    "$(MAKE)" -s git-secret ARGS="tell $(EMAIL)"

.PHONY: secret-show-users
secret-show-users: ## Show all users that have access to git secret 
                   ## via `git-secret whoknows`
    "$(MAKE)" -s git-secret ARGS="whoknows"

.PHONY: secret-remove-user
secret-remove-user: ## Remove a user from git secret via `git-secret killperson $EMAIL`
    @$(if $(EMAIL),,$(error EMAIL is undefined))
    "$(MAKE)" -s git-secret ARGS="killperson $(EMAIL)"

.PHONY: secret-diff
secret-diff: ## Show the diff between the content of encrypted and 
             ## decrypted files via `git-secret changes`
    "$(MAKE)" -s git-secret ARGS="changes"

Workflow

Working with git-secret is pretty straight forward:

Initialize git-secret
Add all users
Add all secret files and make sure they are ignored via .gitignore
Encrypt the files
Commit the encrypted files like "any other file"
If any changes were made by other team members to the files:
- => Decrypt to get the most up-to-date ones
If any modifications are required from your side:
- => make the changes to the decrypted files and then re-encrypt them again

But: The devil is in the details. The Process challenges section explains some of the pitfalls that we have encountered and the Scenarios section gives some concrete examples for common scenarios.

Process Challenges

From a process perspective, we've encountered some challenges that I'd like to mention - including how we deal with them.

Updating Secrets

When updating secrets, you must ensure to always decrypt the files first in order to avoid using "stale" files that you might still have locally. I usually check out the latest main branch and run git secret reveal to have the most up-to-date versions of the secret files. You could also use a post-merge git hook to do this automatically, but I personally don't want to risk overwriting my local secret files by accident.

Code Reviews and Merge Conflicts

Since the encrypted files cannot be diffed meaningfully, the code reviews become more difficult when secrets are involved. We use Gitlab for reviews and I usually first check the diff of the .gitsecret/paths/mapping.cfg file to see "which files have changed" directly in the UI.

In addition, I will:

checkout the main branch
decrypt the secrets via git secret reveal -f
checkout the feature-branch
run git secret changes to see the differences between the decrypted files from main and the encrypted files from feature-branch

Things get even more complicated when multiple team members need to modify secret files at the same time on different branches, as the encrypted files cannot be compared - i.e., git cannot be smart about delta updates. The only way around this is coordinating the pull requests, i.e., merge the first, update the secrets of the second and then merge the second.

Fortunately, this has only happened very rarely so far.

Local git-secret and gpg Setup

Currently, all developers in our team have git-secret installed locally (instead of using it through Docker) and use their own gpg keys.

This means more onboarding overhead, because:

a new dev must
- install git-secret locally (*)
- install and setup gpg locally (*)
- create a gpg key pair
the public key must be added by every other team member (*)
the user of the key must be added via git secret tell
the secrets must be re-encrypted

And for offboarding:

the public key must be removed by every other team member (*)
the user of the key must be removed via git secret killperson
the secrets must be re-encrypted

Plus, we need to ensure that the git-secret and gpg versions are kept up-to-date for everyone to not run into any compatibility issues.

As an alternative, I'm currently leaning more towards handling everything through docker (as presented in this tutorial). All steps marked with (*) are then obsolete, i.e., there is no need to setup git-secret and gpg locally.

But the approach also comes with some downsides, because:

the secret key and all public keys have to be imported every time the container is started
each dev needs to put his private gpg key "in the codebase" (ignored by .gitignore) so it can be shared with docker and imported by gpg (in Docker). The alternative would be using a single secret key that is shared within the team - which feels very wrong :P

To make this a little more convenient, we put the public gpg keys of every dev in the repository under .dev/gpg-keys/ and the private key has to be named secret.gpg and put in the root of the codebase.

In this setup, secret.gpg must also be added to the .gitignore file.

# File: .gitignore
#...
vendor/
secret.gpg

The import can now be be simplified with make targets:

# gpg

DEFAULT_SECRET_GPG_KEY?=secret.gpg
DEFAULT_PUBLIC_GPG_KEYS?=.dev/gpg-keys/*

.PHONY: gpg
gpg: ## Run gpg commands. Specify the command e.g. via ARGS="--list-keys"
    $(EXECUTE_IN_APPLICATION_CONTAINER) gpg $(ARGS)

.PHONY: gpg-import
gpg-import: ## Import a gpg key file
            ## e.g. via GPG_KEY_FILES="/path/to/file /path/to/file2"
    @$(if $(GPG_KEY_FILES),,$(error GPG_KEY_FILES is undefined))
    "$(MAKE)" -s gpg ARGS="--import --batch --yes 
              --pinentry-mode loopback $(GPG_KEY_FILES)"

.PHONY: gpg-import-default-secret-key
gpg-import-default-secret-key: ## Import the default secret key
    "$(MAKE)" -s gpg-import GPG_KEY_FILES="$(DEFAULT_SECRET_GPG_KEY)"

.PHONY: gpg-import-default-public-keys
gpg-import-default-public-keys: ## Import the default public keys
    "$(MAKE)" -s gpg-import GPG_KEY_FILES="$(DEFAULT_PUBLIC_GPG_KEYS)" 

.PHONY: gpg-init
gpg-init: gpg-import-default-secret-key gpg-import-default-public-keys ## Initialize 
              ## gpg in the container, i.e. import all public and private keys

"Everything" can now be handled via:

make gpg-init

that needs to be run one single time after a container has been started.

Scenarios

The scenarios assume the following preconditions:

You have checked out branch part-6-git-secret-encrypt-repository-docker:
```
git checkout part-6-git-secret-encrypt-repository-docker
```
and no running docker containers:
```
make docker-down
```
You have deleted the existing git-secret folder, the keys in .dev/gpg-keys, the secret.gpg key and the passwords.* files
```
rm -rf .gitsecret/ .dev/gpg-keys/* secret.gpg passwords.*
```

Due to technical constraints, this article is capped at 40000 characters. Read the full content at Use git-secret to encrypt secrets in the repository [Tutorial Part 6].

This article was originally posted at https://www.pascallandau.com/blog/git-secret-encrypt-repository-docker?utm_source=blog&utm_medium=rss&utm_campaign=development-feed

License

This article, along with any associated source code and files, is licensed under The MIT License

Written By

PascalLandau

Technical Lead

Germany

Technical Director at ABOUT YOU - mostly into Software Engineering in PHP and Data Engineering with Google BigQuery. I maintain a blog at https://www.pascallandau.com/ and post regularly on twitter under https://twitter.com/PascalLandau

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Use git-secret to Encrypt Secrets in the Repository [Tutorial Part 6]

Published Parts of the Docker PHP Tutorial

Introduction

Tooling

gpg

gpg installation

gpg usage

Create GPG Key Pair

Export, List and Import Private GPG Keys

Export, List and Import Public GPG Keys

git-secret

git-secret installation

git-secret usage

Initialize git-secret

The git-secret Directory and the gpg-agent Socket

Adding, Listing and Removing Users

Reminder: Rotate the encrypted secrets

Adding, Listing and Removing Files for Encryption

Encrypt files

Decrypting Files

Show Changes Between Encrypted and Decrypted Files

Makefile Adjustments

Workflow

Process Challenges

Updating Secrets

Code Reviews and Merge Conflicts

Local git-secret and gpg Setup

Scenarios

License

Comments and Discussions