This is the third part of the blog series on how to move from Azure Pipelines to GitHub Actions.

This is a multi part article - find the other parts at the links below:

Table of Contents

  1. Deployments / Environments
    1. Azure Pipelines
    2. GitHub Actions
  2. Variables
    1. Azure Pipelines
  3. Artifacts
    1. Azure Pipelines
    2. GitHub Actions
  4. Secrets / Credentials
    1. Azure DevOps
      1. Secrets as Variables
      2. Libraries
    2. GitHub Actions
      1. Organization Secrets
      2. Repository Secrets
      3. Environment Secrets
  5. Next

Deployments / Environments

Deployments in Azure Pipelines and GitHub Actions are treated a little differently than “regular” CI pipelines, because they can reference Environments. The concept of environments is available in both tools and they cover a lot of the same things but are also a little different. I will focus on the approval part for the environments.

Azure Pipelines


Azure Pipelines EnvironmentsAzure Pipelines Environments

Azure Pipelines EnvironmentsAzure Pipelines Environments

Azure Pipelines EnvironmentsAzure Pipelines Environments

Azure Pipelines EnvironmentsAzure Pipelines Environments

To reference the environment, use the following snippet: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
...
jobs:
- deployment: deploy
  displayName: deploy
  pool:
    vmImage: 'ubuntu-20.04'
  environment: 'Dev'
  strategy:
    runOnce:
      deploy:
        steps:
        - checkout: self 
        - script: echo my first deployment
...

The entire deployment schema can be found here.
Deployments do cover much more than just environments in Azure Pipelines. You can configure canary deployments, different hooks to react to failed deployments, run pre-deployment tasks and so forth.

GitHub Actions

Environments are create at repository level, in organization repositories to be precise.

GitHub Actions EnvironmentGitHub Actions Environment

GitHub Actions EnvironmentGitHub Actions Environment

GitHub Actions EnvironmentGitHub Actions Environment

Environments allow the configuration of two parts, protection policies and secrets. Secrets are mentioned in more detail in the Secrets / Credentials section of this post. Protection policies allow the configuration of up to six approvers (user/teams) and to set a wait timer.

1
2
3
4
5
6
7
8
9
10
11
12
...
jobs:
  deployment:
    runs-on: ubuntu-20.04
    environment: dev
    steps:
      - shell: bash
        env:
          mySecret: ${{ secrets.mySecret }}
        run: |
          example-command "$mySecret"
...

During a run, it looks like this:

GitHub Actions Environment ApprovalGitHub Actions Environment Approval

GitHub Actions Environment ApprovalGitHub Actions Environment Approval

You can read more about the use of environments here.

Variables

Well, variables in CI/CD environments are bit of a rabbit hole because there is a lot built in and you can also set them yourself or create them with a step. There are environment variables, system variables and user defined variables. I cannot go over all of them but I will go into more detail about how you can set and consume them and in what way.

Azure Pipelines

When you take a look at the docs you can find all of them.

As shown in the Secrets / Credentials section of this post, you can define variables for each pipeline in the portal. I would recommend using this only, if you have to because it can be hard to debug since it is not part of the pipeline definition.

Azure Pipelines VariablesAzure Pipelines Variables

This is a good option for testing, however, in production, I would not recommend it.

The better way is, to set the variables as part of the pipeline definition:

1
2
3
4
5
...
variables:
- name: foo
  value: bar
...

or: 

1
2
3
4
...
variables:
  foo: bar
...

You can even use conditions to set variables. For instance, based on the branch:

1
2
3
4
5
6
7
...
variables:
  ${{ if eq(variables['Build.SourceBranchName'], 'main') }}: 
    environment: prod
  ${{ if eq(variables['Build.SourceBranchName'], 'develop') }}: 
    environment: dev
...

This can be very convenient. The value variables[‘Build.SourceBranchName’] is a reference to a build variable.
Now, you can pass the variable to a step:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
...
stages:
  - stage: build_app
    pool:
      vmImage: ${{ parameters.VM }}
    variables:
    - group: secrets
    jobs:
      - job: build
        steps:
          - script: echo Hello, world!
            displayName: 'Run a one-line script'
          - powershell: |
              Write-Output "$env:environment"
            env:
              environment: $(environment)
...

Keep in mind, variables are referenced with the following syntax: $(<variable name>)

You can create environment variables within scripts and use them in later steps:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
...
stages:
  - stage: build_app
    pool:
      vmImage: ${{ parameters.VM }}
    variables:
    - group: secrets
    jobs:
      - job: build
        steps:
          - script: echo Hello, world!
            displayName: 'Run a one-line script'
          - powershell: |
              Write-Output "##vso[task.setvariable variable=variable_name]value"
            env:
              environment: $(environment)
          - powershell: |
              Write-Output "$env:created_variable'
            env:
              created_variable: $(variable_name)
...

Here, you also reference them with the following syntax: $(<variable name>)

Artifacts

Artifacts can be used to publish software packages (NuGet, NPM, Maven, …) or pass build artifact files (e.g. .exe, .jar, … files) to other jobs in the pipeline - You can build a .jar-file in the build step of your pipeline and consume it during the build of a docker container.
I will focus on the part of passing artifacts to other jobs of the build pipeline.

Azure Pipelines

Azure Pipelines has a step for creating (publishing) an artifact and one for consuming it.

The example below is from the Microsoft Docs and creates a .txt file and publishes the artifact called drop.

1
2
3
4
5
6
7
8
...
- powershell: gci env:* | sort-object name | Format-Table -AutoSize | Out-File $env:BUILD_ARTIFACTSTAGINGDIRECTORY/environment-variables.txt

- task: PublishBuildArtifacts@1
  inputs:
    pathToPublish: '$(Build.ArtifactStagingDirectory)'
    artifactName: drop
...

The example below shows the consumption of the **drop** artifact. The name drop is the default value, it can be called as whatever you want. 
1
2
3
4
5
6
7
8
...
- task: DownloadBuildArtifacts@0
  inputs:
    buildType: 'current'
    downloadType: 'single'
    artifactName: 'drop'
    downloadPath: '$(System.ArtifactsDirectory)'
...

Find the full example below: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
...
stages:
  - stage: build_app
    pool:
      vmImage: ${{ parameters.VM }}
    variables:
    - group: secrets
    jobs:
      - job: build
        steps:
          - script: echo Hello, world!
            displayName: 'Run a one-line script'
          - powershell: |
              Connect-ToService -User 'myUser' -Password $env:password
            env:
              password: $(password)
          - task: PublishBuildArtifacts@1
            inputs:
              pathToPublish: '$(Build.ArtifactStagingDirectory)'
              artifactName: drop
  - stage: build_container
    pool:
      vmImage: ${{ parameters.VM }}
    jobs:
      - job: build
        steps:
          - task: DownloadBuildArtifacts@0
            inputs:
              buildType: 'current'
              downloadType: 'single'
              artifactName: 'drop'
              downloadPath: '$(System.ArtifactsDirectory)'
          # steps to build docker container
...

In some cases, it makes sense to create a zip-archive first and publishing only the zip file.

Find the full reference of Azure Pipeline Artifacts here.

GitHub Actions

GitHub has a very similar approach. For instance, you create .jar-file during the build and publish it as an artifact for the next job to consume it. It also gets a name. If you publish multiple artifacts during a build, you can download all at once when you remove the name parameter from the download step.
By default, artifacts are stored for 90 days, but I would recommend specifying the retention time to the lowest possible value that you require, since this increases the price of your organization and most of the time, a long retention is not necessary.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
...
jobs: 
  build_app:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v2
      
      # steps to create an artifact and store it locally on the runner

      - name: 'Upload Artifact'
        uses: actions/upload-artifact@v2
        with:
          name: my-artifact
          path: my_file.txt
          retention-days: 5

  build_container:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v2
      
      - name: Download a single artifact
        uses: actions/download-artifact@v2
        with:
          name: my-artifact

      # steps to build the docker container
...

You can find the GitHub docs about artifacts here.

Secrets / Credentials

Accessing and passing secrets is entirely different in both tools - well, sort of. Ok, it can be 😉 Let me explain.
Azure DevOps has several ways to store secrets at rest, within the tool itself, you store them per pipeline as (secret) variables, you can create libraries, in which you can create variable groups and secure files to store them grouped together. You can even link a variable group to an Azure KeyVault and allow the variable group to access the secrets there.
In GitHub Actions, it is a little easier - you can create secrets per repository, organization wide secrets and within environments - thats it. It is worth mentioning, organization wide secrets for private repositories require an enterprise license, which might take away that option entirely and leaves you only with secrets stored in your repositories.
Personally, I like the simple approach of GitHub here, but the KeyVault integration of Azure Pipelines is really nice too, so take this switch with a grain of salt.

Azure DevOps

Secrets as Variables

You can defince secrets as variables per pipeline and flag them as secret values:

Azure Pipelines Secrets as variableAzure Pipelines Secrets as variable

Azure Pipelines Secrets as variableAzure Pipelines Secrets as variable

The screenshot above also shows, how you can reference them in the pipeline.

Libraries

Libraries on the other hand have several ways to store and access secrets as well as regular variables. You can create a variable group that stores several entries, each entry can be marked as a secret and later in your pipeline, you can reference the entire variable group.

Azure Pipelines Variable GroupAzure Pipelines Variable Group

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
...
stages:
  - stage: build
    pool:
      vmImage: ${{ parameters.VM }}
    variables:
    - group: secrets
    jobs:
      - job: build
        steps:
    - script: echo Hello, world!
      displayName: 'Run a one-line script'
    - powershell: |
        Connect-ToService -User 'myUser' -Password $env:password
      env:
        password: $(password)
...

Azure Pipelines Variable Group - Key VaultAzure Pipelines Variable Group - Key Vault

You can connect the variable group to an existing Azure Key Vault by using the toggle and selecting the Vault you want to use.

Azure Pipelines Variable Group - Key VaultAzure Pipelines Variable Group - Key Vault

Azure Pipelines Variable Group - Key VaultAzure Pipelines Variable Group - Key Vault

Afterwards, you can reference the secrets the same way as shown above.

Another way is to use secure files. Secure files are just regular files that will be treated as secrets. Usually, secrets are key-value pairs but there are other forms too, a certificate for instance cannot be stored as key-value pair because of its format and this is where secure files come into play.
You can upload a secret file and download it during a pipeline run to use it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
...
stages:
  - stage: deploy
    pool:
      vmImage: ${{ parameters.VM }}
    jobs:
      - job: deploy
        steps:
        - script: echo Hello, world!
          displayName: 'Run a one-line script'

        - task: DownloadSecureFile@1
          inputs:
            secureFile: 'cert.pem'
...

You can read more about Variable Groups.

GitHub Actions

In GitHub Actions, you can use secrets as key-value pairs only. They get a name and the secret value itself. Storing and accessing them is where it gets interesting.

If your secrets have the same name at different scopes (organization, repo, environment), the lowest level takes precedence

If a secret with the same name exists at multiple levels, the secret at the lower level takes precedence. For example, if an organization-level secret has the same name as a repository-level secret, then the repository-level secret takes precedence. Similarly, if an organization, repository, and environment all have a secret with the same name, the environment-level secret takes precedence. >source
Environment > Repository > Organization

Organization Secrets

As mentioned above, if you want to use organization wide secrets within private repos, you need an enterprise license, however, it is free to use in public repos.

GitHub Actions SecretsGitHub Actions Secrets

GitHub Actions SecretsGitHub Actions Secrets

GitHub Actions SecretsGitHub Actions Secrets

GitHub Actions SecretsGitHub Actions Secrets

You can also select repositories, the secret can be used from, it is either all or just selected ones.

You can reference secrets as followed:

1
2
3
4
5
6
7
8
...
steps:
  - shell: bash
    env:
      mySecret: ${{ secrets.mySecret }}
    run: |
      example-command "$mySecret"
...

Repository Secrets

Repository secrets are declared on a repository level. Go to the Settings tab on your repository.

GitHub Actions SecretsGitHub Actions Secrets

GitHub Actions SecretsGitHub Actions Secrets

GitHub Actions SecretsGitHub Actions Secrets

GitHub Actions SecretsGitHub Actions Secrets

You can reference them the same way as mentioned above in the organization section 
1
2
3
4
5
6
7
8
...
steps:
  - shell: bash
    env:
      mySecret: ${{ secrets.mySecret }}
    run: |
      example-command "$mySecret"
...

Environment Secrets

To create an environment, check the Deployments / Environments section of this post.
You can mange secrets there:

GitHub Actions Environment SecretsGitHub Actions Environment Secrets

GitHub Actions Environment SecretsGitHub Actions Environment Secrets

You can reference them the same ways as mentioned above, however, you must reference the environment: 
1
2
3
4
5
6
7
8
9
10
11
12
...
jobs:
  deployment:
    runs-on: ubuntu-20.04
    environment: dev
    steps:
      - shell: bash
        env:
          mySecret: ${{ secrets.mySecret }}
        run: |
          example-command "$mySecret"
...

Next

In part 4 we will check out templates, draw a conclusion and go over examples.