Drone Cache

This plugin requires Volume configuration if you enable filesystem backend with configuration. This means your repository Trusted flag must be enabled. This should not be enabled in untrusted environments.

A Drone plugin for caching current workspace files between builds to reduce your build times. drone-cache is a small CLI program, written in Go without any external OS dependencies (such as tar, etc).

With drone-cache, you can provide your own cache key templates, specify archive format (tar, tar.gz, etc) and you can use an S3 bucket or a mounted volume as storage for your cached files, even better you can implement your own storage backend to cover your use case.

How does it work

drone-cache stores mounted directories and files under a key at the specified backend (by default S3).

Use this plugin to cache data that makes your builds faster. In the case of a cache miss or zero cache restore it will fail silently in won’t break your running pipeline.

The best example would be to use this with your package managers such as Mix, Bundler or Maven. After your initial download, you can build a cache and then you can restore that cache in your next build.

With restored dependencies from a cache, commands like mix deps.get will only need to download new dependencies, rather than re-download every package on each and every build.

Using Cache Key Templates

Cache key template syntax is very basic. You just need to provide a string. In that string you can use variables by prefixing them with a . in {{ }} construct, from provided metadata object (see below).

Also following helper functions provided for your use:

• checksum: Provides md5 hash of a file for given path
• epoch: Provides Unix epoch
• arch: Provides Architecture of running system
• os: Provides Operation system of running system

For further information about this syntax please see official docs from Go standard library.

Template Examples

"{{ .Repo.Name }}-{{ .Commit.Branch }}-{{ checksum "go.mod" }}-yadayadayada"

"{{ .Repo.Name }}_{{ checksum "go.mod" }}_{{ checksum "go.sum" }}_{{ arch }}_{{ os }}"

Metadata

Following metadata object is available and pre-populated with current build information for you to use in cache key templates.

{
  Repo {
    Avatar  string "repository avatar [$DRONE_REPO_AVATAR]"
    Branch  string "repository default branch [$DRONE_REPO_BRANCH]"
    Link    string "repository link [$DRONE_REPO_LINK]"
    Name    string "repository name [$DRONE_REPO_NAME]"
    Owner   string "repository owner [$DRONE_REPO_OWNER]"
    Private bool   "repository is private [$DRONE_REPO_PRIVATE]"
    Trusted bool   "repository is trusted [$DRONE_REPO_TRUSTED]"
  }

  Build {
    Created  int    "build created (default: 0) [$DRONE_BUILD_CREATED]"
    Deploy   string "build deployment target [$DRONE_DEPLOY_TO]"
    Event    string "build event (default: 'push') [$DRONE_BUILD_EVENT]"
    Finished int    "build finished (default: 0) [$DRONE_BUILD_FINISHED]"
    Link     string "build link [$DRONE_BUILD_LINK]"
    Number   int    "build number (default: 0) [$DRONE_BUILD_NUMBER]"
    Started  int    "build started (default: 0) [$DRONE_BUILD_STARTED]"
    Status   string "build status (default: 'success') [$DRONE_BUILD_STATUS]"
  }

  Commit {
    Author {
      Avatar string "git author avatar [$DRONE_COMMIT_AUTHOR_AVATAR]"
      Email  string "git author email [$DRONE_COMMIT_AUTHOR_EMAIL]"
      Name   string "git author name [$DRONE_COMMIT_AUTHOR]"
    }
    Branch  string "git commit branch (default: 'master') [$DRONE_COMMIT_BRANCH]"
    Link    string "git commit link [$DRONE_COMMIT_LINK]"
    Message string "git commit message [$DRONE_COMMIT_MESSAGE]"
    Ref     string "git commit ref (default: 'refs/heads/master') [$DRONE_COMMIT_REF]"
    Remote  string "git remote url [$DRONE_REMOTE_URL]"
    Sha     string "git commit sha [$DRONE_COMMIT_SHA]"
  }
}

Examples

The following is a sample configuration in your .drone.yml file:

Simple

kind: pipeline
name: default

steps:
  - name: restore-cache
    image: meltwater/drone-cache:dev
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: aws_access_key_id
      AWS_SECRET_ACCESS_KEY:
        from_secret: aws_secret_access_key
    pull: true
    settings:
      restore: true
      bucket: drone-cache-bucket
      region: eu-west-1
      mount:
        - 'vendor'

  - name: build
    image: golang:1.11-alpine
    pull: true
    commands:
      - apk add --update make git
      - make drone-cache

  - name: rebuild-cache
    image: meltwater/drone-cache:dev
    pull: true
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: aws_access_key_id
      AWS_SECRET_ACCESS_KEY:
        from_secret: aws_secret_access_key
    settings:
      rebuild: true
      bucket: drone-cache-bucket
      region: eu-west-1
      mount:
        - 'vendor'

Simple (Filesystem/Volume)

kind: pipeline
name: default

steps:
  - name: restore-cache-with-filesystem
    image: meltwater/drone-cache:dev
    pull: true
    settings:
      backend: "filesystem"
      restore: true
      cache_key: "volume"
      archive_format: "gzip"
      # filesystem_cache_root: "/tmp/cache"
      mount:
        - 'vendor'
    volumes:
    - name: cache
      path: /tmp/cache

  - name: build
    image: golang:1.11-alpine
    pull: true
    commands:
      - apk add --update make git
      - make drone-cache

  - name: rebuild-cache-with-filesystem
    image: meltwater/drone-cache:dev
    pull: true
    settings:
      backend: "filesystem"
      rebuild: true
      cache_key: "volume"
      archive_format: "gzip"
      # filesystem_cache_root: "/tmp/cache"
      mount:
        - 'vendor'
    volumes:
    - name: cache
      path: /tmp/cache

volumes:
  - name: cache
    temp: {}

With custom cache key template

See cache key templates section for further information and to learn about syntax.

kind: pipeline
name: default

steps:
  - name: restore-cache-with-key
    image: meltwater/drone-cache:dev
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: aws_access_key_id
      AWS_SECRET_ACCESS_KEY:
        from_secret: aws_secret_access_key
    settings:
      pull: true
      restore: true
      cache_key: '{{ .Repo.Name }}_{{ checksum "go.mod" }}_{{ checksum "go.sum" }}_{{ arch }}_{{ os }}'
      bucket: drone-cache-bucket
      region: eu-west-1
      mount:
        - 'vendor'

  - name: build
    image: golang:1.11-alpine
    pull: true
    commands:
      - apk add --update make git
      - make drone-cache

  - name: rebuild-cache-with-key
    image: meltwater/drone-cache:dev
    pull: true
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: aws_access_key_id
      AWS_SECRET_ACCESS_KEY:
        from_secret: aws_secret_access_key
    settings:
      rebuild: true
      cache_key: '{{ .Repo.Name }}_{{ checksum "go.mod" }}_{{ checksum "go.sum" }}_{{ arch }}_{{ os }}'
      bucket: drone-cache-bucket
      region: eu-west-1
      mount:
        - 'vendor'

With gzip compression

kind: pipeline
name: default

steps:
  - name: restore-cache-with-gzip
    image: meltwater/drone-cache:dev
    pull: true
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: aws_access_key_id
      AWS_SECRET_ACCESS_KEY:
        from_secret: aws_secret_access_key
    settings:
      restore: true
      cache_key: "gzip"
      archive_format: "gzip"
      bucket: drone-cache-bucket
      region: eu-west-1
      mount:
        - 'vendor'

  - name: build
    image: golang:1.11-alpine
    pull: true
    commands:
      - apk add --update make git
      - make drone-cache

  - name: rebuild-cache-with-gzip
    image: meltwater/drone-cache:dev
    pull: true
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: aws_access_key_id
      AWS_SECRET_ACCESS_KEY:
        from_secret: aws_secret_access_key
    settings:
      rebuild: true
      cache_key: "gzip"
      archive_format: "gzip"
      bucket: drone-cache-bucket
      region: eu-west-1
      mount:
        - 'vendor'

Debug

kind: pipeline
name: default

steps:
  - name: restore-cache-debug
    image: meltwater/drone-cache:dev
    settings:
      pull: true
      restore: true
      debug: true

  - name: build
    image: golang:1.11-alpine
    pull: true
    commands:
      - apk add --update make git
      - make drone-cache

  - name: restore-cache-debug
    image: meltwater/drone-cache:dev
    settings:
      pull: true
      rebuild: true
      debug: true

Parameter Reference

backend

cache backend to use in plugin (s3, filesystem) (default: s3)

mount

cache directories, an array of folders to cache

rebuild

rebuild the cache directories

restore

restore the cache directories

cache-key

cache key to use for the cache directories

archive-format

archive format to use to store the cache directories (tar, gzip) (default: tar)

debug

enable debug

filesystem-cache-root

local filesystem root directory for the filesystem cache (default: /tmp/cache)

endpoint

endpoint for the s3 connection

access-key

AWS access key

secret-key

AWS secret key

bucket

AWS bucket name

region

AWS bucket region. (us-east-1, eu-west-1, …)

path-style

use path style for bucket paths. (true for minio, false for aws)

acl

upload files with acl (private, public-read, …) (default: private)

encryption

server-side encryption algorithm, defaults to none. (AES256, aws:kms)