Publish from Jenkins

View as Markdown

A Jenkins Pipeline publishes to CloudRepo the way your laptop does: the build tool sends a repository token with the upload. What changes is where the token lives. Keep it as a Jenkins secret text credential, bind it to an environment variable inside the one stage that publishes, and let a committed config file name that variable instead of holding the token.

Before you start

A repository, and a repository token that reaches it. A token that publishes is Read + write; a build that only pulls needs Read only.
A token you create under Repository Tokens belongs to your account. Deleting a user from your organization deletes that user’s tokens, and any build using one stops authenticating. So create the token as an account that outlives any one person: a user you make for builds (see Create Your First User), signed in to the admin portal when it mints the token. The owner can instead create a service key, which belongs to your organization rather than to a user.
Your repository URL, https://your-org.mycloudrepo.io/repositories/your-repo, and the username that Maven, Gradle, Python and Docker clients send: the email address of the account that created the token. npm sends the token alone.
The tools your pipeline calls (mvn, gradle, npm, pip, docker) installed on the agent that runs it.

Store the token as a credential

In Jenkins, go to Manage Jenkins, then Credentials. Under Stores scoped to Jenkins, select System, then Global credentials (unrestricted), and click Add Credentials.
For Kind, choose Secret text. Paste the token into Secret. For ID, enter cloudrepo-token.
Save the credential.

A Pipeline refers to the credential by its ID, cloudrepo-token, and never holds the value. See Using credentials in Jenkins’ documentation.

The Jenkinsfiles below bind the credential in an environment block inside the stage that publishes, which sets CLOUDREPO_TOKEN for that stage’s steps and no others. Keep it inside the stage: an environment block at the top of the pipeline applies to every step in the Pipeline, so a stage that builds or tests your code would see the token too. The steps read the variable in a single-quoted sh string. The quotes matter: Jenkins’ documentation says Groovy string interpolation should never be used with credentials, because the secret is then copied into the process arguments, where ps can show it, and shell metacharacters in it are executed. A single-quoted string leaves the variable for the shell to read from its environment.

Publish from a Pipeline

Pick your client. The choice is remembered on every page of these docs. Each tab has the same parts: a file that names the token’s variable, the repository settings of your build, the command the pipeline runs, and the Jenkinsfile that runs it with the credential bound. Commit the files; none holds the token.

Maven reads the credential from a settings file. Commit this one as .mvn/settings.xml; ${env.CLOUDREPO_TOKEN} is replaced by the environment variable when Maven runs. Replace you@example.com with the email address of the account that created the token.

<settings>
  <servers>
    <server>
      <id>cloudrepo</id>
      <username>you@example.com</username>
      <password>${env.CLOUDREPO_TOKEN}</password>
    </server>
  </servers>
</settings>

The <id> must match the repository’s <id> in your pom.xml. <distributionManagement> is where mvn deploy publishes:

<distributionManagement>
  <repository>
    <id>cloudrepo</id>
    <url>https://your-org.mycloudrepo.io/repositories/your-repo</url>
  </repository>
</distributionManagement>

This is the command the pipeline runs. With CLOUDREPO_TOKEN set in your shell, it publishes from your machine the way the pipeline will, which is a quick way to check the files above:

mvn --batch-mode --settings .mvn/settings.xml deploy

The Jenkinsfile binds the credential and runs the same command:

pipeline {
    agent any
    stages {
        stage('Publish') {
            environment {
                CLOUDREPO_TOKEN = credentials('cloudrepo-token')
            }
            steps {
                sh 'mvn --batch-mode --settings .mvn/settings.xml deploy'
            }
        }
    }
}

Expected: the build ends with BUILD SUCCESS, and the artifact is listed in the repository in the admin portal. To resolve dependencies from the same repository in a build, add a <repositories> block with the same URL; see Maven Repositories.

Gradle reads the token from the environment in the build script. Replace you@example.com with the email address of the account that created the token.

publishing {
    publications {
        create<MavenPublication>("library") {
            from(components["java"])
        }
    }
    repositories {
        maven {
            url = uri("https://your-org.mycloudrepo.io/repositories/your-repo")
            credentials {
                username = "you@example.com"
                password = providers.environmentVariable("CLOUDREPO_TOKEN").orNull
            }
        }
    }
}

.orNull keeps every other Gradle command working on a machine where the variable is not set; only publish needs it. Add the java-library and maven-publish plugins to the plugins block if your build does not have them.

This is the command the pipeline runs. With CLOUDREPO_TOKEN set in your shell, it publishes from your machine the way the pipeline will:

gradle publish

pipeline {
    agent any
    stages {
        stage('Publish') {
            environment {
                CLOUDREPO_TOKEN = credentials('cloudrepo-token')
            }
            steps {
                sh 'gradle publish'
            }
        }
    }
}

If your project has the Gradle wrapper, run ./gradlew publish instead. A Groovy build script takes the same maven { ... } block under publishing { repositories { ... } }, with the credential read from System.getenv("CLOUDREPO_TOKEN").

npm replaces ${CLOUDREPO_TOKEN} in .npmrc with the environment variable of that name, so this file can be committed. Keep the trailing / on both lines, and keep the second line identical to the registry URL without https:.

registry=https://your-org.mycloudrepo.io/repositories/your-repo/
//your-org.mycloudrepo.io/repositories/your-repo/:_authToken=${CLOUDREPO_TOKEN}

npm sends the token on its own, so there is no username to set. This is the command the pipeline runs; with CLOUDREPO_TOKEN set in your shell, it publishes from your machine the way the pipeline will:

npm publish

pipeline {
    agent any
    stages {
        stage('Publish') {
            environment {
                CLOUDREPO_TOKEN = credentials('cloudrepo-token')
            }
            steps {
                sh 'npm publish'
            }
        }
    }
}

The same .npmrc and credential serve npm ci and npm install in a build that only pulls. For a scoped package, or a repository that sits beside the public registry, see npm Repositories.

twine uploads a built package. It reads its username and password from TWINE_USERNAME and TWINE_PASSWORD, which the script sets from the credential’s variable, and takes the repository URL without /simple/. Replace you@example.com with the email address of the account that created the token. This is what the pipeline runs; with CLOUDREPO_TOKEN set in your shell, it uploads from your machine the way the pipeline will:

pip wheel . --no-deps -w dist
pip install twine
export TWINE_USERNAME=you@example.com
export TWINE_PASSWORD="$CLOUDREPO_TOKEN"
twine upload --repository-url https://your-org.mycloudrepo.io/repositories/your-repo dist/*

pipeline {
    agent any
    stages {
        stage('Publish') {
            environment {
                CLOUDREPO_TOKEN = credentials('cloudrepo-token')
            }
            steps {
                sh '''
                    pip wheel . --no-deps -w dist
                    pip install twine
                    export TWINE_USERNAME=you@example.com
                    export TWINE_PASSWORD="$CLOUDREPO_TOKEN"
                    twine upload --repository-url https://your-org.mycloudrepo.io/repositories/your-repo dist/*
                '''
            }
        }
    }
}

To install from the repository in a build, set PIP_INDEX_URL from the credential rather than passing --index-url, which would put the token on the command line. See Repository Tokens for the URL, and write the @ in your email address as %40.

Log in to the host alone, with your email address and the token on standard input, then build and push. The image reference carries the repository after the host. Last, log out: docker login keeps the credential in the Docker configuration on the agent, and a Jenkins agent can outlive the build. The agent needs a Docker daemon it can reach. This is what the pipeline runs; with CLOUDREPO_TOKEN set in your shell, it works from your machine the way the pipeline will:

echo "$CLOUDREPO_TOKEN" | docker login your-org.mycloudrepo.io --username you@example.com --password-stdin
docker build -t your-org.mycloudrepo.io/repositories/your-repo/my-app:1.0.0 .
docker push your-org.mycloudrepo.io/repositories/your-repo/my-app:1.0.0
docker logout your-org.mycloudrepo.io

In the Jenkinsfile the logout is in a post section with the always condition, so it runs even when the build or the push fails:

pipeline {
    agent any
    stages {
        stage('Publish image') {
            environment {
                CLOUDREPO_TOKEN = credentials('cloudrepo-token')
            }
            steps {
                sh '''
                    echo "$CLOUDREPO_TOKEN" | docker login your-org.mycloudrepo.io --username you@example.com --password-stdin
                    docker build -t your-org.mycloudrepo.io/repositories/your-repo/my-app:1.0.0 .
                    docker push your-org.mycloudrepo.io/repositories/your-repo/my-app:1.0.0
                '''
            }
            post {
                always {
                    sh 'docker logout your-org.mycloudrepo.io'
                }
            }
        }
    }
}

Use your own image name and tag. The password is the repository token, not a portal password. More: Docker Repositories.

When publishing fails

401 Unauthorized. The credential was refused. The username must be the email address of the account that created the token, and the token must not be expired or revoked. Check that the credential’s ID is the one the Jenkinsfile names, that it is a Secret text credential, and that its value is the whole token. The full list is on Repository Tokens.
403 Forbidden. The token does not reach this repository, or it is Read only and the build publishes. Create a Read + write token that includes the repository.
409 Conflict from mvn deploy. The release version you are publishing already exists, and Overwrite Protection refuses to replace it. Publish a new version, or use a -SNAPSHOT version while you iterate. See Maven Overwrite Protection. Python repositories have the same setting: Python Repositories.