Move your CI off the JFrog CLI and Nexus staging

View as Markdown

A job that publishes through the JFrog CLI, an Artifactory build plugin or nexus-staging-maven-plugin keeps working after your move to CloudRepo once each of those steps is a plain command. For each tool, this page shows the step you have now, read from the vendor’s own documentation and linked beside it, and the plain command that replaces it. Each block for CloudRepo is copied from the CloudRepo page for that format or that CI platform.

JFrog’s --build-name and --build-number options and its jf rt build-publish step collect and publish build information to Artifactory (JFrog). The plain commands below send none, so remove those options and that step from the job.

This page changes your jobs. It does not copy your files. To copy them, see Migrate from JFrog Artifactory or Migrate from Sonatype Nexus Repository.

Before you start

A repository in CloudRepo for each format you publish. See Creating a repository. Files that are not a package go in a Maven repository: see Any file.
A repository token with Read + write on those repositories, from Repository tokens. The username is the email address of the account that created the token. npm sends the token alone.
The token in your CI’s secret store, named CLOUDREPO_TOKEN, handed to the one step that publishes. The CI/CD overview shows where each platform keeps it, and why to create the token as an account that outlives any one person.
Artifactory or Nexus still running. Leave it up until a job has published to CloudRepo and a second machine has installed what it published, as Migrate from JFrog Artifactory does.

Maven

Maven publishes to CloudRepo with mvn deploy, a <server> entry in settings.xml and a <distributionManagement> repository that uses the same id. Start from what your job runs now.

From the JFrog CLI

JFrog documents jf mvn-config as the one-time step that tells jf mvn which Artifactory repositories to deploy to, and jf mvn as Maven run with Artifactory’s build integration:

jf mvn-config --server-id-deploy=my-server --repo-deploy-releases=libs-release-local --repo-deploy-snapshots=libs-snapshot-local
jf mvn deploy --build-name=my-java-app --build-number=1

The same page says that without that build integration you can use plain mvn pointed at Artifactory through settings.xml.

JFrog documents that jf mvn deploys at the install phase and later. Maven’s own lifecycle copies to a remote repository only at deploy, and install copies to the local repository. So a job that publishes with jf mvn clean install must run mvn deploy after the move, not mvn install.

From nexus-staging-maven-plugin

Sonatype’s README says that a plugin declared with <extensions>true</extensions> disables the maven-deploy-plugin:deploy executions and injects its own nexus-staging-maven-plugin:deploy executions instead. Declared like this, mvn deploy runs the staging goal in place of maven-deploy-plugin, and a release is staged on the Nexus server that <nexusUrl> names, not uploaded to CloudRepo:

<build>
  <plugins>
    <plugin>
      <groupId>org.sonatype.plugins</groupId>
      <artifactId>nexus-staging-maven-plugin</artifactId>
      <version>1.6.8</version>
      <extensions>true</extensions>
      <configuration>
        <nexusUrl>https://nexus.example.com/</nexusUrl>
        <serverId>nexus</serverId>
      </configuration>
    </plugin>
  </plugins>
</build>

Remove the whole <plugin> entry, or at least its <extensions>true</extensions> line. Maven then runs maven-deploy-plugin again, and mvn deploy uploads to the repository in <distributionManagement>. Two settings leave the build green and publish nothing, so look for both in your pom, your parent pom and your job’s -D options:

skipNexusStagingDeployMojo. Sonatype documents it as completely skipping the plugin’s deploy goal. With the extension still in place that goal is the only deploy, so the build ends in BUILD SUCCESS and nothing is uploaded.
<skip>true</skip> on maven-deploy-plugin. The README’s configuration for Maven 2 sets it beside the plugin’s own deploy execution. Without the staging plugin, Maven prints Skipping artifact deployment and uploads nothing.

To plain `mvn deploy`

Put the credential in ~/.m2/settings.xml. The <id> is a name you choose, and the repository in your pom.xml must use the same one. ${env.CLOUDREPO_TOKEN} reads the token from an environment variable, so the file holds no token. Export the variable in the shell that runs Maven, or in your CI’s secret store.

<settings>
  <servers>
    <server>
      <id>cloudrepo</id>
      <username>you@example.com</username>
      <password>${env.CLOUDREPO_TOKEN}</password>
    </server>
  </servers>
</settings>

The <distributionManagement> repository is where mvn deploy publishes. Put it in a parent POM if you have one, so a project states it once.

<distributionManagement>
  <repository>
    <id>cloudrepo</id>
    <url>https://your-org.mycloudrepo.io/repositories/your-repo</url>
  </repository>
</distributionManagement>

Run this in the directory of your pom.xml:

mvn --batch-mode deploy

Expected: Maven prints Uploading to cloudrepo: and Uploaded to cloudrepo: for the POM, the JAR and maven-metadata.xml, and ends with BUILD SUCCESS. If it ends with BUILD SUCCESS and prints neither line, a skip above is still in your build.

More: Publish Maven artifacts, for snapshots and releases, and Pull Maven artifacts, for resolving dependencies from CloudRepo.

Gradle

CloudRepo is a Maven repository, so Gradle publishes to it through its maven-publish plugin and gradle publish.

JFrog documents the com.jfrog.artifactory plugin as adding an artifactoryPublish task that deploys your publications, and says an artifactory block alone is not enough: maven-publish or ivy-publish and a publication must be applied too.

plugins {
    id 'java'
    id 'maven-publish'
    id 'com.jfrog.artifactory' version '6.+'
}

publishing {
    publications {
        mavenJava(MavenPublication) {
            from components.java
        }
    }
}

artifactory {
    publish {
        contextUrl = 'https://yourcompany.jfrog.io/artifactory'
        repository {
            repoKey = 'libs-snapshot-local'
            username = project.property('artifactory_user')
            password = project.property('artifactory_password')
        }
        defaults {
            publications('ALL_PUBLICATIONS')
        }
    }
}

The job runs ./gradlew artifactoryPublish. A job that runs the same task through jf gradle, as in jf gradle clean artifactoryPublish, changes the same way.

Your maven-publish plugin and your publication stay. Remove the com.jfrog.artifactory plugin line and the artifactory block, add the CloudRepo repository to the publishing block, and change the job to gradle publish. Your project needs a group and a version. If you already have a publication, keep yours and add only the repositories block:

publishing {
    publications {
        library(MavenPublication) {
            from components.java
        }
    }
    repositories {
        maven {
            url = "https://your-org.mycloudrepo.io/repositories/your-repo"
            credentials {
                username = findProperty("cloudrepoUsername")
                password = findProperty("cloudrepoToken")
            }
        }
    }
}

Keep the token in your CI’s secret store and hand it to Gradle through the environment, so no file is written. Gradle turns an environment variable named ORG_GRADLE_PROJECT_ plus a property name into that project property, so the build script above works unchanged:

export ORG_GRADLE_PROJECT_cloudrepoUsername="you@example.com"
export ORG_GRADLE_PROJECT_cloudrepoToken="$CLOUDREPO_TOKEN"
gradle publish

More: Publish Gradle artifacts, for the Kotlin form and for snapshots and releases.

npm

JFrog documents jf npm-config as the one-time step that names the Artifactory repositories, and jf npm publish as packing the package and uploading it itself, not through npm’s own publish command (unless it runs in native mode):

jf npm-config --server-id-resolve=my-server --repo-resolve=npm-virtual --server-id-deploy=my-server --repo-deploy=npm-local
jf npm publish --build-name=my-app --build-number=42

npm publishes to CloudRepo the way it publishes to any registry: the registry’s address and your token go in .npmrc, and npm publish uploads the package. npm replaces ${CLOUDREPO_TOKEN} with the environment variable of that name, so the file holds no token and can be copied or committed.

registry=https://your-org.mycloudrepo.io/repositories/your-repo/
//your-org.mycloudrepo.io/repositories/your-repo/:_authToken=${CLOUDREPO_TOKEN}

Keep the trailing / on both lines, and keep the second line identical to the registry URL without https:. npm sends the token only to the registry that prefix names, so if the two differ, even by the slash, npm sends no token at all. npm sends the token on its own, so there is no username. Run this in the directory of your package:

npm publish

More: Publish npm packages, for scoped packages and for keeping npmjs.org as your default registry.

Python

JFrog documents jf pip-config as writing the repositories jf twine reads, and jf twine upload as publishing the files in dist with build information:

jf pip-config --server-id-resolve=my-server --repo-resolve=pypi-virtual --server-id-deploy=my-server --repo-deploy=pypi-local
python3 -m build
jf twine upload dist/* --build-name=my-python-lib --build-number=$BUILD_NUMBER

A job that uploads with plain twine instead names an entry of ~/.pypirc whose repository is Artifactory’s PyPI address (JFrog):

twine upload --repository pypi-local dist/*

Both become the same twine command pointed at CloudRepo. twine reads its username and password from TWINE_USERNAME and TWINE_PASSWORD, which the script sets from your secret, and takes the repository’s address without /simple/. The username is the email address of the account that created the token, not __token__, and the password is the token. --non-interactive makes a missing credential fail the job instead of waiting for someone to type.

export TWINE_USERNAME="you@example.com"
export TWINE_PASSWORD="$CLOUDREPO_TOKEN"
twine upload --non-interactive \
  --repository-url https://your-org.mycloudrepo.io/repositories/your-repo \
  dist/*

More: Publish Python packages, for a ~/.pypirc entry in place of the variables.

Docker

JFrog documents that jf docker logs in to the registry for you before it builds, pushes or pulls:

jf docker build -t yourcompany.jfrog.io/docker-local/my-app:1.0.0 .
jf docker push yourcompany.jfrog.io/docker-local/my-app:1.0.0 --build-name=my-app --build-number=1

Plain Docker logs in once, to your CloudRepo host and nothing more, with the token on standard input. Then it builds and pushes an image whose name carries the repository: the host, the literal repositories/ segment, your repository, then the image and its tag.

echo "$CLOUDREPO_TOKEN" | docker login your-org.mycloudrepo.io \
  --username you@example.com \
  --password-stdin

docker build -t your-org.mycloudrepo.io/repositories/your-repo/my-app:1.0.0 .
docker push your-org.mycloudrepo.io/repositories/your-repo/my-app:1.0.0

docker login takes your-org.mycloudrepo.io and nothing more. The push name adds /repositories/your-repo/. A name without the repositories/ segment is refused as invalid.

More: Push Docker images, for a refused push.

Any file

JFrog documents jf rt upload as taking a source path and a target path, and a pattern as uploading every file it matches:

jf rt upload "*.zip" my-repo/packages/

CloudRepo has no Raw repository type. A Maven repository stores a file at any path you name and returns it from the same path, so a plain curl replaces the command (Raw files over HTTP). curl reads the credential from ~/.netrc, so the token is on no command line. Keep the file readable by you alone (chmod 600 ~/.netrc).

machine your-org.mycloudrepo.io
login you@example.com
password YOUR_REPOSITORY_TOKEN

curl --netrc --fail --upload-file hello.txt \
  https://your-org.mycloudrepo.io/repositories/your-repo/uploads/hello.txt

Replace hello.txt and uploads/hello.txt with your file and the path you want it at. The command sends one file, so run it once for each file the pattern matched. A second upload to a path that already holds a file is refused with 409 Conflict while Overwrite Protection is on: put each version at its own path. See Remove or replace a file.

GitHub Actions

JFrog’s GitHub Actions page says the Setup JFrog CLI action installs and configures JFrog CLI for a job, from a JF_URL variable and one credential: an access token, a user and password, or OIDC. The steps after it run jf commands:

steps:
  - uses: actions/checkout@v4
  - name: Setup JFrog CLI
    uses: jfrog/setup-jfrog-cli@v4
    env:
      JF_URL: ${{ vars.JF_URL }}
      # JFrog's example also sets JF_ACCESS_TOKEN here, from a secret
  - name: Configure Maven
    run: jf mvn-config --server-id-resolve=setup-jfrog-cli-server --repo-resolve-releases=libs-release --repo-resolve-snapshots=libs-snapshot --server-id-deploy=setup-jfrog-cli-server --repo-deploy-releases=libs-release-local --repo-deploy-snapshots=libs-snapshot-local
  - name: Build with Maven
    run: jf mvn clean install -DskipTests --build-name=my-java-app --build-number=${{ github.run_number }}
  - name: Publish build info
    run: jf rt build-publish my-java-app ${{ github.run_number }}

Delete the setup step, the JF_URL variable and the JFrog credential. Add a repository secret named CLOUDREPO_TOKEN, and hand it to the step that publishes through that step’s env, where the tool reads it as CLOUDREPO_TOKEN. The step runs the plain command from the sections above. This is the Maven workflow:

name: Publish
on:
  push:
    tags: ["release-*"]
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: "17"
      - name: Publish to CloudRepo
        env:
          CLOUDREPO_TOKEN: ${{ secrets.CLOUDREPO_TOKEN }}
        run: mvn --batch-mode --settings .mvn/settings.xml deploy

It reads .mvn/settings.xml and the repository in pom.xml, which Publish from GitHub Actions shows with the Gradle, npm, Python and Docker workflows. For GitLab CI/CD, CircleCI and Bitbucket Pipelines, see the CI/CD overview.

Jenkins

JFrog’s current documentation calls the Jenkins Artifactory plugin legacy and points new pipelines at the Jenkins JFrog plugin, whose jf step runs the JFrog CLI commands from the sections above. Replace those the way those sections do.

A pipeline on the legacy plugin uses the steps in JFrog’s Declarative Pipeline Syntax page, archived on 7 January 2023. rtMavenDeployer names the Artifactory repositories Maven deploys to, rtMavenRun runs the build with it, and rtUpload uploads the files its file spec matches:

rtMavenDeployer (
    id: 'deployer-unique-id',
    serverId: 'Artifactory-1',
    releaseRepo: 'libs-release-local',
    snapshotRepo: 'libs-snapshot-local'
)
rtMavenRun (
    tool: MAVEN_TOOL,
    pom: 'maven-example/pom.xml',
    goals: 'clean install',
    deployerId: 'deployer-unique-id'
)
rtUpload (
    serverId: 'Artifactory-1',
    spec: '''{
          "files": [
            {
              "pattern": "bazinga/*froggy*.zip",
              "target": "bazinga-repo/froggy-files/"
            }
         ]
    }'''
)

An rtMavenDeployer and the rtMavenRun that uses it become the sh step of Publish from Jenkins. The credential is a Secret text credential with the ID cloudrepo-token, bound in the environment block of the one stage that publishes, and the step runs the plain Maven command. The files it reads, .mvn/settings.xml and the repository in pom.xml, are on that page.

pipeline {
    agent any
    stages {
        stage('Publish') {
            environment {
                CLOUDREPO_TOKEN = credentials('cloudrepo-token')
            }
            steps {
                sh 'mvn --batch-mode --settings .mvn/settings.xml deploy'
            }
        }
    }
}

An rtUpload becomes the curl command of Any file in an sh step. That command reads a netrc file, so keep the file from that section as a Jenkins Secret file credential with the ID cloudrepo-netrc. Jenkins documents that credentials() in an environment directive supports secret files, and sets the variable to the location of a temporary file. curl’s --netrc-file option names that file:

pipeline {
    agent any
    stages {
        stage('Upload') {
            environment {
                CLOUDREPO_NETRC = credentials('cloudrepo-netrc')
            }
            steps {
                sh 'curl --netrc-file "$CLOUDREPO_NETRC" --fail --upload-file hello.txt https://your-org.mycloudrepo.io/repositories/your-repo/uploads/hello.txt'
            }
        }
    }
}

When the first run fails

401 Unauthorized or 403 Forbidden. The credential was refused, or the token cannot publish here. The username is the email address of the account that created the token, and a Read only token cannot publish. See When a client answers 401, 403 or 404.
409 Conflict. CloudRepo refuses a second publish of a release version, or a second upload to a path that holds a file, while Overwrite Protection is on, so a job that publishes the same version twice fails the second time. Publish a new version, or see Snapshots and releases for Maven, Publish a version once for npm, Publish a file once for Python and Remove or replace a file for other files.
BUILD SUCCESS and nothing uploaded. A skip is still in your Maven build: see From nexus-staging-maven-plugin.