Skip to content

Migrate from JFrog Artifactory

View as Markdown

This page moves the files in your Artifactory repositories to CloudRepo and repoints your builds, with JFrog left running until your builds pass against CloudRepo. It copies files and nothing else: your users, permissions and every other piece of data that is not a file stay in Artifactory.

  1. List what you have and decide where each repository goes.
  2. Create the repositories in CloudRepo.
  3. Export the files from Artifactory.
  4. Import them into CloudRepo.
  5. Repoint your builds.
  6. Check, then cut over.

1. List what you have

Section titled “1. List what you have”

You need a CloudRepo organization. If you have none, sign up.

List your repositories from Artifactory, with their package types, using JFrog’s repository list API. JFrog documents it as needing admin privileges, or a scoped token with artifact:*:r or system:info:r. Give curl the credential through the netrc file described in step 3.

Terminal
curl --netrc-file ~/.jfrog.netrc --silent --fail \
  "https://yourcompany.jfrog.io/artifactory/api/repositories?type=local" | jq -r '.[] | [.key, .packageType] | @tsv'

Then decide where each one goes. The left column uses JFrog’s names for its repository types and package types:

In Artifactory In CloudRepo
Local repository, package type Maven, Gradle, npm, PyPI or Docker A repository of that format. Its files are what you export and import.
Remote repository A proxy repository, if the upstream is one CloudRepo offers. Nothing to export: a proxy fills itself from the upstream.
Virtual repository No copy. JFrog describes a virtual repository as an aggregate of local and remote repositories, so you move those. Point builds at the CloudRepo repositories directly. For Maven and Docker, a group repository can serve several repositories under one address.
Any other package type, or a Generic repository No CloudRepo repository for it. CloudRepo hosts Maven, Python, npm and Docker, and cannot create a raw (generic) repository.

JFrog’s repository list API also names federated and distribution repositories (JFrog). They are not covered here. Email support@cloudrepo.io if you rely on them.

2. Create the repositories

Section titled “2. Create the repositories”

For each local repository, create a repository of the same format in CloudRepo, and give it the name your builds already use if you can, so a URL changes in one place only. See Creating a repository. Then add whoever needs access: add users, and create a repository token with Read + write for the import and Read only tokens for builds that only pull.

3. Export the files

Section titled “3. Export the files”

Give your Artifactory credential to curl through a netrc file, so it is on no command line. JFrog documents that a reference token can be used instead of a password for basic authentication, as long as the username is the one the token was created with (Access Tokens).

~/.jfrog.netrc
machine yourcompany.jfrog.io
login jfrog-user
password JFROG_REFERENCE_TOKEN

Keep the file readable by you alone (chmod 600 ~/.jfrog.netrc). Then, for each repository, list its files with Artifactory’s file list API and download each one into a directory named for the repository:

Terminal
HOST=https://yourcompany.jfrog.io
REPO=libs-release-local
curl --netrc-file ~/.jfrog.netrc --silent --fail \
  "$HOST/artifactory/api/storage/$REPO/?list&deep=1&listFolders=0" | jq -r '.files[].uri' > "$REPO.txt"
while IFS= read -r p; do
  curl --netrc-file ~/.jfrog.netrc --silent --fail --create-dirs \
    --output "export/$REPO$p" "$HOST/artifactory/$REPO$p"
done < "$REPO.txt"

Expected: export/<repository>/ holds every file, at the path it has in Artifactory. JFrog documents the file list API as needing a non-anonymous privileged user and Artifactory Pro. JFrog’s own JFrog CLI copies files down too (jf rt download), if you have it installed.

4. Import the files

Section titled “4. Import the files”

Import artifacts into CloudRepo has one loop for each format. Run it for each repository, then check the bytes arrived. You can run a loop again after a failure, and what a second run skips and what it replaces depends on the format: see Running a loop again.

5. Repoint your builds

Section titled “5. Repoint your builds”

In each build, replace the Artifactory URL and credential with the CloudRepo repository’s URL and a repository token. The page for your build tool shows the file to change:

Pull Maven artifacts and Publish Maven artifacts: ~/.m2/settings.xml holds the credential, and pom.xml holds the repository URL.

The username for every client except npm is the email address of the account that created the token, and the password is the token. In CI, store the token in the CI system’s secret store and expose it as an environment variable, as the pages above show; do not write it into a script or a committed file.

6. Check, then cut over

Section titled “6. Check, then cut over”

Do this in order, and leave Artifactory running until the last step:

  1. Build against CloudRepo, with Artifactory still up. Point one project at CloudRepo and run its full build, then the projects that depend on it. A 401, 403 or 404 has the same causes as when you publish by hand: see “When a Client Answers 401, 403 or 404” on Repository tokens.
  2. Publish one new version to CloudRepo from CI and install it from a second machine.
  3. Switch every build that still names Artifactory, and every CI job’s secret, to CloudRepo.
  4. Stop publishing to Artifactory. From now on a new version exists in CloudRepo only.
  5. Keep an export of Artifactory until you are sure nothing reads it. Step 3 of this page is that export.

Webhooks, if you used any, are not copied. CloudRepo’s are in Webhooks.

If you are stuck, email support@cloudrepo.io with your organization name, the repositories involved, the command you ran and the error it printed.