# Import artifacts into CloudRepo

> Copy files you exported from another repository manager into CloudRepo with a re-runnable loop for Maven, npm, Python and Docker, then check the bytes arrived.

You have the files on disk, exported from another repository manager. This page copies them into CloudRepo: one loop per format, each of which you can run again after a failure.

## Before you start

- A CloudRepo repository for each repository whose files you are moving, of the same format. If you have none, [create them](/docs/manage/repositories.html#creating-a-repository). Import only those. For each other kind of repository in your old system, the page for that system says what to create in CloudRepo, if anything, such as a [proxy repository](/docs/consume/proxy-repositories.html).
- A repository token with **Read + write** that reaches those repositories. See [Repository tokens](/docs/authenticate/repository-tokens.html). Your username is the email address of the account that created the token.
- The exported files in a directory, `./export`, each repository in its own subdirectory. The page for the system you are leaving tells you how to get them: [JFrog Artifactory](/docs/migrate/from-jfrog.html), [Sonatype Nexus](/docs/migrate/from-nexus.html), [AWS CodeArtifact](/docs/migrate/from-aws-codeartifact.html). For Azure Artifacts, see the [Azure Artifacts guide](/docs/migration-advanced/migration-from-azure.html).

## Running a loop again

What a second run does depends on the format and on whether Overwrite Protection is on. It is on for every Maven, npm and Python repository unless someone turned it off for that repository. See [Overwrite Protection](/docs/manage/repositories.html#overwrite-protection).

- **Maven.** While Overwrite Protection is on, CloudRepo answers `409 Conflict` for a release file, or a checksum or signature file that belongs to one, that is already in the repository, and leaves it as it is. `maven-metadata.xml`, its own checksum and signature files, and every file under a `-SNAPSHOT` version are rewritten and never refused.
- **npm and Python.** While Overwrite Protection is on, CloudRepo answers `409 Conflict` for an npm version or a Python file that is already in the repository, and leaves it as it is.
- **Docker.** A Docker repository has no Overwrite Protection. Pushing a tag that already exists moves the tag to the new image.
- **Overwrite Protection turned off.** The repository accepts overwrites, so a second run replaces what it already holds.

So a second run skips the Maven releases, npm versions and Python files the first run finished and uploads the rest, only while Overwrite Protection is on, and it re-points every Docker tag it pushes again.

## Import

**Maven**

A Maven repository is a directory tree, and CloudRepo’s Maven door takes a `PUT` at any path in that tree, which is what `mvn deploy` sends. So the whole tree goes up file by file, checksum files and `maven-metadata.xml` included, with the same paths.

**1. Put the credential in `~/.netrc`,** so curl reads it from there and the token is on no command line. Keep the file readable by you alone (`chmod 600 ~/.netrc`).

`~/.netrc`

```ini
machine your-org.mycloudrepo.io
login you@example.com
password YOUR_REPOSITORY_TOKEN
```

**2. Upload the tree.** Run this from the repository’s directory in `./export`. It prints one line per file: the status CloudRepo answered, then the path.

**Terminal**

```bash
BASE=https://your-org.mycloudrepo.io/repositories/your-repo
cd export/your-repo
find . -type f | sort | while IFS= read -r f; do
  code=$(curl --netrc --proto =https --silent --output /dev/null --write-out '%{http_code}' \
    --upload-file "$f" "$BASE/${f#./}")
  printf '%s %s\n' "$code" "$f"
done | tee ../import.log
```

`2xx` means the file is in. With Overwrite Protection on, `409` means it was already there: a release file, or a checksum file that belongs to one, from an earlier run. `maven-metadata.xml` and `-SNAPSHOT` versions are rewritten, never refused. Look at every other status in `import.log`: `401` is the credential, `403` is a token that cannot write here, `413` is a file over 50 GB.

**npm**

Each package version is a tarball (`.tgz`), and `npm publish` can publish a tarball directly. Put the registry and the token in `~/.npmrc`, as on [Publish npm packages](/docs/publish/npm.html):

`~/.npmrc`

```ini
registry=https://your-org.mycloudrepo.io/repositories/your-repo/
//your-org.mycloudrepo.io/repositories/your-repo/:_authToken=${CLOUDREPO_TOKEN}
```

Then publish every tarball, oldest version first:

**Terminal**

```bash
BASE=https://your-org.mycloudrepo.io/repositories/your-repo/
find ./export -name '*.tgz' | sort -V | while IFS= read -r t; do
  v=$(tar -xzOf "$t" package/package.json | jq -r .version)
  case "$v" in *-*) tag=next ;; *) tag=latest ;; esac
  npm publish "$t" --registry "$BASE" --tag "$tag" || echo "FAILED $t" >&2
done
```

Three things in that loop matter:

- **`--registry` is not optional.** A package’s own `package.json` can name a registry in `publishConfig`, and npm publishes there, whatever your `.npmrc` says. A package that came from your old registry may carry that line, and without `--registry` npm would send it straight back.
- **`--tag`.** npm refuses a prerelease version such as `1.0.0-beta.1` unless you name a tag, so the loop gives prereleases the tag `next`. Everything else gets `latest`.
- **Oldest first.** Each publish sets the tag it carries, so `latest` ends on the version you published last. Version-sorting the paths puts the highest version last. Check a package afterwards with `npm view <name> dist-tags`, and move `latest` with `npm dist-tag add <name>@<version> latest`.

While Overwrite Protection is on, an `npm publish` of a version already in the repository fails with `409`, and the loop prints it and carries on.

**pip**

A Python repository holds wheels (`.whl`) and source distributions (`.tar.gz`), and twine uploads them. Give twine its credential through its environment, so no file is written and no token is on a command line, and upload one file at a time, so one refusal does not stop the rest:

**Terminal**

```bash
BASE=https://your-org.mycloudrepo.io/repositories/your-repo
export TWINE_USERNAME="you@example.com"
export TWINE_PASSWORD="$CLOUDREPO_TOKEN"
find ./export \( -name '*.whl' -o -name '*.tar.gz' \) | sort | while IFS= read -r f; do
  twine upload --non-interactive --repository-url "$BASE" "$f" || echo "FAILED $f" >&2
done
```

`--repository-url` is the repository’s URL without `/simple/`. While Overwrite Protection is on, a file already in the repository is refused with `409`; twine prints only the status line, so rerun one file with `--verbose` to read the explanation. Python has no tags and no versions to order: the files can go up in any order.

**Docker**

A Docker repository holds images by name and tag. The Docker CLI copies one image at a time: pull it from the old registry, give it its new name, push it. Log in to both hosts with the token on standard input (`docker login`, as on [Push Docker images](/docs/publish/docker.html)), then list the images as `name:tag` lines in `images.txt` and run:

**Terminal**

```bash
OLD=registry.example.com/old-repo
NEW=your-org.mycloudrepo.io/repositories/your-repo
while IFS= read -r img; do
  docker pull "$OLD/$img" \
    && docker tag "$OLD/$img" "$NEW/$img" \
    && docker push "$NEW/$img" \
    || echo "FAILED $img" >&2
  docker rmi "$OLD/$img" "$NEW/$img" > /dev/null 2>&1
done < images.txt
```

The image name on CloudRepo carries `repositories/` and your repository after the host; the login does not. `docker rmi` frees the disk after each image.

`docker pull` fetches the image for your machine’s platform, so an image published for several platforms arrives for one. Rebuild and push those from your CI instead, as on [Push Docker images](/docs/publish/docker.html). Pushing a tag that already exists moves the tag to the new image, so a second run replaces what a tag points at.

## Check the bytes arrived

A count is not proof. Fetch a few files back from CloudRepo and compare them with the originals:

**Terminal**

```bash
f=com/example/my-library/1.0.0/my-library-1.0.0.jar
curl --netrc --fail --silent "https://your-org.mycloudrepo.io/repositories/your-repo/$f" | sha256sum
sha256sum "export/your-repo/$f"
```

The two lines must match. For npm, Python and Docker, install or pull one package or image from CloudRepo in a clean directory or on a clean machine, as on [Pull npm packages](/docs/consume/npm.html), [Pull Python packages](/docs/consume/python.html) and [Pull Docker images](/docs/consume/docker.html). The files are also listed in the repository in the [admin portal](https://admin.cloudrepo.io).

## When something fails

Each refusal has one cause, and it is the same as when you publish by hand. See the list at the end of [Publish Maven artifacts](/docs/publish/maven.html), [Publish npm packages](/docs/publish/npm.html), [Publish Python packages](/docs/publish/python.html) or [Push Docker images](/docs/publish/docker.html). If you are stuck, email <support@cloudrepo.io> with your organization name, the repository, the failing line from `import.log` and the command you ran.

---

The page: https://www.cloudrepo.io/docs/migrate/import-artifacts.html