--- name: cloudrepo-migrate description: Move a team's repositories from Sonatype Nexus 2, Nexus 3 or JFrog Artifactory into CloudRepo with the cloudrepo-migrate command, every file checksum-verified. Use when someone says they are done with Nexus, wants to get off Artifactory, or asks to migrate their artifact repositories to CloudRepo. --- # cloudrepo-migrate `cloudrepo-migrate` is one command, `cloudrepo-migrate ...`, that copies a server's repositories into CloudRepo and checks every file against the checksum the server recorded for it. It only reads the source server. It is on pypi.org, one file for Python 3.8 or later with no dependencies: `pipx install cloudrepo-migrate`. ## Tested set Tested against a real server of each kind: - Nexus 2: hosted Maven and npm repositories. - Nexus 3: hosted Maven, npm, PyPI and Docker repositories. - Artifactory: local Maven repositories. Move only what this list covers. If the dry run lists a repository outside it, ask the human before moving it, and name the repositories to move with `--repo`. From version 0.9.0, an `artifactory` run moves Maven by default, and moves a Generic repository the human names with `--repo` into a CloudRepo Maven repository. A run that finds a repository it is not moving says so: `Skipped N ... repositories: add --include ... (experimental)` once at the start and again in its report, where each one has the action `skipped`. The run has left them out on purpose. `--include docker`, `--include pypi` or `--include npm` adds that format (commas work too, as in `--include docker,pypi`). Each is experimental: tested against a stand-in for the server's API, not yet against a real one. Never add `--include` unless the human asks for that format. Then repeat the same `--include` in every run of that move, a resume included: the tool stores nothing between runs, so a run moves what its own command line asks. A `--repo` that names a repository of one of those formats, without its `--include`, is refused with exit code `2` and nothing moves. Ask the human whether to add the flag or leave that repository out. ## Ask the human to set up 1. A CloudRepo organization ([sign up](https://www.cloudrepo.io/docs/get-started/sign-up.html) if they have none). 2. A CloudRepo repository for each one that moves, of the same format and with the source repository's name (or the names to map with `--repo SOURCE=TARGET`), at https://admin.cloudrepo.io/repositories/create. 3. A CloudRepo repository token with Download and publish packages on those repositories. 4. Read access to the Nexus or Artifactory repositories: a Nexus login that can read them (none when Nexus allows anonymous reads), or an Artifactory access token (a legacy API key works too). Then ask them for the server's base URL (ending in `/nexus` on a default Nexus 2 install, the bare host on Nexus 3, ending in `/artifactory` for Artifactory) and the CloudRepo organization. ## Credentials Every credential is an environment variable the human sets in their own shell: never on a command line, never in a file, and never pasted into the chat. Never print one, and never ask for one in the chat. The tool reads only these: - `NEXUS_USERNAME` and `NEXUS_PASSWORD`, for `nexus2` and `nexus3` (leave both unset to read anonymously). - `ARTIFACTORY_ACCESS_TOKEN`, for `artifactory`, or `ARTIFACTORY_API_KEY` for a legacy API key. - `CLOUDREPO_TOKEN`, the repository token. The human can set each one without it showing or landing in the shell's history (bash or zsh), such as this, where `IFS=` keeps a space or tab at either end of a password: ```sh IFS= read -rs CLOUDREPO_TOKEN && export CLOUDREPO_TOKEN ``` Your commands see only the environment you were started with. Check each variable without printing it, such as `test -n "$CLOUDREPO_TOKEN"`. If one is missing, ask the human to set it in their shell and start you again from that shell, or to run the commands themselves. If a credential is pasted into the chat anyway, do not use it: ask the human to replace it. ## Steps 1. Check that `python3 --version` is 3.8 or later. If `cloudrepo-migrate --version` does not answer, run `pipx install cloudrepo-migrate` (or `pip install cloudrepo-migrate` in a virtual environment); if it answers a version older than 0.9.0, run `pipx upgrade cloudrepo-migrate` (before 0.9.0, an `artifactory` run moves more than the tested set by default). 2. Check that the credentials are set, without printing them. 3. Run the dry run for the human's server, with their address and organization. Expect exit code `0` and a last line that starts `Would move`. On exit code `2`, read the message: if a target repository is missing, ask the human to create it, then repeat the dry run. If the message says `--repo` names a repository that moves only with `--include`, ask the human whether to add that flag or leave the repository out. A dry run that prints `Skipped N ... repositories: add --include ... (experimental)` has left those out on purpose: tell the human, and add no `--include` unless they ask for that format. ```sh cloudrepo-migrate nexus2 --nexus-url https://nexus.example.com/nexus --org your-org --dry-run cloudrepo-migrate nexus3 --nexus-url https://nexus.example.com --org your-org --dry-run cloudrepo-migrate artifactory --artifactory-url https://artifactory.example.com/artifactory --org your-org --dry-run ``` 4. Run the same command with `--report-json report.json` in place of `--dry-run`. 5. Verify all of these: the exit code is `0`; the last line starts `Moved` and ends `V of F verified, 0 failed.` with `V` equal to `F`; in `report.json`, `failures` is empty, `totals.failed` is `0` and `totals.verified` equals `totals.files`; and no entry in `repositories` has the action `blocked`. 6. Tell the human the last line, every repository with the action `skipped` and its reason (the `Skipped N` lines among them), and every entry in `kept` (a dist-tag CloudRepo already holds, left as it was). On exit code `1`, read each failure, remove its cause, and run the same command again, `--include` and all: it resumes, and a file CloudRepo already holds with the same bytes is not uploaded again. If the same file fails twice, stop and report its path and reason. Change nothing on the Nexus or Artifactory server, and do not repoint builds or CI to CloudRepo until the human says so. ## The guides Each guide has these steps for its server under "For agents", beside the options, what moves and what does not, and what each error message means: - The Nexus 2 guide: https://www.cloudrepo.io/docs/migrate/from-nexus-2.html#for-agents - The Nexus 3 guide: https://www.cloudrepo.io/docs/migrate/from-nexus-3.html#for-agents - The Artifactory guide: https://www.cloudrepo.io/docs/migrate/from-artifactory.html#for-agents