Add --cargo passthrough support for cargo commands

[isdaniel]

Motivation

Fixes #2135.

In monorepo/submodule layouts, cargo resolves .cargo/config.toml by walking up from the CWD, so cargo pgrx install picks up the parent repo's config and fails. cargo build solves this with --config, but cargo pgrx install had no equivalent. (PGRX_BUILD_FLAGS only reaches the build step, not the cargo metadata call that runs first — so source/registry/proxy config still breaks it.)

How to fix

Add a repeatable --config <KEY=VALUE | path> flag to cargo pgrx install, mirroring cargo's own flag, and forward it to every cargo call install makes: cargo metadata (via MetadataCommand::other_options), cargo rustc, and the schema generator's cargo run.

Results

cargo pgrx install now accepts a repeatable --config <KEY=VALUE | path>, forwarded to every cargo call install, run.

Example: a dependency from a private registry

A child extension depends on a crate from an internal registry, defined only in the child's .cargo/config.toml:

# child/.cargo/config.toml
[registries.mycorp]
index = "sparse+https://cargo.mycorp.com/index/"
# child/Cargo.toml
[dependencies]
internal-crate = { version = "1.0", registry = "mycorp" }

Before

building from the parent repo picks up the parent's config (which lacks [registries.mycorp]), and install dies in the very first step, cargo metadata:

$ cargo pgrx install --pg-config ...
Error: couldn't get cargo metadata
Caused by:
registry index was not found in any configuration: `mycorp`

(PGRX_BUILD_FLAGS can't help here — it only reaches the build step, never cargo metadata.)

After

point install at the right config:

$ cargo pgrx install --pg-config ... --config child/.cargo/config.toml
# or inline:
$ cargo pgrx install --pg-config ... \
  --config 'registries.mycorp.index="sparse+https://cargo.mycorp.com/index/"'
  Updating `mycorp` index
  Building extension ...

cargo metadata now resolves internal-crate against the correct registry and install proceeds.

[eeeebbbbrrrr]

What's the use-case for this? I'd like to see a real-world example of where this is needed. It's curious to me that after all these years this is now a concern.

EDIT: I see the original bug report, of course. @hakashya can you provide more details about your project layout?

[isdaniel]

Hi @eeeebbbbrrrr I don't know what's the use case from @hakashya, but below is the used case which I noticed that why I create this PR, please kindly check

Real use-case: private-registry crate in a monorepo submodule

An extension is a submodule of a larger monorepo and depends on an internal crate from a private registry. Per cargo convention, that registry is declared in the submodule's own .cargo/config.toml (so the submodule stays buildable standalone, and to avoid clashing with other submodules in the shared root config).

Cargo resolves .cargo/config.toml from the CWD upward, so building from the monorepo root inherits the parent config which doesn't know the private registry — and cargo pgrx install fails at its first step, cargo metadata:

cargo metadata ... error: registry index was not found in any configuration: `acme`

A runnable end-to-end repro lives at repro/install-config-2135 , we could refer more detail from this commit (39a2fa0)

cargo build solves this with --config; this PR gives cargo pgrx install (and package/run) the same flag, forwarded to the cargo invocations install makes: cargo metadata, cargo rustc, the get_version metadata, and the schema-gen cargo run.

Output:

$ ./repro.sh
==> [1/6] build & start the private registry (Docker)
  Up 4 seconds
==> [2/6] publish acme-internal to the private registry
  published acme-internal v0.1.0
==> [3/6] stage the monorepo (childext depends on acme-internal)
==> [4/6] BEFORE: cargo pgrx install from the monorepo root (no --config)
  Error:
	 0: couldn't get cargo metadata
	 1: `cargo metadata` exited with an error: error: failed to parse manifest at `/tmp/acme-demo/monorepo/childext/Cargo.toml`
		Caused by:
		  registry index was not found in any configuration: `acme`
	 Location: cargo-pgrx/src/metadata.rs:36
	 0: cargo_pgrx::command::install::execute at cargo-pgrx/src/command/install.rs:68
==> [5/6] AFTER: same command + --config childext/.cargo/config.toml
	 Running command "cargo" "rustc" "--lib" "--config" "childext/.cargo/config.toml" "--manifest-path" "childext/Cargo.toml"
...
	 Compiling acme-internal v0.1.0 (registry `acme`)
	 Compiling childext v0.0.0
  Finished installing childext
==> [6/6] SQL proof: the private crate runs inside Postgres
  CREATE EXTENSION
   acme_secret
  -------------
			42
  (1 row)
==> DONE — acme_secret() == 42 came from the private-registry crate.

Generally speaking, When the root project's .cargo/config.toml is not ours to change, this PR lets us point cargo pgrx install at the submodule's own .cargo/config.toml via --config, so cargo resolves the private-registry crate from the right registry.

[eeeebbbbrrrr]

I guess what I don't like about this is that it's plumbing through specialized support for just one cargo argument.

Should we take a step back and try to generalize this so that cargo pgrx exposes some way to pass through any of cargo's arguments?

What I don't want to end up with is a situation where I say that this particular approach is fine and in a week/month someone PRs another and then another and then another and we've ended up making a mess of pgrx's primary user-facing tool.

If this is a general problem class, perhaps we ought to solve it generally.

[isdaniel]

Hi @eeeebbbbrrrr

Understood and I agree with your point, and I also noticed there is an Env PGRX_BUILD_FLAGS, but it's not suitable here it's build-only, and forwarding it to cargo metadata would be breaking: cargo metadata rejects build-only flags (--release, --jobs, --target, …) that users may already have set. So PGRX_BUILD_FLAGS is left untouched, and PGRX_CARGO_FLAGS is new + defaults to empty no behavior change for anyone, no new CLI flag.

What: a new env var PGRX_CARGO_FLAGS, forwarded to every cargo invocation cargo pgrx install/package/run makes (cargo metadata, cargo rustc, schema cargo run). For cargo's global flags (--config, --offline, --frozen, --locked, -Z):

PGRX_CARGO_FLAGS="--config <submodule>/.cargo/config.toml" cargo pgrx install ...

I also change the repro example to demonstrate real use case which we need PGRX_CARGO_FLAGS to support more parameters, please let me know if that work and I can remove that example before merged PR.

[eeeebbbbrrrr]

I'm not sure I like having another envvar for this. Can we not support something like cargo pgrx install --cargo f1 --cargo f2 or even cargo pgrx install --cargo "<cargo flags go here>"?

I think this will be easier for robots and humans and also easier for cargo-pgrx to validate, if it needs to.

[isdaniel]

HI @eeeebbbbrrrr

Ok, I changed mechanism to parameter flag as you mentioned, please let me know if you have any question about that.

--cargo on install / package / run / test, forwarded to every cargo call those commands make (cargo metadata, cargo rustc, get_version, cargo test). cargo metadata matters most — it's the first call and where the private-registry case fails.

Both forms you suggested work:

• cargo pgrx install --cargo=--config=child/.cargo/config.toml --cargo=--offline
• cargo pgrx install --cargo "--config child/.cargo/config.toml --offline"

~/pgrx$ cargo run -p cargo-pgrx --bin cargo-pgrx -- pgrx install --help
Install the crate as an extension into the Postgres specified by `pg_config`

Usage: cargo pgrx install [OPTIONS]

Options:
  -p, --package <PACKAGE>              Package to build (see `cargo help pkgid`)
      --manifest-path <MANIFEST_PATH>  Path to Cargo.toml
  -v, --verbose...                     Enable info logs, -vv for debug, -vvv for trace
  -r, --release                        Compile for release mode (default is debug)
      --profile <PROFILE>              Specific profile to use (conflicts with `--release`)
      --test                           Build in test mode (for `cargo pgrx test`)
  -c, --pg-config <PG_CONFIG>          The `pg_config` path (default is first in $PATH)
  -s, --sudo                           Use `sudo` to install the extension artifacts
      --all-features                   Activate all available features
      --no-default-features            Do not activate the `default` feature
  -F, --features <FEATURES>            Space-separated list of features to activate
      --target <TARGET>                
      --cargo <FLAG>                   Extra cargo flags forwarded to every `cargo` invocation. Repeatable and split on whitespace:
                                       `--cargo=--config=foo` or `--cargo "--offline --frozen"`
  -h, --help                           Print help
  -V, --version                        Print version

Supported commands

--cargo is exposed on the commands that drive a cargo build / metadata resolution, and is forwarded to every cargo invocation they make (cargo metadata, cargo rustc, the schema generator's cargo run, and — for test/bench — cargo test):

Command	--cargo	Notes
install	✅
package	✅
run	✅
test	✅	forwarded to cargo test
bench	✅	builds, so it forwards to cargo metadata / cargo rustc / cargo test

[eeeebbbbrrrr]

cargo pgrx regress does a build too.

But I like where this is going!

[isdaniel]

Good call regress builds too, so I've added --cargo to it and wired it through both cargo paths it hits.

--cargo is now on all six build-driving commands: install / package / run / test / bench / regress, I also noticed schema should build as well, which I implement in same commit;

[isdaniel]

Hi @eeeebbbbrrrr hope you are doing well, please let me know if you have any question regarding this PR, thanks.