New test stories added in ci/journey.sh#48
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds tiered-mode journey coverage for database-backed archiver configuration, partitioner commands, hot-table lifecycle behavior, empty partitions, invalid S3 endpoints, and composite partition-key rejection. The new stories are appended to tiered orchestration. ChangesArchiver and partitioner behavior
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Up to standards ✅🟢 Issues
|
…, hot table rename, cold-tier E2E, retention update, disable/enable, and unregister
…040 INSERT timestamp and restore assertion
… mesh mode The previous main.go change removed DETACH CONCURRENTLY from cleanupAlreadyArchived and replaced it with a direct DROP TABLE to work around TC-024 failing in mesh mode. The failure was in partMgr.Detach which fans out to Spock peers by connecting to each peer DSN from spock.node_interface — those hostnames are not resolvable from the archiver host in the CI mesh environment. That introduced a locking regression: DROP TABLE on an attached partition takes ACCESS EXCLUSIVE on the entire parent table, blocking all reads and writes across all partitions. The DETACH CONCURRENTLY mechanism was specifically built to avoid this. The correct fix is scoped to the test: story_partitioner_fk_drop already skips in mesh mode for the exact same reason. TC-024 follows the same pattern.
a900708 to
19885ac
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ci/journey.sh`:
- Around line 2062-2081: Update the TC-119 round-trip flow after the ARCHIVER
export so the import consumes the exported artifact’s cli_events configuration
rather than the hand-written /tmp/journey-roundtrip.yaml entry. Extract the
complete cli_events entry, preserving serialized fields such as
partition_period, hot_period, and enablement, then use that derived artifact for
import while retaining the export validation.
- Around line 2545-2563: The TC-053 archiver invocations must be validated
independently instead of masking failures or reusing prior output. Update both
commands around the disabled and re-enabled checks to write to separate fresh
log files, fail immediately when the archiver exits non-zero, and grep only the
corresponding invocation’s log for the [events] assertion; retain the existing
enable command behavior and messages.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: c713f966-d15f-4a62-88be-86c11dbfa244
📒 Files selected for processing (1)
ci/journey.sh
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ci/journey.sh`:
- Around line 2542-2569: The TC-053 disabled and re-enabled archiver invocations
must not fail the story solely because they return non-zero for an expected
no-op. Keep the fresh per-run log files and append non-fatal handling such as
`|| true` to both `"$ARCHIVER" --config /tmp/journey-disen.yaml` calls before
the existing grep-based log assertions.
- Around line 2062-2085: Update the TC-119 flow around the existing cli_events
deletion and re-import checks to snapshot partition_period, hot_period,
retention_period, and enabled before deletion, then query and compare each value
after import. Keep the existing row-existence and import-success assertions, but
make the test fail when any restored configuration value differs from its
pre-deletion snapshot.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 774a790e-7ef7-4870-a427-095b6f55735f
📒 Files selected for processing (1)
ci/journey.sh
| # TC-119: export, delete the row, then re-import the EXPORTED archiver.tables | ||
| # to restore it. export omits connection config (per-node), so wrap the real | ||
| # exported block with a DSN before importing. import upserts (ON CONFLICT DO | ||
| # UPDATE), so the rows still present are no-ops and cli_events is restored from | ||
| # its exported entry, validating the serialization a hand-written YAML wouldn't. | ||
| "$ARCHIVER" export --config /tmp/journey-conn.yaml >/tmp/journey-export.yaml 2>&1 | ||
| if grep -q "source_table: cli_events" /tmp/journey-export.yaml; then | ||
| pass "TC-119: export produced YAML with cli_events" | ||
| else | ||
| fail "TC-119: export missing cli_events"; tail -3 /tmp/journey-export.yaml | ||
| fi | ||
| cat > /tmp/journey-roundtrip.yaml <<EOF | ||
| postgres: { dsn: "host=${DB_IP} port=5432 dbname=coldfront user=coldfront password=coldfront sslmode=disable" } | ||
| iceberg: { warehouse: "${WAREHOUSE}", lakekeeper_endpoint: "http://${LK_IP}:8181/catalog" } | ||
| $(storage_yaml) | ||
| EOF | ||
| cat /tmp/journey-export.yaml >>/tmp/journey-roundtrip.yaml | ||
| q "$HOST" "DELETE FROM coldfront.partition_config WHERE schema_name='public' AND table_name='cli_events';" >/dev/null | ||
| assert_eq "TC-119: cli_events deleted from partition_config" "0" \ | ||
| "$(q "$HOST" "SELECT count(*) FROM coldfront.partition_config WHERE table_name='cli_events';")" | ||
| if "$ARCHIVER" import --config /tmp/journey-roundtrip.yaml >/tmp/journey-roundtrip.log 2>&1; then | ||
| pass "TC-119: re-import of exported artifact succeeded" | ||
| else | ||
| fail "TC-119: import failed (see /tmp/journey-roundtrip.log)"; tail -5 /tmp/journey-roundtrip.log |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Assert restored configuration values, not only row existence.
A valid export can serialize an incorrect but acceptable partition_period, hot_period, retention_period, or enabled value; import still succeeds and the row-count assertion passes. Snapshot these fields before deletion and compare them after import.
Proposed fix
+ local cli_events_config_before
+ cli_events_config_before=$(q "$HOST" "SELECT jsonb_build_object(
+ 'partition_period', partition_period,
+ 'hot_period', hot_period,
+ 'retention_period', retention_period,
+ 'enabled', enabled
+ )::text FROM coldfront.partition_config WHERE schema_name='public' AND table_name='cli_events';")
+
q "$HOST" "DELETE FROM coldfront.partition_config WHERE schema_name='public' AND table_name='cli_events';" >/dev/null
assert_eq "TC-119: cli_events deleted from partition_config" "0" \
"$(q "$HOST" "SELECT count(*) FROM coldfront.partition_config WHERE table_name='cli_events';")"
@@
assert_eq "TC-119: cli_events row restored in partition_config" "1" \
"$(q "$HOST" "SELECT count(*) FROM coldfront.partition_config WHERE table_name='cli_events';")"
+ assert_eq "TC-119: cli_events configuration round-tripped" "$cli_events_config_before" \
+ "$(q "$HOST" "SELECT jsonb_build_object(
+ 'partition_period', partition_period,
+ 'hot_period', hot_period,
+ 'retention_period', retention_period,
+ 'enabled', enabled
+ )::text FROM coldfront.partition_config WHERE schema_name='public' AND table_name='cli_events';")"🧰 Tools
🪛 ast-grep (0.44.1)
[warning] 2066-2066: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/journey-conn.yaml
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
[warning] 2066-2066: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/journey-export.yaml
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
[warning] 2067-2067: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/journey-export.yaml
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
[warning] 2070-2070: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/journey-export.yaml
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
[warning] 2072-2072: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/journey-roundtrip.yaml
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
[warning] 2077-2077: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/journey-export.yaml
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
[warning] 2077-2077: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/journey-roundtrip.yaml
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
[warning] 2081-2081: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/journey-roundtrip.yaml
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
[warning] 2081-2081: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/journey-roundtrip.log
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
[warning] 2084-2084: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/journey-roundtrip.log
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ci/journey.sh` around lines 2062 - 2085, Update the TC-119 flow around the
existing cli_events deletion and re-import checks to snapshot partition_period,
hot_period, retention_period, and enabled before deletion, then query and
compare each value after import. Keep the existing row-existence and
import-success assertions, but make the test fail when any restored
configuration value differs from its pre-deletion snapshot.
1983b15 to
5d61c6e
Compare
|
@coderabbitai review |
✅ Action performedReview finished.
|
2 participants
TC-022 — Verifies that a wrong S3 endpoint causes the archiver to fail at Phase 2 with a clear connection error naming the bad host
TC-024 — Verifies that an empty (0-row) partition completes all six archiver phases cleanly and is dropped from PostgreSQL
TC-040 — Verifies that renaming the hot table triggers the DDL hook to update the registry and rebuild the view, and DML via the view continues to route correctly
TC-052 — Verifies that partitioner set --retention updates partition_config in the database without touching the YAML file
TC-053 — Verifies that a disabled table is silently excluded from archiver runs and re-enabling it restores normal processing
TC-054 — Verifies that partitioner remove deletes the partition_config row while leaving the physical table intact
TC-071 — Verifies that a composite PARTITION BY RANGE (col1, col2) key is rejected by the archiver before any Iceberg or S3 work
TC-118 — Verifies that an archiver.tables block in the YAML config is ignored at runtime; the archiver always drives off partition_config
TC-119 — Verifies that export → delete row → import restores a partition_config entry (backup/restore cycle works correctly)
TC-120 — Verifies that archiver set writes only to partition_config; the YAML config file on disk is not modified