Sina Rezaei

Exactly. And sometimes the query isn't even the real problem. In PostgreSQL, bad row-count estimates can push the planner toward a poor join strategy, so adding an index won't necessarily fix the endpoint. You need to look at the actual execution plan, compare estimated vs. actual rows, and measure the change under realistic data. AI can suggest the fix, but you still need to know what evidence would prove it worked.

The migration angle gets even trickier with multi-tenant data. Imagine cleaning up orphaned membership records using an anti-join against an external identity mapping table. If the query only checks the user ID and ignores the tenant ID, you can accidentally classify a valid membership as orphaned because the same user exists under a different tenant. I'd test the NULL cases separately, but also make the tenant boundary part of the correlation itself. Something like NOT EXISTS (... WHERE mapping.tenant_id = membership.tenant_id AND mapping.external_id = membership.external_id) makes the intended relationship explicit. Whether a NULL external ID should count as unmatched is a separate business rule, not something I'd leave to SQL's three-valued logic. I've seen people focus on swapping NOT IN for NOT EXISTS and miss the more serious bug: the query is technically valid but defines the wrong relationship. Do you usually test these anti-joins against cross-tenant collisions as well as NULLs?

One thing I’d watch in a document-processing pipeline is what happens when a batch partially fails while the retry queue is already backing up. Say you're processing millions of support events, and each result updates a customer record plus emits a downstream event. If a worker crashes after saving the record but before acknowledging the batch item, replaying it can duplicate the downstream event even if the database write itself is idempotent. Meanwhile, retrying every failed item immediately can push the API straight back into the same rate limit. I’d track each item by a stable ID, persist its processing state, and use an outbox or equivalent mechanism for downstream side effects. Then retries can target only failed items, with concurrency adjusted against token consumption rather than request count alone. The part I find tricky is deciding when to requeue an item versus parking it for investigation. How do you handle that boundary when the batch contains a mix of transient API failures and failures in your own processing pipeline?

The op limitation is an interesting one because exposing the CLI isn't quite the same as granting safe access to secrets. Imagine an agent working on a deployment that needs a database credential from 1Password. If it can run arbitrary op commands with a broadly privileged service account, a prompt injection hidden in a dependency README could try to turn that convenience into access to unrelated vault items. Even if the agent never sends an email, it could still leak a secret through a build log or an outbound request. I'd be tempted to put a small host-side broker between the agent and op: allow specific secret references, inject values only into the process that needs them, and keep the vault token out of the agent's environment. Add an outbound network allowlist, and you've reduced the blast radius without taking away the workflow. The tricky part is preserving that boundary when the agent can execute arbitrary shell commands. Have you considered exposing a narrow secret-access interface rather than the full CLI?

There’s another edge case here: what if the documents needed to resolve a conflict exist, but the user isn’t allowed to retrieve them? Take a fraud investigation where an analyst can access the standard escalation procedure, but a case-specific exception is restricted to the compliance team. The retrieved procedure may look authoritative, and the answer may be perfectly grounded in what the analyst can see, yet still lead to the wrong action because a material exception is missing. I’d test this by running the same question under different permission scopes and checking whether the system can detect insufficient evidence without exposing the restricted content. That’s a different failure from a retrieval miss, and reranking won’t fix it. How do you distinguish a genuine conflict from a conflict the current user simply can’t see?

One edge case worth covering is a timeout after the database commits but before the executor records success. If the queue retries the migration blindly, you can end up applying a non-idempotent operation twice or marking a completed change as failed. I'd persist an execution ID and reconcile the actual resource state before retrying, so recovery doesn't depend on guessing whether the previous attempt made it through.

The rollback section is especially important for event-driven systems. Imagine a Java 25 deployment starts publishing a new enum value to Kafka, then you roll the service back to Java 11 because of a runtime issue. The old consumer may fail to deserialize events that were already written, even though the old JAR itself is perfectly recoverable. I'd add a compatibility test that replays messages produced by both versions against both consumers before rollout. Rolling back binaries is one thing; rolling back the data contract is another.

A tricky case would be SLA reporting in a support platform. You can calculate resolution time from timestamps, but enterprise tickets may have paused SLA clocks while waiting on the customer, and different support tiers may have different targets. A computed column can be perfectly valid and still flag the wrong tickets if those rules aren't reflected in the metric. I'd want to test the calculation against edge cases before letting users group and chart the results for operational decisions.

The interesting trade-off here is how much of the local environment an agent actually needs. Take a monorepo where a task requires running a specific Homebrew-installed tool, generating code, and spinning up dependent services for integration tests. A sandbox that blocks those workflows pushes you toward workarounds, but letting the agent escape its boundaries too easily creates a different problem. I’d rather expose the exact binaries, writable paths, and network endpoints the task needs than make broader permissions the default.

The interesting part is that the agent can improve against the test suite while the test suite itself is still missing important cases. For meeting summaries, I’d keep a holdout set with things like corrected figures, negated decisions, and action items assigned to the wrong person, then check whether changes actually improve those cases too. Otherwise, you can end up optimizing for the harness rather than the product. The test harness needs testing as well.