Repository navigation
fix: apply quant bridge host rewrite and file mounts to selected environment - #691
Conversation
…ronment lean live deploy stores brokerage settings under lean_config["environments"][<env>], so the top-level-only checks in LeanRunner never saw them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- the localhost to host.docker.internal rewrite was copied in both docker config builders, so a later fix could land in one and miss the other - the method takes the config and returns it, so the change is visible at the call site instead of hidden inside a helper - debug lines say which host was replaced and which file was mounted where, for live deploy troubleshooting
There was a problem hiding this comment.
The fix is correct and matches #690. Nothing here blocks the code itself. What's missing is evidence, listed below.
What I checked
lean live deploycreateslean_config["environments"][environment_name](lean/commands/live/deploy.py:207-210) and the module settings go into it (lean/components/util/json_modules_handler.py:39).get_complete_lean_configsetslean_config["environment"](lean_config_manager.py:227), so_get_top_level_and_selected_environmentfinds the right scope.- In the published modules JSON,
quant-bridge-symbol-map-fileis aninternal-input(valuequant-bridge-symbol-map.json), not apath-parameter. Somount_paths(step 1 of_mount_common_directories) never writes/Files/...into the environment before step 6 reads it, and nothing gets mounted twice or from the wrong source. - A common case is a
lean.jsonthat already holds top-level Quant Bridge keys, saved by an earlierlean data download. Here, step 6 now mounts the environment's value and rewrites both scopes to/Files/<key>. The host rewrite also runs on both scopes. Before this change, Lean read the environment's unrewritten values, because environment entries override top-level ones. The new single-mount test covers this case. - The top-level-only paths (
lean backtest,lean data download, research, optimize) behave as before: the scope list is just[lean_config]when there's no selected environment, and.get()handles a selected environment missing fromenvironments. - The environment lookup follows the same pattern
mount_pathsand_mount_lean_config_and_finalizealready use. No other host keys or file keys inLeanRunnerhave the same problem.
Changes requested
- CI hasn't run. The Build and Regression Testing workflows for both pushes are in
action_required, waiting for a maintainer to approve them. Because the PR author isn't a member, I only read the code and didn't run it. A maintainer needs to approve the runs, and they need to pass. - Description proof. "They fail before the change and pass after it" needs the commands and their output, e.g.
pytest tests/components/docker/test_lean_runner.pyonmasterwith the new tests (failing) and on this branch (passing), plus the wider suite you ran. The description also predates the third commit: it still says_get_config_scopes(now_get_top_level_and_selected_environment), and it doesn't mention_replace_localhost_with_docker_hostor the new debug log lines. - End-to-end run. This changes what
lean live deploypasses to the Lean container. Please runlean live deploywith--brokerage "Quant Bridge" --data-provider-live "Quant Bridge" --quant-bridge-server-host localhost. Then report the outcome: the generatedconfig.jsonshould havehost.docker.internaland/Files/quant-bridge-symbol-map-fileunder the environment, and the container should have the symbol map mount. The command should also run with a top-levelquant-bridge-symbol-map-filealready inlean.json. Lean doesn't need to connect to a Quant Bridge server for this check: the config and mounts show the result. If the run isn't possible, say why in the description.
Affected repositories: none. Lean already applies environment overrides, and the modules JSON needs no change.
Risk: I set risk:high. The change is small, backwards compatible and easy to revert, but CI hasn't run and there's no end-to-end run yet. Passing CI and the run in item 3 would support risk:low.
There was a problem hiding this comment.
Follow-up to my review of 2210781. There are no new commits, comments or description edits since then, so this only covers CI.
What changed
- Build passed on 2210781, on every OS and Python version in the matrix. The pull request's own Build run is still
action_required, but the same commit was also pushed to a branch in this repository, and that push ran Build. So the unit tests, the newtest_lean_runner.pycases included, pass in CI. - Regression Testing still hasn't run. Its
pushtrigger isbranches: ['*'], and*doesn't match/, so pushingfix/live-deploy-brokerage-env-settingsdidn't start it. Itspull_requestrun is stillaction_required. It's the workflow that runstests/test_cli.py, which drives the CLI through Docker, and that's the path this change touches. A maintainer needs to approve that run, and it needs to pass.
My earlier read of the code still holds. I found nothing new in it.
Still requested
- Regression Testing: approve and pass the run, as above.
- Description proof (unchanged from last time): the
pytestcommands and their output, showing the new tests failing onmasterand passing on this branch, plus the wider suite you ran. The description also needs to match the code: it names_get_config_scopes, which is now_get_top_level_and_selected_environment, and it doesn't mention_replace_localhost_with_docker_hostor the new debug log lines. - End-to-end run (unchanged from last time):
lean live deploywith--brokerage "Quant Bridge" --data-provider-live "Quant Bridge" --quant-bridge-server-host localhost. Report two things: that the generated config hashost.docker.internaland/Files/quant-bridge-symbol-map-fileunder the environment, and that the container has the symbol map mount. Also do the same run with a top-levelquant-bridge-symbol-map-filealready inlean.json. If the run isn't possible, say why in the description.
Affected repositories: none.
Risk: still risk:high. Build is green now, but Regression Testing and the end-to-end run are both missing. With those two, the change is small, backwards compatible and easy to revert, which would support risk:low.
There was a problem hiding this comment.
Follow-up to my review of 2210781. The head commit and the description are the same as last time. The one change is that the pull request's CI runs were approved and finished.
What changed
- CI is green. Build and Regression Testing both passed on 2210781 for the pull request, so item 1 from my last review is done.
- Keep in mind what Regression Testing actually ran. In
Run CLI tests,tests/test_cli.pyreports1 skipped("API credentials not specified"), because a pull request from a fork gets no repository secrets. That suite doesn't coverlean live deployanyway, so the skip doesn't hide anything this change touches. But the green run doesn't exercise the deploy path through Docker either. The newtest_lean_runner.pycases in the unit run are the only CI coverage of the change.
My earlier read of the code still holds. I found nothing new in it.
Still requested
- Description proof (unchanged): the
pytestcommands and their output, showing the new tests failing onmasterand passing on this branch, plus the wider suite you ran. The description also needs to match the code: it names_get_config_scopes, which is now_get_top_level_and_selected_environment, and it doesn't mention_replace_localhost_with_docker_hostor the new debug log lines. - End-to-end run (unchanged):
lean live deploywith--brokerage "Quant Bridge" --data-provider-live "Quant Bridge" --quant-bridge-server-host localhost. Report two things: that the generated config hashost.docker.internaland/Files/quant-bridge-symbol-map-fileunder the environment, and that the container has the symbol map mount. Then do the same run with a top-levelquant-bridge-symbol-map-filealready inlean.json. Lean doesn't need to connect to a Quant Bridge server for this. Anyone with Docker can run it, including a maintainer. If it can't be run, say why in the description.
Affected repositories: none.
Risk: still risk:high. CI is green now, and the change is small, backwards compatible and easy to revert. What risk:low still lacks is the end-to-end run in item 2, because no test in CI exercises the deploy path through Docker.
|
Thanks. The description is updated: it now uses the current helper names ( This comment was drafted with Claude Code. |
There was a problem hiding this comment.
Follow-up to my review of 2210781. The head commit is the same. Since then the description has been updated, which covers both items I had left open.
What changed
- Description proof: done. The description now has the
pytestcommands and their output: the 4 new cases fail onmasterand pass on this branch (35 passed), and the wider suite passes (2783 passed). It also uses the current helper names and mentions the debug log lines. - End-to-end run: done. A QuantConnect maintainer ran
lean live deploywith Quant Bridge as brokerage and live data provider and--quant-bridge-server-host localhost, from alean.jsonthat already had top-levelquant-bridge-*keys. The run checks what I asked for and would have failed if the bug were still there:localhostis rewritten tohost.docker.internalboth at the top level and underenvironments.lean-cli.- The container has the bind mount to
/Files/quant-bridge-symbol-map-file, and the environment entry points at it. - With a top-level
quant-bridge-symbol-map-fileadded, only the environment's file is mounted, and both scopes point at/Files/quant-bridge-symbol-map-file.
This matches the code: in_mount_common_directories,configs[-1]is the environment's value, and every scope that holds the key gets rewritten.
- CI: Build (push and pull request) and Regression Testing both pass on 2210781.
My earlier read of the code still holds: the environment lookup matches what mount_paths already does, and commands that don't select an environment (lean backtest, lean data download, research, optimize) behave as before.
A note, not blocking: the run also showed that Lean can't load QuantBridgeBrokerage yet, because the published brokerage module package predates the plugin rename in #685. That's a release step for the brokerage plugin, outside this repository and outside this fix. Quant Bridge live deployments won't start until that package is published.
Affected repositories: none for this change. The brokerage plugin package needs publishing, as noted above.
Risk: I set risk:low. The change is small, backwards compatible and easy to revert. It only touches how LeanRunner applies two existing rewrites to the selected environment. It has unit tests that fail without it, green CI and an end-to-end run that exercises the deploy path.
3393cbd
into
QuantConnect:master
Fixes #690
Cause
LeanRunneronly looked at the top level of the Lean config when rewritingquant-bridge-server-host(localhost/127.0.0.1->host.docker.internal) and when mountingtransaction-log/quant-bridge-symbol-map-file.lean live deploywrites brokerage and live data provider settings underlean_config["environments"][<environment>], and Lean gives those precedence, so the container got the unrewrittenlocalhostand no symbol map mount.Fix
_get_top_level_and_selected_environmentreturns the top-level scope and the selected environment's entries._replace_localhost_with_docker_hostapplies thehost.docker.internalrewrite to both scopes (get_basic_docker_config,get_basic_docker_config_without_algo)._mount_common_directoriesmounts file keys (/Files/<key>) from both scopes. A key set in both places is mounted once, using the environment's value.Commands that use no environment (
lean backtest,lean data download, research, optimize) only have the top-level scope, so their behavior is unchanged.Tests
Four new cases in
tests/components/docker/test_lean_runner.py: host rewrite with the setting under the selected environment (localhostand127.0.0.1), symbol map mount from the environment, and a key set in both places (single mount).lean/frommasterwith the new tests:This branch:
tests/test_cli.py(the Regression Testing workflow) drives the CLI through Docker with QuantConnect credentials, so it was not run locally. Its run on this PR needs maintainer approval.End-to-end run
Run by a maintainer on Windows, Docker Desktop 29.5.2, image
quantconnect/lean:latest(Lean 18222), from a CLI root whoselean.jsonalready carries top-levelquant-bridge-*keys from an earlier run, including"quant-bridge-server-host": "localhost":quant-bridge-symbol-map-file:--verboseoutput:_replace_localhost_with_docker_host(): replacing 'quant-bridge-server-host' value 'localhost' with 'host.docker.internal'twice (top level and environment), and_mount_common_directories(): mounting 'D:\Source\data\symbol-properties\quant-bridge-symbol-map.json' to '/Files/quant-bridge-symbol-map-file'.docker inspect: bind mountD:\Source\data\symbol-properties\quant-bridge-symbol-map.json -> /Files/quant-bridge-symbol-map-file.config.jsoninside the container: top level"quant-bridge-server-host": "host.docker.internal";environments.lean-clihas"quant-bridge-server-host": "host.docker.internal","quant-bridge-symbol-map-file": "/Files/quant-bridge-symbol-map-file","live-mode-brokerage": "QuantBridgeBrokerage"."quant-bridge-symbol-map-file": "root-symbol-map.json"added at the top level oflean.json:quant-bridge-symbol-map.json;root-symbol-map.jsonwas neither created nor mounted;environments.lean-cliread"quant-bridge-symbol-map-file": "/Files/quant-bridge-symbol-map-file", both hostshost.docker.internal.Not related to this change: Lean then logs
Not able to fetch brokerage factory with name: QuantBridgeBrokerage, because the latest published module packageQuantConnect.Brokerages.TerminalLink 2.5.18217(built 2026-10-29) predates the plugin rename and only containsTerminalLinkBrokerageFactory. The config and mounts above are produced before that point. A package built from the renamed plugin is needed for Quant Bridge live deployments to start.This change was prepared with the assistance of an AI agent (Claude, by Anthropic).