Augment Local Env With Coordinator-side Remote Env

[Bastian-Krause]

Description
In larger labs, maintaining and distributing environment configs for all the different places can become cumbersome. Defining resources, drivers, feature and options per place in the coordinator could help solve that problem.

This prototype adds a remote_env attribute to the Place class, a SetPlaceRemoteEnv method to the Coordinator class and a new sub command labgrid-client edit to edit the remote environment. The remote environment is printed in labgrid-client show and labgrid-client env. The RemotePlaceManager creates resources and drivers after expanding resource matches. If a local environment exists, it is augmented with the remote environment.

This is a prototype. Open tasks:

• Decide how to store configs on the coordinator (DB, git, yaml?)
• Allow defining features and options on target level
• Ignore RemotePlace resource in remote env
• Ignore/override drivers from the coordinator in the local env via RemotePlace.ignore_drivers/RemotePlace.ignore_resources
• Add tests
• Add documentation

Follow-up tasks could be

• Allow editing tags via labgrid-client edit
• Allow editing matches via labgrid-client edit
• Maybe allow imports in remote env

---

Minimal test with labgrid-client

$ export LG_PLACE=test
$ labgrid-client create
$ labgrid-client edit

Save this in your editor:

resources:          
- NetworkService:
    address: mydut.lab.example.com
    username: user
drivers:
- SSHDriver: {}

$ labgrid-client lock
$ labgrid-client ssh

---

Minimal test with pytest (using the place created above in "Minimal test with labgrid-client")
env.yaml:

targets:                          
  main:
    resources:
      RemotePlace:
        name: "test"

$ pytest --lg-env env.yaml

---

Checklist

• Documentation for the feature
• Tests for the feature

• The arguments and description in doc/configuration.rst have been updated

• Add a section on how to use the feature to doc/usage.rst

• Add a section on how to use the feature to doc/development.rst

• PR has been tested

• Man pages have been regenerated

Related to #1866

[codecov]

Codecov Report

❌ Patch coverage is 25.00000% with 78 lines in your changes missing coverage. Please review.
✅ Project coverage is 45.8%. Comparing base (915b591) to head (2c4f2dc).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
labgrid/remote/client.py	6.8%	54 Missing ⚠️
labgrid/remote/coordinator.py	0.0%	16 Missing ⚠️
labgrid/remote/common.py	58.3%	5 Missing ⚠️
labgrid/factory.py	80.0%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##           master   #1889     +/-   ##
========================================
- Coverage    46.0%   45.8%   -0.2%     
========================================
  Files         180     180             
  Lines       14464   14561     +97     
========================================
+ Hits         6654    6674     +20     
- Misses       7810    7887     +77     

Flag	Coverage Δ
3.10	45.8% <25.0%> (-0.2%)	⬇️
3.11	45.8% <25.0%> (-0.2%)	⬇️
3.12	45.8% <25.0%> (-0.2%)	⬇️
3.13	?
3.14	45.8% <25.0%> (-0.2%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[ozan956]

Hi @Bastian-Krause,

Sorry for the late reply, I only found time to look at this now.

Overall, the approach makes sense to me. In the meeting I mentioned defining this kind of configuration via the exporters, mainly because I had the case in mind where the person changing the hardware is physically close to the setup and could update the configuration directly there. But if the remote environment can be edited via the client, then that requirement mostly goes away.

I have a few questions/comments:

First, I am wondering about the relationship between places and targets in this model. So far I have thought of a place mostly as a named group of hardware resources, while a target describes how that hardware is used for a specific use case. With the current implementation, is it possible to store more than one remote environment for a single place? I think this could be useful, for example if the same hardware setup can be used in different ways. On the CLI side, selecting one of multiple remote environments might be relatively straightforward, but I am not sure yet what the best interface would be.

Second, I would like to better understand the intended scope of this feature. Is the goal mainly to augment a local environment with configuration from the coordinator, or is it also intended to allow some users to avoid local env files completely? I think the latter is already achieved within the PR. If one user has configured the remote environment for a place, another user can directly use it without interacting with any env file at all. I think that would be a really nice property for larger labs.

One more related point: in the example, I saw a NetworkService with a username. Currently the username is required there, but in larger labs I think it would be better if user-specific details could stay local to each user. I opened a PR for that: #1863. With that change, a NetworkService could become more easily shareable within the environment, since the username can still be resolved from each user's local SSH config.

Regarding storage, I do not have a very strong opinion yet, but my first preference would probably be something versioned :)

Those were the main things that came to my mind. I would also be happy to help with some of the open tasks if needed.