Concept of checkbeam device

[oliwenmandiamond]

Fixes #ISSUE

Instructions to reviewer on how to test:

1. Do thing x
2. Confirm thing y happens

Checks for reviewer

• Would the PR title make sense to a scientist on a set of release notes
• If a new device has been added does it follow the standards
• If changing the API for a pre-existing device, ensure that any beamlines using this device have updated their Bluesky plans accordingly
• Have the connection tests for the relevant beamline(s) been run via dodal connect ${BEAMLINE}

[codecov]

Codecov Report

❌ Patch coverage is 91.22807% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.07%. Comparing base (e243681) to head (b349dee).

Files with missing lines	Patch %	Lines
src/dodal/devices/pause_plan_device.py	93.75%	3 Missing ⚠️
src/dodal/beamlines/i09.py	77.77%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1975      +/-   ##
==========================================
- Coverage   99.10%   99.07%   -0.04%     
==========================================
  Files         318      319       +1     
  Lines       12133    12190      +57     
==========================================
+ Hits        12025    12077      +52     
- Misses        108      113       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[rtuck99]

I wonder if there would be any benefit in supporting WatchableAsyncStatus with this - I have done something similar for hyperion waiting-for-beam, because it enables the supervisor to monitor it for progress to determine whether the plan is genuinely stuck or merely waiting for beam.

[DominicOram]

Have you looked at using suspenders for this? I think it's the more Bluesky-native way of doing it

[oliwenmandiamond]

Yeah, I've had a look and they do look to be what we want. My only concern is that they are installed globally on the RunEngine so applied to every plan. This is the opposite of what GDA currently does where it is only applied when user specifies it in the scan. We would also need BlueAPI to support being able to add them.

e.g

@devices.suspender
def checkbeam() -> MySuspender:
     ...

then on blueapiclient side we could maybe do this

bc = BlueapiClient()
devs = bc.devices
plans = bc.plans

# Installed suspenders by default
`>>> plans.scan(...)`

#Run a plan without the suspender
>>> bc.remove_suspender("checkbeam")
>>> bc.my_plan()

>>> bc.install_suspender("checkbeam")
>>> bc.scan(...)

As I mentioned in one of the previous athena drop in sessions, it feels like DeviceManager will have to evolve to a BeamlineManager to being able to support things that aren't devices. Or maybe we create lots of sub managers to manage specific things e.g SuspenderManager which blueapi can also register like DeviceManager. However, suspenders need access to device and their signals to construct one.

[DominicOram]

My only concern is that they are installed globally on the RunEngine so applied to every plan.

You can do it inside the plan. Using install or remove e.g.

def my_plan():
   yield from bps.install_suspender(checkbeam)
   ...
   yield from bps.remove_suspender(checkbeam)

or more cleanly (and to cover the finally case):

@bpp.suspend_wrapper(checkbeam)
def my_plan():
    ...

[oliwenmandiamond]

My only concern is that they are installed globally on the RunEngine so applied to every plan.

You can do it inside the plan. Using install or remove e.g.

def my_plan():
   yield from bps.install_suspender(checkbeam)
   ...
   yield from bps.remove_suspender(checkbeam)

or more cleanly (and to cover the finally case):

@bpp.suspend_wrapper(checkbeam)
def my_plan():
    ...

Okay I didn't know this. My issue with doing it like this though is that you now have to wrap every plan with this where as with it being a device like GDA, it can be used with any plan whenever required dynamically as you just specify the device to the plan arguments. This way is less flexible.

Also, how do we apply this with BlueAPI?

Do we have to define a suspender instance in dodal and then have our plan repos import the suspender and add this as a decorator?

[oliwenmandiamond]

To make it generic, I think we would have to have blueapi support it as server object so then from client side we could do

>>> plans.install_suspender(bc.suspenders.checkbeam)

So we can turn on / off whenever user requires it.

If it is a server object too, again we can use inject for plans to auto include them. Another use case for #1997 and DiamondLightSource/blueapi#1462

[DominicOram]

My issue with doing it like this though is that you now have to wrap every plan with this where as with it being a device like GDA, it can be used with any plan whenever required dynamically as you just specify the device to the plan arguments. This way is less flexible.

Ok, in which case having an endpoint or configuration option on BlueAPI to add it to the RE seems like a better way to go.

Also, how do we apply this with BlueAPI?

I'm not sure I understand, BlueAPI runs your plan, your plan adds the suspender. Do you mean how do we apply it to the out of the box scans? In which case, yh, creating a way of asking BlueAPI to hook it into the RE would be the way to do it.

Do we have to define a suspender instance in dodal and then have our plan repos import the suspender and add this as a decorator?
Another use case for #1997 and DiamondLightSource/blueapi#1462

Yh, I agree we should do this.

I really think we should go down the suspender route if possible though, this is literally what they're designed for. And using a device like this seems like we're going back down the route of having "Scannables are everything". @DiamondLightSource/developers-daq-core how do you envisage suspenders being added to plans?

[oliwenmandiamond]

I really think we should go down the suspender route if possible though, this is literally what they're designed for. And using a device like this seems like we're going back down the route of having "Scannables are everything". @DiamondLightSource/developers-daq-core how do you envisage suspenders being added to plans?

Agreed, we should find a way to support suspenders to do checkbeam and any other conditions.

[fajinyuan]

Agreed that we should make use of bluesky offered mechanism - suppender - for this use case. BlueAPI client has to support this requirement to enable user control of this. As to the implementation as Device or not is debateable, in my personal view Ophyd device can be a virtual device implementing special logics using other devices. After all, suspender is to pause data collection under certain detected conditions. This can be done by suspend Run Engine or making a virtual device busy (at least for step scan this will work).

[DominicOram]

I attempted to use suspenders for a similar problem on i22 today and found some issues:

• Suspenders do not currently work with ophyd-async (Implement support for bluesky suspenders bluesky/ophyd-async#508). I worked around this on the beamline
• There seems to be something odd where the RE will go into "suspending" briefly then back into "running" even though the suspender is still asking it to be paused - I wonder if this is BlueAPI doing something wacky on top of the RE, needs more investigation

[oliwenmandiamond]

I attempted to use suspenders for a similar problem on i22 today and found some issues:

* Suspenders do not currently work with ophyd-async ([Implement support for bluesky suspenders bluesky/ophyd-async#508](https://github.com/bluesky/ophyd-async/issues/508)). I worked around this on the beamline

* There seems to be something odd where the RE will go into "suspending" briefly then back into "running" even though the suspender is still asking it to be paused - I wonder if this is BlueAPI doing something wacky on top of the RE, needs more investigation

Interesting, can you link me the code you used to test with? How did you get this to work with BlueAPI? Did you build the suspender inside a plan from devices and then install it to run engine?

[DominicOram]

Interesting, can you link me the code you used to test with? How did you get this to work with BlueAPI? Did you build the suspender inside a plan from devices and then install it to run engine?

Initial implementation is at DiamondLightSource/saxs-bluesky#104. I just added it to the plan I needed it for using bpp.suspend_decorator. I appreciate that won't work for the out of the box scan commands but it's enough for i22 for now.

[fajinyuan]

this checkbeam is missing front end shutter checking. Is this done somewhere else?

[oliwenmandiamond]

This was just a proof of concept so not everything is added yet

[oliwenmandiamond]

However, I've made it flexible enough so that you can add any custom logic. So you would just need to inject the front end shutter here, add check to see if closed is signals_to_condions

[oliwenmandiamond]

To be clear though, I agree with @DominicOram and I don't think a device is the way to go about this. This was just a proof of concept on how we could possibly translate the GDA code to bluesky with this being a possibility. If we make it a device and it "pauses" a scan, it won't pause the RunEngine and therefore BlueAPI will not be able to tell the scan is paused either. It will probably look like the plan is hanging as it is blocking and it may not be obvious why to the user. Using the Suspenders will give BlueAPI a lot more natural context and feedback as to what is happening with the plan.

[fajinyuan]

with checkbeam for data collection in user mode, I would ensure it is built into RE directly as Suspender for any plans to be run as default as users data collection will always need these. However for developers or beamline staffs who will work during shutdown this must be not as deafult in RE.

[oliwenmandiamond]

It is very flexible. the RunEngine has install_suspender and remove_suspender hooks to remove it globally. E.g by default, run_engine.install_suspender(checkbeam) and in dummy we can easily do run_engine.remove_suspender(checkbeam) for shutdown.

We also have the option for the suspender to only suspender if synchrotron.synchtron_mode is not "Shutdown" or "Mach. Dev" as well. So this will not be a problem.

The two things we need from core team though will be for this to be supported in BlueAPI. We will need #1997 to be supported, HasName on objects or something equivalent.

We also need the run engine hooks to be present so that from blueapi client we can do something like so

dodal side

@devices.fixture
def checkbeam(
     synchrotron: Synchrotron,
     shutter: Shutter,
     # Anything else,
) -> MySuspender:
     return MySuspender(synchrotron, shutter, ...)

Blueapi client side

bc = BlueapiClient(...)
plans = bc.plans
devs = bc.devices
fixtures = bc.fixtures

# Applied globally
bc.install_suspender(fixtures.checkbeam)
...
bc.remove_suspender(fixtures.checkbeam)

and also inside plans, we can control it locally (inject has to be able to support things with names that aren't devices DiamondLightSource/blueapi#1462)

from bluesky.plan_stubs import install_suspender, remove_suspender

def my_plan(a, b, checkbeam: MySuspender = inject("checkbeam")):
      # Applied only for this plan
      yield from install_suspender(checkbeam)
      ...
      yield from remove_suspender(checkbeam)

[fajinyuan]

I prefer your 2nd offer to auto detect if synchrotron in user mode or not. This will eliminate the need for users to know how to install and remove suspender as checkbeam will be permenantly installed in run engine which can be configured by beamline support software engineers.
However we still need to extend blueapi to support this instal/remove suspender feature for other use cases.