Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Qs main #39

Draft
wants to merge 78 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
12d452d
improve logging code
gwbischof Jan 8, 2021
daebac3
Update ci to test latest environment
AbbyGi Mar 2, 2021
9bd6c19
Update beamline specific steps
AbbyGi Mar 2, 2021
4e42daa
Add timeout fix for CI
AbbyGi Mar 2, 2021
62a2e37
Added config file for CI
AbbyGi Mar 9, 2021
07bcfb4
Added glbl.yml file
AbbyGi Mar 9, 2021
2b50cbd
Changed where glbl.yml is located
AbbyGi Mar 9, 2021
182884b
Added xpd_beamline_config.yml file
AbbyGi Mar 17, 2021
f02293a
HACK: make the profile not ask for config confirmation
tacaswell Feb 26, 2021
d644a5d
MNT: make forgiving of missing slack token
tacaswell Feb 26, 2021
6d5305a
MNT: add missing commas
tacaswell Feb 26, 2021
97ea48b
Merge pull request #24 from NSLS-II-PDF/update-ci
tacaswell Mar 19, 2021
21c7868
MNT: avoid bare except
tacaswell Feb 26, 2021
669832d
MNT: lift missing value from _py file
tacaswell Feb 26, 2021
ad96016
MNT: add missing import to keep the linter happy
tacaswell Feb 26, 2021
172036d
MNT: put back in an RE API compatibly wrapper of xrun
tacaswell Feb 26, 2021
da32e93
MNT: files for queueserver startup
tacaswell Feb 26, 2021
ace7731
ENH: add plan that looks up sample metadata from xpdacq
tacaswell Feb 27, 2021
fbabb22
ENH: update and curate published plans
tacaswell Feb 27, 2021
0ae9e28
rapid development of sorensen power supply
tacaswell Mar 1, 2021
64dfab5
ENH: add caproto IOC for ramp plan
tacaswell Mar 1, 2021
b50c7d4
tweak controlled ramp plan
tacaswell Mar 2, 2021
cf77222
MNT: make writing the CSV optional
tacaswell Mar 2, 2021
bb6a421
FIX: the grid objects are in globals not locals
tacaswell Mar 2, 2021
4b937bf
BUG: add ability to optionally write out a CSV of temperatures
tacaswell Mar 2, 2021
5daa121
Collection of on-the-fly changes to thermo control
tacaswell Mar 10, 2021
db73d88
formatting + changes db insertion
tacaswell Mar 10, 2021
ff3ac6b
add flag to IOC to control xrd collection
tacaswell Mar 10, 2021
21860b9
HACK: overnight script to stop experiment after it melts
tacaswell Mar 10, 2021
2934cd8
ignore vscode files
tacaswell Mar 10, 2021
5daee8a
CI: collection-2021-1.2
mrakitin Mar 19, 2021
038420f
Do not install xpdacq (already in the env)
mrakitin Mar 19, 2021
173489e
Run tests for both collection-2021-1.0/1.2
mrakitin Mar 19, 2021
261e535
Use collection-2021-1.2 from main
mrakitin Mar 23, 2021
ff9de2e
Merge pull request #25 from NSLS-II-PDF/collection-2021-1.2
AbbyGi Mar 23, 2021
7473d51
remove suspenders
tacaswell Apr 7, 2021
b3fff96
Add plans for demo
tacaswell Apr 7, 2021
d8ebcde
Regenerate list of known plans + use existing filtering
tacaswell Apr 7, 2021
bc08329
STY: fix indentation of comments
tacaswell Apr 7, 2021
e30ad53
ENH: skeletons of the plans we need for the demo
tacaswell Apr 7, 2021
daa3f2c
FIX: make positions actually optional
tacaswell Apr 7, 2021
755714f
MNT: simplify signature of sample_aware_count
tacaswell Apr 9, 2021
7fa0664
Merge remote-tracking branch 'origin/master' into mnt_prep_for_demo
tacaswell Nov 29, 2021
656b44f
Merge remote-tracking branch 'origin/20211007-snapshot' into mnt_prep…
tacaswell Nov 29, 2021
5706800
MNT: remove plans that QS can not describe
tacaswell Dec 1, 2021
9eb87fc
ENH: add sample aware move + ct plan
tacaswell Dec 1, 2021
a2d0cb3
ENH: update QS config with new plans
tacaswell Dec 1, 2021
0399644
MNT: update detector / beamstop position
tacaswell Dec 1, 2021
fd77248
FIX: pre-move in rocking_ct
tacaswell Dec 1, 2021
e6f689e
STY: apply black
tacaswell Dec 1, 2021
a132e67
MNT: add envs for qserver startup
tacaswell Dec 1, 2021
7c1d672
Merge remote-tracking branch 'ws2/20211007-snapshot' into qs_main
tacaswell Dec 1, 2021
eab8e74
STY: apply black
tacaswell Dec 1, 2021
e4cc3bb
MNT: improve local name
tacaswell Feb 8, 2022
7c16054
merge changes from last cycle
tacaswell Feb 8, 2022
67804a3
Set connection timeout back to 1s
tacaswell Feb 23, 2022
7d322dc
Update to new allowed plan schema
tacaswell Feb 23, 2022
b281e57
Add "take a nap" demo plan
tacaswell Feb 23, 2022
42489bb
ENH: upgrade to using v34 versions of plugins
tacaswell Feb 23, 2022
9fd169e
Comment out Pilatus because it in not available
tacaswell Feb 23, 2022
b284d73
Correct PYTHONPATH location
tacaswell Feb 23, 2022
1ca4898
Remove test user from permissions file
tacaswell Feb 23, 2022
c3295df
Add client side script for setting up environment
maffettone Feb 23, 2022
06065f9
Merge remote-tracking branch 'DANGER/qs_main' into qs_main
tacaswell Feb 23, 2022
76f1680
Fix for the 'pe1c' detector's descriptor's shape (copy-pasted)
mrakitin Feb 24, 2022
823e4e5
Add publishing to kafka
mrakitin Apr 15, 2022
2892c92
because tom asked
Jul 5, 2022
7e257f4
Merge remote-tracking branch 'update-2022-c2' into qs_main
tacaswell Jul 5, 2022
4bc71a6
MNT: the pilatus is unplugged
tacaswell Jul 5, 2022
825ee9f
Updated permissions for all plans starting with agent_
Jul 5, 2022
bc58c75
add agent permissions
Jul 9, 2022
8405832
Add plans for experiment
maffettone Jul 9, 2022
f4d2d5d
updated plans
Jul 9, 2022
63aab20
update plan to vary motor
maffettone Jul 9, 2022
bcfbad3
update pdf plans
Jul 10, 2022
1b08d6d
debugging msg hook on RE
maffettone Jul 10, 2022
3059f88
debugging msg hook on RE
maffettone Jul 10, 2022
674cd1c
uncommited changes
tacaswell Dec 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion .ci/bl-specific.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,17 @@

# cp -v <...> ~/.ipython/profile_${TEST_PROFILE}/

conda install -y -c ${CONDA_CHANNEL_NAME} 28-id-1-pdf-collection
# xpdacq is already installed in the 'collection-2021-1.2' conda env.
if [ "$CONDA_ENV_NAME" == "collection-2021-1.0" ]; then
conda install -y -c ${CONDA_CHANNEL_NAME} xpdacq
fi

mkdir -v -p ~/.config/acq/
mkdir -v -p ~/user_data/config_base/yml/
sudo mkdir -v -p /mnt/data/bnl/xpdacq_special/data/xpdConfig/
sudo chown -Rv $USER: /mnt/data/bnl/xpdacq_special/data/xpdConfig/

cp -v .ci/pdf.yml ~/.config/acq/
cp -v .ci/glbl.yml ~/user_data/config_base/yml/
cp -v .ci/xpd_beamline_config.yml /mnt/data/bnl/xpdacq_special/data/xpdConfig/

158 changes: 158 additions & 0 deletions .ci/glbl.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
_dark_dict_list:
- acq_time: 5
exposure: 5
timestamp: 1614460063.2800422
uid: 37d192fb-df2c-44a9-b41b-2a05c9751247
- acq_time: 5
exposure: 5
timestamp: 1614460242.3911414
uid: cf374ae7-2ed5-4e5d-9563-f4841a3759e9
- acq_time: 5
exposure: 5
timestamp: 1614460464.3035522
uid: 5fc90d2e-ef96-4ca7-9207-ae72121923d0
- acq_time: 5
exposure: !!python/object/apply:numpy.core.multiarray.scalar
- &id001 !!python/object/apply:numpy.dtype
args:
- f8
- false
- true
state: !!python/tuple
- 3
- <
- null
- null
- null
- -1
- -1
- 0
- !!binary |
AAAAAAAAFEA=
timestamp: 1614460566.1485345
uid: 12c2f3ca-b459-4180-a12e-2b2d4b6d5aa8
- acq_time: 5
exposure: !!python/object/apply:numpy.core.multiarray.scalar
- *id001
- !!binary |
AAAAAAAAFEA=
timestamp: 1614464054.0055833
uid: 5bf51ebe-6439-4815-a9bb-0c824fd3afe8
- acq_time: 5
exposure: !!python/object/apply:numpy.core.multiarray.scalar
- *id001
- !!binary |
AAAAAAAAFEA=
timestamp: 1614464263.5876548
uid: 25d95820-b20f-4d37-901d-6c96c86462a1
- acq_time: 5
exposure: !!python/object/apply:numpy.core.multiarray.scalar
- *id001
- !!binary |
AAAAAAAAFEA=
timestamp: 1614464433.3411603
uid: bb969c1c-2e41-4914-91db-3440b576c752
- acq_time: 5
exposure: !!python/object/apply:numpy.core.multiarray.scalar
- *id001
- !!binary |
AAAAAAAAFEA=
timestamp: 1614464443.6444564
uid: 319ccd2f-7483-459c-96ac-e308b150006e
_exclude_dir:
- /mnt/data/bnl/xpdacq_special/data/user_data
- /mnt/data/bnl/xpdacq_special/data/xpdConfig
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml
_export_tar_dir:
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base
- /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
all_folders: &id002
- /mnt/data/bnl/xpdacq_special/data/user_data
- /mnt/data/bnl/xpdacq_special/data/xpdConfig
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/samples
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/scanplans
- /mnt/data/bnl/xpdacq_special/data/user_data/tiff_base
- /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
- /mnt/data/bnl/xpdacq_special/data/user_data/Import
- /mnt/data/bnl/xpdacq_special/data/user_data/userAnalysis
allfolders: *id002
archive_base_dir: /mnt/data/bnl/xpdacq_special/archive/.userbeamtimearchive
archive_base_dir_name: .userbeamtimearchive
archive_dir: /mnt/data/bnl/xpdacq_special/archive/.userbeamtimearchive/2021
archive_root_dir: /mnt/data/bnl/xpdacq_special/archive
auto_dark: true
auto_load_calib: true
base: /mnt/data/bnl/xpdacq_special/data
base_dir: /mnt/data/bnl/xpdacq_special/data
beamline_host_name:
- jupiter
beamline_id: 28-ID-1
blconfig_dir: /mnt/data/bnl/xpdacq_special/data/xpdConfig
blconfig_dir_name: xpdConfig
blconfig_name: xpd_beamline_config.yml
blconfig_path: /mnt/data/bnl/xpdacq_special/data/xpdConfig/xpd_beamline_config.yml
bt_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml
calib_config_name: xpdAcq_calib_info.yml
config_base: /mnt/data/bnl/xpdacq_special/data/user_data/config_base
dark_field_key: sc_dk_field_uid
dark_window: 0.1
det_image_field: &id003
- pe1c_image
- pe2c_image
- dexela_image
- dexela
diffraction_dets:
- pe1c
- pe1
- pe2c
- pe2
- dexela_image
- dexela
dk_window: 0.1
exp_broker_name: pdf
exp_hash_uid: f4be92ca-79c1-49ae-9bcd-59c7a8185aa3
facility: NSLS-II
frame_acq_time: 0.1
frame_acquire_time: 0.1
glbl_yaml_name: glbl.yml
glbl_yaml_path: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/glbl.yml
group: XPD
home: /mnt/data/bnl/xpdacq_special/data/user_data
home_dir: /mnt/data/bnl/xpdacq_special/data/user_data
home_dir_name: user_data
image_field: pe1c_image
image_fields: *id003
import_dir: /mnt/data/bnl/xpdacq_special/data/user_data/Import
inbound_proxy_address: xf28id1-ca1:5577
is_simulation: false
mask_kwargs:
alpha: 3.0
bs_width: 13
edge: 30
lower_thresh: 0.0
tri_offset: 13
v_asym: 0
name: glbl
outbound_proxy_address: xf28id1-ca1:5578
owner: PDF
radiogram_dets:
- blackfly_det
radiograph_names:
- blackfly_det
sample_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/samples
scanplan_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/scanplans
shutter_conf:
close: 0
open: 1
shutter_control: true
shutter_sleep: 0
simulation: false
tiff_base: /mnt/data/bnl/xpdacq_special/data/user_data/tiff_base
user_backup_dir_name: '2021'
userscript_dir: /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
usrAnalysis_dir: /mnt/data/bnl/xpdacq_special/data/user_data/userAnalysis
usrScript_dir: /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
xpdconfig: /mnt/data/bnl/xpdacq_special/data/xpdConfig
yaml_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml
35 changes: 35 additions & 0 deletions .ci/pdf.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
archive_base_dir_name: .userbeamtimearchive
archive_root_dir: /mnt/data/bnl/xpdacq_special/archive
base_dir: /mnt/data/bnl/xpdacq_special/data
beamline_host_name: [jupiter]
beamline_id: 28-ID-1
blconfig_dir_name: xpdConfig
blconfig_name: xpd_beamline_config.yml
calib_config_name: xpdAcq_calib_info.yml
dark_window: 0.1
facility: NSLS-II
frame_acquire_time: 0.1
glbl_yaml_name: glbl.yml
group: XPD
home_dir_name: user_data
image_field: pe1c_image
owner: PDF
simulation: false
shutter_conf: {close: 0, open: 1}
mask_kwargs:
edge: 30
lower_thresh: 0.0
bs_width: 13
tri_offset: 13
v_asym: 0
alpha: 3.0
dark_field_key : sc_dk_field_uid
det_image_field : pe1_image
exp_broker_name: pdf
outbound_proxy_address: 'xf28id1-ca1:5578'
inbound_proxy_address: 'xf28id1-ca1:5577'
shutter_sleep: 0
diffraction_dets: ['pe1c', 'pe1', 'pe2c', 'pe2','dexela_image', 'dexela']
radiograph_names : ['blackfly_det']
radiogram_dets: ['blackfly_det']
image_fields: ['pe1c_image', 'pe2c_image', 'dexela_image', 'dexela']
4 changes: 4 additions & 0 deletions .ci/xpd_beamline_config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Verification time: '2021-01-09 17:45:59'
Verified by: AUTO VERIFIED IN TEST
hadtoedit: false
is_pytest: false
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -86,3 +86,5 @@ db/
static/
history.sqlite
Link\ to\ startup

.vscode/
11 changes: 7 additions & 4 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,13 @@ resources:
- repository: templates
type: github
name: NSLS-II/profile-collection-ci
# ref: refs/heads/<branch-name> # for future testings on a branch of NSLS-II/profile-collection-ci
ref: refs/heads/main
endpoint: github

jobs:
- template: azure-linux.yml@templates # Template reference
parameters:
beamline_acronym: PDF
- template: collection-2021-1.0.yml@templates
parameters:
beamline_acronym: PDF
- template: collection-2021-1.2.yml@templates
parameters:
beamline_acronym: PDF
7 changes: 7 additions & 0 deletions env_vars/client_side_qserver_extra
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
conda activate 2021-3.1-py37
OVERLAY=/nsls2/data/pdf/shared/config/bluesky_overlays/2021-3.1-py37
PATH=$OVERLAY/bin:$PATH
PYTHONPATH=$OVERLAY/lib/python3.7/site-packages:$PYTHONPATH
export PATH
export PYTHONPATH
export BS_ENV=/nsls2/data/pdf/shared/config/profile_collection
2 changes: 2 additions & 0 deletions env_vars/qserver_extra
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export PYTHONPATH=/home/xf28id1/bluesky_overlays/lib/python3.9/site-packages/
export QSERVER_STARTUP_DIR=/nsls2/data/pdf/shared/config/profile_collection/startup
35 changes: 35 additions & 0 deletions scripts/ramp_ioc.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/usr/bin/env python3
from textwrap import dedent

from caproto.server import PVGroup, ioc_arg_parser, pvproperty, run


class SimpleIOC(PVGroup):
"""
An IOC with three uncoupled read/writable PVs

Scalar PVs
----------
A (int)
B (float)

Vectors PVs
-----------
C (vector of int)
"""

done = pvproperty(
value=0,
doc="An integer to track if the ramp should be done.",
name="RampDone-Cmd",
)
delta = pvproperty(value=0.1, doc="The delta of the ramp rate.", name="RampDelta")
take = pvproperty(value=0, doc="If XRD data should be taken", name="TakeXRD-Cmd",)


if __name__ == "__main__":
ioc_options, run_options = ioc_arg_parser(
default_prefix="OvenRampControl:", desc=dedent(SimpleIOC.__doc__)
)
ioc = SimpleIOC(**ioc_options)
run(ioc.pvdb, **run_options)
48 changes: 48 additions & 0 deletions scripts/robo_dan.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
from ophyd import Device, Component as Cpt, EpicsSignal
import time
import datetime


class RampControl(Device):
delta = Cpt(EpicsSignal, "RampDelta")
done = Cpt(EpicsSignal, "RampDone-Cmd")
take_xrd = Cpt(EpicsSignal, "TakeXRD-Cmd")


temperature = EpicsSignal("XF:28ID1-ES{LS336:1-Chan:C}T-I")
ramp_control = RampControl("OvenRampControl:", name="ramp_control")
power_rbv = EpicsSignal("XF:28ID1-ES{LS336:1-Out:3}Out:Man-RB")
power_sp = EpicsSignal("XF:28ID1-ES{LS336:1-Out:3}Out:Man-SP")

print(f"{datetime.datetime.now()} Good morning! Robo-dan going to work!")

while True:
T = temperature.get()
if T is not None and T > 1025:
break
time.sleep(60)
print(f"{datetime.datetime.now()} temperature at {T:.2f}, keep going!")

print(f"{datetime.datetime.now()} temperature at {T}, Done!!")

ramp_control.delta.put(0)
print(f"{datetime.datetime.now()} holding for 5 minutes")
time.sleep(60 * 5)

print(f"{datetime.datetime.now()} starting cooling")
ramp_control.delta.put(-2.5)

while True:
p = power_rbv.get()
print(f"{datetime.datetime.now()} power currently at {p}, still cooling")
if p < 1:
break
time.sleep(3 * 60)

time.sleep(5 * 60)
print(f"{datetime.datetime.now()} power low, declare done")
ramp_control.done.put(1)

time.sleep(5 * 60)
print(f"{datetime.datetime.now()} putting power to 0 just in case")
power_sp.put(0)
Loading
Loading