Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to copier #159

Merged
merged 24 commits into from
Aug 19, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .copier-answers.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ author_email: [email protected]
author_name: Dominic Oram
component_owner: group:data-acquisition
description: Bluesky tools for MX Beamlines at DLS
distribution_name: mx_bluesky
distribution_name: mx-bluesky
docker: true
docs_type: sphinx
git_platform: github.com
github_org: DiamondLightSource
package_name: mx_bluesky
pypi: true
repo_name: mx_bluesky
repo_name: mx-bluesky
type_checker: pyright
2 changes: 1 addition & 1 deletion .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,5 @@ COPY --from=build /venv/ /venv/
ENV PATH=/venv/bin:$PATH

# change this entrypoint if it is not the same as the repo
ENTRYPOINT ["mx_bluesky"]
ENTRYPOINT ["mx-bluesky"]
CMD ["--version"]
4 changes: 2 additions & 2 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Contribute to the project

Contributions and issues are most welcome! All issues and pull requests are
handled through [GitHub](https://github.com/DiamondLightSource/mx_bluesky/issues). Also, please check for any existing issues before
handled through [GitHub](https://github.com/DiamondLightSource/mx-bluesky/issues). Also, please check for any existing issues before
filing a new one. If you have a great idea but it involves big changes, please
file a ticket before making a pull request! We want to make sure you don't spend
your time coding something that might not fit the scope of the project.

## Issue or Discussion?

Github also offers [discussions](https://github.com/DiamondLightSource/mx_bluesky/discussions) as a place to ask questions and share ideas. If
Github also offers [discussions](https://github.com/DiamondLightSource/mx-bluesky/discussions) as a place to ask questions and share ideas. If
your issue is open ended and it is not obvious when it can be "closed", please
raise it as a discussion instead.

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
strategy:
matrix:
runs-on: ["ubuntu-latest"] # can add windows-latest, macos-latest
python-version: ["3.10", "3.11"] # , "3.12"] # when the dependency thing is resolved
python-version: ["3.10", "3.11"] # , "3.12"] # add 3.12 when p4p #145 is fixed
include:
# Include one that runs in the dev environment
- runs-on: "ubuntu-latest"
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# The devcontainer should use the developer target and run as root with podman
# or docker with user namespaces.
ARG PYTHON_VERSION=3.11
FROM python:${PYTHON_VERSION} as developer

Check warning on line 4 in Dockerfile

View workflow job for this annotation

GitHub Actions / container / build

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


# Add any system dependencies for the developer/build environment here
RUN apt-get update && apt-get install -y --no-install-recommends \
Expand All @@ -13,17 +13,17 @@
ENV PATH=/venv/bin:$PATH

# The build stage installs the context into the venv
FROM developer as build

Check warning on line 16 in Dockerfile

View workflow job for this annotation

GitHub Actions / container / build

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/
COPY . /context
WORKDIR /context
RUN touch dev-requirements.txt && pip install -c dev-requirements.txt .

# The runtime stage copies the built venv into a slim runtime container
FROM python:${PYTHON_VERSION}-slim as runtime

Check warning on line 22 in Dockerfile

View workflow job for this annotation

GitHub Actions / container / build

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/
# Add apt-get system dependecies for runtime here if needed
COPY --from=build /venv/ /venv/
ENV PATH=/venv/bin:$PATH

# change this entrypoint if it is not the same as the repo
ENTRYPOINT ["mx_bluesky"]
ENTRYPOINT ["mx-bluesky"]
CMD ["--version"]
28 changes: 14 additions & 14 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
mx_bluesky
mx-bluesky
===========================

|code_ci| |docs_ci| |coverage| |pypi_version| |license|

Contains code for working with Bluesky on MX beamlines at Diamond

============== ==============================================================
PyPI ``pip install mx_bluesky``
Source code https://github.com/DiamondLightSource/mx_bluesky
Documentation https://DiamondLightSource.github.io/mx_bluesky
Releases https://github.com/DiamondLightSource/mx_bluesky/releases
PyPI ``pip install mx-bluesky``
Source code https://github.com/DiamondLightSource/mx-bluesky
Documentation https://DiamondLightSource.github.io/mx-bluesky
Releases https://github.com/DiamondLightSource/mx-bluesky/releases
============== ==============================================================

Getting Started
Expand All @@ -26,20 +26,20 @@ If you're doing more in-depth development we recommend developing with VSCode. Y
$ module load vscode
$ code .vscode/mx-bluesky.code-workspace

.. |code_ci| image:: https://github.com/DiamondLightSource/mx_bluesky/actions/workflows/code.yml/badge.svg?branch=main
:target: https://github.com/DiamondLightSource/mx_bluesky/actions/workflows/code.yml
.. |code_ci| image:: https://github.com/DiamondLightSource/mx-bluesky/actions/workflows/code.yml/badge.svg?branch=main
:target: https://github.com/DiamondLightSource/mx-bluesky/actions/workflows/code.yml
:alt: Code CI

.. |docs_ci| image:: https://github.com/DiamondLightSource/mx_bluesky/actions/workflows/docs.yml/badge.svg?branch=main
:target: https://github.com/DiamondLightSource/mx_bluesky/actions/workflows/docs.yml
.. |docs_ci| image:: https://github.com/DiamondLightSource/mx-bluesky/actions/workflows/docs.yml/badge.svg?branch=main
:target: https://github.com/DiamondLightSource/mx-bluesky/actions/workflows/docs.yml
:alt: Docs CI

.. |coverage| image:: https://codecov.io/gh/DiamondLightSource/mx_bluesky/branch/main/graph/badge.svg
:target: https://codecov.io/gh/DiamondLightSource/mx_bluesky
.. |coverage| image:: https://codecov.io/gh/DiamondLightSource/mx-bluesky/branch/main/graph/badge.svg
:target: https://codecov.io/gh/DiamondLightSource/mx-bluesky
:alt: Test Coverage

.. |pypi_version| image:: https://img.shields.io/pypi/v/mx_bluesky.svg
:target: https://pypi.org/project/mx_bluesky
.. |pypi_version| image:: https://img.shields.io/pypi/v/mx-bluesky.svg
:target: https://pypi.org/project/mx-bluesky
:alt: Latest PyPI version

.. |license| image:: https://img.shields.io/badge/License-Apache%202.0-blue.svg
Expand All @@ -50,4 +50,4 @@ If you're doing more in-depth development we recommend developing with VSCode. Y
Anything below this line is used when viewing README.rst and will be replaced
when included in index.rst

See https://DiamondLightSource.github.io/mx_bluesky for more detailed documentation.
See https://DiamondLightSource.github.io/mx-bluesky for more detailed documentation.
6 changes: 3 additions & 3 deletions catalog-info.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: mx_bluesky
title: mx_bluesky
name: mx-bluesky
title: mx-bluesky
description: Bluesky tools for MX Beamlines at DLS
annotations:
backstage.io/techdocs-ref: dir:.
github.com/project-slug: DiamondLightSource/mx_bluesky
github.com/project-slug: DiamondLightSource/mx-bluesky
tags:
- python
- bluesky
Expand Down
4 changes: 2 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# -- General configuration ------------------------------------------------

# General information about the project.
project = "mx_bluesky"
project = "mx-bluesky"

# The full version, including alpha/beta/rc tags.
release = mx_bluesky.__version__
Expand Down Expand Up @@ -83,7 +83,7 @@
default_role = "any"

# The suffix of source filenames.
source_suffix = ".rst"
source_suffix = [".rst", ".md"]

# The master toctree document.
master_doc = "index"
Expand Down
2 changes: 1 addition & 1 deletion docs/developer/general/how-to/create-a-release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Release

1. Create a new branch from main named pre followed by the release version e.g. pre_v0.1.0. The release versions should look like ``v{major}.{minor}.{patch}``.
2. On this branch pin the up-to-date version of dodal and the latest release of nexgen if necessary.
3. Go `here <https://github.com/DiamondLightSource/mx_bluesky/releases/new>`_.
3. Go `here <https://github.com/DiamondLightSource/mx-bluesky/releases/new>`_.
4. Select Choose a new tag and type the version of the release, then select the branch created in step 1 as the target.
5. Click on Generate release notes. This will create a starting set of release notes based on PR titles since the last release.
6. You should now manually go through each line on the release notes and read it from the perspective of a beamline scientist. It should be clear from each what the change means to the beamline and should have links to easily find further info.
Expand Down
6 changes: 3 additions & 3 deletions docs/developer/general/how-to/deploy-a-release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Deploy a New Release

**Remember to discuss any new deployments with the appropriate beamline scientist.**

The ``/deploy/deploy_mxbluesky.py`` script will deploy the latest Hyperion version to a specified beamline. Deployments live in ``/dls_sw/ixx/software/bluesky/mx_bluesky_vXXX``. To do a new deployment you should run the deploy script from your mx_bluesky dev environment with e.g.
The ``/deploy/deploy_mxbluesky.py`` script will deploy the latest Hyperion version to a specified beamline. Deployments live in ``/dls_sw/ixx/software/bluesky/mx-bluesky_vXXX``. To do a new deployment you should run the deploy script from your mx-bluesky dev environment with e.g.

``python ./deploy/deploy_mxbluesky.py --beamline i24``

Expand All @@ -21,5 +21,5 @@ If you need a specific beamline test deployment you can also run:
which will create the beamline deployment (eg. I24) in the specified test directory ``/your-path/mxbluesky_release_test``.


**Note:** When deploying on I24, the edm screens for serial crystallography will be deployed automatically along with the mx_bluesky release.
When running a ``dev`` deployment instead, `this script <https://github.com/DiamondLightSource/mx_bluesky/wiki/Serial-Crystallography-on-I24#deploying-a-local-version-of-the-edm-screens>`_ will also need to be run to get the latest version of the screens.
**Note:** When deploying on I24, the edm screens for serial crystallography will be deployed automatically along with the mx-bluesky release.
When running a ``dev`` deployment instead, `this script <https://github.com/DiamondLightSource/mx-bluesky/wiki/Serial-Crystallography-on-I24#deploying-a-local-version-of-the-edm-screens>`_ will also need to be run to get the latest version of the screens.
10 changes: 5 additions & 5 deletions docs/developer/general/how-to/get-started.rst
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
Get Started with mx_bluesky
Get Started with mx-bluesky
---------------------------
=======================
Development Environment
=======================

- Clone this repo: ``[email protected]:DiamondLightSource/mx_bluesky.git``
- To install a ``dev`` environment run ``./dls_dev_setup.sh``. Note that this will also clone and install a local version of ``dodal``, as the ``mx_bluesky`` package makes use of the devices instantiated there.
- Clone this repo: ``[email protected]:DiamondLightSource/mx-bluesky.git``
- To install a ``dev`` environment run ``./dls_dev_setup.sh``. Note that this will also clone and install a local version of ``dodal``, as the ``mx-bluesky`` package makes use of the devices instantiated there.

The recommended IDE is vscode, and a workspace which includes dodal has been set up in the repo. This can be used on a DLS machine as follows:

.. code-block:: bash

cd /path/to/mx_bluesky
cd /path/to/mx-bluesky
module load vscode
code ./.vscode/mx_bluesky.code-workspace
code ./.vscode/mx-bluesky.code-workspace

- If you use vs code, you may need to set the python interpreter for both repositories to the one from the virtual environment created in ``.venv``

Expand Down
6 changes: 3 additions & 3 deletions docs/developer/general/tutorials/dev-install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Clone the repository
First clone the repository locally using `Git
<https://git-scm.com/downloads>`_::

$ git clone git://github.com/DiamondLightSource/mx_bluesky.git
$ git clone git://github.com/DiamondLightSource/mx-bluesky.git

Install dependencies
--------------------
Expand All @@ -25,7 +25,7 @@ requires python 3.9 or later) or to run in a container under `VSCode

.. code::

$ cd mx_bluesky
$ cd mx-bluesky
$ python3 -m venv venv
$ source venv/bin/activate
$ pip install -e .[dev]
Expand All @@ -34,7 +34,7 @@ requires python 3.9 or later) or to run in a container under `VSCode

.. code::

$ vscode mx_bluesky
$ vscode mx-bluesky
# Click on 'Reopen in Container' when prompted
# Open a new terminal

Expand Down
2 changes: 1 addition & 1 deletion docs/developer/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Documentation is split general and topic-specific sections. Each section is spli

+++

General information for working on mx_bluesky
General information for working on mx-bluesky

.. grid-item-card:: :material-regular:`apps;3em`

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ The architecture of how Murko is integrated is still in flux but as of 07/08 the

.. image:: ../images/murko_setup.drawio.png

The mx_bluesky code is deployed into the `beamline kubernetes cluster <https://k8s-i04.diamond.ac.uk/>`_ behind a `blueAPI <https://github.com/DiamondLightSource/blueapi>`_ REST interface. Alongside this an instance of murko is also deployed to the k8s cluster.
The mx-bluesky code is deployed into the `beamline kubernetes cluster <https://k8s-i04.diamond.ac.uk/>`_ behind a `blueAPI <https://github.com/DiamondLightSource/blueapi>`_ REST interface. Alongside this an instance of murko is also deployed to the k8s cluster.

When GDA reaches a point where it will thaw the pin (usually just after robot load), if the ``gda.mx.bluesky.thaw`` property has been set to True it will instead call out to the thaw_and_center plan inside mx_bluesky.
When GDA reaches a point where it will thaw the pin (usually just after robot load), if the ``gda.mx.bluesky.thaw`` property has been set to True it will instead call out to the thaw_and_center plan inside mx-bluesky.

The thawing plan will first create a ``MurkoCallback`` callback, this will stream metadata into a key in a redis database running on i04-control (this should be moved in `#145 <https://github.com/DiamondLightSource/mx_bluesky/issues/145>`_).
The thawing plan will first create a ``MurkoCallback`` callback, this will stream metadata into a key in a redis database running on i04-control (this should be moved in `#145 <https://github.com/DiamondLightSource/mx-bluesky/issues/145>`_).

It will then trigger the ``OAVToRedisForwarder`` device in ``dodal`` that will stream image data to redis in the form of pickled RGB numpy arrays. Each image is given a uuid by this device, which is used to correlate the images with the other metadata, which could be streamed with a different frequency.

The image streaming must be done with an ophyd device as there is too much data for it all to be emitted in bluesky documents.

When the data is entered into redis it will publish a message to the redis ``murko`` channel. This will get picked up by the `socket_handler <https://github.com/DiamondLightSource/mx_auto_mjpeg_capture/tree/main/socket_handler>`_, which will forward the data to murko. Currently, during testing the socket_handler is just manually run on a workstation, `#146 <https://github.com/DiamondLightSource/mx_bluesky/issues/146>`_ should fix this.
When the data is entered into redis it will publish a message to the redis ``murko`` channel. This will get picked up by the `socket_handler <https://github.com/DiamondLightSource/mx_auto_mjpeg_capture/tree/main/socket_handler>`_, which will forward the data to murko. Currently, during testing the socket_handler is just manually run on a workstation, `#146 <https://github.com/DiamondLightSource/mx-bluesky/issues/146>`_ should fix this.
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Once this is done, the environment can be started by running:

.. code:: bash

cd /path/to/mx_bluesky
cd /path/to/mx-bluesky
source .venv/bin/activate

On beamline I24, the package will be saved in
Expand All @@ -25,7 +25,7 @@ shell commands pointing to the correct scripts/edm locations.

.. code:: bash

./path/to/mx_bluesky/deploy/deploy_edm_for_ssx.sh
./path/to/mx-bluesky/deploy/deploy_edm_for_ssx.sh

Setting the current visit directory
===================================
Expand All @@ -37,7 +37,7 @@ point to the current visit and then running the command:

.. code:: bash

./path/to/mx_bluesky/src/mx_bluesky/i24/serial/set_visit_directory.sh
./path/to/mx-bluesky/src/mx-bluesky/i24/serial/set_visit_directory.sh

Note that the default experiment type for the script setting the
directory will be ``fixed-target``. In case of an extruder collection,
Expand All @@ -46,4 +46,4 @@ the command line.

.. code:: bash

./path/to/mx_bluesky/src/mx_bluesky/i24/serial/set_visit_directory.sh extruder
./path/to/mx-bluesky/src/mx-bluesky/i24/serial/set_visit_directory.sh extruder
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ A couple of entry points have been set up so that:

Before opening the experiment specific edm, each of these entry points
will start a ``BlueAPI`` server. The configuration used by ``BlueAPI``
is saved in ``src/mx_bluesky/i24/serial/blueapi_config.yaml``.
is saved in ``src/mx-bluesky/i24/serial/blueapi_config.yaml``.
d-perl marked this conversation as resolved.
Show resolved Hide resolved

Detector choice
===============
Expand Down Expand Up @@ -75,7 +75,7 @@ set up the coordinate system.

Before this step remember to reset the scale and skew factors as well as
the motor directions as needed. Current values are saved in
``src/mx_bluesky/i24/serial/parameters/fixed_target`` in the
``src/mx-bluesky/i24/serial/parameters/fixed_target`` in the
d-perl marked this conversation as resolved.
Show resolved Hide resolved
``cs_maker.json`` and ``motor_direction.txt`` files.

1. From the main edm screen open the ``viewer`` and ``moveonclick``.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ in a couple of different ways.

.. code:: python

yield from bps.mv(pmac.x, 0, pmac.y, 1)https://github.com/DiamondLightSource/mx_bluesky/wiki/Serial-Crystallography-on-I24#cs_reset-custom-chips
yield from bps.mv(pmac.x, 0, pmac.y, 1)https://github.com/DiamondLightSource/mx-bluesky/wiki/Serial-Crystallography-on-I24#cs_reset-custom-chips

Notes on the coordinate system for a fixed-target collection
============================================================
Expand Down Expand Up @@ -81,8 +81,8 @@ Theory for this computation

The plan needs information stored in a few files:

* The motor directions are stored in ``src/mx_bluesky/i24/serial/parameters/fixed_target/cs/motor_directions.txt.`` The motor number multiplied by the motor direction should give the positive chip direction.
* The scale values for x,y,z, the skew value and the sign of Sx, Sy, Sz are all stored in ``src/mx_bluesky/i24/serial/parameters/fixed_target/cs/cs_maker.json``
* The motor directions are stored in ``src/mx-bluesky/i24/serial/parameters/fixed_target/cs/motor_directions.txt.`` The motor number multiplied by the motor direction should give the positive chip direction.
* The scale values for x,y,z, the skew value and the sign of Sx, Sy, Sz are all stored in ``src/mx-bluesky/i24/serial/parameters/fixed_target/cs/cs_maker.json``
d-perl marked this conversation as resolved.
Show resolved Hide resolved
* The fiducials 1 and 2 positions are written to file when selecting the fiducials (Setting fiducial 0 instead sends a homing command directly to the pmac_string PV)

NOTE. The ``motor_direction.txt`` and ``cs_maker.json`` files should
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ board <https://github.com/orgs/DiamondLightSource/projects/10/views/2>`__

Ongoing list of things needed:

1. Solution for enabling users to run mx_bluesky instead of old scripts:
1. Solution for enabling users to run mx-bluesky instead of old scripts:
blocked by permission issues. Preferred solution would be run on
beamline kubernetes cluster - in the meantime also loooking into
procserv as a possibility. (Fixing this should also allow us to stop
Expand Down Expand Up @@ -59,7 +59,7 @@ Ongoing list of things needed:
+---------------------------------------+----------------+---------------------------------+
| Permissions issues - run as a service | Dec. 23 | :material-regular:`check;2em` |
+---------------------------------------+----------------+---------------------------------+
| Deploy a first version of MX_BLUESKY | Nov. 23 | |
| Deploy a first version of mx-bluesky | Nov. 23 | |
| with the current iteration - tested | | :material-regular:`check;2em` |
| on the beamline - of the serial | | |
| tools. Set up a ``module load`` that | | |
Expand Down
4 changes: 2 additions & 2 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ The documentation is split into 2 sections:
:link: user/index
:link-type: doc

The User Guide contains documentation on how to install and use mx_bluesky.
The User Guide contains documentation on how to install and use mx-bluesky.

.. grid-item-card:: :material-regular:`code;4em`
:link: developer/index
:link-type: doc

The Developer Guide contains documentation on how to develop and contribute changes back to mx_bluesky.
The Developer Guide contains documentation on how to develop and contribute changes back to mx-bluesky.

.. toctree::
:hidden:
Expand Down
6 changes: 3 additions & 3 deletions docs/user/how-to/run-container.rst
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
Run in a container
==================

Pre-built containers with mx_bluesky and its dependencies already
Pre-built containers with mx-bluesky and its dependencies already
installed are available on `Github Container Registry
<https://ghcr.io/DiamondLightSource/mx_bluesky>`_.
<https://ghcr.io/DiamondLightSource/mx-bluesky>`_.

Starting the container
----------------------

To pull the container from github container registry and run::

$ docker run ghcr.io/DiamondLightSource/mx_bluesky:main --version
$ docker run ghcr.io/DiamondLightSource/mx-bluesky:main --version

To get a released version, use a numbered release instead of ``main``.
Loading
Loading