Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add py 3.12 to tests #570

Merged
merged 10 commits into from
Dec 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/test_conda_env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ channels:
- conda-forge
- defaults
dependencies:
- conda-forge::compilers
- numpy>=1.12
- matplotlib>=1.3.0
- scipy
Expand Down
36 changes: 0 additions & 36 deletions .github/test_conda_env_macOS.yml

This file was deleted.

46 changes: 15 additions & 31 deletions .github/workflows/runtest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,31 +8,21 @@ jobs:
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ['3.8', '3.9', '3.10', '3.11']
python-version: ['3.9', '3.10', '3.11', '3.12']
fail-fast: false
# continue-on-error: true

steps:
- uses: actions/checkout@v3

- name: Get conda env file
shell: bash -l {0}
run: |
if [ "$RUNNER_OS" == "macOS" ]; then
cp .github/test_conda_env_macOS.yml .github/test_conda_env.yml
fi
- uses: actions/checkout@v4

- name: Setup conda
uses: conda-incubator/setup-miniconda@v2.1.1
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
miniforge-version: latest
python-version: ${{ matrix.python-version }}
activate-environment: eqcorrscan-test
use-mamba: true

- name: Update Env
run: mamba env update -n eqcorrscan-test -f .github/test_conda_env.yml
run: conda env update -n eqcorrscan-test -f .github/test_conda_env.yml

- name: install eqcorrscan
shell: bash -l {0}
Expand Down Expand Up @@ -72,7 +62,7 @@ jobs:
py.test -v -m "slow and not serial and not network" --cov-report=xml --cov-append

- name: upload coverage
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ./coverage.xml
Expand All @@ -86,23 +76,20 @@ jobs:
runs-on: "ubuntu-latest"
strategy:
matrix:
python-version: ['3.10']
python-version: ['3.11']
fail-fast: false

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Setup conda
uses: conda-incubator/setup-miniconda@v2.1.1
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
miniforge-version: latest
python-version: ${{ matrix.python-version }}
activate-environment: eqcorrscan-test
use-mamba: true

- name: Update Env
run: mamba env update -n eqcorrscan-test -f .github/test_conda_env.yml
run: conda env update -n eqcorrscan-test -f .github/test_conda_env.yml

- name: install eqcorrscan
shell: bash -l {0}
Expand Down Expand Up @@ -131,7 +118,7 @@ jobs:
py.test -m "superslow" -s eqcorrscan/tests/tutorials_test.py eqcorrscan/tests/subspace_test.py --cov-report=xml --cov-append

- name: upload coverage
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ./coverage.xml
Expand All @@ -144,23 +131,20 @@ jobs:
runs-on: "ubuntu-latest"
strategy:
matrix:
python-version: ['3.9']
python-version: ['3.11']
fail-fast: false

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Setup conda
uses: conda-incubator/setup-miniconda@v2.1.1
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
miniforge-version: latest
python-version: ${{ matrix.python-version }}
activate-environment: eqcorrscan-test
use-mamba: true

- name: Update Env
run: mamba env update -n eqcorrscan-test -f .github/test_conda_env.yml
run: conda env update -n eqcorrscan-test -f .github/test_conda_env.yml

- name: install fmf
shell: bash -l {0}
Expand Down Expand Up @@ -189,7 +173,7 @@ jobs:
py.test eqcorrscan/tests/correlate_test.py --cov-report=xml

- name: upload coverage
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ./coverage.xml
Expand Down
22 changes: 22 additions & 0 deletions eqcorrscan/core/match_filter/helpers/processes.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
from queue import Empty

from obspy import Stream
from obspy.clients.fdsn import Client

Check warning on line 28 in eqcorrscan/core/match_filter/helpers/processes.py

View check run for this annotation

Codecov / codecov/patch

eqcorrscan/core/match_filter/helpers/processes.py#L28

Added line #L28 was not covered by tests

from eqcorrscan.core.match_filter.helpers import (
_pickle_stream, _unpickle_stream)
Expand All @@ -40,6 +41,25 @@

Logger = logging.getLogger(__name__)

# FDSN_CLIENT_ARGS = inspect.signature(Client).parameters.keys()
# Only allow a limited subset to be parsed through - this means
# that authenticated clients won't work.
FDSN_CLIENT_ARGS = {"base_url", "debug", "timeout"}

Check warning on line 47 in eqcorrscan/core/match_filter/helpers/processes.py

View check run for this annotation

Codecov / codecov/patch

eqcorrscan/core/match_filter/helpers/processes.py#L47

Added line #L47 was not covered by tests


class _FDSN_parseable_Client:
def __init__(self, **kwargs):
self.kwargs = kwargs

Check warning on line 52 in eqcorrscan/core/match_filter/helpers/processes.py

View check run for this annotation

Codecov / codecov/patch

eqcorrscan/core/match_filter/helpers/processes.py#L50-L52

Added lines #L50 - L52 were not covered by tests

@classmethod
def from_client(cls, client):
kwargs = {key: client.__dict__.get(key, None)

Check warning on line 56 in eqcorrscan/core/match_filter/helpers/processes.py

View check run for this annotation

Codecov / codecov/patch

eqcorrscan/core/match_filter/helpers/processes.py#L54-L56

Added lines #L54 - L56 were not covered by tests
for key in FDSN_CLIENT_ARGS}
return cls(**kwargs)

Check warning on line 58 in eqcorrscan/core/match_filter/helpers/processes.py

View check run for this annotation

Codecov / codecov/patch

eqcorrscan/core/match_filter/helpers/processes.py#L58

Added line #L58 was not covered by tests

def to_client(self):
return Client(**self.kwargs)

Check warning on line 61 in eqcorrscan/core/match_filter/helpers/processes.py

View check run for this annotation

Codecov / codecov/patch

eqcorrscan/core/match_filter/helpers/processes.py#L60-L61

Added lines #L60 - L61 were not covered by tests


###############################################################################
# Process handlers
Expand Down Expand Up @@ -242,6 +262,8 @@
:param samp_rate: See utils.pre_processing.multi_process
:param process_length: See utils.pre_processing.multi_process
"""
if isinstance(client, _FDSN_parseable_Client):
client = client.to_client()

Check warning on line 266 in eqcorrscan/core/match_filter/helpers/processes.py

View check run for this annotation

Codecov / codecov/patch

eqcorrscan/core/match_filter/helpers/processes.py#L266

Added line #L266 was not covered by tests
while True:
killed = _wait_on_output_to_be_available(
poison_queue=poison_queue, output_queue=output_filename_queue,
Expand Down
2 changes: 1 addition & 1 deletion eqcorrscan/core/match_filter/matched_filter.py
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ def match_filter(template_names, template_list, st, threshold,

.. math::

av\_chan\_corr\_thresh=threshold \\times (cccsum\ /\ len(template))
av_chan_corr_thresh=threshold \\times (cccsum/len(template))

where :math:`template` is a single template from the input and the
length is the number of channels within this template.
Expand Down
2 changes: 1 addition & 1 deletion eqcorrscan/core/match_filter/party.py
Original file line number Diff line number Diff line change
Expand Up @@ -381,7 +381,7 @@ def plot(self, plot_grouped=False, dates=None, min_dets=1, rate=False,
:type rate: bool
:param rate: Whether or not to plot the daily rate of detection as
opposed to cumulative number. Only works with plot_grouped=True.
:param \**kwargs: Any other arguments accepted by
:param kwargs: Any other arguments accepted by
:func:`eqcorrscan.utils.plotting.cumulative_detections`

.. rubric:: Examples
Expand Down
2 changes: 1 addition & 1 deletion eqcorrscan/core/match_filter/template.py
Original file line number Diff line number Diff line change
Expand Up @@ -521,7 +521,7 @@ def detect(self, stream, threshold, threshold_type, trig_int,

.. math::

av\_chan\_corr\_thresh=threshold \\times (cccsum /
av_chan_corr_thresh=threshold \\times (cccsum /
len(template))

where :math:`template` is a single template from the input and the
Expand Down
12 changes: 9 additions & 3 deletions eqcorrscan/core/match_filter/tribe.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@

from obspy import Catalog, Stream, read, read_events
from obspy.core.event import Comment, CreationInfo
from obspy.clients.fdsn import Client as FDSNClient

from eqcorrscan.core import template_gen
from eqcorrscan.core.match_filter.template import (
Expand Down Expand Up @@ -817,7 +818,7 @@

.. math::

av\_chan\_corr\_thresh=threshold \\times (cccsum /
av_chan_corr_thresh=threshold \\times (cccsum /
len(template))

where :math:`template` is a single template from the input and the
Expand Down Expand Up @@ -1364,15 +1365,15 @@

.. math::

av\_chan\_corr\_thresh=threshold \\times (cccsum /
av_chan_corr_thresh=threshold \\times (cccsum /
len(template))

where :math:`template` is a single template from the input and the
length is the number of channels within this template.
"""
from eqcorrscan.core.match_filter.helpers.tribe import _download_st
from eqcorrscan.core.match_filter.helpers.processes import (
_get_detection_stream)
_get_detection_stream, _FDSN_parseable_Client)

# This uses get_waveforms_bulk to get data - not all client types have
# this, so we check and monkey patch here.
Expand Down Expand Up @@ -1457,6 +1458,11 @@
time_queue = Queue()
stream_queue = Queue(maxsize=1)

# Convert client to pickleable client if needed - issue with
# Python 3.12 Windows and MacOS
if isinstance(client, FDSNClient):
client = _FDSN_parseable_Client.from_client(client)

Check warning on line 1464 in eqcorrscan/core/match_filter/tribe.py

View check run for this annotation

Codecov / codecov/patch

eqcorrscan/core/match_filter/tribe.py#L1464

Added line #L1464 was not covered by tests

downloader = Process(
target=_get_detection_stream,
kwargs=dict(
Expand Down
4 changes: 2 additions & 2 deletions eqcorrscan/tests/matched_filter/helper_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
get_waveform_client, _test_event_similarity)
from eqcorrscan.core.match_filter.helpers.processes import (
_get_detection_stream, Poison, _prepper, _pre_processor,
_make_detections)
_make_detections, _FDSN_parseable_Client)

try:
from pytest_cov.embed import cleanup_on_sigterm
Expand Down Expand Up @@ -285,7 +285,7 @@ def setUpClass(cls):
process_length = 360
cls.global_kwargs = dict(
template_channel_ids=[("NZ", "WVZ", "10", "HHZ")],
client=Client("GEONET"),
client=_FDSN_parseable_Client.from_client(Client("GEONET")),
retries=3,
min_gap=0.0,
buff=3,
Expand Down
Loading