Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support python3.13 #4100

Draft
wants to merge 58 commits into
base: master
Choose a base branch
from
Draft
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
2ca9d02
explicitly test 3.13 in test workflow
DanielYang59 Oct 10, 2024
47b4cde
add 3.13 classifier
DanielYang59 Oct 10, 2024
bb4554d
drop install for openff-toolkit for now
DanielYang59 Oct 10, 2024
3661359
bump chgnet to support numpy 2
DanielYang59 Oct 10, 2024
7a8d6ff
loose torch pin
DanielYang59 Oct 10, 2024
f3176b4
try to remove manual install of torch
DanielYang59 Oct 10, 2024
02fbe6a
bump github action version
DanielYang59 Oct 10, 2024
249821b
revert bump of chgnet version
DanielYang59 Oct 10, 2024
2673ce1
bump chgnet to 0.4.0 to support np2
DanielYang59 Oct 10, 2024
dc47253
migrate deprecated config
DanielYang59 Oct 10, 2024
300f4e4
pin python < 3.13 for chgnet and matgl
DanielYang59 Oct 10, 2024
464defe
try to exclude some python313 incompatible packages
DanielYang59 Oct 10, 2024
d01bf59
should work for python 3.13 now
DanielYang59 Oct 10, 2024
16caa14
revert migrated changes
DanielYang59 Oct 10, 2024
5fb01a7
NEED CONFIRM: bump numpy to 1.26.2 to support python 3.13
DanielYang59 Oct 10, 2024
960a44b
bump h5py to support python 3.13
DanielYang59 Oct 10, 2024
a8a765a
bump scipy to support python 3.13
DanielYang59 Oct 10, 2024
44fda8d
merge master
DanielYang59 Nov 19, 2024
0f02418
remove duplicate from merging
DanielYang59 Nov 19, 2024
687effa
clean up comments
DanielYang59 Nov 20, 2024
cb0adc9
skip netcdf4 for python3.13 as 1.6.5 cannot be installed
DanielYang59 Nov 20, 2024
ccda40d
make wheel build quiet
DanielYang59 Nov 20, 2024
0f9d2a1
test openff-toolkit
DanielYang59 Nov 21, 2024
18175db
still cannot install openff-toolkit
DanielYang59 Nov 21, 2024
e3f69e5
clean up comment
DanielYang59 Nov 21, 2024
f2cebdf
Merge branch 'master' into support-python313
shyuep Dec 11, 2024
e7f9b28
Merge branch 'master' into support-python313
DanielYang59 Dec 12, 2024
cf318de
remove manual torch install
DanielYang59 Dec 12, 2024
54bbc6b
fix python version pin
DanielYang59 Dec 20, 2024
b3ca5f2
Merge branch 'master' into support-python313
DanielYang59 Dec 24, 2024
c0d36b1
Merge remote-tracking branch 'upstream/master' into support-python313
DanielYang59 Jan 10, 2025
e3d26b9
Merge branch 'master' into support-python313
DanielYang59 Jan 24, 2025
50f4622
un-skip netcdf4 and BoltzTraP2
DanielYang59 Jan 24, 2025
56108b6
unskip hiphive
DanielYang59 Jan 24, 2025
dcd65e1
unskip numba
DanielYang59 Jan 24, 2025
b7051d7
try to un-skip openff-toolkit
DanielYang59 Jan 24, 2025
4fbf58f
Revert "try to un-skip openff-toolkit"
DanielYang59 Jan 24, 2025
6a6be7b
Merge branch 'master' into support-python313
DanielYang59 Jan 29, 2025
300129b
Merge branch 'master' into support-python313
DanielYang59 Jan 31, 2025
05e98d2
Merge branch 'master' into support-python313
DanielYang59 Jan 31, 2025
5ea26e3
fix mypy error
DanielYang59 Feb 8, 2025
26007e9
add types for analysis.eos
DanielYang59 Feb 8, 2025
c9cc898
TODO: to be reverted, test 3.13 across all platforms
DanielYang59 Feb 9, 2025
3e55250
add requires decorator to is_valid_bibtex
DanielYang59 Feb 9, 2025
5fd64ab
skip prototype test if pybtex is not available
DanielYang59 Feb 9, 2025
b04042c
more readable error msg from the start
DanielYang59 Feb 9, 2025
a73d390
Merge branch 'support-python313' of https://github.com/DanielYang59/p…
DanielYang59 Feb 9, 2025
9f2dda1
make pybtex optional
DanielYang59 Feb 9, 2025
ea66fe3
Merge branch 'master' into support-python313
DanielYang59 Feb 14, 2025
4825b12
Revert "add types for analysis.eos"
DanielYang59 Feb 14, 2025
2692696
revert: make pybtex optional
DanielYang59 Feb 14, 2025
769b2fb
Revert "skip prototype test if pybtex is not available"
DanielYang59 Feb 14, 2025
ebfd917
reapply requires to is_valid_bibtex
DanielYang59 Feb 14, 2025
af559fa
insert debug tags
DanielYang59 Feb 14, 2025
4cf635a
Revert "insert debug tags"
DanielYang59 Feb 14, 2025
f11974f
add PR tag
DanielYang59 Feb 14, 2025
f8830ee
Merge branch 'master' into support-python313
DanielYang59 Feb 18, 2025
26a6558
Merge branch 'master' into support-python313
DanielYang59 Feb 25, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -30,7 +30,7 @@ jobs:
- name: Install Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
python-version: "3.13"

- name: Build source distribution
run: |
@@ -76,7 +76,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
python-version: "3.13"

- name: Get build artifacts
uses: actions/download-artifact@v4
20 changes: 9 additions & 11 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -28,22 +28,22 @@ jobs:
# Maximize CI coverage of different platforms and python versions while minimizing the
# total number of jobs. We run all pytest splits with the oldest supported python
# version (currently 3.10) on windows (seems most likely to surface errors) and with
# newest version (currently 3.12) on ubuntu (to get complete coverage on unix).
# newest version (currently 3.13) on ubuntu (to get complete coverage on unix).
config:
- os: windows-latest
python: "3.10"
python: "3.13"
resolution: highest
extras: ci,optional
- os: windows-latest
python: "3.10"
python: "3.13"
resolution: highest
extras: ci,optional,numpy-v1 # Test NP1 on Windows (quite buggy ATM)
- os: ubuntu-latest
python: "3.12"
python: "3.13"
resolution: lowest-direct
extras: ci,optional
- os: macos-latest
python: "3.11"
python: "3.13"
resolution: lowest-direct
extras: ci # test with only required dependencies installed

@@ -70,19 +70,17 @@ jobs:
- name: Install ubuntu-only conda dependencies
if: matrix.config.os == 'ubuntu-latest'
run: |
micromamba install -n pmg -c conda-forge bader enumlib \
openff-toolkit packmol pygraphviz tblite --yes
micromamba install -n pmg -c conda-forge --yes \
bader enumlib packmol pygraphviz tblite
# openff-toolkit
# TODO: openff-toolkit doesn't support Python 3.13 yet

- name: Install uv
uses: astral-sh/setup-uv@v4

- name: Install pymatgen and dependencies via uv
run: |
micromamba activate pmg
# TODO1 (use uv over pip) uv install torch is flaky, track #3826
# TODO2 (pin torch version): DGL library (matgl) doesn't support torch > 2.2.1,
# see: https://discuss.dgl.ai/t/filenotfounderror-cannot-find-dgl-c-graphbolt-library/4302
pip install torch==2.2.1

# Install from wheels to test the content
uv build --wheel --no-build-logs
14 changes: 8 additions & 6 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -21,7 +21,7 @@ Python Materials Genomics is a robust materials analysis code that defines core
and molecules with support for many electronic structure codes. It is currently the core analysis code powering the
Materials Project (https://materialsproject.org)."""
readme = "README.md"
requires-python = ">=3.10,<3.13"
requires-python = ">=3.10,<3.14"
keywords = [
"ABINIT",
"VASP",
@@ -48,6 +48,7 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: Scientific/Engineering :: Chemistry",
"Topic :: Scientific/Engineering :: Information Analysis",
"Topic :: Scientific/Engineering :: Physics",
@@ -61,12 +62,11 @@ dependencies = [
"palettable>=3.3.3",
"pandas>=2",
"plotly>=5.0.0",
"pybtex>=0.24.0",
"requests>=2.32",
"ruamel.yaml>=0.17.0",
"scipy>=1.13.0",
# scipy<1.14.1 is incompatible with NumPy 2.0 on Windows
# https://github.com/scipy/scipy/issues/21052
"scipy>=1.13.0",
"scipy>=1.14.1; platform_system == 'Windows'",
"spglib>=2.5",
"sympy>=1.3", # PR #4116
@@ -92,9 +92,9 @@ ase = ["ase>=3.23.0"]
ci = ["pytest-cov>=4", "pytest-split>=0.8", "pytest>=8", "pymatgen[symmetry]"]
docs = ["invoke", "sphinx", "sphinx_markdown_builder", "sphinx_rtd_theme"]
electronic_structure = ["fdint>=2.0.2"]
mlp = ["chgnet>=0.3.8", "matgl>=1.1.3"]
mlp = ["chgnet>=0.4.0; python_version<'3.13'", "matgl>=1.1.3; python_version<'3.13'"]
numba = ["numba>=0.55"]
numpy-v1 = ["numpy>=1.25.0,<2"] # Test NP1 on Windows (quite buggy ATM)
numpy-v1 = ["numpy>=1.25.0,<2"] # Test NP1 on Windows (buggy at this moment)
optional = [
"pymatgen[abinit,ase,mlp,tblite]",
"beautifulsoup4",
@@ -107,14 +107,16 @@ optional = [
"hiphive>=1.3.1",
"jarvis-tools>=2020.7.14",
"matplotlib>=3.8",
"openbabel-wheel>=3.1.1.20",
"phonopy>=2.33.3",
"pybtex>=0.24.0", # TODO: don't work on MacOS Python 3.13
"seekpath>=2.0.1",
]
# moyopy[interface] includes ase
symmetry = ["moyopy[interface]>=0.3", "spglib>=2.5"]
# tblite only support Python 3.12+ through conda-forge
# https://github.com/tblite/tblite/issues/175
tblite = [ "tblite[ase]>=0.3.0; platform_system=='Linux' and python_version<'3.12'"]
tblite = ["tblite[ase]>=0.3.0; platform_system=='Linux' and python_version<'3.12'"]
vis = ["vtk>=6.0.0"]

[project.scripts]
5 changes: 4 additions & 1 deletion src/pymatgen/analysis/prototypes.py
Original file line number Diff line number Diff line change
@@ -25,7 +25,10 @@
from pymatgen.core.structure import Structure

MODULE_DIR = os.path.dirname(os.path.abspath(__file__))
AFLOW_PROTOTYPE_LIBRARY = loadfn(f"{MODULE_DIR}/aflow_prototypes.json")
try:
AFLOW_PROTOTYPE_LIBRARY = loadfn(f"{MODULE_DIR}/aflow_prototypes.json")
except RuntimeError as exc:
raise ImportError("pybtex is needed to load AFLOW library") from exc
Copy link
Contributor Author

@DanielYang59 DanielYang59 Feb 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this would be breaking, this just re-raise the exception from requires to provide more context, otherwise the error message when pybtex is not available would be:

tests/analysis/test_prototypes.py:6: in <module>
    from pymatgen.analysis.prototypes import AflowPrototypeMatcher
src/pymatgen/analysis/prototypes.py:28: in <module>
    AFLOW_PROTOTYPE_LIBRARY = loadfn(f"{MODULE_DIR}/aflow_prototypes.json")
venv312/lib/python3.12/site-packages/monty/serialization.py:79: in loadfn
    return json.load(fp, *args, **kwargs)
../../opt/miniconda3/envs/pmg/lib/python3.12/json/__init__.py:293: in load
    return loads(fp.read(),
../../opt/miniconda3/envs/pmg/lib/python3.12/json/__init__.py:359: in loads
    return cls(**kw).decode(s)
venv312/lib/python3.12/site-packages/monty/json.py:879: in decode
    return self.process_decoded(d)
venv312/lib/python3.12/site-packages/monty/json.py:857: in process_decoded
    return [self.process_decoded(x) for x in d]
venv312/lib/python3.12/site-packages/monty/json.py:853: in process_decoded
    self.process_decoded(k): self.process_decoded(v) for k, v in d.items()
venv312/lib/python3.12/site-packages/monty/json.py:777: in process_decoded
    return cls_.from_dict(data)
src/pymatgen/util/provenance.py:340: in from_dict
    return cls(
src/pymatgen/util/provenance.py:229: in __init__
    if references and not is_valid_bibtex(references):
venv312/lib/python3.12/site-packages/monty/dev.py:167: in decorated
    raise self.err_cls(self.message)
E   RuntimeError: pybtex is not available

Before adding the requires decorator to is_valid_bibtex:

tests/analysis/test_prototypes.py:6: in <module>
    from pymatgen.analysis.prototypes import AflowPrototypeMatcher
src/pymatgen/analysis/prototypes.py:29: in <module>
    AFLOW_PROTOTYPE_LIBRARY = loadfn(f"{MODULE_DIR}/aflow_prototypes.json")
venv312/lib/python3.12/site-packages/monty/serialization.py:79: in loadfn
    return json.load(fp, *args, **kwargs)
../../opt/miniconda3/envs/pmg/lib/python3.12/json/__init__.py:293: in load
    return loads(fp.read(),
../../opt/miniconda3/envs/pmg/lib/python3.12/json/__init__.py:359: in loads
    return cls(**kw).decode(s)
venv312/lib/python3.12/site-packages/monty/json.py:879: in decode
    return self.process_decoded(d)
venv312/lib/python3.12/site-packages/monty/json.py:857: in process_decoded
    return [self.process_decoded(x) for x in d]
venv312/lib/python3.12/site-packages/monty/json.py:853: in process_decoded
    self.process_decoded(k): self.process_decoded(v) for k, v in d.items()
venv312/lib/python3.12/site-packages/monty/json.py:777: in process_decoded
    return cls_.from_dict(data)
src/pymatgen/util/provenance.py:340: in from_dict
    return cls(
src/pymatgen/util/provenance.py:229: in __init__
    if references and not is_valid_bibtex(references):
src/pymatgen/util/provenance.py:53: in is_valid_bibtex
    parser = bibtex.Parser()
E   AttributeError: 'NoneType' object has no attribute 'Parser'

Update: I just realized pybtex is not an optional dependency, need to put more thoughts into this:

pymatgen/pyproject.toml

Lines 56 to 64 in 5b997f7

dependencies = [
"joblib>=1",
"matplotlib>=3.8",
"monty>=2025.1.9",
"networkx>=2.7", # PR4116
"palettable>=3.3.3",
"pandas>=2",
"plotly>=4.5.0,<6.0.0",
"pybtex>=0.24.0",

Copy link
Contributor Author

@DanielYang59 DanielYang59 Feb 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird, pybtex is installed successfully in MacOS CI but some or all parts of pybtex is still not available, need to look into the reason later as I don't have a MacOS machine at hand right now


Update:

On MacOS, from pybtex.database.input import bibtex gives:

Traceback (most recent call last):
  File "/Users/yang/developer/pymatgen/debug/test_pybtex_install.py", line 8, in <module>
    from pybtex.database.input import bibtex
  File "/Users/yang/developer/pymatgen/venv313/lib/python3.13/site-packages/pybtex/database/__init__.py", line 44, in <module>
    from pybtex.plugin import find_plugin
  File "/Users/yang/developer/pymatgen/venv313/lib/python3.13/site-packages/pybtex/plugin/__init__.py", line 26, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

Which is an already fixed issue from pybtex's side but they haven't got a release with this patch yet. In fact they haven't released any issue since Jan 2021



@due.dcite(
4 changes: 2 additions & 2 deletions src/pymatgen/io/cif.py
Original file line number Diff line number Diff line change
@@ -1357,8 +1357,8 @@ def get_bibtex_string(self) -> str:
"""
try:
from pybtex.database import BibliographyData, Entry
except ImportError:
raise RuntimeError("Bibliographic data extraction requires pybtex.")
except ImportError as exc:
raise RuntimeError("Bibliographic data extraction requires pybtex.") from exc

bibtex_keys: dict[str, tuple[str, ...]] = {
"author": ("_publ_author_name", "_citation_author_name"),
5 changes: 2 additions & 3 deletions src/pymatgen/io/vasp/outputs.py
Original file line number Diff line number Diff line change
@@ -4732,9 +4732,8 @@ def concatenate(
raise ValueError("preamble is None")
poscar = Poscar.from_str("\n".join([*preamble, "Direct", *coords_str]))

if (
(ionicstep_end is None and ionicstep_cnt >= ionicstep_start)
or ionicstep_start <= ionicstep_cnt < ionicstep_end # type: ignore[operator]
if (ionicstep_end is None and ionicstep_cnt >= ionicstep_start) or (
ionicstep_end is not None and ionicstep_start <= ionicstep_cnt < ionicstep_end
):
structures.append(poscar.structure)
self.structures = structures
2 changes: 2 additions & 0 deletions src/pymatgen/util/provenance.py
Original file line number Diff line number Diff line change
@@ -9,6 +9,7 @@
from io import StringIO
from typing import TYPE_CHECKING, NamedTuple

from monty.dev import requires
from monty.json import MontyDecoder, MontyEncoder

from pymatgen.core.structure import Molecule, Structure
@@ -36,6 +37,7 @@
MAX_BIBTEX_CHARS: int = 20_000 # maximum number of characters for BibTeX reference


@requires(bibtex is not None, "pybtex is not available")
def is_valid_bibtex(reference: str) -> bool:
"""Use pybtex to validate that a reference is in proper BibTeX format.

8 changes: 8 additions & 0 deletions tests/analysis/test_cost.py
Original file line number Diff line number Diff line change
@@ -2,11 +2,19 @@

from unittest import TestCase

import pytest
from pytest import approx

from pymatgen.analysis.cost import CostAnalyzer, CostDBCSV, CostDBElements
from pymatgen.util.testing import TEST_FILES_DIR

try:
# Not using find_spec because it would error out during import
import pybtex.database # noqa: F401

except ImportError:
pytest.skip("pybtex is not available", allow_module_level=True)

TEST_DIR = f"{TEST_FILES_DIR}/analysis/cost"


2 changes: 1 addition & 1 deletion tests/analysis/test_eos.py
Original file line number Diff line number Diff line change
@@ -426,7 +426,7 @@ def test_numerical_eos_values(self):
assert_allclose(self.num_eos_fit.e0, -10.84749, atol=1e-3)
assert_allclose(self.num_eos_fit.v0, 40.857201, atol=1e-1)
assert_allclose(self.num_eos_fit.b0, 0.55, atol=1e-2)
assert_allclose(self.num_eos_fit.b0_GPa, 89.0370727, atol=1e-1)
assert_allclose(self.num_eos_fit.b0_GPa, 89.0370727, atol=0.3) # PR 4100
assert_allclose(self.num_eos_fit.b1, 4.344039, atol=1e-2)

def test_eos_func(self):
8 changes: 7 additions & 1 deletion tests/analysis/test_prototypes.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
from __future__ import annotations

from pymatgen.analysis.prototypes import AflowPrototypeMatcher
import pytest

try:
from pymatgen.analysis.prototypes import AflowPrototypeMatcher
except ImportError:
pytest.skip("pybtex is not available", allow_module_level=True)

from pymatgen.util.testing import PymatgenTest


2 changes: 2 additions & 0 deletions tests/io/test_cif.py
Original file line number Diff line number Diff line change
@@ -14,6 +14,7 @@

try:
import pybtex
import pybtex.database
except ImportError:
pybtex = None

@@ -166,6 +167,7 @@ def test_long_loop(self):


class TestCifIO(PymatgenTest):
@pytest.mark.skipif(pybtex is None, reason="pybtex not present")
def test_cif_parser(self):
parser = CifParser(f"{TEST_FILES_DIR}/cif/LiFePO4.cif")
for struct in parser.parse_structures():
7 changes: 7 additions & 0 deletions tests/util/test_provenance.py
Original file line number Diff line number Diff line change
@@ -11,6 +11,13 @@
from pymatgen.core.structure import Molecule, Structure
from pymatgen.util.provenance import Author, HistoryNode, StructureNL

try:
# Not using find_spec because it would error out during import
import pybtex.database # noqa: F401

except ImportError:
pytest.skip("pybtex is not available", allow_module_level=True)

__author__ = "Anubhav Jain"
__credits__ = "Shyue Ping Ong"
__copyright__ = "Copyright 2012, The Materials Project"