Librarify! (#320)

* Move everything into the src folder

* start fixing paths

* Remaining paths, hopefully

* continue fixing stuff

* start fixing some import stuff

* Fix get_base_segment_class

* Allow extension segments to work again

* try to simplify imports

* Update mypy stuff

* black

* forgor

* gitignore

* update expected test

* make it installable

* move split.py contents to scripts/split.py

* Add split subcommand

* capy

* move create_config too

* mypy & black

* Move capy to the bottom

* changelog

* forgor to remove this

* fix ci

* Add release CI

* yeet authors and maintainers

---------

Co-authored-by: Ethan <[email protected]>

.github/workflows/mypy.yml

@@ -5,19 +5,50 @@ on:
   pull_request:
 
 jobs:
-  mypy_checks:
+  mypy_splat_checks:
     runs-on: ubuntu-latest
-    name: mypy
+    name: mypy splat lib
     steps:
-    - uses: actions/checkout@v1
+    - name: Checkout repo
+      uses: actions/checkout@v4
+
+    - name: Set up Python 3.8
+      uses: actions/setup-python@v1
+      with:
+        python-version: 3.8
+
+    - name: Install Dependencies
+      run: |
+        pip install mypy
+        pip install -r requirements.txt
+        pip install types-PyYAML
+
+    - name: mypy splat lib
+      run: mypy --show-column-numbers --hide-error-context src/splat
+
+  mypy_programs_checks:
+    runs-on: ubuntu-latest
+    name: mypy splat programs
+    steps:
+    - name: Checkout repo
+      uses: actions/checkout@v4
+
     - name: Set up Python 3.8
       uses: actions/setup-python@v1
       with:
         python-version: 3.8
+
     - name: Install Dependencies
       run: |
         pip install mypy
         pip install -r requirements.txt
         pip install types-PyYAML
-    - name: mypy
-      run: mypy --show-column-numbers --hide-error-context .
+
+    - name: mypy split
+      run: mypy --show-column-numbers --hide-error-context split.py
+
+    - name: mypy create_config
+      run: mypy --show-column-numbers --hide-error-context create_config.py
+
+    - name: mypy test
+      run: mypy --show-column-numbers --hide-error-context test.py

.github/workflows/pypi.yml

@@ -0,0 +1,44 @@
+name: Build and upload to PyPI
+
+# Build on every branch push, tag push, and pull request change:
+on: [push, pull_request]
+
+jobs:
+  build_wheel:
+    name: Build wheel
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v2
+
+      - name: Install build module
+        run: pip install build
+
+      - name: Build wheel and source
+        run: python -m build --sdist --wheel --outdir dist/ .
+
+      - uses: actions/upload-artifact@v4
+        with:
+          path: dist/*
+
+  upload_pypi:
+    name: Upload to PyPI
+    needs: [build_wheel]
+    runs-on: ubuntu-latest
+    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
+    environment:
+      name: pypi
+      url: https://pypi.org/p/splat64
+    permissions:
+        id-token: write
+
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: artifact
+          path: dist
+
+      - uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          user: __token__
+          password: ${{ secrets.PYPI_PASSWORD }}

.github/workflows/test_lib.yml

@@ -0,0 +1,20 @@
+name: Test splat lib
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  test_splat_lib:
+    name: Test lib
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout reposistory
+      uses: actions/checkout@v4
+
+    - name: Install local splat
+      run: python3 -m pip install .
+
+    - name: Test
+      run: splat capy

.gitignore

@@ -1,5 +1,6 @@
 .idea/
 venv/
+.venv/
 .vscode/
 __pycache__/
 .mypy_cache/

CHANGELOG.md

@@ -1,5 +1,57 @@
 # splat Release Notes
 
+### 0.21.0
+
+* BREAKING: Extension segments will need adjustments to continue to work.
+  * Due to splat working as a library now, absolute imports on extension segments no longer work.
+  * Here's an example on how to fix this. Before version 0.21.0 you would have something like this
+
+    ```py
+    from util import log, options
+
+    from segtypes.n64.segment import N64Segment
+    from segtypes.common.data import CommonSegData
+    ```
+
+    There are two ways to fix this, depending on how the user uses splat:
+    * Installing splat as a Python package:
+
+      If the user decides to use splat as a package instead of as a subrepo/submodule then fixing this issue becomes very easy, just prefix the imports with `splat.`:
+
+      ```py
+      from splat.util import log, options
+
+      from splat.segtypes.n64.segment import N64Segment
+      from splat.segtypes.common.data import CommonSegData
+      ```
+
+    * Using splat as a submodule/subrepo:
+
+      This option is a bit more complex since it requires relative imports, as if the extension segment were part of splat.
+
+      splat will load extension segment as if it were in `segtypes/{PLATFORM}/{EXTENSION}.py`, so imports should be relative to that folder.
+
+      Assuming the extension is for an `n64` project, the fixed version would look like this:
+
+      ```py
+      from ...util import log, options
+
+      from .segment import N64Segment
+      from ..common.data import CommonSegData
+      ```
+
+* splat has been librarified!
+  * splat can now be installed as a Python package and used as a library.
+  * The normal way of invoking `./split.py` still works as usual.
+* Installing the splat package allows to use it as a cli tool besides using it as a library.
+  * Check `python3 -m splat --help` (or simply `splat --help`) to the options.
+  * `splat split` has the same functionality as the plain `./split.py` script.
+  * `splat create_config` has the same functionality as the plain `./create_config.py` script.
+  * `splat capy`.
+* DO NOT use splat as both an installed Python package and a submodule/subrepo.
+  * This may be very problematic if the version of both splats go out of sync.
+  * This warning is mainly for users that want use their own extension segments or use splat as a library.
+
 ### 0.20.1
 
 * splat now uses [crunch64](https://github.com/decompals/crunch64) as a dependency for handling decompression of various formats, starting with Yay0 and MIO0

create_config.py

@@ -4,9 +4,9 @@
 import sys
 from pathlib import Path
 
-from util.gc import gcinfo
-from util.n64 import find_code_length, rominfo
-from util.psx import psxexeinfo
+from src.splat.util.gc import gcinfo
+from src.splat.util.n64 import find_code_length, rominfo
+from src.splat.util.psx import psxexeinfo
 
 parser = argparse.ArgumentParser(
     description="Create a splat config from an N64 ROM, PSX executable, or a GameCube disc image."

pyproject.toml

@@ -0,0 +1,28 @@
+[project]
+name = "splat"
+# Should be synced with src/splat/__init__.py
+version = "0.21.0"
+description = "A binary splitting tool to assist with decompilation and modding projects"
+readme = "README.md"
+license = {file = "LICENSE"}
+requires-python = ">=3.8"
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+]
+dynamic = ["dependencies"]
+
+[project.urls]
+Repository = "https://github.com/ethteck/splat"
+Issues = "https://github.com/ethteck/splat/issues"
+Changelog = "https://github.com/ethteck/splat/blob/master/CHANGELOG.md"
+
+[build-system]
+requires = ["hatchling", "hatch-requirements-txt"]
+build-backend = "hatchling.build"
+
+[tool.hatch.metadata.hooks.requirements_txt]
+files = ["requirements.txt"]
+
+[project.scripts]
+splat = "splat.__main__:splat_main"