Skip to content

Commit

Permalink
Merge pull request #9 from miguelcarcamov/development
Browse files Browse the repository at this point in the history
Development
  • Loading branch information
miguelcarcamov authored Oct 26, 2022
2 parents a67e09d + 2e5a88a commit 6273f0d
Show file tree
Hide file tree
Showing 28 changed files with 284 additions and 194 deletions.
12 changes: 6 additions & 6 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,17 +47,17 @@ jobs:
run: python3 -m build
- name: Build bdist package
run: python3 setup.py bdist_wheel
- name: Install selfcalframework
- name: Install snow
run: pip install dist/*.tar.gz
- name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: selfcalframework-artifacts
name: snow-artifacts
path: dist/*
- name: Run commands to test that everything is OK
run: |
pip3 freeze | grep "selfcalframework"
python3 -c "import selfcalframework"
pip3 freeze | grep "snow"
python3 -c "import snow"
test:
runs-on: ubuntu-latest
needs: build
Expand All @@ -81,9 +81,9 @@ jobs:
- uses: actions/download-artifact@v3
id: download
with:
name: selfcalframework-artifacts
name: snow-artifacts
path: dist/
- name: Install selfcalframework from artifacts
- name: Install snow from artifacts
run: pip3 install dist/*.tar.gz
#- name: Test with pytest and pycov
# run: pytest --cov=csromer --cov-report xml:coverage.xml --cov-report term-missing tests/ -W "ignore" -v
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/build_latest_container.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,16 +51,16 @@ jobs:
run: python3 -m build
- name: Build bdist package
run: python3 setup.py bdist_wheel
- name: Install selfcalframework
- name: Install snow
run: pip install dist/*.tar.gz
- name: Upload artifacts
uses: actions/upload-artifact@v3
with:
path: dist/*
- name: Run commands to test that everything is finished
run: |
pip3 freeze | grep "selfcalframework"
python3 -c "import selfcalframework"
pip3 freeze | grep "snow"
python3 -c "import snow"
build-latest-container:
runs-on: ubuntu-latest
needs: build
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -128,4 +128,4 @@ dmypy.json

# Pyre type checker
.pyre/
!/src/selfcalframework/_version.py
!/src/snow/_version.py
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,5 +29,5 @@ repos:
rev: v0.31.1
hooks:
- id: markdownlint-fix
entry: markdownlint -f '**/*.md' --disable MD013 MD036 MD046 MD033 MD041
entry: markdownlint -f '**/*.md' --disable MD013 MD036 MD046 MD033 MD041 MD040
# args: [--disable MD013 MD036]
6 changes: 4 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -38,5 +38,7 @@ RUN python3 --version
RUN pip3 --version
RUN python3 -m pip install --upgrade pip
RUN python3 -m pip install mpi4py --no-cache-dir
RUN echo "Hello from selfcalframework base image"
LABEL org.opencontainers.image.source="https://github.com/miguelcarcamov/selfcalframework"
RUN echo "Hello from SNOW base image"
LABEL org.opencontainers.image.source="https://github.com/miguelcarcamov/snow"
LABEL org.opencontainers.image.description="Container image for SNOW"
LABEL org.opencontainers.image.licenses=GPL3
12 changes: 7 additions & 5 deletions Dockerfile.prod
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
FROM ghcr.io/miguelcarcamov/selfcalframework:base
FROM ghcr.io/miguelcarcamov/snow:base

RUN python3 --version
RUN pip3 --version
RUN echo "Installing selfcalframework"
RUN pip3 install git+https://github.com/miguelcarcamov/selfcalframework.git
RUN echo "Hello from selfcalframework production image"
LABEL org.opencontainers.image.source="https://github.com/miguelcarcamov/selfcalframework"
RUN echo "Installing snow"
RUN pip3 install git+https://github.com/miguelcarcamov/snow.git
RUN echo "Hello from snow production image"
LABEL org.opencontainers.image.source="https://github.com/miguelcarcamov/snow"
LABEL org.opencontainers.image.description="Container image for SNOW"
LABEL org.opencontainers.image.licenses=GPL3
29 changes: 15 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Object Oriented Framework for Self-calibration of radio-interferometric datasets
# **SNOW**

Many radioastronomers repeat the process of writing different scripts for self-calibration
## ca**S**a pytho**N** self-calibrati**O**n frame**W**ork

Many radio-astronomers repeat the process of writing different scripts for self-calibration
depending on their datasets. This repository holds an object-oriented Framework for self-calibration
of radio-interferometric datasets that will help radio astronomers to minimize the tedious work of
writing self-calibration scripts once again. The idea is to call just one main Python script that
Expand All @@ -17,42 +19,41 @@ objects (phase, amplitude, amplitude-phase) having the self-calibrated dataset a

### From PYPI repository

- `pip install selfcalframework`
- `pip install snow`

### From Github

- `pip install -U git+https://github.com/miguelcarcamov/selfcalframework`
- `pip install -U git+https://github.com/miguelcarcamov/snow`

### From source

```bash
git clone https://github.com/miguelcarcamov/selfcalframework
cd selfcalframework
git clone https://github.com/miguelcarcamov/snow
cd snow
pip install .
```

### From source as developer

```bash
git clone https://github.com/miguelcarcamov/selfcalframework
cd selfcalframework
git clone https://github.com/miguelcarcamov/snow
cd snow
pip install e .
```

## Using docker container

```bash
docker pull ghcr.io/miguelcarcamov/selfcalframework:latest
docker pull ghcr.io/miguelcarcamov/snow:latest
```

## Run selfcalframework
## Run snow

```Python
```python
# Import the modules that you want to use
import sys
import numpy as np
from src.selfcalframework.selfcalibration import Phasecal, AmpPhasecal
from src.selfcalframework.imaging import Tclean
from snow.selfcalibration import Phasecal, AmpPhasecal
from snow.imaging import Tclean

if __name__ == '__main__':
# This step is up to you, and option to capture your arguments from terminal is using sys.argv
Expand Down
20 changes: 20 additions & 0 deletions config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import time

datapath = ["~/.casa/data"]
rundata = "~/.casa/data/rsync"
logfile = 'casalog-%s.log' % time.strftime("%Y%m%d-%H", time.localtime())
telemetry_enabled = True
crashreporter_enabled = True
nologfile = False
log2term = True
nologger = True
nogui = False
colors = "LightBG"
agg = False
pipeline = False
iplog = True
user_site = False
telemetry_log_directory = "/tmp"
telemetry_log_limit = 20000
telemetry_log_size_interval = 60
telemetry_submit_interval = 604800
4 changes: 2 additions & 2 deletions main_files/main_ALMA.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import sys
from src.selfcalframework.imaging import Tclean
from src.selfcalframework.selfcalibration import AmpPhasecal, Phasecal
from snow.imaging import Tclean
from snow.selfcalibration import AmpPhasecal, Phasecal

if __name__ == '__main__':
visfile = sys.argv[3]
Expand Down
4 changes: 2 additions & 2 deletions main_files/main_VLA.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import sys
from src.selfcalframework.imaging import Tclean
from src.selfcalframework.selfcalibration import AmpPhasecal, Phasecal
from snow.imaging import Tclean
from snow.selfcalibration import AmpPhasecal, Phasecal

if __name__ == '__main__':
visfile = sys.argv[3]
Expand Down
4 changes: 2 additions & 2 deletions main_files/main_eMERLIN.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
import sys
from casatasks import flagdata, mstransform, flagmanager

from src.selfcalframework.imaging import Tclean
from src.selfcalframework.selfcalibration import AmpPhasecal
from snow.imaging import Tclean
from snow.selfcalibration import AmpPhasecal

if __name__ == '__main__':

Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = ["setuptools>=62.6.0", "setuptools-scm", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "selfcalframework"
name = "snow"
authors = [
{name = "Miguel Cárcamo", email = "[email protected]"},
]
Expand All @@ -25,8 +25,8 @@ Source = "https://github.com/miguelcarcamov/selfcalframework"

[tool.setuptools.packages.find]
where = ["src"]
include = ["selfcalframework*"]
include = ["snow*"]
namespaces = false

[tool.setuptools_scm]
write_to = "src/selfcalframework/_version.py"
write_to = "src/snow/_version.py"
18 changes: 9 additions & 9 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
almatasks==1.5.2
astropy==5.1
casadata==2022.1.17
casampi==0.5.0
casaplotms==1.6.2
casaplotserver==1.2.1
casashell==6.5.0.15
casatasks==6.5.0.15
casatestutils==6.5.0.15
casatools==6.5.0.15
casaviewer==1.5.2
casadata==2022.9.5
casampi==0.5.01
casaplotms==1.8.7
casaplotserver==1.4.6
casashell==6.5.2.26
casatasks==6.5.2.26
casatestutils==6.5.2.26
casatools==6.5.2.26
casaviewer==1.6.6
numpy==1.22.4
reproject==0.9
scipy==1.8.1
9 changes: 7 additions & 2 deletions setup.cfg
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[metadata]
name = selfcalframework
version = attr: selfcalframework.__version__
name = snow
version = attr: snow.__version__
description = A Python object oriented framework to do self-calibration
long_description = file: README.md
long_description_content_type = text/markdown
Expand All @@ -11,3 +11,8 @@ column_limit = 100
dedent_closing_brackets = true
allow_split_before_dict_value = false
split_before_bitwise_operator = false

[isort]
multi_line_output = 3
include_trailing_comma = true
line_length = 100
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -5,34 +5,15 @@

from casatasks import exportfits, fixvis, immath, importfits, tclean
from casatools import image
from dataclasses import dataclass, field

from ..utils.image_utils import reproject
from .imager import Imager


@dataclass(init=False, repr=True)
class GPUvmem(Imager):

def __init__(
self,
executable: str = "gpuvmem",
gpu_blocks: list = [16, 16, 256],
initial_values: list = [],
regfactors: list = [],
gpuids: list = [0],
residual_output: str = "residuals.ms",
model_input: str = None,
model_out: str = "mod_out.fits",
user_mask: str = None,
force_noise: float = None,
gridding_threads: int = 4,
positivity: bool = True,
ftol: float = 1e-12,
noise_cut: float = 10.0,
gridding: bool = False,
print_images: bool = False,
**kwargs
):
"""
"""
gpuvmem imager object
Parameters
Expand All @@ -55,9 +36,54 @@ def __init__(
noise_cut : Mask threshold based one the inverse of the primary beam
gridding : Whether to grid visibilities or not to increase computation speed
print_images : Whether to output the intermediate images during the optimization
kwargs : General imager parameters
"""
"""
executable: str = "gpuvmem"
gpu_blocks: list = None
initial_values: list = field(init=True, repr=True, default_factory=list)
regfactors: list = field(init=True, repr=True, default_factory=list)
gpuids: list = None
residual_output: str = "residuals.ms"
model_input: str = None
model_out: str = "mod_out.fits"
user_mask: str = None
force_noise: float = None
gridding_threads: int = 4
positivity: bool = True
ftol: float = 1e-12
noise_cut: float = 10.0
gridding: bool = False
print_images: bool = False

def __init__(
self,
executable: str = "gpuvmem",
gpu_blocks: list = [16, 16, 256],
initial_values: list = [],
regfactors: list = [],
gpuids: list = [0],
residual_output: str = "residuals.ms",
model_input: str = None,
model_out: str = "mod_out.fits",
user_mask: str = None,
force_noise: float = None,
gridding_threads: int = 4,
positivity: bool = True,
ftol: float = 1e-12,
noise_cut: float = 10.0,
gridding: bool = False,
print_images: bool = False,
**kwargs
):

super().__init__(**kwargs)
super().__post_init__()

if self.gpu_blocks is None:
self.gpu_blocks = [16, 16, 256]

if self.gpuids is None:
self.gpuids = [0]

self.name = "GPUvmem"
self.executable = executable
self.gpu_blocks = gpu_blocks
Expand Down
Loading

0 comments on commit 6273f0d

Please sign in to comment.