Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bjorn1 sitrep #15

Open
wants to merge 135 commits into
base: master
Choose a base branch
from
Open

Bjorn1 sitrep #15

wants to merge 135 commits into from

Conversation

AlaaALatif
Copy link
Member

@AlaaALatif AlaaALatif commented May 16, 2021 •
edited
Loading

  • change snakemake pipeline to specify all chunk data files as temporary in order to minimize disk space usage
  • fixes outbreak-info/outbreak.info-issues-362
  • TODO: after merge, replace shapefiles with minimal versions for speed-up in runtime

@AlaaALatif
script for downloading GISAID API feed, generating fasta file and pro…
832fc8d 
…cessed metadata
@AlaaALatif
separate env for api data processing, due to issues installing geopan…
caa3064 
…das in the original env
@AlaaALatif
fix strain processing in metadata, require username as input arg
bfa480b
@AlaaALatif
modify aggregate_replacements() and aggregate_deletions() to avoid pa…
9c3d6d0 
…ndas copy warnings
@AlaaALatif
always add ref seq to input fasta to avoid downstream failures, teste…
713703e 
…d with new GISAID API data
@AlaaALatif
fix aggregate funcs to convert date to str to prevent NaN issues
a61afa2
@AlaaALatif
minor formatting fixes
d7b2a70
@AlaaALatif
dev nb, transfer to local machine
a91940d
@AlaaALatif
generate lowercased versions of location info
2a0df49
@AlaaALatif
handle exceptions with mutation aggregation
75a1bdf
@AlaaALatif
fix mutation aggregation funcs
fd5365a
@AlaaALatif
fix field names of normalized locations
d085e0a
@AlaaALatif
migrate to local machine
7fe5cdf
@AlaaALatif
fix bjorn.py to adapt to breaking changes made for outbreak, allowing…
29ca63c 
… it to be used for internal sequence submissions
@AlaaALatif
updated script to tabulate mutations seen in A-lab sequences, excludi…
09e0605 
…ng insertions :(
@AlaaALatif
bash scripts used for sequencing and submission pipelines
1d0bb65
@AlaaALatif
not now...
6bea875
@AlaaALatif
added batch_iterator() function for chunking big fasta files into sma…
b8bd0fe 
…ller ones
@AlaaALatif
script for splitting input fasta into smaller chunks with a user-spec…
5da440b 
…ified maximum number of sequences in each
@AlaaALatif
baseline rules, still need to add chunk_fasta rule and run tests
82aa1c3
@AlaaALatif
tested successfully, takes ~5mins to split file containing ~600K seqs…
30fea4b 
… into 60 files
@AlaaALatif
move current codebase to separate module
65f5b68
@AlaaALatif
tested successfully, still need to add API pull (rule_0) and gcloud u…
100eaaf 
…pload (rule_n)
@AlaaALatif
initial readme for pipeline
f486717
@AlaaALatif
replace hard-coded numbers with dynamic ones, which was used for rapi…
09cb203 
…d testing
@AlaaALatif
fixed chunk_fasta to use single core only, fixed msa_2_mutations to a…
fa171cd 
…void error due to NaN dates
@AlaaALatif
python scripts for pulling API data and merging final results
8dcafcf
@AlaaALatif
added rule_0 and rule_n for pulling API data and generating json for …
9a83c7c 
…upload, still need to test on outbreak.info
@AlaaALatif
changed json2fasta to function def to run inline within pipeline (fas…
0bc1894 
…ter)
@AlaaALatif
add minimap2 to the env, which was the source of most cron errors...
1321ada
AlaaALatif and others added 26 commits April 27, 2021 12:57
@AlaaALatif
loc norm umlauted divisions in sweden, fixes outbreak-info/outbreak.i…
43ea6e0 
…nfo-issues-334
@AlaaALatif
fix codec error
00393be
@AlaaALatif
fix issue with del name assignment for short frame-shifting dels, e.g…
0bb2cfa 
…. S:DEL144/145
@AlaaALatif
Merge pull request #9 from andersen-lab/bjorn1_sitrep
4233a16 
Bjorn1 sitrep
@corneliusroemer
Fixed Swiss normalisation for GADM
c7c80b8
@gkarthik
Merge pull request #11 from corneliusroemer/swiss_test
a1b2fab 
Fixed Swiss normalisation for GADM
@corneliusroemer
Further Swiss Normalization fixes
0cc453f
@gkarthik
Merge pull request #12 from corneliusroemer/patch-1
b578e7c 
Further Swiss Normalization fixes
@AlaaALatif
fix puerto rico location normalization
699f476
@AlaaALatif
Merge branch 'master' into bjorn1_sitrep
59c553d
@AlaaALatif
Merge pull request #13 from andersen-lab/bjorn1_sitrep
a528dd4 
fix puerto rico location normalization
@AlaaALatif
loc norm testing lab
5c56b7f
@AlaaALatif
Merge branch 'bjorn1_sitrep' of https://github.com/andersen-lab/bjorn …
3b93153 
…into bjorn1_sitrep
@AlaaALatif
code cleanup, removing old commented statements
2ee4941
@AlaaALatif
Merge branch 'bjorn1_sitrep' of https://github.com/andersen-lab/bjorn …
c9b994e 
…into bjorn1_sitrep
@AlaaALatif
specify chunk data files as temp to prevent excess disk usage
ea83812
@AlaaALatif
incorporate test mode with correct filenaming and behavior
3871c33
@AlaaALatif
change to gadm_mini.tsv for quicker runtime, incorporate test mode wi…
69ff4ff 
…th correct behavior
@AlaaALatif
incorporate test mode with correct behavior, does NOT upload to outbr…
dae5518 
…eak nor generate an API file
@AlaaALatif
skip api metadata file generation during test mode
aedbaa7
@AlaaALatif
qc filters are now applied separately within a single function, seqs …
5db6df9 
…with long dels get filtered correctly, find_replacements() better captures ambiguous calls
@AlaaALatif
apply qc filtering before identifying mutations
056911b
@AlaaALatif
consolidate qc filters into a single function, correctly filter seqs …
5174b72 
…that are too short and/or low coverage
@AlaaALatif
apply qc filters prior to identifying mutations
455d16f
@AlaaALatif
fixes outbreak-info/outbreak.info-issues-362
5dd5a1b
@AlaaALatif
fix small error in comment
412d7dc
@AlaaALatif AlaaALatif mentioned this pull request Jun 1, 2021
Open
@gkarthik gkarthik force-pushed the master branch 2 times, most recently from af33f06 to 2cb3655 Compare October 18, 2021 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@AlaaALatif @corneliusroemer @gkarthik