Code

All of naip#326

Code

Draft

yellowcap wants to merge 84 commits intomainClay-foundation/model:mainfrom all-of-naipClay-foundation/model:all-of-naipCopy head branch name to clipboard

+2,474-240

Commits

Commits on Jul 25, 2024

• Compile clay model encoder

  srmsoumya

  committed
  48748f1

  Copy full SHA for 48748f1

• Add benchmark & test files for the compiled clay encoder

  srmsoumya

  committed
  97eb19a

  Copy full SHA for 97eb19a

• Revert changes to Encoder, don't change the API

  srmsoumya

  committed
  eba1867

  Copy full SHA for eba1867

• Add embedder to load clay encoder & save in onnx/ep format

  srmsoumya

  committed
  73171dd

  Copy full SHA for 73171dd

• Remove files from src, fix utils to run everything on same device

  srmsoumya

  committed
  1f2fcc9

  Copy full SHA for 1f2fcc9

• Bump torch==2.3.1 & torchvision==0.18.1, add onnx & onnxsxript as dependency

  srmsoumya

  committed
  0a3ce9d

  Copy full SHA for 0a3ce9d

• Release few contraints on env

  srmsoumya

  committed
  37503fe

  Copy full SHA for 37503fe

• Add notebook to show how to embed using compiled embedders

  srmsoumya

  committed
  db8a3f2

  Copy full SHA for db8a3f2

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for c0552bd
  pre-commit-ci[bot]

  committed
  c0552bd

  Copy full SHA for c0552bd

• Clear outputs from the notebook

  srmsoumya

  committed
  1e97506

  Copy full SHA for 1e97506

• Merge branch 'compile' of https://github.com/Clay-foundation/model into compile

  srmsoumya

  committed
  0c37c58

  Copy full SHA for 0c37c58

• Add torchdata as a pip dependency

  srmsoumya

  committed
  1803954

  Copy full SHA for 1803954

Commits on Aug 6, 2024

• Randomly pass time & latlon as zeros 20% of time

  srmsoumya

  committed
  fdf4e80

  Copy full SHA for fdf4e80

• Add modis band info to metadata.yaml

  srmsoumya

  committed
  519a171

  Copy full SHA for 519a171

• Add prefetch factor as an arg to DataModule

  srmsoumya

  committed
  a9efd2b

  Copy full SHA for a9efd2b

• Change wavelength to millimeter scale for modis

  srmsoumya

  committed
  6434473

  Copy full SHA for 6434473

• change batch_first to True

  srmsoumya

  committed
  7021ab4

  Copy full SHA for 7021ab4

• Add transformer code from vit_pytorch as module

  srmsoumya

  committed
  5770cd6

  Copy full SHA for 5770cd6

• Add MRL

  srmsoumya

  committed
  6bbc902

  Copy full SHA for 6bbc902

• SAM as teacher, MRL, split code into modules

  srmsoumya

  committed
  94f2b75

  Copy full SHA for 94f2b75

• Remove outdated README, all in docs

  srmsoumya

  committed
  08df662

  Copy full SHA for 08df662

Commits on Aug 7, 2024

• Fix trainer

  srmsoumya

  committed
  c6df3e6

  Copy full SHA for c6df3e6

• Fix mean, std for s1

  srmsoumya

  committed
  c6151c6

  Copy full SHA for c6151c6

• update config for 1 node run

  srmsoumya

  committed
  a8cee0f

  Copy full SHA for a8cee0f

Commits on Aug 8, 2024

• Modify Sentinel 1 from raw pixels to dB scale

  srmsoumya

  committed
  4dceba3

  Copy full SHA for 4dceba3

• Cluster template for multi-node training

  srmsoumya

  committed
  fe01673

  Copy full SHA for fe01673

Commits on Aug 12, 2024

• Fix distributed DataLoader, add new env & slurm script

  srmsoumya

  committed
  940fbab

  Copy full SHA for 940fbab

• Fix docs

  srmsoumya

  committed
  f12e221

  Copy full SHA for f12e221

Commits on Aug 13, 2024

• Scale down recontruction loss for MODIS, change alpha to 0.9 for rec/rep split

  srmsoumya

  committed
  ea4136b

  Copy full SHA for ea4136b

Commits on Aug 21, 2024

• Use groups in wandb

  srmsoumya

  committed
  224c547

  Copy full SHA for 224c547

• Add random dropping for channels

  srmsoumya

  committed
  79712a7

  Copy full SHA for 79712a7

• Add script to check sanity of npz files

  srmsoumya

  committed
  380e814

  Copy full SHA for 380e814

• Add script to split npz files of batch 128 to 32

  srmsoumya

  committed
  4fe7138

  Copy full SHA for 4fe7138

Commits on Sep 20, 2024

• Adapt script to large model size

  srmsoumya

  committed
  bb6d13c

  Copy full SHA for bb6d13c

Commits on Nov 4, 2024

• Modify classify, segment examples for clay v1.5

  srmsoumya

  committed
  c638972

  Copy full SHA for c638972

• Check non MRL loss for the model (#331)

  Show description for 05c167e
  srmsoumya

  authored
  05c167e

  Copy full SHA for 05c167e

Commits on Nov 5, 2024

• Document MODIS data sampling

  yellowcap

  committed
  38a3c27

  Copy full SHA for 38a3c27

• intermediate commit

  yellowcap

  committed
  4f464b9

  Copy full SHA for 4f464b9

• intermediate

  yellowcap

  committed
  8ecff64

  Copy full SHA for 8ecff64

• Update to v1.5 and add logging

  yellowcap

  committed
  e701052

  Copy full SHA for e701052

• Fix for v1.5 module input

  yellowcap

  committed
  d46522a

  Copy full SHA for d46522a

• Update adding sentinel-2

  yellowcap

  committed
  c7283d7

  Copy full SHA for c7283d7

• Upgrade stacchip

  yellowcap

  committed
  c71bebf

  Copy full SHA for c71bebf

• Update adding sentinel-2

  yellowcap

  committed
  97404e8

  Copy full SHA for 97404e8

• Update docker file

  yellowcap

  committed
  308eb80

  Copy full SHA for 308eb80

• Update docker file

  yellowcap

  committed
  c9af8e6

  Copy full SHA for c9af8e6

• Update docker file

  yellowcap

  committed
  8ce1515

  Copy full SHA for 8ce1515

• update dockerfile

  yellowcap

  committed
  300d978

  Copy full SHA for 300d978

• Fix import path

  yellowcap

  committed
  9b5b7a6

  Copy full SHA for 9b5b7a6

• Fix paths again

  yellowcap

  committed
  d79f93e

  Copy full SHA for d79f93e

• Fix paths in docker file

  yellowcap

  committed
  4d8a1a5

  Copy full SHA for 4d8a1a5

• Fix docker file

  yellowcap

  committed
  70030ba

  Copy full SHA for 70030ba

• Fix docker file

  yellowcap

  committed
  7beb600

  Copy full SHA for 7beb600

• Use relative path

  yellowcap

  committed
  d442661

  Copy full SHA for d442661

• Make checkpoint path relative

  yellowcap

  committed
  dd0d0f9

  Copy full SHA for dd0d0f9

• Make path relative

  yellowcap

  committed
  40659da

  Copy full SHA for 40659da

• Make metadata dynamic

  yellowcap

  committed
  2b1a567

  Copy full SHA for 2b1a567

• Make metadata dynamic fix

  yellowcap

  committed
  3ad580c

  Copy full SHA for 3ad580c

• Log device early

  yellowcap

  committed
  c08926e

  Copy full SHA for c08926e

• Use pip instead of conda to avoid re-install of torch

  yellowcap

  committed
  c14f8a1

  Copy full SHA for c14f8a1

• Use pip instead of conda to avoid re-install of torch

  yellowcap

  committed
  9952c4e

  Copy full SHA for 9952c4e

• Use pip instead of conda to avoid re-install of torch

  yellowcap

  committed
  f2773ee

  Copy full SHA for f2773ee

• Use pip instead of conda to avoid re-install of torch

  yellowcap

  committed
  cdf6e41

  Copy full SHA for cdf6e41

• Use pip instead of conda to avoid re-install of torch

  yellowcap

  committed
  bdb2ca3

  Copy full SHA for bdb2ca3

• Remove patch level embeddings storing

  yellowcap

  committed
  9146bdd

  Copy full SHA for 9146bdd

• Flaten path to mirror naip-analytic bucket

  yellowcap

  committed
  b093ec5

  Copy full SHA for b093ec5

• Add option to limit to state

  yellowcap

  committed
  ab703ce

  Copy full SHA for ab703ce

• Remove stale files

  yellowcap

  committed
  e902414

  Copy full SHA for e902414

• Update model checkpoint name

  yellowcap

  committed
  2f33b44

  Copy full SHA for 2f33b44

• Update model checkpoint and s3 sign strategy

  yellowcap

  committed
  d85a664

  Copy full SHA for d85a664

• Adapt to rio-stac 0.10.0 and pin requirement

  yellowcap

  committed
  85f0544

  Copy full SHA for 85f0544

• Fix datetime bug for files that have date stamps in them

  yellowcap

  committed
  89f1be1

  Copy full SHA for 89f1be1

Commits on Nov 6, 2024

• Add check for files that are already processed

  yellowcap

  committed
  eefae20

  Copy full SHA for eefae20

• Move embeddings to CPU early

  yellowcap

  committed
  2496953

  Copy full SHA for 2496953

• Remove patch embedding extraction

  yellowcap

  committed
  097e291

  Copy full SHA for 097e291

Commits on Nov 12, 2024

• Sentinel-2 2024 run preparation

  yellowcap

  committed
  0a6f476

  Copy full SHA for 0a6f476

• Fix sentinel paths for output

  yellowcap

  committed
  6c867ee

  Copy full SHA for 6c867ee

Commits on Nov 22, 2024

• Pre-download S2 scene, batch pixel load and embeding generation

  yellowcap

  committed
  ac1434b

  Copy full SHA for ac1434b

• Use torch tensor for normalization

  yellowcap

  committed
  c270a4c

  Copy full SHA for c270a4c

• Use custom endpoint on demand

  yellowcap

  committed
  88ed3c0

  Copy full SHA for 88ed3c0

Commits on Nov 24, 2024

• Check exists for Sentinel-2 process

  yellowcap

  committed
  ceecb61

  Copy full SHA for ceecb61

Commits on Nov 26, 2024

• Make Dockerfile cacheable

  yellowcap

  committed
  55a82a8

  Copy full SHA for 55a82a8

Commits on Dec 21, 2024

• Update all-sentinel.py

  Show description for 085f371
  MaceGrim

  authored
  085f371

  Copy full SHA for 085f371

• Update utils.py

  Show description for 2a48007
  MaceGrim

  authored
  2a48007

  Copy full SHA for 2a48007