Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

978 Migrate Local Users, Datasets, Folders and Files #1058

Open
wants to merge 47 commits into
base: release/v2.0-beta-3
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
511923d
adding new dependency and class
tcnichol Mar 28, 2024
f763cb4
starting a user migration
tcnichol Mar 28, 2024
84d43bc
adding script for migrate metadata definitions
tcnichol Mar 28, 2024
e82e023
adding option of temporary password (for use with users who are being…
tcnichol Mar 28, 2024
dcdbea1
reverting changes
tcnichol Mar 28, 2024
79e56b4
temporary - for users that are migrated
tcnichol Mar 28, 2024
33f1cab
creates users
tcnichol Apr 23, 2024
93509e7
does not actually create the dataset
tcnichol Apr 25, 2024
aba43a3
using beanie might not work here due to async issues
tcnichol Apr 25, 2024
727685f
using nest asyncio to fix problem with even loop closing
tcnichol Apr 25, 2024
d4e9526
entry from right table for file bytes
tcnichol Apr 26, 2024
8c7d6bd
dependencies for migration, adding file entry and indexing files
tcnichol May 20, 2024
2ba7df5
wrong api key
tcnichol May 20, 2024
f624784
api key works for user,
tcnichol May 21, 2024
adc6c63
file upload works, sloppy needs fixed
tcnichol May 21, 2024
171b2b3
we are only getting partial files, not sure how to fix yet
tcnichol May 21, 2024
1f6a2ed
files upload right now
tcnichol May 26, 2024
2bf100c
delete file
tcnichol May 26, 2024
4eccae1
adding folders
tcnichol May 27, 2024
c01ce96
folder hierarchy - this should work
tcnichol May 30, 2024
c2ab289
works for now, folders need to be fixed
tcnichol May 31, 2024
2d7cdce
previous is no longer selectable when moving to my datasets. we start…
tcnichol Jun 1, 2024
65599bb
Merge branch 'main' into 978-migrate-users
tcnichol Jun 1, 2024
5be97bf
new folder hierarchy method works
tcnichol Jun 1, 2024
f479284
need a license for datasets
tcnichol Jun 2, 2024
cb4b835
method for getting all the folders
tcnichol Jun 2, 2024
e885a2a
need to remove logging, but folders are created and files uploaded to…
tcnichol Jun 3, 2024
b4db0f4
should work now
tcnichol Jun 3, 2024
72e0845
fixing typos
tcnichol Jun 3, 2024
a6cd25a
formatting
tcnichol Jun 3, 2024
54747ca
sample .env
tcnichol Jun 3, 2024
55ff21a
formatting
tcnichol Jun 3, 2024
e216405
Merge branch 'main' into 978-migrate-users
tcnichol Jul 13, 2024
851503e
refactoring migrate_user
tcnichol Jul 13, 2024
074154e
formatting
tcnichol Jul 15, 2024
909b8f6
Merge branch 'main' into 978-migrate-users
tcnichol Aug 5, 2024
b309023
Merge branch 'release/v2.0-beta-3' into 978-migrate-users
tcnichol Aug 6, 2024
58ada46
Merge remote-tracking branch 'origin/main' into 978-migrate-users
tcnichol Aug 14, 2024
62d5b02
rename
tcnichol Aug 22, 2024
8772325
Merge branch 'release/v2.0-beta-3' into 978-migrate-users
longshuicy Aug 22, 2024
ec56aa4
migrate metadata definition; modify migration script (#1184)
longshuicy Aug 26, 2024
8050460
fix clowder user api key creation issue
longshuicy Aug 27, 2024
c936760
fix user api key if user already exists
tcnichol Aug 27, 2024
5547715
Merge remote-tracking branch 'origin/978-migrate-users' into 978-migr…
tcnichol Aug 27, 2024
86c8493
switch headers and files
tcnichol Aug 27, 2024
2a52c7c
Migrate metadata (#1192)
longshuicy Aug 29, 2024
4e68f21
1187 migrate collections and spaces as v2 metadata. (#1190)
tcnichol Sep 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,17 @@ deployments/kubernetes/charts/clowder2/*clowder2-software-dev.yaml
*secret*.yaml

# Environments
**/*.env
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
*/.env
scripts/migration/.env


# Test folder contect except for test source code and metadata document.
backend/app/tests/*
Expand Down
5 changes: 4 additions & 1 deletion backend/app/keycloak_auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -403,7 +403,9 @@ async def get_current_user_id(identity: Json = Depends(get_token)) -> str:
return keycloak_id


async def create_user(email: str, password: str, firstName: str, lastName: str):
async def create_user(
email: str, password: str, firstName: str, lastName: str, temporary: bool = False
):
"""Create a user in Keycloak."""
keycloak_admin = KeycloakAdmin(
server_url=settings.auth_server_url,
Expand All @@ -427,6 +429,7 @@ async def create_user(email: str, password: str, firstName: str, lastName: str):
{
"value": password,
"type": "password",
"temporary": temporary,
}
],
},
Expand Down
Loading
Loading