Download UI (#176)

#59 #60 #120 #121 Implements the download modals for the dataset and run pages: - Adds remix-i18next for i18n interpolation and component nesting - Refactor tabs to use MUI - Add layout context - Use loremflickr for cat images - Add `<semibold />` to `<I18n />` component - Add hook for storing state in query parameters - download modal components - configure download - download options ## Demo ### Dataset page ### via S3 <img width="649" alt="image" src="https://github.com/chanzuckerberg/cryoet-data-portal/assets/2176050/55e215bc-c721-46be-b092-fee9b311955a"> ### via API <img width="650" alt="image" src="https://github.com/chanzuckerberg/cryoet-data-portal/assets/2176050/603d6b11-60fc-4fc3-b53c-6820916ba9c9"> ## Run page ### Configure download - download tomogram <img width="651" alt="image" src="https://github.com/chanzuckerberg/cryoet-data-portal/assets/2176050/29b3f0bc-3f3e-4239-91ab-89e5af137dbe"> ### Configure download - download all annotations <img width="636" alt="image" src="https://github.com/chanzuckerberg/cryoet-data-portal/assets/2176050/51333cec-bc49-42ce-94f7-0a9a72913dba"> ### Download options - download tomogram - via direct download <img width="636" alt="image" src="https://github.com/chanzuckerberg/cryoet-data-portal/assets/2176050/9a5409e2-496f-4251-bad1-fdfaf5aa7378"> ### Download options - download tomogram - via curl <img width="637" alt="image" src="https://github.com/chanzuckerberg/cryoet-data-portal/assets/2176050/7242883c-8b92-4734-8065-996cd6e14994"> ### Download options - download tomogram - via S3 <img width="641" alt="image" src="https://github.com/chanzuckerberg/cryoet-data-portal/assets/2176050/5605b89d-c094-4ed8-926b-48961008c030"> ### Download options - download tomogram - via API <img width="654" alt="image" src="https://github.com/chanzuckerberg/cryoet-data-portal/assets/2176050/a31f0010-8441-460a-988e-c07009dec6a3"> ### Download options - all annotations - via S3 <img width="633" alt="image" src="https://github.com/chanzuckerberg/cryoet-data-portal/assets/2176050/914f4307-fedc-40f9-a738-fb51e1896f9e"> ### Download options - all annotations - via API <img width="641" alt="image" src="https://github.com/chanzuckerberg/cryoet-data-portal/assets/2176050/b3029885-e585-474f-8969-04e910439ce6">
chanzuckerberg · Nov 28, 2023 · 76a3063 · 76a3063
1 parent 720aeb7
commit 76a3063
Show file tree

Hide file tree

Showing 54 changed files with 2,035 additions and 197 deletions.
diff --git a/docs/cryoet_data_portal_docsite_aws_download.md b/docs/cryoet_data_portal_docsite_aws_download.md
@@ -1,15 +1,17 @@
 # Download Data Using Amazon Web Services (AWS)
 
-This page provides details on how to use Amazon Web Services (AWS) Command Line Interface (CLI) tool to download data from the CryoET Data Portal. 
+This page provides details on how to use Amazon Web Services (AWS) Command Line Interface (CLI) tool to download data from the CryoET Data Portal.
 
 **The Data Portal's S3 bucket is public**, so it can be used without credential sign in by specifying `--no-sign-request` in your commands. In only a few minutes, you can get started downloading data following the steps in the [Quickstart Guide](#quickstart), and for more details, refer to the [Installation](#installation), [Download Data](#download-data), and [Optimize Download Speed](#optimize-download-speed) in-depth explanations.
 
 ## Quickstart
+
 1. Download the installer: [MacOS Installer Download](https://awscli.amazonaws.com/AWSCLIV2.pkg) / [Windows Installer Download](https://awscli.amazonaws.com/AWSCLIV2.msi)
 2. Open installer and complete installation following prompts. (No further steps, since credentials ARE NOT needed to use the tool.)
 3. Open terminal (MacOS) or command prompt (Windows).
 4. Copy and paste the command from the download prompt for the desired data into terminal / command prompt and hit enter.
 5. Alternatively, create a custom command inserting the S3 URL of the data and the desired download destination in the spaces provided.
+
 ```
 aws s3 cp --no-sign-request [S3 bucket URL] [Local destination path]
 ```
@@ -20,6 +22,7 @@ For example, to download a particular JSON file of tomogram metadata into a fold
 aws s3 cp --no-sign-request s3://cryoet-data-portal-public/10000/TS_026/Tomograms/VoxelSpacing13.48/CanonicalTomogram/tomogram_metadata.json ~/Downloads/
 
 ```
+
 In the above example, the download happened very quickly because the file was only about 1 kB in size. However, typical tomograms are multiple GB, so expect downloading to take 30-60 mins for a single tomogram for a given run, but downloading could take as long as days depending on the number and sizes of the files. To speed up download, you can follow [these instructions to optimize download speed](#optimize-download-speed)
 
 For more detailed instructions, please refer to the sections below.
@@ -39,32 +42,36 @@ Once AWS CLI is installed, you will be able to use it in terminal (MacOS) or com
 1. Download the installer pkg file using this URL: [https://awscli.amazonaws.com/AWSCLIV2.pkg](https://awscli.amazonaws.com/AWSCLIV2.pkg)
 2. Open the file and follow the instructions provided in the installer window.
 
-To confirm successful installation, open terminal and type `aws --version` to list the version of the AWS CLI installed. If installation was successful, you should see an output like: 
+To confirm successful installation, open terminal and type `aws --version` to list the version of the AWS CLI installed. If installation was successful, you should see an output like:
+
 ```
 aws-cli/2.7.25 Python/3.10.6 Darwin/23.0.0 source/arm64 prompt/off
 ```
 
 ### Windows Installation
 
 1. Download the installer pkg file using this URL: [https://awscli.amazonaws.com/AWSCLIV2.msi](https://awscli.amazonaws.com/AWSCLIV2.msi)
-2. Open the file and follow the instructions provided in the installer window. 
+2. Open the file and follow the instructions provided in the installer window.
+
+To confirm successful installation, open a command prompt window (open the Start menu and search for cmd) and type `aws --version` to list the version of the AWS CLI installed. If installation was successful, you should see an output like:
 
-To confirm successful installation, open a command prompt window (open the Start menu and search for cmd) and type `aws --version` to list the version of the AWS CLI installed. If installation was successful, you should see an output like: 
 ```
 aws-cli/2.10.0 Python/3.11.2 Windows/10 exe/AMD64 prompt/off
 ```
 
 ## Download Data
 
 To download data, we'll run commands in terminal (MacOS) or command prompt (Windows). The basic structure of these commands is below:
+
 ```
 aws <command> <subcommand> <flags> [options and parameters (often S3 URL)]
 ```
+
 If you followed the above installation instructions, which did not include setting up credentials, use `--no-sign-request` as a `<flag>` in all of your AWS CLI commands to indicate that you are accessing the bucket without signing in.
 
 The URL of the CryoET Data Portal is `s3://cryoet-data-portal-public`, and each dataset in the bucket has its own unique URL such as `s3://cryoet-data-portal-public/10000/TS_026`.
 
-To list all files in a directory, use the `s3` and `ls` as the `<command>` and `<subcommand>`, respectively. 
+To list all files in a directory, use the `s3` and `ls` as the `<command>` and `<subcommand>`, respectively.
 
 The basic structure of this command is `aws s3 ls --no-sign-request [s3 bucket URL]`. For example, to list all data in the portal use:
 
@@ -86,16 +93,19 @@ To download a file, We can use the `s3` and `cp` as the `<command>` and `<subcom
 aws s3 cp --no-sign-request s3://cryoet-data-portal-public/10000/TS_026/Tomograms/VoxelSpacing13.48/CanonicalTomogram/tomogram_metadata.json ~/Downloads/
 
 ```
+
 The file should appear in your specified directory and the output in terminal / command prompt should be something like:
+
 ```
 download: s3://cryoet-data-portal-public/10000/TS_026/Tomograms/VoxelSpacing13.48/CanonicalTomogram/tomogram_metadata.json to ./tomogram_metadata.json
 ```
 
-In the above example, the download happened very quickly because the file was only about 1 kB in size. However, typical tomograms are multiple GB, so expect downloading to take 30-60 mins for a single tomogram for a given run, but downloading could take as long as days depending on the number and sizes of the files. 
+In the above example, the download happened very quickly because the file was only about 1 kB in size. However, typical tomograms are multiple GB, so expect downloading to take 30-60 mins for a single tomogram for a given run, but downloading could take as long as days depending on the number and sizes of the files.
 
 ## Optimize Download Speed
 
-You can optimize your download speed by configuring your AWS CLI with the below command, which will increase your transfer rate to ~50 MB/s if your connection has sufficient bandwidth. 
+You can optimize your download speed by configuring your AWS CLI with the below command, which will increase your transfer rate to ~50 MB/s if your connection has sufficient bandwidth.
+
 ```
 aws configure set default.s3.max_concurrent_requests 30
 ```

diff --git a/docs/docs/cryoet_data_portal_docsite_github_issue.md.md b/docs/docs/cryoet_data_portal_docsite_github_issue.md.md
@@ -2,18 +2,21 @@
 
 Did you encounter a bug, error, or other issue while using the portal? [Submit an issue on Github](https://github.com/chanzuckerberg/cryoet-data-portal/issues) to let us know!
 
-To submit an issue, you'll need to create a [free Github account](https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F&source=header-home). 
-This allows our team to followup with you on Github if we have a question about the problem you encountered. Then, [fill out this form](https://github.com/chanzuckerberg/cryoet-data-portal/issues/new). 
+To submit an issue, you'll need to create a [free Github account](https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F&source=header-home).
+This allows our team to followup with you on Github if we have a question about the problem you encountered. Then, [fill out this form](https://github.com/chanzuckerberg/cryoet-data-portal/issues/new).
 We suggest you use a descriptive title, paste an error messages using the `<>` icon on the form, and provide as many details as possible about the problem, including what you expected to happen and what type of machine you were using.
 
 An example issue is below:
 
 Title: Tomogram TS_026 cannot be downloaded
 
-Body: 
+Body:
 I have the AWS CLI tool installed on a Mac computer. I copied the download command from the prompt on the tomogram page. Instead of downloading, I received this error message:
+
 ```
 ERROR MESSAGE COPIED FROM TERMINAL
 ```
-------------------------------------------
+
+---
+
 For more information about submiting issues on Github, please refer to [Github's documentation](https://docs.github.com/en/issues/tracking-your-work-with-issues/creating-an-issue#creating-an-issue-from-a-repository).
diff --git a/frontend/packages/data-portal/app/axios.ts b/frontend/packages/data-portal/app/axios.ts
@@ -0,0 +1,5 @@
+import Axios from 'axios'
+import { setupCache } from 'axios-cache-interceptor'
+
+export const axios =
+  process.env.NODE_ENV === 'production' ? setupCache(Axios) : Axios
diff --git a/frontend/packages/data-portal/app/components/BrowseData/DatasetTable.tsx b/frontend/packages/data-portal/app/components/BrowseData/DatasetTable.tsx
@@ -89,7 +89,7 @@ export function DatasetTable() {
               <KeyPhoto
                 title={dataset.title}
                 // TODO use dataset keyphoto
-                src="https://cataas.com/cat"
+                src="https://loremflickr.com/400/400/cat"
                 loading={isLoadingDebounced}
               />
 
@@ -240,7 +240,6 @@ export function DatasetTable() {
     <Table
       data={isLoadingDebounced ? LOADING_DATASETS : datasets}
       columns={columns}
-      withFiltersSidebar
     />
   )
 }
diff --git a/frontend/packages/data-portal/app/components/CopyBox.tsx b/frontend/packages/data-portal/app/components/CopyBox.tsx
@@ -0,0 +1,50 @@
+import { Button } from '@czi-sds/components'
+import { ReactNode } from 'react'
+
+import { useI18n } from 'app/hooks/useI18n'
+import { cns } from 'app/utils/cns'
+
+export function CopyBox({
+  className,
+  content,
+  title,
+  titleClassName,
+}: {
+  className?: string
+  content: ReactNode
+  title?: ReactNode
+  titleClassName?: string
+}) {
+  const { t } = useI18n()
+
+  return (
+    <div className={className}>
+      {title && (
+        <p
+          className={cns(
+            'text-sds-header-xs leading-sds-header-xs font-semibold mb-sds-xxs',
+            titleClassName,
+          )}
+        >
+          {title}:
+        </p>
+      )}
+
+      <div
+        className={cns(
+          'bg-sds-gray-100 border-[0.5px] border-sds-gray-300',
+          'p-sds-default flex gap-sds-s',
+        )}
+      >
+        <pre className="whitespace-normal break-all flex-grow">{content}</pre>
+
+        <Button
+          className="!min-w-0 uppercase !p-0"
+          onClick={() => navigator.clipboard.writeText(String(content))}
+        >
+          {t('copy')}
+        </Button>
+      </div>
+    </div>
+  )
+}
diff --git a/frontend/packages/data-portal/app/components/Dataset/DatasetHeader.tsx b/frontend/packages/data-portal/app/components/Dataset/DatasetHeader.tsx
@@ -4,12 +4,15 @@ import { DatasetDescription } from 'app/components/Dataset/DatasetDescription'
 import { KeyPhoto } from 'app/components/KeyPhoto'
 import { PageHeader } from 'app/components/PageHeader'
 import { useDatasetById } from 'app/hooks/useDatasetById'
-import { i18n } from 'app/i18n'
+import { useDownloadModalQueryParamState } from 'app/hooks/useDownloadModalQueryParamState'
+import { useI18n } from 'app/hooks/useI18n'
 import { useDrawer } from 'app/state/drawer'
 
 export function DatasetHeader() {
   const { dataset } = useDatasetById()
   const drawer = useDrawer()
+  const { t } = useI18n()
+  const { openDatasetDownloadModal } = useDownloadModalQueryParamState()
 
   return (
     <PageHeader
@@ -18,14 +21,15 @@ export function DatasetHeader() {
           startIcon={<Icon sdsIcon="download" sdsType="button" sdsSize="l" />}
           sdsType="primary"
           sdsStyle="rounded"
+          onClick={openDatasetDownloadModal}
         >
-          {i18n.downloadDataset}
+          {t('downloadDataset')}
         </Button>
       }
       lastModifiedDate={dataset.last_modified_date ?? dataset.deposition_date}
       metadata={[
         {
-          key: i18n.portalIdBlank,
+          key: t('portalIdBlank'),
           value: String(dataset.id),
           uppercase: true,
         },
@@ -40,7 +44,10 @@ export function DatasetHeader() {
         </div>
 
         <div className="flex-1 w-full max-w-[465px]">
-          <KeyPhoto title={dataset.title} src="https://cataas.com/cat" />
+          <KeyPhoto
+            title={dataset.title}
+            src="https://loremflickr.com/400/400/cat"
+          />
         </div>
       </div>
     </PageHeader>

diff --git a/frontend/packages/data-portal/app/components/Dataset/RunsTable.tsx b/frontend/packages/data-portal/app/components/Dataset/RunsTable.tsx
@@ -52,7 +52,7 @@ export function RunsTable() {
               <KeyPhoto
                 title={run.name}
                 // TODO use dataset keyphoto
-                src="https://cataas.com/cat"
+                src="https://loremflickr.com/400/400/cat"
                 loading={isLoadingDebounced}
               />
 
@@ -122,10 +122,6 @@ export function RunsTable() {
   }, [isLoadingDebounced, location.pathname, location.search])
 
   return (
-    <Table
-      data={isLoadingDebounced ? LOADING_RUNS : runs}
-      columns={columns}
-      withFiltersSidebar
-    />
+    <Table data={isLoadingDebounced ? LOADING_RUNS : runs} columns={columns} />
   )
 }
diff --git a/...end/packages/data-portal/app/components/DatasetFilter/AnnotationMetadataFilterSection.tsx b/...end/packages/data-portal/app/components/DatasetFilter/AnnotationMetadataFilterSection.tsx
@@ -2,7 +2,7 @@ import { useSearchParams } from '@remix-run/react'
 import { useMemo } from 'react'
 
 import { FilterSection, SelectFilter } from 'app/components/Filters'
-import { DatasetFilterQueryParams } from 'app/constants/query'
+import { QueryParams } from 'app/constants/query'
 import { useDatasets } from 'app/hooks/useDatasets'
 import { i18n } from 'app/i18n'
 import { BaseFilterOption } from 'app/types/filter'
@@ -18,9 +18,7 @@ export function AnnotationMetadataFilterSection() {
 
   const objectNameValue = useMemo<BaseFilterOption[]>(
     () =>
-      searchParams
-        .getAll(DatasetFilterQueryParams.ObjectName)
-        .map((value) => ({ value })),
+      searchParams.getAll(QueryParams.ObjectName).map((value) => ({ value })),
     [searchParams],
   )
 
@@ -32,7 +30,7 @@ export function AnnotationMetadataFilterSection() {
   const objectShapeTypeValue = useMemo<BaseFilterOption[]>(
     () =>
       searchParams
-        .getAll(DatasetFilterQueryParams.ObjectShapeType)
+        .getAll(QueryParams.ObjectShapeType)
         .map((value) => ({ value })),
     [searchParams],
   )
@@ -46,10 +44,10 @@ export function AnnotationMetadataFilterSection() {
         label={i18n.objectName}
         onChange={(options) =>
           setSearchParams((prev) => {
-            prev.delete(DatasetFilterQueryParams.ObjectName)
+            prev.delete(QueryParams.ObjectName)
 
             options?.forEach((option) =>
-              prev.append(DatasetFilterQueryParams.ObjectName, option.value),
+              prev.append(QueryParams.ObjectName, option.value),
             )
 
             return prev
@@ -64,13 +62,10 @@ export function AnnotationMetadataFilterSection() {
         label={i18n.objectShapeType}
         onChange={(options) =>
           setSearchParams((prev) => {
-            prev.delete(DatasetFilterQueryParams.ObjectShapeType)
+            prev.delete(QueryParams.ObjectShapeType)
 
             options?.forEach((option) =>
-              prev.append(
-                DatasetFilterQueryParams.ObjectShapeType,
-                option.value,
-              ),
+              prev.append(QueryParams.ObjectShapeType, option.value),
             )
 
             return prev

diff --git a/frontend/packages/data-portal/app/components/DatasetFilter/HardwareFilterSection.tsx b/frontend/packages/data-portal/app/components/DatasetFilter/HardwareFilterSection.tsx
@@ -1,7 +1,7 @@
 import { useMemo } from 'react'
 
 import { FilterSection, SelectFilter } from 'app/components/Filters'
-import { DatasetFilterQueryParams } from 'app/constants/query'
+import { QueryParams } from 'app/constants/query'
 import { useDatasetFilter } from 'app/hooks/useDatasetFilter'
 import { useDatasets } from 'app/hooks/useDatasets'
 import { i18n } from 'app/i18n'
@@ -31,7 +31,7 @@ export function HardwareFilterSection() {
         value={cameraManufacturerValue}
         label={i18n.cameraManufacturer}
         onChange={(option) =>
-          updateValue(DatasetFilterQueryParams.CameraManufacturer, option)
+          updateValue(QueryParams.CameraManufacturer, option)
         }
       />
     </FilterSection>

diff --git a/frontend/packages/data-portal/app/components/DatasetFilter/IncludedContentsFilterSection.tsx b/frontend/packages/data-portal/app/components/DatasetFilter/IncludedContentsFilterSection.tsx
@@ -5,7 +5,7 @@ import {
   FilterSection,
   SelectFilter,
 } from 'app/components/Filters'
-import { DatasetFilterQueryParams } from 'app/constants/query'
+import { QueryParams } from 'app/constants/query'
 import { useDatasetFilter } from 'app/hooks/useDatasetFilter'
 import { i18n } from 'app/i18n'
 import {
@@ -60,10 +60,7 @@ export function IncludedContentsFilterSection() {
       <BooleanFilter
         label={i18n.groundTruthAnnotation}
         onChange={(value) =>
-          updateValue(
-            DatasetFilterQueryParams.GroundTruthAnnotation,
-            value ? 'true' : null,
-          )
+          updateValue(QueryParams.GroundTruthAnnotation, value ? 'true' : null)
         }
         value={isGroundTruthEnabled}
       />
@@ -73,9 +70,7 @@ export function IncludedContentsFilterSection() {
         options={AVAILABLE_FILES_OPTIONS}
         value={availableFilesOptions}
         label={i18n.availableFiles}
-        onChange={(options) =>
-          updateValue(DatasetFilterQueryParams.AvailableFiles, options)
-        }
+        onChange={(options) => updateValue(QueryParams.AvailableFiles, options)}
       />
 
       <SelectFilter
@@ -84,7 +79,7 @@ export function IncludedContentsFilterSection() {
         label={i18n.numberOfRuns}
         onChange={(option) =>
           updateValue(
-            DatasetFilterQueryParams.NumberOfRuns,
+            QueryParams.NumberOfRuns,
             option ? JSON.stringify(option.value) : null,
           )
         }