Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DataGridPro] Server side data source lazy loading #13878

Open
wants to merge 66 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
1c75ed9
row indexes query params take precedence over pagination
arminmeh Jul 18, 2024
7dfca4f
Add basic (dev) demo for server side lazy loading
arminmeh Jul 18, 2024
1aaf275
Extend data source model
arminmeh Jul 18, 2024
5b73693
Update API docs
arminmeh Jul 18, 2024
bf5819d
Add new event to be sent when new batch of rows should be loaded from…
arminmeh Jul 18, 2024
976c80f
Data source cache takes start and end params into account
arminmeh Jul 18, 2024
a541a73
Add server side version of lazy loading pre-processor and hook
arminmeh Jul 18, 2024
d9267a4
Prvent both client and server side lazy loading setup
arminmeh Jul 18, 2024
377eef3
Data source hook supports new way of loading data and replaces lazy l…
arminmeh Jul 18, 2024
ded846d
Lazy loading flag moved to the root props instead of it being a part …
arminmeh Jul 22, 2024
0e8b8f2
Update docs
arminmeh Jul 22, 2024
a25b6b8
Check for skeleton rows with the sorting and filtering enabled
arminmeh Aug 2, 2024
c1e96cb
Loading overlay for the initial load only
arminmeh Aug 2, 2024
81c13df
Skeleton rows should be added after data load to be able to know the …
arminmeh Aug 7, 2024
f399015
Rename and repurpose the event
arminmeh Aug 7, 2024
40be521
Publish event even for data source updates from cache
arminmeh Aug 7, 2024
a387468
Fix duplicate key issues. Add initial throttling
arminmeh Aug 7, 2024
e916157
Move events to pro lookup. Do not document
arminmeh Aug 8, 2024
f2faec4
Add default value
arminmeh Aug 8, 2024
c25cb99
Add hook to the premium package
arminmeh Aug 8, 2024
2e85c2b
Update demo
arminmeh Aug 8, 2024
662bd05
Remove encoding
arminmeh Aug 14, 2024
b3f3b72
Remove non-existing import
arminmeh Aug 16, 2024
da06e25
Fix the mock data slice calculation
arminmeh Aug 16, 2024
e992aea
Import throttle from internals package. Do not reset rendering contex…
arminmeh Aug 16, 2024
1525b2e
Adjust start and end params to cover whole page(s). Cache responses i…
arminmeh Aug 16, 2024
63fea21
Do not reset render context on filtering
arminmeh Aug 16, 2024
55d841c
Params should always be adjusted to the whole page to allow proper ex…
arminmeh Aug 16, 2024
e03e40e
Add support for infinite loading
arminmeh Sep 6, 2024
3907e2a
Update docs
arminmeh Sep 6, 2024
af46ed5
Update docs and API. Add missing types
arminmeh Sep 6, 2024
40be195
Cleanup
arminmeh Sep 6, 2024
f86d6b3
Change the way current row count is retrieved to support controlled r…
arminmeh Sep 12, 2024
a3d2976
Get current rows from the state instead of prop when data source is used
arminmeh Sep 19, 2024
5bf02fd
Make sure row count is not undefined
arminmeh Sep 19, 2024
1aac03d
Support switching between lazy loading and infinite loading mode
arminmeh Sep 19, 2024
9decdee
Extend fetchRows API to enable request retries. Keep compatibility wi…
arminmeh Sep 23, 2024
3320dea
Pagination params adjustment inside lazyLoading hook. Use the same ap…
arminmeh Oct 1, 2024
f1db48b
Update prop validation
arminmeh Oct 1, 2024
52c0f91
Clear the cache when grid is reset
arminmeh Oct 2, 2024
39ef57a
Show loading indicator when grid is reset
arminmeh Oct 2, 2024
68ab9a3
Update existing examples
arminmeh Oct 2, 2024
7b64c36
Update docs and add new examples
arminmeh Oct 2, 2024
6e3c78a
Fix docs
arminmeh Oct 2, 2024
e8993e6
Update description
arminmeh Oct 2, 2024
c5855d7
Fix type error
arminmeh Oct 2, 2024
555b48e
Update API docs
arminmeh Oct 3, 2024
9d9ab54
Align toolbar customization
arminmeh Oct 3, 2024
34cd639
Update future feature links
arminmeh Oct 3, 2024
9ca68b7
Do not export GridDataSourceCacheDefaultConfig
arminmeh Oct 3, 2024
5f6461a
Add prop to control throttling
arminmeh Oct 3, 2024
e9c310e
Update API docs
arminmeh Oct 3, 2024
1ee5d0a
Fix: Sorting in viewport mode should fill the grid with skeleton rows
arminmeh Oct 4, 2024
314a3d5
Add tests
arminmeh Oct 4, 2024
74749cc
Fix test
arminmeh Oct 4, 2024
fcaaeba
Use default delay for the mock server in the demos
arminmeh Oct 4, 2024
0a4b2af
Add loading overlay on sorting
arminmeh Oct 4, 2024
0869dd3
Put back a bit more delay on the mock server
arminmeh Oct 4, 2024
5d0fb34
Update API docs to distinguish client and server side event
arminmeh Oct 25, 2024
32af550
Example updates
arminmeh Oct 30, 2024
71bb345
Remove new label from the sub-page
arminmeh Oct 30, 2024
92c920c
Error example updates
arminmeh Oct 30, 2024
6deb4ae
put back parentId param to fetchRows
arminmeh Nov 1, 2024
7d367e1
Update error example to work with the new fetchRows definition
arminmeh Nov 1, 2024
42a9bc0
Fix check parent condition
arminmeh Nov 1, 2024
bcf300f
Update API
arminmeh Nov 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/data/data-grid/events/events.json
Copy link
Member

@MBilalShafi MBilalShafi Nov 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Server-side sorting, filtering, pagination, etc. doesn't seem to work in the existing demos.
Tested with plain-data, tree-data, row grouping
Working on live versions: https://mui.com/x/react-data-grid/server-side-data/

Is it because of the usage of firstRowToRender, lastRowToRender in useMockServer?
Maybe simplifying it to the same interface as GridDataSource would be better?

Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@
{
"projects": ["x-data-grid-pro", "x-data-grid-premium"],
"name": "fetchRows",
"description": "Fired when a new batch of rows is requested to be loaded. Called with a GridFetchRowsParams object.",
"description": "Fired when a new batch of rows is requested to be loaded. Called with a GridFetchRowsParams object. Used to trigger <code>onFetchRows</code>.",
"params": "GridFetchRowsParams",
"event": "MuiEvent<{}>",
"componentProp": "onFetchRows"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
import * as React from 'react';
import {
DataGridPro,
useGridApiRef,
GridToolbar,
GRID_ROOT_GROUP_ID,
} from '@mui/x-data-grid-pro';
import Checkbox from '@mui/material/Checkbox';
import FormControlLabel from '@mui/material/FormControlLabel';
import { useMockServer } from '@mui/x-data-grid-generator';
import Alert from '@mui/material/Alert';
import Button from '@mui/material/Button';

function ErrorAlert({ onClick }) {
return (
<Alert
sx={{
position: 'absolute',
bottom: '0',
paddingX: 2,
paddingY: 1,
width: '100%',
zIndex: 10,
}}
severity="error"
action={
<Button color="inherit" size="small" onClick={onClick}>
Retry
</Button>
}
>
Could not fetch the data
</Alert>
);
}

function ServerSideLazyLoadingErrorHandling() {
const apiRef = useGridApiRef();
const [retryParams, setRetryParams] = React.useState(null);
const [shouldRequestsFail, setShouldRequestsFail] = React.useState(false);

const { fetchRows, ...props } = useMockServer(
{ rowLength: 100 },
{ useCursorPagination: false, minDelay: 300, maxDelay: 800 },
shouldRequestsFail,
);

const dataSource = React.useMemo(
() => ({
getRows: async (params) => {
const urlParams = new URLSearchParams({
filterModel: JSON.stringify(params.filterModel),
sortModel: JSON.stringify(params.sortModel),
firstRowToRender: `${params.start}`,
lastRowToRender: `${params.end}`,
});
const getRowsResponse = await fetchRows(
`https://mui.com/x/api/data-grid?${urlParams.toString()}`,
);
return {
rows: getRowsResponse.rows,
rowCount: getRowsResponse.rowCount,
};
},
}),
[fetchRows],
);

return (
<div style={{ width: '100%' }}>
<FormControlLabel
control={
<Checkbox
checked={shouldRequestsFail}
onChange={(event) => setShouldRequestsFail(event.target.checked)}
/>
}
label="Make the requests fail"
/>
<div style={{ height: 400, position: 'relative' }}>
{retryParams && (
<ErrorAlert
onClick={() => {
apiRef.current.unstable_dataSource.fetchRows(
GRID_ROOT_GROUP_ID,
retryParams,
);
setRetryParams(null);
}}
/>
)}
<DataGridPro
{...props}
apiRef={apiRef}
unstable_dataSource={dataSource}
unstable_onDataSourceError={(_, params) => setRetryParams(params)}
unstable_dataSourceCache={null}
lazyLoading
paginationModel={{ page: 0, pageSize: 10 }}
slots={{ toolbar: GridToolbar }}
/>
</div>
</div>
);
}

export default ServerSideLazyLoadingErrorHandling;
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
import * as React from 'react';
import {
DataGridPro,
useGridApiRef,
GridToolbar,
GridDataSource,
GridGetRowsParams,
GRID_ROOT_GROUP_ID,
} from '@mui/x-data-grid-pro';
import Checkbox from '@mui/material/Checkbox';
import FormControlLabel from '@mui/material/FormControlLabel';
import { useMockServer } from '@mui/x-data-grid-generator';
import Alert from '@mui/material/Alert';
import Button from '@mui/material/Button';

function ErrorAlert({ onClick }: { onClick: () => void }) {
return (
<Alert
sx={{
position: 'absolute',
bottom: '0',
paddingX: 2,
paddingY: 1,
width: '100%',
zIndex: 10,
}}
severity="error"
action={
<Button color="inherit" size="small" onClick={onClick}>
Retry
</Button>
}
>
Could not fetch the data
</Alert>
);
}
Comment on lines +16 to +37
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
function ErrorAlert({ onClick }: { onClick: () => void }) {
return (
<Alert
sx={{
position: 'absolute',
bottom: '0',
paddingX: 2,
paddingY: 1,
width: '100%',
zIndex: 10,
}}
severity="error"
action={
<Button color="inherit" size="small" onClick={onClick}>
Retry
</Button>
}
>
Could not fetch the data
</Alert>
);
}
function ErrorSnackbar(props: SnackbarProps & { onRetry: () => void }) {
const { onRetry, ...rest } = props;
return (
<Snackbar {...rest}>
<Alert
severity="error"
variant="filled"
sx={{ width: '100%' }}
action={
<Button color="inherit" size="small" onClick={onRetry}>
Retry
</Button>
}
>
Failed to fetch row data
</Alert>
</Snackbar>
);
}


function ServerSideLazyLoadingErrorHandling() {
const apiRef = useGridApiRef();
const [retryParams, setRetryParams] = React.useState<GridGetRowsParams | null>(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to hide the Alert when the data is fetched? Maybe by using rowsFetched event?

error-demo.mp4

That being said, I think passing getRowsParams as additional information with the rowsFetched event could be useful.

What do you think?

null,
);
const [shouldRequestsFail, setShouldRequestsFail] = React.useState(false);

const { fetchRows, ...props } = useMockServer(
{ rowLength: 100 },
{ useCursorPagination: false, minDelay: 300, maxDelay: 800 },
shouldRequestsFail,
);

const dataSource: GridDataSource = React.useMemo(
() => ({
getRows: async (params) => {
const urlParams = new URLSearchParams({
filterModel: JSON.stringify(params.filterModel),
sortModel: JSON.stringify(params.sortModel),
firstRowToRender: `${params.start}`,
lastRowToRender: `${params.end}`,
Comment on lines +58 to +59
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use similar terminology in the mock server to keep things more consistent?

Suggested change
firstRowToRender: `${params.start}`,
lastRowToRender: `${params.end}`,
start: params.start,
end: params.end,

});
const getRowsResponse = await fetchRows(
`https://mui.com/x/api/data-grid?${urlParams.toString()}`,
);
return {
rows: getRowsResponse.rows,
rowCount: getRowsResponse.rowCount,
};
},
}),
[fetchRows],
);

return (
<div style={{ width: '100%' }}>
<FormControlLabel
control={
<Checkbox
checked={shouldRequestsFail}
onChange={(event) => setShouldRequestsFail(event.target.checked)}
/>
}
label="Make the requests fail"
/>
<div style={{ height: 400, position: 'relative' }}>
{retryParams && (
<ErrorAlert
onClick={() => {
apiRef.current.unstable_dataSource.fetchRows(
GRID_ROOT_GROUP_ID,
retryParams,
);
setRetryParams(null);
}}
/>
)}
Comment on lines +86 to +95
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<ErrorAlert
onClick={() => {
apiRef.current.unstable_dataSource.fetchRows(
GRID_ROOT_GROUP_ID,
retryParams,
);
setRetryParams(null);
}}
/>
)}
<ErrorSnackbar
open={!!retryParams}
onRetry={() => {
apiRef.current.unstable_dataSource.fetchRows(
GRID_ROOT_GROUP_ID,
retryParams,
);
setRetryParams(null);
}}
/>

<DataGridPro
{...props}
apiRef={apiRef}
unstable_dataSource={dataSource}
unstable_onDataSourceError={(_, params) => setRetryParams(params)}
unstable_dataSourceCache={null}
lazyLoading
paginationModel={{ page: 0, pageSize: 10 }}
slots={{ toolbar: GridToolbar }}
/>
</div>
</div>
);
}

export default ServerSideLazyLoadingErrorHandling;
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import * as React from 'react';
import { DataGridPro } from '@mui/x-data-grid-pro';
import { useMockServer } from '@mui/x-data-grid-generator';

function ServerSideLazyLoadingInfinite() {
const { fetchRows, ...props } = useMockServer(
{ rowLength: 100 },
{ useCursorPagination: false, minDelay: 200, maxDelay: 500 },
);

const dataSource = React.useMemo(
() => ({
getRows: async (params) => {
const urlParams = new URLSearchParams({
filterModel: JSON.stringify(params.filterModel),
sortModel: JSON.stringify(params.sortModel),
firstRowToRender: `${params.start}`,
lastRowToRender: `${params.end}`,
});
const getRowsResponse = await fetchRows(
`https://mui.com/x/api/data-grid?${urlParams.toString()}`,
);

return {
rows: getRowsResponse.rows,
};
},
}),
[fetchRows],
);

return (
<div style={{ width: '100%', height: 400 }}>
<DataGridPro
{...props}
unstable_dataSource={dataSource}
lazyLoading
paginationModel={{ page: 0, pageSize: 15 }}
/>
</div>
);
}

export default ServerSideLazyLoadingInfinite;
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
import * as React from 'react';
import {
DataGridPro,
GridDataSource,
GridGetRowsParams,
} from '@mui/x-data-grid-pro';
import { useMockServer } from '@mui/x-data-grid-generator';

function ServerSideLazyLoadingInfinite() {
const { fetchRows, ...props } = useMockServer(
{ rowLength: 100 },
{ useCursorPagination: false, minDelay: 200, maxDelay: 500 },
);

const dataSource: GridDataSource = React.useMemo(
() => ({
getRows: async (params: GridGetRowsParams) => {
const urlParams = new URLSearchParams({
filterModel: JSON.stringify(params.filterModel),
sortModel: JSON.stringify(params.sortModel),
firstRowToRender: `${params.start}`,
lastRowToRender: `${params.end}`,
});
const getRowsResponse = await fetchRows(
`https://mui.com/x/api/data-grid?${urlParams.toString()}`,
);

return {
rows: getRowsResponse.rows,
};
},
}),
[fetchRows],
);

return (
<div style={{ width: '100%', height: 400 }}>
<DataGridPro
{...props}
unstable_dataSource={dataSource}
lazyLoading
paginationModel={{ page: 0, pageSize: 15 }}
/>
</div>
);
}

export default ServerSideLazyLoadingInfinite;
Loading
Loading