Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sort includes #7326

Merged
merged 15 commits into from
Nov 23, 2023
Merged

Sort includes #7326

merged 15 commits into from
Nov 23, 2023

Conversation

thanodnl
Copy link
Member

@thanodnl thanodnl commented Nov 6, 2023

This change adds a script to programatically group all includes in a specific order. The script was used as a one time invocation to group and sort all includes throught our formatted code. The grouping is as follows:

  • System includes (eg. #include<...>)
  • Postgres.h (eg. #include "postgres.h")
  • Toplevel imports from postgres, not contained in a directory (eg. #include "miscadmin.h")
  • General postgres includes (eg . #include "nodes/...")
  • Toplevel citus includes, not contained in a directory (eg. #include "citus_verion.h")
  • Columnar includes (eg. #include "columnar/...")
  • Distributed includes (eg. #include "distributed/...")

Because it is quite hard to understand the difference between toplevel citus includes and toplevel postgres includes it hardcodes the list of toplevel citus includes. In the same manner it assumes anything not prefixed with columnar/ or distributed/ as a postgres include.

The sorting/grouping is enforced by CI. Since we do so with our own script there are not changes required in our uncrustify configuration.

Copy link

codecov bot commented Nov 6, 2023

Codecov Report

Merging #7326 (8900f46) into main (3b556cb) will decrease coverage by 0.53%.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7326      +/-   ##
==========================================
- Coverage   89.57%   89.05%   -0.53%     
==========================================
  Files         278      278              
  Lines       59973    59974       +1     
  Branches     7469     7469              
==========================================
- Hits        53723    53409     -314     
- Misses       4101     4346     +245     
- Partials     2149     2219      +70     

ci/include-grouping.py Outdated Show resolved Hide resolved
ci/include-grouping.py Outdated Show resolved Hide resolved
ci/include-grouping.py Outdated Show resolved Hide resolved
ci/include-grouping.py Outdated Show resolved Hide resolved
JelteF
JelteF previously requested changes Nov 8, 2023
Copy link
Contributor

@JelteF JelteF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really good overall. I left some small suggestions.

ci/include-grouping.py Outdated Show resolved Hide resolved
ci/include-grouping.py Outdated Show resolved Hide resolved
ci/include-grouping.py Outdated Show resolved Hide resolved
@thanodnl
Copy link
Member Author

thanodnl commented Nov 9, 2023

Looks really good overall. I left some small suggestions.

Initially I was thinking of not enforcing it on CI, especially since it was looking like a need to introduce a new tool.
Given it is a python script without any dependencies, do you (@JelteF ) think it is reasonable to add this to our style checker?

I'd like to keep the includes in a reasonable shape after we merge this. And as it turns out, it might actually find issues where includes are not always self contained, like the one causing the identification of PG16 RC1 being used by parts of our CI system.

@thanodnl thanodnl force-pushed the sort-includes branch 2 times, most recently from d7a158a to 6f54735 Compare November 9, 2023 15:14
@JelteF
Copy link
Contributor

JelteF commented Nov 9, 2023

Initially I was thinking of not enforcing it on CI, especially since it was looking like a need to introduce a new tool.
Given it is a python script without any dependencies, do you (@JelteF ) think it is reasonable to add this to our style checker?

Yeah sounds, good let's add it to CI and probably also ci/fix_style.sh (which is executed by make reindent).

@thanodnl thanodnl marked this pull request as ready for review November 13, 2023 15:41
ci/include_grouping.py Outdated Show resolved Hide resolved
ci/include_grouping.py Outdated Show resolved Hide resolved
ci/include_grouping.py Outdated Show resolved Hide resolved
ci/include_grouping.py Outdated Show resolved Hide resolved
ci/include_grouping.py Outdated Show resolved Hide resolved
@thanodnl thanodnl force-pushed the sort-includes branch 2 times, most recently from 0d32a3d to b158d22 Compare November 16, 2023 14:23
@thanodnl thanodnl force-pushed the sort-includes branch 2 times, most recently from b055f51 to 88bb019 Compare November 23, 2023 16:59
@thanodnl thanodnl enabled auto-merge (squash) November 23, 2023 17:06
@thanodnl thanodnl dismissed JelteF’s stale review November 23, 2023 17:19

Addressed feedback. He is out of office and not able to re-review. Onur has reviewed instead

@thanodnl thanodnl merged commit 0620c8f into main Nov 23, 2023
125 of 126 checks passed
@thanodnl thanodnl deleted the sort-includes branch November 23, 2023 17:19
JelteF pushed a commit that referenced this pull request Apr 16, 2024
CHERRY-PICK NOTES: This cherry-pick only includes the scripts, not the
actual changes. These are done in a follow up commit to ease further
backporting.

This change adds a script to programatically group all includes in a
specific order. The script was used as a one time invocation to group
and sort all includes throught our formatted code. The grouping is as
follows:

 - System includes (eg. `#include<...>`)
 - Postgres.h (eg. `#include "postgres.h"`)
- Toplevel imports from postgres, not contained in a directory (eg.
`#include "miscadmin.h"`)
 - General postgres includes (eg . `#include "nodes/..."`)
- Toplevel citus includes, not contained in a directory (eg. `#include
"citus_verion.h"`)
 - Columnar includes (eg. `#include "columnar/..."`)
 - Distributed includes (eg. `#include "distributed/..."`)

Because it is quite hard to understand the difference between toplevel
citus includes and toplevel postgres includes it hardcodes the list of
toplevel citus includes. In the same manner it assumes anything not
prefixed with `columnar/` or `distributed/` as a postgres include.

The sorting/grouping is enforced by CI. Since we do so with our own
script there are not changes required in our uncrustify configuration.

(cherry picked from commit 0620c8f)
JelteF pushed a commit that referenced this pull request Apr 16, 2024
CHERRY-PICK NOTES: This cherry-pick only includes the scripts, not the
actual changes. These are done in a follow up commit to ease further
backporting.

This change adds a script to programatically group all includes in a
specific order. The script was used as a one time invocation to group
and sort all includes throught our formatted code. The grouping is as
follows:

 - System includes (eg. `#include<...>`)
 - Postgres.h (eg. `#include "postgres.h"`)
- Toplevel imports from postgres, not contained in a directory (eg.
`#include "miscadmin.h"`)
 - General postgres includes (eg . `#include "nodes/..."`)
- Toplevel citus includes, not contained in a directory (eg. `#include
"citus_verion.h"`)
 - Columnar includes (eg. `#include "columnar/..."`)
 - Distributed includes (eg. `#include "distributed/..."`)

Because it is quite hard to understand the difference between toplevel
citus includes and toplevel postgres includes it hardcodes the list of
toplevel citus includes. In the same manner it assumes anything not
prefixed with `columnar/` or `distributed/` as a postgres include.

The sorting/grouping is enforced by CI. Since we do so with our own
script there are not changes required in our uncrustify configuration.

(cherry picked from commit 0620c8f)
JelteF pushed a commit that referenced this pull request Apr 17, 2024
CHERRY-PICK NOTES: This cherry-pick only includes the scripts, not the
actual changes. These are done in a follow up commit to ease further
backporting.

This change adds a script to programatically group all includes in a
specific order. The script was used as a one time invocation to group
and sort all includes throught our formatted code. The grouping is as
follows:

 - System includes (eg. `#include<...>`)
 - Postgres.h (eg. `#include "postgres.h"`)
- Toplevel imports from postgres, not contained in a directory (eg.
`#include "miscadmin.h"`)
 - General postgres includes (eg . `#include "nodes/..."`)
- Toplevel citus includes, not contained in a directory (eg. `#include
"citus_verion.h"`)
 - Columnar includes (eg. `#include "columnar/..."`)
 - Distributed includes (eg. `#include "distributed/..."`)

Because it is quite hard to understand the difference between toplevel
citus includes and toplevel postgres includes it hardcodes the list of
toplevel citus includes. In the same manner it assumes anything not
prefixed with `columnar/` or `distributed/` as a postgres include.

The sorting/grouping is enforced by CI. Since we do so with our own
script there are not changes required in our uncrustify configuration.

(cherry picked from commit 0620c8f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants