Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review GHA for i18n consistency check #551

Merged
merged 13 commits into from
Aug 29, 2023
Merged
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 52 additions & 71 deletions .github/workflows/i18n-consistency-checker.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
name: i18n check consistency and notify on GitHub Issues
# -------------------------
# To add a new language, do the following:
# - add the 2-letter language code in matrix.language
# - add the flag and language name that you want to use at the beginning of the #check-consistency step
# -------------------------
name: i18n Consistency Check

on:
pull_request:
branches:
- master
paths:
- .github/workflows/i18n-consistency-check.yml
schedule:
- cron: '0 0 1 * *'
- cron: '0 0 1 * *' # run at midnight, on day 1 of the month
workflow_dispatch:

jobs:
Expand All @@ -17,88 +17,69 @@ jobs:
matrix:
language: [ja, zh]
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-depth: '0'
- name: Check consistency
- name: Check consistency and create issue
id: check-consistency
run: |
# Set the issue header
issue="\
# i18n Contents Consistency Issue\\n\
\\n\
The following files may have consistency issues with the English version. Please check and update the files.\\n\
\\n\
This issue is created when there is an update to content/en. It compares the git update history to let you know what updates are overdue. The issue should be closed when the update is complete.\\n"

for file in $(find patterns/2-structured -name '*.md'); do

# Get the translated file name and check if it exists
i18n_filename=$(echo "$file" | sed 's/patterns\/2-structured/translation\/${{matrix.language}}\/patterns/g')
# Declare the flags
declare -A flags=( ["ja"]=":jp: Japanese" ["zh"]=":cn: Chinese")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where to lookup the icons to use here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yuhattor this is the only question remaining that I think we should add to the documentation.

If you could share where to look up the strings like :jp:, then I can add that to the documentation.
I might do when working on #582.


issue_title="${flags['${{matrix.language}}']}: Content Consistency Issue"

# Heredoc for issue header
cat <<- EOM > issue.md
# i18n Contents Consistency Issue

The following files may have consistency issues with the English version. Please check and update the files.

This issue is created when any of the English patterns have changed (in folder `patterns/`). It compares the git update history to let you know what updates are overdue. The issue should be closed when the update is complete.
EOM

# Loop through all files in the English directory
for file in $(find patterns/{2-structured,3-validated} -name '*.md'); do
[[ $file =~ "3-validated" ]] && continue # if the file is under 3-validated, skip (one liner) - 2023/08/26
i18n_filename=$(echo "$file" | sed "s/patterns\/\(2-structured\|3-validated\)/translation\/${{matrix.language}}\/patterns/g")

if [[ ! -e "$i18n_filename" ]]; then
continue
continue
fi
echo $file
echo $i18n_filename
# Loop through all the files in the English directory
# Get the last updated date of the original file and the translated file
original_updated_at=$(date -d "$(git log -1 --format=%cd --date=iso $file)" +%s)

original_updated_at=$(date -d "$(git log -1 --format=%cd --date=iso $file)" +%s)
i18n_content_updated_at=$(date -d "$(git log -1 --format=%cd --date=iso $i18n_filename)" +%s)

# print the last updated date of the original file and the translated file

# Check if the translated file is updated after the original file
if [[ $(($original_updated_at - $i18n_content_updated_at)) -ge 1 ]]; then
# Get the title of the content
content_header=$(echo "$(cat "$file")" | grep -E '^title+' | sort -r | head -n1)

if [[ $((original_updated_at - i18n_content_updated_at)) -ge 1 ]]; then
content_header=$(grep -E '^title+' "$file" | sort -r | head -n1)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't look like the title extract works right now

However let's leave this in here, as we may be adding a frontmatter to all patterns anyways, at which point this should work :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No wonder this thing doesn't work.
I was getting the title from the "title: foobar" meta info to match hugo's md style... But this time it's gitbook, so I need to extract from ## Title.
I didn't realize this when I ported...

I've changed it to use awk to extract it, so it should work now!

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very pretty!

content_title=$(echo "$content_header" | sed 's/title: //g')

# Get the days since the translated file is overdue
days_since_overdue_updates=$(($(( $(date +%s) - $original_updated_at))/ 60 / 60 / 24))
# Get the diff between the original file and the translated file
days_since_overdue_updates=$(( ($(date +%s) - original_updated_at) / 60 / 60 / 24 ))
original_last_update_hash=$(git log -1 --format=%H $file)
# Get the parent hash of the original last update hash
parent_hash=$(git log -1 --format=%P $original_last_update_hash)
# Get the diff between the original file and the translated file
result=$(git diff "${parent_hash}" HEAD "$file" | sed '1,4 s/^/# /')

# Append to the issue.md file
cat <<- EOM >> issue.md
<details><summary><b>$content_title</b> ($file)</summary>

result=$(echo "$(git diff ${parent_hash} HEAD $file)" | sed '1,4 s/^/# /')
echo -e "$result"

# Add the contents to the issue.md file
issue+="<details><summary><b>$content_title</b> ($file)</summary>\\n\\n"
issue+="- Original File(en): [$file](https://github.com/$GITHUB_REPOSITORY/blob/master/$file)\\n"
issue+="- Translated File(${{matrix.language}}): [$i18n_filename](https://github.com/$GITHUB_REPOSITORY/blob/master/$i18n_filename)\\n"
issue+="- [Diff on GitHub](https://github.com/yuhattor/innersourcecommons.org/compare/$i18n_last_update_hash...$original_last_update_hash)\\n"
issue+="- Days since overdue updates: $days_since_overdue_updates days\\n"
issue+="\`\`\`diff\\n"
issue+="$result"
issue+="\\n\`\`\`\\n"
issue+="</details>\\n"
echo -e "$issue" >> issue.md
issue=""
For more information, please compare [the original file(en)](https://github.com/$GITHUB_REPOSITORY/blob/master/$file) with [the translated file](https://github.com/$GITHUB_REPOSITORY/blob/master/$i18n_filename). You can view [the differences](https://github.com/$GITHUB_REPOSITORY/compare/$i18n_last_update_hash...$original_last_update_hash) on GitHub. The number of days since overdue updates is **$days_since_overdue_updates** days.

\`\`\`diff
$result
\`\`\`
</details>
EOM
fi
done
- name: Create Issue
run: |
# Declare the flags
declare -A flags=(
["ja"]="🇯🇵 Japanese"
["zh"]="🇨🇳 Chinese"
)

# Set the issue title
issue_title="${flags[${{matrix.language}}]}: Content Consistency Issue"


# Get the existing issue ID
existing_issue_id=$(gh issue list -S "state:open type:issue title:$issue_title" | cut -f1)

# If the issue.md file exists, create a new issue or comment on the existing issue
existing_issue_id=$(gh issue list -S "is:issue is:open $issue_title" | cut -f1)
echo "existing_issue_id: $existing_issue_id"
# Create a new issue or comment on the existing one
if [ -f issue.md ]; then
if expr "$existing_issue_id" : '^[0-9]*$' >/dev/null; then
gh issue comment "$existing_issue_id" -F issue.md -R $GITHUB_REPOSITORY
else
gh issue create -t "$issue_title" -F issue.md -R $GITHUB_REPOSITORY -l "Type - Translation"
fi
fi
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}