Implement Crowdin Translation Automation

[ManindraDeMel]

Introducing Automated README Translation Workflow 🌍 #180

Language issues relevant #122 #176 #43 #129 #70 #194

Based on insights gathered from this issue, we have now integrated our translation process with Crowdin, a leading cloud-based localization management platform. This new integration automates the translation process, streamlining our efforts and ensuring that our global community always has access to the latest translated content.

You can find the open source translation project for ChatDev here. You can also view the quality of the machine translations at that same link

🚀 Key Benefits:

• Automated Translation Management: With Crowdin, any update to our README.md (or other specified files) gets instantly uploaded and ready for translation.

• Efficiency for Maintainers: This process eliminates the need for maintainers to handle translation PRs directly. This way, we can focus on core development, yet still cater to our international users.

• Machine Translation with Crowdin: While community translations remain our primary focus, Crowdin's Machine Translation (MT) feature offers a quick and sometimes helpful alternative. However, we understand MT may not always grasp the subtle nuances, so community involvement remains essential.

• FREE

📚 Crowdin Overview:

Crowdin not only automates our translation workflow but also provides a feature-rich interface for our translators:

1. Source Upload: Any update to our main documents, like README.md, gets uploaded to Crowdin instantly.
2. Community Participation: Our translators can then work on Crowdin, translating the content.
3. Automated PR Creation: Once translations are complete, Crowdin auto-creates a PR with the translated content. Manual translation PRs are now a thing of the past!

The integration with the Crowdin GitHub Action ensures that our repository stays updated with the freshest translations.

⚙️ Workflow Breakdown:

Below is a detailed breakdown of the GitHub Action workflow that powers this automation:

name: Translate Readme

on: # This workflow triggers on pushes and PRs to the main branch, specifically affecting the README.md
  push:
    branches: [ main ]
    paths:
      - 'README.md'
  pull_request:
    branches: [ main ]
    paths:
      - 'README.md'
  workflow_dispatch: # Also provides a manual trigger option

jobs:
  Translate:
    runs-on: ubuntu-latest
    permissions:
      contents: write
      pull-requests: write

    steps:
      - name: Checkout # Fetches the current repo's content
        uses: actions/checkout@v3 

      - name: Upload to Crowdin # Uploads any changes in specified files to Crowdin
        uses: crowdin/github-action@v1
        with:
          command: 'upload'
          upload_sources: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
          CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}

      - name: Apply Machine Translation # Applies machine translation to the new content in various languages
        uses: crowdin/github-action@v1
        with:
          command: 'pre-translate'
          command_args: '-l zh-CN -l nl -l fr -l ja -l ar -l pt-PT -l ru -l sv-SE -l de -l es-ES --method mt --engine-id 409836'
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
          CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}

      - name: Download from Crowdin # Downloads the translated files from Crowdin
        uses: crowdin/github-action@v1
        with:
          download_translations: true
          localization_branch_name: l10n_crowdin_translations
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
          CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}

      - name: Set Git Identity  # This sets the identity for the git user to clearly indicate that changes are made by the GitHub Action.
        run: |
          git config --local user.email "[email protected]"
          git config --local user.name "GitHub Action"

      - name: Adjust .git permissions # Adjusting permissions ensures that the GitHub Action has the required permissions to perform subsequent operations.
        run: sudo chmod -R 777 .git

      - name: Switch to new branch # It switches to a dedicated branch for Crowdin translations. This keeps translations separate until they're ready to be merged.
        run: |
          git checkout l10n_crowdin_translations

      - name: Update paths for translated READMEs # This step updates the paths in the translated READMEs, ensuring links and image sources point to the correct locations.
        run: |
          find readme -name "README-*.md" -exec sed -i 's|href="\./|href="../|g' {} \;
          find readme -name "README-*.md" -exec sed -i 's|src="\./|src="../|g' {} \;
          find readme -name "README-*.md" -exec sed -i "s|href='\./|href='../|g" {} \;
          find readme -name "README-*.md" -exec sed -i "s|src='\./|src='../|g" {} \;
          
      - name: Commit changes # After updating paths, it commits and pushes the changes to the branch designated for translations.
        run: |
          git add -A
          git commit -m "Update paths in translated READMEs"
          git push origin l10n_crowdin_translations

      - name: Create PR with new translations # The final step in the process is to create a Pull Request with the new translations. Once reviewed and approved, the translations can be merged into the main branch.
        uses: crowdin/github-action@v1
        with:
          create_pull_request: true
          pull_request_title: 'New Crowdin Translations'
          pull_request_body: 'New Crowdin translations by [Crowdin GH Action](https://github.com/crowdin/github-action)'
          pull_request_base_branch_name: 'main'
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
          CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}

Examples from workflow running

[ImBIOS]

Wow, it's great! I'm initialized 3 translation of my languages and I think it's good to go:

• docs: init turkish translation #243
• docs: init arabic translation #242
• docs: init bahasa indonesia translation #241

[ManindraDeMel]

Could i get a review on this, since it was used for translations #243 and #242 which have been merged..

[andrii-bodnar]

Hi, I just stumbled across this PR by accident 🙂

Well done, I'm glad to know that my PR in the github-readme-stats helped you with the integration!

By the way, we have a separate action for readme translation - https://github.com/crowdin/translate-readme. Have you considered it? If so, I'd love to hear your thoughts on it.

Thanks in advance!