Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docusaurus preparations #2703

Merged
merged 14 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Contributing Guidelines

Thank you for considering contributing to this project! To ensure a smooth and efficient process, please follow these guidelines.

## Adding a New Example

1. **Create a Directory**: Create a new directory for your example in the root of the repository. Please do not use a "fiber" prefix in the directory name.

2. **Add a `README.md`**: Each example must include a `README.md` file in its directory. This file should contain the following:

- **Docusaurus Metadata**: Add the following metadata at the top of the `README.md` file:
```markdown
---
title: Your Example Title
keywords: [keyword1, keyword2, keyword3]
---
```

- `title`: A short and descriptive title for your example.
- `keywords`: A list of relevant keywords (excluding "fiber").

- **Content**: The `README.md` should provide a detailed explanation of the example, including:
- The idea behind the example.
- The components used in the example.
- Instructions on how to run the example.
- Any other relevant information.

3. **Update the Overview**: After adding your example, run the following command in the root directory to update the overview table of contents:
```bash
make generate
```
ReneWerner87 marked this conversation as resolved.
Show resolved Hide resolved

By following these guidelines, you help maintain the quality and consistency of the project. Thank you for your contributions!
55 changes: 55 additions & 0 deletions .github/scripts/sync_docs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#!/usr/bin/env bash
set -e

# Some env variables
BRANCH="main"
REPO_URL="github.com/gofiber/docs.git"
AUTHOR_EMAIL="github-actions[bot]@users.noreply.github.com"
AUTHOR_USERNAME="github-actions[bot]"
REPO_DIR="recipes"
COMMIT_URL="https://github.com/gofiber/recipes"

# Set commit author
git config --global user.email "${AUTHOR_EMAIL}"
git config --global user.name "${AUTHOR_USERNAME}"

ReneWerner87 marked this conversation as resolved.
Show resolved Hide resolved
git clone https://${TOKEN}@${REPO_URL} fiber-docs

latest_commit=$(git rev-parse --short HEAD)
ReneWerner87 marked this conversation as resolved.
Show resolved Hide resolved

# remove all files in the docs directory
rm -rf $ROOT/../fiberDocs/docs/${REPO_DIR}/*

for f in $(find -E . -type f -iregex '.*\.(md|png|jpe?g|gif|bmp|svg|webp)$' -not -path "./(fiberDocs)/*" -not -path "*/vendor/*" -not -path "*/.github/*" -not -path "*/.*"); do
log_output=$(git log --oneline "${BRANCH}" HEAD~1..HEAD --name-status -- "${f}")

if [[ $log_output != "" || ! -f "fiber-docs/docs/${REPO_DIR}/$f" ]]; then
mkdir -p fiber-docs/docs/${REPO_DIR}/$(dirname $f)
cp "${f}" fiber-docs/docs/${REPO_DIR}/$f
fi
done
Comment on lines +20 to +30
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix security and reliability issues in file synchronization

Several critical issues need to be addressed:

  1. Unsafe file handling in find loop
  2. Undefined ROOT variable
  3. Potential path traversal vulnerability
  4. Missing error handling for file operations

Apply these changes:

+# Validate ROOT variable
+if [ -z "${ROOT:-}" ]; then
+    ROOT="."
+fi
+
 # remove all files in the docs directory
-rm -rf $ROOT/../fiberDocs/docs/${REPO_DIR}/*
+rm -rf "${ROOT}/../fiberDocs/docs/${REPO_DIR:?}/"*
 
-for f in $(find -E . -type f -iregex '.*\.(md|png|jpe?g|gif|bmp|svg|webp)$' -not -path "./(fiberDocs)/*" -not -path "*/vendor/*" -not -path "*/.github/*" -not -path "*/.*"); do
-  log_output=$(git log --oneline "${BRANCH}" HEAD~1..HEAD --name-status -- "${f}")
-
-    if [[ $log_output != "" || ! -f "fiber-docs/docs/${REPO_DIR}/$f" ]]; then
-      mkdir -p fiber-docs/docs/${REPO_DIR}/$(dirname $f)
-      cp "${f}" fiber-docs/docs/${REPO_DIR}/$f
-  fi
-done
+# Use find -exec for safer file handling
+find -E . -type f \
+    -iregex '.*\.(md|png|jpe?g|gif|bmp|svg|webp)$' \
+    -not -path "./(fiberDocs)/*" \
+    -not -path "*/vendor/*" \
+    -not -path "*/.github/*" \
+    -not -path "*/.*" \
+    -exec bash -c '
+        f="$1"
+        log_output=$(git log --oneline "${BRANCH}" HEAD~1..HEAD --name-status -- "${f}")
+        target_dir="fiber-docs/docs/${REPO_DIR}/$(dirname "${f}")"
+        target_file="${target_dir}/$(basename "${f}")"
+        
+        if [[ $log_output != "" || ! -f "${target_file}" ]]; then
+            mkdir -p "${target_dir}"
+            if ! cp "${f}" "${target_file}"; then
+                echo "ERROR: Failed to copy ${f}"
+                exit 1
+            fi
+        fi
+    ' bash {} \;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# remove all files in the docs directory
rm -rf $ROOT/../fiberDocs/docs/${REPO_DIR}/*
for f in $(find -E . -type f -iregex '.*\.(md|png|jpe?g|gif|bmp|svg|webp)$' -not -path "./(fiberDocs)/*" -not -path "*/vendor/*" -not -path "*/.github/*" -not -path "*/.*"); do
log_output=$(git log --oneline "${BRANCH}" HEAD~1..HEAD --name-status -- "${f}")
if [[ $log_output != "" || ! -f "fiber-docs/docs/${REPO_DIR}/$f" ]]; then
mkdir -p fiber-docs/docs/${REPO_DIR}/$(dirname $f)
cp "${f}" fiber-docs/docs/${REPO_DIR}/$f
fi
done
# Validate ROOT variable
if [ -z "${ROOT:-}" ]; then
ROOT="."
fi
# remove all files in the docs directory
rm -rf "${ROOT}/../fiberDocs/docs/${REPO_DIR:?}/"*
# Use find -exec for safer file handling
find -E . -type f \
-iregex '.*\.(md|png|jpe?g|gif|bmp|svg|webp)$' \
-not -path "./(fiberDocs)/*" \
-not -path "*/vendor/*" \
-not -path "*/.github/*" \
-not -path "*/.*" \
-exec bash -c '
f="$1"
log_output=$(git log --oneline "${BRANCH}" HEAD~1..HEAD --name-status -- "${f}")
target_dir="fiber-docs/docs/${REPO_DIR}/$(dirname "${f}")"
target_file="${target_dir}/$(basename "${f}")"
if [[ $log_output != "" || ! -f "${target_file}" ]]; then
mkdir -p "${target_dir}"
if ! cp "${f}" "${target_file}"; then
echo "ERROR: Failed to copy ${f}"
exit 1
fi
fi
' bash {} \;
🧰 Tools
🪛 Shellcheck (0.10.0)

[warning] 23-23: For loops over find output are fragile. Use find -exec or a while read loop.

(SC2044)


[warning] 27-27: Quote this to prevent word splitting.

(SC2046)



# Push changes
cd fiber-docs/ || true
git add .

git commit -m "Add docs from ${COMMIT_URL}/commit/${latest_commit}"

MAX_RETRIES=5
DELAY=5
retry=0

while ((retry < MAX_RETRIES))
do
git push https://${TOKEN}@${REPO_URL} && break
retry=$((retry + 1))
git pull --rebase
sleep $DELAY
done

if ((retry == MAX_RETRIES))
then
echo "Failed to push after $MAX_RETRIES attempts. Exiting with 1."
exit 1
fi
ReneWerner87 marked this conversation as resolved.
Show resolved Hide resolved
25 changes: 25 additions & 0 deletions .github/scripts/sync_local.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
set -e

if [ "$#" -eq 1 ]; then
REPO_DIR="$1"
else
REPO_DIR="recipes" # default value
fi

if [[ ! "$REPO_DIR" =~ ^[a-zA-Z0-9_-]+$ ]]; then
echo "Error: REPO_DIR must contain only alphanumeric characters, underscores, and hyphens" >&2
exit 1
fi

# determine root repo directory
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd -P)"

# remove all files in the docs directory
rm -rf $ROOT/../fiberDocs/docs/${REPO_DIR}/*
ReneWerner87 marked this conversation as resolved.
Show resolved Hide resolved

for f in $(find -E . -type f -iregex '.*\.(md|png|jpe?g|gif|bmp|svg|webp)$' -not -path "./(fiberDocs)/*" -not -path "*/vendor/*" -not -path "*/.github/*" -not -path "*/.*"); do
echo "Copying $f"
mkdir -p $ROOT/../fiberDocs/docs/${REPO_DIR}/$(dirname $f)
cp "${f}" $ROOT/../fiberDocs/docs/${REPO_DIR}/$f
done
Comment on lines +21 to +25
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Improve file handling safety and portability

The current implementation has several issues with special characters in filenames and portability. Additionally, Shellcheck warns about fragile find usage.

-for f in $(find -E . -type f -iregex '.*\.(md|png|jpe?g|gif|bmp|svg|webp)$' -not -path "./(fiberDocs)/*" -not -path "*/vendor/*" -not -path "*/.github/*" -not -path "*/.*"); do
-  echo "Copying $f"
-    mkdir -p $ROOT/../fiberDocs/docs/${REPO_DIR}/$(dirname $f)
-    cp "${f}" $ROOT/../fiberDocs/docs/${REPO_DIR}/$f
-done
+# Use portable find syntax and handle special characters properly
+find . -type f \( \
+    -iname "*.md" -o \
+    -iname "*.png" -o \
+    -iname "*.jpg" -o \
+    -iname "*.jpeg" -o \
+    -iname "*.gif" -o \
+    -iname "*.bmp" -o \
+    -iname "*.svg" -o \
+    -iname "*.webp" \
+  \) \
+  -not -path "./fiberDocs/*" \
+  -not -path "*/vendor/*" \
+  -not -path "*/.github/*" \
+  -not -path "*/.*" \
+  -print0 | while IFS= read -r -d '' file; do
+    echo "Copying: ${file}"
+    target_dir="${ROOT}/../fiberDocs/docs/${REPO_DIR}/$(dirname "${file}")"
+    if ! mkdir -p "${target_dir}"; then
+        echo "Error: Failed to create directory: ${target_dir}" >&2
+        exit 1
+    fi
+    if ! cp "${file}" "${ROOT}/../fiberDocs/docs/${REPO_DIR}/${file}"; then
+        echo "Error: Failed to copy: ${file}" >&2
+        exit 1
+    fi
+done

This change:

  1. Uses portable find syntax instead of -E
  2. Properly handles filenames with spaces and special characters
  3. Adds error handling for mkdir and cp operations
  4. Properly quotes all variables
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for f in $(find -E . -type f -iregex '.*\.(md|png|jpe?g|gif|bmp|svg|webp)$' -not -path "./(fiberDocs)/*" -not -path "*/vendor/*" -not -path "*/.github/*" -not -path "*/.*"); do
echo "Copying $f"
mkdir -p $ROOT/../fiberDocs/docs/${REPO_DIR}/$(dirname $f)
cp "${f}" $ROOT/../fiberDocs/docs/${REPO_DIR}/$f
done
# Use portable find syntax and handle special characters properly
find . -type f \( \
-iname "*.md" -o \
-iname "*.png" -o \
-iname "*.jpg" -o \
-iname "*.jpeg" -o \
-iname "*.gif" -o \
-iname "*.bmp" -o \
-iname "*.svg" -o \
-iname "*.webp" \
\) \
-not -path "./fiberDocs/*" \
-not -path "*/vendor/*" \
-not -path "*/.github/*" \
-not -path "*/.*" \
-print0 | while IFS= read -r -d '' file; do
echo "Copying: ${file}"
target_dir="${ROOT}/../fiberDocs/docs/${REPO_DIR}/$(dirname "${file}")"
if ! mkdir -p "${target_dir}"; then
echo "Error: Failed to create directory: ${target_dir}" >&2
exit 1
fi
if ! cp "${file}" "${ROOT}/../fiberDocs/docs/${REPO_DIR}/${file}"; then
echo "Error: Failed to copy: ${file}" >&2
exit 1
fi
done
🧰 Tools
🪛 Shellcheck (0.10.0)

[warning] 21-21: For loops over find output are fragile. Use find -exec or a while read loop.

(SC2044)


[warning] 23-23: Quote this to prevent word splitting.

(SC2046)

34 changes: 34 additions & 0 deletions .github/workflows/sync-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: 'Sync docs'

on:
push:
branches:
- master
- main
paths:
- '**/*.md'

jobs:
sync-docs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 2

- name: Setup Node.js environment
uses: actions/setup-node@v4
with:
node-version: '18'

- name: Install JQ
run: sudo apt-get install jq

- name: Sync docs
run: ./.github/scripts/sync_docs.sh
env:
EVENT: ${{ github.event_name }}
TAG_NAME: ${{ github.ref_name }}
TOKEN: ${{ secrets.DOC_SYNC_TOKEN }}
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Thumbs.db
*.out

# Dependency directories (remove the comment below to include it)
# vendor/
vendor/

# Go workspace file
go.work
Expand Down
Loading