build_docs does not work for Windows

[justincr-elastic]

Requirements for local builds of docs are listed as Python 3 and Docker.

• https://github.com/elastic/docs/blob/master/README.asciidoc

There is no mentioned of OS requirements. Since Elasticsearch dev is supported on Linux, Mac, and Windows, my assumption is docs dev is supported on those same platforms.

However, when I try running build_docs in Git Bash for Windows, but I ran into issues. I can get build_docs to run, but it appears to have code which depends on os.* imports that are missing from Python 3 for Windows (https://www.python.org/downloads/).

Steps to reproduce:

1. Install Git Bash for Windows.
2. Install Python 3; check the option to add to Windows PATH.
3. Run Git Bash shell, and verify python3 --version works.

$ python3 --version
Python 3.10.3
$ python --version
Python 3.10.3

4. Clone two repos: git clone [email protected]:elastic/docs.git and git clone [email protected]:elastic/elasticsearch.git
5. Execute build_docs for an Elasticsearch index.asciidoc: /c/GitHub/docs/build_docs --doc index.asciidoc --open

Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

6. Open Windows settings, search for "Managed app execution aliases", disable Python and Python3, and re-run build_docs.

/usr/bin/env: 'python3': No such file or directory

7. Prepend python3 to the command: python3 /c/GitHub/docs/build_docs --doc index.asciidoc --open

Traceback (most recent call last):
  File "C:\GitHub\docs\build_docs", line 20, in <module>
    from os import environ, getgid, getuid
ImportError: cannot import name 'getgid' from 'os' (C:\Users\JustinCranford\AppData\Local\Programs\Python\Python310\lib\os.py)

8. Edit build_docs to import all from os.

import os
#from os import environ, getgid, getuid
#from os.path import basename, dirname, exists, expanduser, isdir
#from os.path import join, normpath, realpath

9. Execute build_docs again: python3 /c/GitHub/docs/build_docs --doc index.asciidoc --open

Traceback (most recent call last):
  File "C:\GitHub\docs\build_docs", line 35, in <module>
    DIR = dirname(realpath(__file__))
NameError: name 'dirname' is not defined

I can get build_docs to execute, but dependencies os.getgit, os.getuid, and os.dirname are not found.

---

Please update https://github.com/elastic/docs/blob/master/README.asciidoc with a workaround.

If it helps, some options might be:

1. If execution on Windows supported, fix cross-platform compatibility in build_docs, and add the python3 cmd and Managed app execution aliases workarounds to the README.

2. If execution on Windows is not supported, add a docker run command to bootstrap build_docs into a Linux container.

No code changes required, and Docker is already a dependency. This might be the easier option. Include cloning the repos, and mounting them into the container, plus a third mount for the output folder.

Thank you.

[elasticmachine]

Pinging @elastic/es-docs (Team:Docs)

[justincr-elastic]

Here are notes of what I tried to bootstrap build_docs from Windows into a Docker container. Prerequisite is enable tcp 2375 listener in Docker Desktop settings, plus access to C:\docs_build, C:\build_docs, and C:\elasticsearch.

The commands below start a container to run build_docs, and send requests to Docker host. It creates container images to run, but asking Docker host to start the images fails. Hard-coded Linux path /doc_builds/build_docs.pl is visible inside my container, but not in the Docker host because it is Windows.

I think I got close, but I could not get fully unblocked.

cd /c
git clone [email protected]:elastic/docs.git docs_build
git clone [email protected]:elastic/elasticsearch.git elasticsearch
winpty docker run --name build_docs --network=bridge -v "C:\elasticsearch":/elasticsearch -v "C:\docs_build":/docs_build -v "C:\build_docs":/build_docs --rm -it debian bash

# Install Docker & Python3 prerequisites
apt -y update && apt -y upgrade && apt install -y docker.io python3-pip

# Verify Python 3 and Docker installed ok, and Docker client has access to Docker host
export DOCKER_HOST=host.docker.internal
python3 --version && docker --version && docker ps -a

# Output directory
mkdir -p /build_docs/mybuild1
chmod 777 /build_docs/mybuild1

# build_docs refuses to run as root, so create doc1 user and run as that user
useradd doc1 --create-home --shell /bin/bash
su - doc1
export DOCKER_HOST=host.docker.internal
ls -l /docs_build/build_docs.pl

# Container sees /docs_build/build_docs.pl, but Docker Host does not see it.
python3 /docs_build/build_docs --doc /elasticsearch/x-pack/docs/en/security/index.asciidoc --out /build_docs/mybuild1

# Empty
ls -l /build_docs/mybuild1

[debadair]

We are not implementing any further enhancements to the the current doc build system.

[gtback]

Thanks for the thorough description, @justincr-elastic .

As Deb mentioned, we aren't planning to make any significant changes to the current system (this one). That said, I'm not opposed to adding some helpful notes to the README. I believe @ollyhowell got everything set up on Windows, so I'm curious how his approach differs from this. Maybe just using WSL?

[ollyhowell]

Yeah, I used WSL to get this running. It needed that one tweak to the python shebang you made, Greg - #2274

I didn't get very far with trying to run it via Windows...

[justincr-elastic]

I switched to mounting the repos into a Debian VM. I opened a separate issue #2405 for an unrelated problem.