diff --git a/404.html b/404.html index 39b5ed1..dc20677 100644 --- a/404.html +++ b/404.html @@ -454,7 +454,7 @@ - Using jinja2 + Use jinja2 for automation diff --git a/assets/tables/markdown_table/index.html b/assets/tables/markdown_table/index.html index 857ba61..1ba8a3c 100644 --- a/assets/tables/markdown_table/index.html +++ b/assets/tables/markdown_table/index.html @@ -454,7 +454,7 @@ - Using jinja2 + Use jinja2 for automation diff --git a/howto/alternatives/index.html b/howto/alternatives/index.html index 972cc48..f644c30 100644 --- a/howto/alternatives/index.html +++ b/howto/alternatives/index.html @@ -468,7 +468,7 @@ - Using jinja2 + Use jinja2 for automation diff --git a/howto/customize_tables/index.html b/howto/customize_tables/index.html index 4d3d205..812177e 100644 --- a/howto/customize_tables/index.html +++ b/howto/customize_tables/index.html @@ -544,7 +544,7 @@ - Using jinja2 + Use jinja2 for automation diff --git a/howto/docker/index.html b/howto/docker/index.html index 3de8da7..d9e8607 100644 --- a/howto/docker/index.html +++ b/howto/docker/index.html @@ -468,7 +468,7 @@ - Using jinja2 + Use jinja2 for automation diff --git a/howto/preprocess_tables/index.html b/howto/preprocess_tables/index.html index e2e4de0..4edc02b 100644 --- a/howto/preprocess_tables/index.html +++ b/howto/preprocess_tables/index.html @@ -526,7 +526,7 @@ - Using jinja2 + Use jinja2 for automation @@ -735,6 +735,10 @@
Alternative: use jinja
+You can also use jinja2 to display a list of tables. See how to use jinja2 for automation.
+table-reader
supports mkdocs-macros-plugin
, which enables you to use jinja2 syntax inside markdown files (among other things).
To enable macros
, specify the plugin before table-reader
in your mkdocs.yml
file:
plugins:
- macros
- table-reader
Now you can do cool things like dynamically load a list of tables:
+Now you can do cool things like:
+# index.md
{% set table_names = ["basic_table.csv","basic_table2.csv"] %}
@@ -685,7 +722,7 @@ Using jinja2
{% endfor %}
If you inserted content has multiple lines, then indentation will be not be retained beyond the first line. This means things like content tabs will not work as expected.
To fix that, you can use the custom filter add_indendation
(a filter add to macros
by table-reader
plugin). For example:
To avoid the tables being inserted into the code example, we replaced {{
with { {
.
If you copy this example, make sure to fix.
mkdocs-macros-plugin
enables you to define additional functions (called macros) that you will be able to use within your markdown files.
+See their documentation on how to set this up. Here's an example with some functions to interact with the filesystem:
def define_env(env):
+ """
+ Register additional mkdocs-macros-plugin functions that can be used as macros in markdown files.
+ """
+ @env.macro
+ def listdir(path):
+ return os.listdir(path)
+
+ @env.macro
+ def path_exists(path):
+ return Path(path).exists()
+
+ @env.macro
+ def is_file(path):
+ return Path(path).is_file()
+
Now you could do something like:
+# index.md
+
+{% for table_name in listdir('docs/assets/my_tables") %}
+
+{ { read_csv(table_name) }}
+
+{% endfor %}
+
Note the space in { {
+To avoid the tables being inserted into the code example, we replaced {{
with { {
.
+If you copy this example, make sure to fix.
MkDocs plugin that enables a markdown tag like {{ read_csv('table.csv') }}
to directly insert various table formats into a page.
For a workflow with other plugins see the blogpost building reproducible reports with MkDocs
"},{"location":"#installation","title":"Installation","text":"Install the plugin using pip
:
pip install mkdocs-table-reader-plugin\n
Next, add the following lines to your mkdocs.yml
:
plugins:\n - search\n - table-reader\n
If you have no plugins
entry in your config file yet, you'll likely also want to add the search
plugin. MkDocs enables it by default if there is no plugins
entry set.
In your markdown files you can now use:
{{ read_csv('path_to_table.csv') }}\n
Where the path is relative to the location of your project's mkdocs.yml
file, or your project's docs/
directory, or the location of your markdown source file (all 3 possible locations will be searched, in that order).
.csv
, .fwf
, .json
, .xls
, .xlsx
, .yaml
, .feather
and .tsv
files. There is also the read_raw()
reader that will allow you to insert tables (or other content) already in markdown format.table-reader
is compatible with mkdocs-macros-plugin
, which means you can dynamically insert tables using jinja2 syntax.See timvink.github.io/mkdocs-table-reader-plugin/
"},{"location":"options/","title":"Options","text":"You can customize the plugin by setting options in mkdocs.yml
. For example:
plugins:\n - table-reader:\n data_path: \".\"\n allow_missing_files: False\n select_readers:\n - read_csv\n - read_json\n enabled: True\n
"},{"location":"options/#data_path","title":"data_path
","text":"Default is .
. Set a default path to the searched directories in order to shorten table filename specifications.
Given a file path, table-reader
will search for that file relative to your your project's mkdocs.yml
and relative to your docs/
folder. If you use a folder for all your table files you can shorten the path specification by setting the data_path
.
For example, if your table is located in docs/assets/tables/basic_table.csv
, you can set data_path
to docs/assets/tables/
. Then you will be able to use {{ read_csv(\"basic_table.csv\") }}
instead of {{ read_csv(\"docs/assets/tables/basic_table.csv\") }}
inside any markdown page.
Info
Note that by default the plugin will also search the page's directory but only when a table is not found.
For more examples see the how to guide on project structure.
"},{"location":"options/#allow_missing_files","title":"allow_missing_files
","text":"Default: False
. When enabled, if a filepath is not found, the plugin will raise a warning instead of an error.
select_readers
","text":"Default: Selects all available readers. Specify a list of readers to improve documentation build times for very large sites.
"},{"location":"options/#enabled","title":"enabled
","text":"Default is True
. Enables you to deactivate this plugin. This option is supported by all plugins since mkdocs 1.6 (see docs). A possible use case is local development where you might want faster build times and/or do not have the tables ready. It's recommended to use this option with an environment variable together with a default fallback (introduced in mkdocs v1.2.1, see docs). Example:
plugins:\n - table-reader:\n enabled: !ENV [ENABLED_TABLE_READER, True]\n
Which enables you to disable the plugin locally using:
export ENABLED_TABLE_READER=false\nmkdocs serve\n
"},{"location":"readers/","title":"Readers","text":"The following table reader functions are available:
"},{"location":"readers/#read_csv","title":"read_csv","text":"{{ read_csv() }}
passed to pandas.read_csv(). Example:
{{ read_csv('tables/basic_table.csv') }}
{{ read_fwf() }}
passed to pandas.read_fwf(). Example:
{{ read_fwf('tables/fixedwidth_table.txt') }}
{{ read_yaml() }}
is parsed with yaml.safe_load() and passed to pandas.json_normalize(). Example:
{{ read_yaml('tables/yaml_table.yml') }}
{{ read_table() }}
passed to pandas.read_table(). Example:
{{ read_table('tables/basic_table.csv', sep = ',') }}
{{ read_json() }}
passed to pandas.read_json(). Example:
{{ read_json('tables/data.json', orient='split') }}
{{ read_feather() }}
passed to pandas.read_feather(). Example:
{{ read_json('tables/data.feather') }}
{{ read_excel() }}
passed to pandas.read_excel(). Example:
{{ read_excel('tables/excel_table.xlsx', engine='openpyxl') }}
Reading xlsx files
You might get a XLRDError('Excel xlsx file; not supported',)
error when trying to read modern excel files. That's because xlrd
does not support .xlsx
files (stackoverflow post). Instead, install openpyxl and use:
{{ read_excel('tables/excel_table.xlsx', engine='openpyxl') }}
{{ read_raw() }}
inserts contents from a file directly. This is great if you have a file with a table already in markdown format. It could also replace a workflow where you use the snippets extension to embed external files.
Example:
InputOutput{{ read_raw('tables/markdown_table.md') }}
This plugin is built to be able to quickly insert table files anywhere in a markdown file.
You could also consider alternative approaches that might fit your use-case better.
"},{"location":"howto/alternatives/#write-tables-to-markdown-files","title":"Write tables to markdown files","text":"You could write a script (maybe triggered by a mkdocs hook) that writes the tables you need into markdown files. It could look something like this:
# write markdown tables\nimport pandas as pd\n\nmd = pd.read_csv(\"your_file.csv\").to_markdown()\nwith open(\"docs/assets/tables/my_file.md\", \"w\") as f:\n f.write(md)\n
You can then use the snippets extension to insert the tables into your markdown pages:
# some_page.md\n\nMy table:\n\n--8<-- \"assets/tables/my_file.md\"\n
Upsides:
Downsides:
You could also choose to insert the markdown for tables dynamically, using packages like markdown-exec or mkdocs-macros-plugin.
For example:
```python exec=\"true\"\nimport pandas as pd\n\nfile_path = \"path/to/file/from/project/root\"\nprint(pd.read_csv(file_path).to_markdown(index=False, disable_numparse=True))\n```\n
"},{"location":"howto/customize_tables/","title":"Customize markdown tables","text":"You can customize the resulting markdown tables!
"},{"location":"howto/customize_tables/#theory","title":"Theory","text":"Under the hood mkdocs-table-reader-plugin
is basically doing:
import pandas as pd\ndf = pd.read_csv('path_to_table.csv')\ndf.to_markdown(index=False, tablefmt='pipe')\n
Any keyword arguments you give to {{ read_csv('path_to_your_table.csv') }}
will be matched and passed the corresponding pandas.read_csv() and/or .to_markdown() functions.
Pandas's .to_markdown()
uses the tabulate package and any keyword arguments that are passed to it. Tabulate in turn offers many customization options, see library usage.
Text columns will be aligned to the left by default, whilst columns which contain only numbers will be aligned to the right. You can override this behaviour using tabulate's custom column alignment. Example:
left center right{{ read_csv('tables/basic_table.csv', colalign=(\"left\",)) }}
{{ read_csv('tables/basic_table.csv', colalign=(\"center\",)) }}
{{ read_csv('tables/basic_table.csv', colalign=(\"right\",)) }}
If you use mkdocs-material, you can configure sortable tables.
"},{"location":"howto/customize_tables/#number-formatting","title":"Number formatting","text":"You can use tabulate's number formatting. Example:
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=\".0f\") }}
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=\".1f\") }}
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=\".2f\") }}
If you follow the mkdocs-material
tutorial on installation with docker
, you will run into an error if you try to add this plugin saying:
\u00d7 Building wheel for numpy (pyproject.toml) did not run successfully.
The reason is that the mkdocs-material
docker image uses the alpine
image. mkdocs-table-reader-plugin
depends on pandas
, which depends on numpy
, which in turn requires C++ to install, which is not part of the alpine
image.
If you need a very small image, you can adapt the alpine image to support numpy, but you are better off using a more complete docker image like python-slim
. So you need to build a different docker image.
As an example, below is a Dockerfile
that is adapted from the mkdocs-material dockerfile. You can use it to create a new Docker image that supports mkdocs-table-reader-plugin
:
git clone https://github.com/squidfunk/mkdocs-material.git material-git/\ncd material-git\n# Manually replace `Dockerfile` with the example below\ndocker build -t YOUR-CONTAINER-NAME .\n# Now, inside your own project, you can use:\ndocker run --rm -it -p 8000:8000 -v ${PWD}:/docs YOUR-CONTAINER-NAME\n
Dockerfile FROM python:3-slim\n\n# Build-time flags\nARG WITH_PLUGINS=true\n\n# Environment variables\nENV PACKAGES=/usr/local/lib/python3.11/site-packages\nENV PYTHONDONTWRITEBYTECODE=1\n\n# Set build directory\nWORKDIR /tmp\n\n# Copy files necessary for build\nCOPY material-git/material material\nCOPY material-git/package.json package.json\nCOPY material-git/README.md README.md\nCOPY material-git/requirements.txt requirements.txt\nCOPY material-git/pyproject.toml pyproject.toml\n\n# Perform build and cleanup artifacts and caches\nRUN \\\napt update \\\n&& \\\napt install -y \\\n libcairo2-dev \\\n libfreetype6-dev \\\n git \\\n libturbojpeg-dev \\\n openssh-server \\\n zlib1g-dev \\\n&& \\\napt install -y \\\n gcc \\\n libffi-dev \\\n musl-dev \\\n&& \\\npip install --no-cache-dir . \\\n&& \\\nif [ \"${WITH_PLUGINS}\" = \"true\" ]; then \\\n pip install --no-cache-dir \\\n \"mkdocs-minify-plugin>=0.3\" \\\n \"mkdocs-redirects>=1.0\" \\\n \"pillow>=9.0\" \\\n \"cairosvg>=2.5\" \\\n \"mkdocs-table-reader-plugin\" \\\n ; \\\nfi \\\n&& \\\nfor theme in mkdocs readthedocs; do \\\n rm -rf ${PACKAGES}/mkdocs/themes/$theme; \\\n ln -s \\\n ${PACKAGES}/material \\\n ${PACKAGES}/mkdocs/themes/$theme; \\\ndone \\\n&& \\\nrm -rf /tmp/* /root/.cache \\\n&& \\\nfind ${PACKAGES} \\\n -type f \\\n -path \"*/__pycache__/*\" \\\n -exec rm -f {} \\;\n\n# Trust directory, required for git >= 2.35.2\nRUN git config --global --add safe.directory /docs &&\\\n git config --global --add safe.directory /site\n\n# Set working directory\nWORKDIR /docs\n\n# Expose MkDocs development server port\nEXPOSE 8000\n\n# Start development server by default\nENTRYPOINT [\"mkdocs\"]\nCMD [\"serve\", \"--dev-addr=0.0.0.0:8000\"]\n
\"mkdocs-table-rea"},{"location":"howto/preprocess_tables/","title":"Preprocess input tables","text":"mkdocs>=1.4
supports hooks, which enable you to run python scripts on mkdocs serve
or mkdocs build
.
Here are some example of workflows that use hooks and the table-reader
plugin:
from os import listdir\nfrom os.path import isfile, join\nimport pandas as pd\n\ndef on_pre_build(config, **kwargs) -> None:\n tables = []\n input_files = [f for f in listdir(mypath) if isfile(join(mypath, f))]\n for table in input_files:\n tables.append(pd.read_csv(table))\n df = pd.concat(tables, ignore_index=True)\n df.to_csv(\"docs/assets/output_table.csv\")\n
My table: {{ read_csv(\"docs/assets/output_table.csv\") }}
# ...\nplugins:\n - table-reader\nhooks:\n - scripts/hooks.py\n# ...\n
.\n\u251c\u2500\u2500 scripts/\n\u2502 \u2514\u2500\u2500\u2500 hooks.py\n\u251c\u2500\u2500 docs/\n\u2502 \u2514\u2500\u2500\u2500 index.md\n\u2502 \u2514\u2500\u2500 assets/tables/\n\u2502 \u2514\u2500\u2500 table1.csv\n\u2502 \u2514\u2500\u2500 table2.csv\n\u2514\u2500\u2500 mkdocs.yml\n
"},{"location":"howto/preprocess_tables/#download-a-table-from-an-api","title":"Download a table from an API","text":"hooks.pyindex.mdmkdocs.ymlProject structure import pandas as pd\n\ndef on_pre_build(config, **kwargs) -> None:\n df = pd.read_csv('https://data.cityofnewyork.us/resource/nu7n-tubp.csv?$limit=100')\n df.to_csv(\"docs/assets/nyc_data.csv\")\n
My table: {{ read_csv(\"docs/assets/nyc_data.csv\") }}
# ...\nplugins:\n - table-reader\nhooks:\n - scripts/hooks.py\n# ...\n
.\n\u251c\u2500\u2500 scripts/\n\u2502 \u2514\u2500\u2500\u2500 hooks.py\n\u251c\u2500\u2500 docs/\n\u2502 \u2514\u2500\u2500\u2500 index.md\n\u2502 \u2514\u2500\u2500 assets/\n\u2514\u2500\u2500 mkdocs.yml\n
Note that during development when you use mkdocs serve
and autoreload, you might not want to run this hook every time you make a change. You could use an environment variable inside your hook, for example something like if os.environ['disable_hook'] == 1: return None
.
You have different possible strategies to store and load your tables. This guide gives some examples.
"},{"location":"howto/project_structure/#one-table-per-markdown-file","title":"One table per markdown file","text":"If you only want to include an occasional table in a specific markdown file, just store it in the same directory as the markdown file:
project structurefolder/page.md.\n\u251c\u2500\u2500 docs/\n\u2502 \u251c\u2500\u2500 index.md\n\u2502 \u2514\u2500\u2500 folder/\n\u2502 \u2514\u2500\u2500 another_table.csv\n\u2502 \u2514\u2500\u2500 page.md\n\u2514\u2500\u2500 mkdocs.yml\n
Here is the table:\n\n\\{\\{ read_csv(\"another_table.csv\") \\}\\}\n
In page.md
, to read another_table.csv
, you can choose to use:
{{ read_csv(\"docs/folder/another_table.csv\") }}
(Path relative to mkdocs.yml){{ read_csv(\"folder/another_table.csv\") }}
(Path relative to docs/ directory){{ read_csv(\"another_table.csv\") }}
(Path relative to page source file)If you want to reuse tables in multiple markdown files, or have many tables, you'll want to store them in a central directory, like docs/assets/tables
. That way, if you restructure your navigation, the links to the tables won't break either. It's also great if you generate tables because the output directory will be the same.
Given the following project structure:
.\n\u251c\u2500\u2500 docs/\n\u2502 \u251c\u2500\u2500 index.md\n\u2502 \u2514\u2500\u2500 folder/\n\u2502 \u2514\u2500\u2500 page.md\n\u2502 \u2514\u2500\u2500 assets/\n\u2502 \u2514\u2500\u2500 tables/\n\u2502 \u2514\u2500\u2500 another_table.csv\n\u2514\u2500\u2500 mkdocs.yml\n
In page.md
, to read another_table.csv
, you can choose to use:
{{ read_csv(\"docs/assets/tables/another_table.csv\") }}
(Path relative to mkdocs.yml){{ read_csv(\"assets/tables/another_table.csv\") }}
(Path relative to docs/ directory){{ read_csv(\"../assets/tables/another_table.csv\") }}
(Path relative to page source file (note that ..
stands for \"one directory up\"))table-reader
supports mkdocs-macros-plugin
, which enables you to use jinja2 syntax inside markdown files (among other things).
To enable macros
, specify the plugin before table-reader
in your mkdocs.yml
file:
plugins:\n - macros\n - table-reader\n
Now you can do cool things like dynamically load a list of tables:
# index.md\n\n{% set table_names = [\"basic_table.csv\",\"basic_table2.csv\"] %}\n{% for table_name in table_names %}\n\n{ { read_csv(table_name) }}\n\n{% endfor %}\n
"},{"location":"howto/use_jinja2/#indented-content-like-content-tabs","title":"Indented content like content tabs","text":"If you inserted content has multiple lines, then indentation will be not be retained beyond the first line. This means things like content tabs will not work as expected.
To fix that, you can use the custom filter add_indendation
(a filter add to macros
by table-reader
plugin). For example:
{% set table_names = [\"basic_table.csv\",\"basic_table2.csv\"] %}\n{% for table_name in table_names %}\n\n=== \"{{ table_name }}\"\n\n { { read_csv(table_name) | add_indentation(spaces=4) }}\n\n{% endfor %}\n
site_name: test git_table_reader site\nuse_directory_urls: true\n\ntheme:\nname: material\n\nplugins:\n - search\n - macros\n - table-reader\n\nmarkdown_extensions:\n - pymdownx.superfences\n - pymdownx.tabbed:\n alternate_style: true\n
Note the space in { {
To avoid the tables being inserted into the code example, we replaced {{
with { {
. If you copy this example, make sure to fix.
MkDocs plugin that enables a markdown tag like {{ read_csv('table.csv') }}
to directly insert various table formats into a page.
For a workflow with other plugins see the blogpost building reproducible reports with MkDocs
"},{"location":"#installation","title":"Installation","text":"Install the plugin using pip
:
pip install mkdocs-table-reader-plugin\n
Next, add the following lines to your mkdocs.yml
:
plugins:\n - search\n - table-reader\n
If you have no plugins
entry in your config file yet, you'll likely also want to add the search
plugin. MkDocs enables it by default if there is no plugins
entry set.
In your markdown files you can now use:
{{ read_csv('path_to_table.csv') }}\n
Where the path is relative to the location of your project's mkdocs.yml
file, or your project's docs/
directory, or the location of your markdown source file (all 3 possible locations will be searched, in that order).
.csv
, .fwf
, .json
, .xls
, .xlsx
, .yaml
, .feather
and .tsv
files. There is also the read_raw()
reader that will allow you to insert tables (or other content) already in markdown format.table-reader
is compatible with mkdocs-macros-plugin
, which means you can dynamically insert tables using jinja2 syntax.See timvink.github.io/mkdocs-table-reader-plugin/
"},{"location":"options/","title":"Options","text":"You can customize the plugin by setting options in mkdocs.yml
. For example:
plugins:\n - table-reader:\n data_path: \".\"\n allow_missing_files: False\n select_readers:\n - read_csv\n - read_json\n enabled: True\n
"},{"location":"options/#data_path","title":"data_path
","text":"Default is .
. Set a default path to the searched directories in order to shorten table filename specifications.
Given a file path, table-reader
will search for that file relative to your your project's mkdocs.yml
and relative to your docs/
folder. If you use a folder for all your table files you can shorten the path specification by setting the data_path
.
For example, if your table is located in docs/assets/tables/basic_table.csv
, you can set data_path
to docs/assets/tables/
. Then you will be able to use {{ read_csv(\"basic_table.csv\") }}
instead of {{ read_csv(\"docs/assets/tables/basic_table.csv\") }}
inside any markdown page.
Info
Note that by default the plugin will also search the page's directory but only when a table is not found.
For more examples see the how to guide on project structure.
"},{"location":"options/#allow_missing_files","title":"allow_missing_files
","text":"Default: False
. When enabled, if a filepath is not found, the plugin will raise a warning instead of an error.
select_readers
","text":"Default: Selects all available readers. Specify a list of readers to improve documentation build times for very large sites.
"},{"location":"options/#enabled","title":"enabled
","text":"Default is True
. Enables you to deactivate this plugin. This option is supported by all plugins since mkdocs 1.6 (see docs). A possible use case is local development where you might want faster build times and/or do not have the tables ready. It's recommended to use this option with an environment variable together with a default fallback (introduced in mkdocs v1.2.1, see docs). Example:
plugins:\n - table-reader:\n enabled: !ENV [ENABLED_TABLE_READER, True]\n
Which enables you to disable the plugin locally using:
export ENABLED_TABLE_READER=false\nmkdocs serve\n
"},{"location":"readers/","title":"Readers","text":"The following table reader functions are available:
"},{"location":"readers/#read_csv","title":"read_csv","text":"{{ read_csv() }}
passed to pandas.read_csv(). Example:
{{ read_csv('tables/basic_table.csv') }}
{{ read_fwf() }}
passed to pandas.read_fwf(). Example:
{{ read_fwf('tables/fixedwidth_table.txt') }}
{{ read_yaml() }}
is parsed with yaml.safe_load() and passed to pandas.json_normalize(). Example:
{{ read_yaml('tables/yaml_table.yml') }}
{{ read_table() }}
passed to pandas.read_table(). Example:
{{ read_table('tables/basic_table.csv', sep = ',') }}
{{ read_json() }}
passed to pandas.read_json(). Example:
{{ read_json('tables/data.json', orient='split') }}
{{ read_feather() }}
passed to pandas.read_feather(). Example:
{{ read_json('tables/data.feather') }}
{{ read_excel() }}
passed to pandas.read_excel(). Example:
{{ read_excel('tables/excel_table.xlsx', engine='openpyxl') }}
Reading xlsx files
You might get a XLRDError('Excel xlsx file; not supported',)
error when trying to read modern excel files. That's because xlrd
does not support .xlsx
files (stackoverflow post). Instead, install openpyxl and use:
{{ read_excel('tables/excel_table.xlsx', engine='openpyxl') }}
{{ read_raw() }}
inserts contents from a file directly. This is great if you have a file with a table already in markdown format. It could also replace a workflow where you use the snippets extension to embed external files.
Example:
InputOutput{{ read_raw('tables/markdown_table.md') }}
This plugin is built to be able to quickly insert table files anywhere in a markdown file.
You could also consider alternative approaches that might fit your use-case better.
"},{"location":"howto/alternatives/#write-tables-to-markdown-files","title":"Write tables to markdown files","text":"You could write a script (maybe triggered by a mkdocs hook) that writes the tables you need into markdown files. It could look something like this:
# write markdown tables\nimport pandas as pd\n\nmd = pd.read_csv(\"your_file.csv\").to_markdown()\nwith open(\"docs/assets/tables/my_file.md\", \"w\") as f:\n f.write(md)\n
You can then use the snippets extension to insert the tables into your markdown pages:
# some_page.md\n\nMy table:\n\n--8<-- \"assets/tables/my_file.md\"\n
Upsides:
Downsides:
You could also choose to insert the markdown for tables dynamically, using packages like markdown-exec or mkdocs-macros-plugin.
For example:
```python exec=\"true\"\nimport pandas as pd\n\nfile_path = \"path/to/file/from/project/root\"\nprint(pd.read_csv(file_path).to_markdown(index=False, disable_numparse=True))\n```\n
"},{"location":"howto/customize_tables/","title":"Customize markdown tables","text":"You can customize the resulting markdown tables!
"},{"location":"howto/customize_tables/#theory","title":"Theory","text":"Under the hood mkdocs-table-reader-plugin
is basically doing:
import pandas as pd\ndf = pd.read_csv('path_to_table.csv')\ndf.to_markdown(index=False, tablefmt='pipe')\n
Any keyword arguments you give to {{ read_csv('path_to_your_table.csv') }}
will be matched and passed the corresponding pandas.read_csv() and/or .to_markdown() functions.
Pandas's .to_markdown()
uses the tabulate package and any keyword arguments that are passed to it. Tabulate in turn offers many customization options, see library usage.
Text columns will be aligned to the left by default, whilst columns which contain only numbers will be aligned to the right. You can override this behaviour using tabulate's custom column alignment. Example:
left center right{{ read_csv('tables/basic_table.csv', colalign=(\"left\",)) }}
{{ read_csv('tables/basic_table.csv', colalign=(\"center\",)) }}
{{ read_csv('tables/basic_table.csv', colalign=(\"right\",)) }}
If you use mkdocs-material, you can configure sortable tables.
"},{"location":"howto/customize_tables/#number-formatting","title":"Number formatting","text":"You can use tabulate's number formatting. Example:
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=\".0f\") }}
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=\".1f\") }}
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=\".2f\") }}
If you follow the mkdocs-material
tutorial on installation with docker
, you will run into an error if you try to add this plugin saying:
\u00d7 Building wheel for numpy (pyproject.toml) did not run successfully.
The reason is that the mkdocs-material
docker image uses the alpine
image. mkdocs-table-reader-plugin
depends on pandas
, which depends on numpy
, which in turn requires C++ to install, which is not part of the alpine
image.
If you need a very small image, you can adapt the alpine image to support numpy, but you are better off using a more complete docker image like python-slim
. So you need to build a different docker image.
As an example, below is a Dockerfile
that is adapted from the mkdocs-material dockerfile. You can use it to create a new Docker image that supports mkdocs-table-reader-plugin
:
git clone https://github.com/squidfunk/mkdocs-material.git material-git/\ncd material-git\n# Manually replace `Dockerfile` with the example below\ndocker build -t YOUR-CONTAINER-NAME .\n# Now, inside your own project, you can use:\ndocker run --rm -it -p 8000:8000 -v ${PWD}:/docs YOUR-CONTAINER-NAME\n
Dockerfile FROM python:3-slim\n\n# Build-time flags\nARG WITH_PLUGINS=true\n\n# Environment variables\nENV PACKAGES=/usr/local/lib/python3.11/site-packages\nENV PYTHONDONTWRITEBYTECODE=1\n\n# Set build directory\nWORKDIR /tmp\n\n# Copy files necessary for build\nCOPY material-git/material material\nCOPY material-git/package.json package.json\nCOPY material-git/README.md README.md\nCOPY material-git/requirements.txt requirements.txt\nCOPY material-git/pyproject.toml pyproject.toml\n\n# Perform build and cleanup artifacts and caches\nRUN \\\napt update \\\n&& \\\napt install -y \\\n libcairo2-dev \\\n libfreetype6-dev \\\n git \\\n libturbojpeg-dev \\\n openssh-server \\\n zlib1g-dev \\\n&& \\\napt install -y \\\n gcc \\\n libffi-dev \\\n musl-dev \\\n&& \\\npip install --no-cache-dir . \\\n&& \\\nif [ \"${WITH_PLUGINS}\" = \"true\" ]; then \\\n pip install --no-cache-dir \\\n \"mkdocs-minify-plugin>=0.3\" \\\n \"mkdocs-redirects>=1.0\" \\\n \"pillow>=9.0\" \\\n \"cairosvg>=2.5\" \\\n \"mkdocs-table-reader-plugin\" \\\n ; \\\nfi \\\n&& \\\nfor theme in mkdocs readthedocs; do \\\n rm -rf ${PACKAGES}/mkdocs/themes/$theme; \\\n ln -s \\\n ${PACKAGES}/material \\\n ${PACKAGES}/mkdocs/themes/$theme; \\\ndone \\\n&& \\\nrm -rf /tmp/* /root/.cache \\\n&& \\\nfind ${PACKAGES} \\\n -type f \\\n -path \"*/__pycache__/*\" \\\n -exec rm -f {} \\;\n\n# Trust directory, required for git >= 2.35.2\nRUN git config --global --add safe.directory /docs &&\\\n git config --global --add safe.directory /site\n\n# Set working directory\nWORKDIR /docs\n\n# Expose MkDocs development server port\nEXPOSE 8000\n\n# Start development server by default\nENTRYPOINT [\"mkdocs\"]\nCMD [\"serve\", \"--dev-addr=0.0.0.0:8000\"]\n
\"mkdocs-table-rea"},{"location":"howto/preprocess_tables/","title":"Preprocess input tables","text":"mkdocs>=1.4
supports hooks, which enable you to run python scripts on mkdocs serve
or mkdocs build
.
Here are some example of workflows that use hooks and the table-reader
plugin:
from os import listdir\nfrom os.path import isfile, join\nimport pandas as pd\n\ndef on_pre_build(config, **kwargs) -> None:\n tables = []\n input_files = [f for f in listdir(mypath) if isfile(join(mypath, f))]\n for table in input_files:\n tables.append(pd.read_csv(table))\n df = pd.concat(tables, ignore_index=True)\n df.to_csv(\"docs/assets/output_table.csv\")\n
My table: {{ read_csv(\"docs/assets/output_table.csv\") }}
# ...\nplugins:\n - table-reader\nhooks:\n - scripts/hooks.py\n# ...\n
.\n\u251c\u2500\u2500 scripts/\n\u2502 \u2514\u2500\u2500\u2500 hooks.py\n\u251c\u2500\u2500 docs/\n\u2502 \u2514\u2500\u2500\u2500 index.md\n\u2502 \u2514\u2500\u2500 assets/tables/\n\u2502 \u2514\u2500\u2500 table1.csv\n\u2502 \u2514\u2500\u2500 table2.csv\n\u2514\u2500\u2500 mkdocs.yml\n
Alternative: use jinja
You can also use jinja2 to display a list of tables. See how to use jinja2 for automation.
"},{"location":"howto/preprocess_tables/#download-a-table-from-an-api","title":"Download a table from an API","text":"hooks.pyindex.mdmkdocs.ymlProject structureimport pandas as pd\n\ndef on_pre_build(config, **kwargs) -> None:\n df = pd.read_csv('https://data.cityofnewyork.us/resource/nu7n-tubp.csv?$limit=100')\n df.to_csv(\"docs/assets/nyc_data.csv\")\n
My table: {{ read_csv(\"docs/assets/nyc_data.csv\") }}
# ...\nplugins:\n - table-reader\nhooks:\n - scripts/hooks.py\n# ...\n
.\n\u251c\u2500\u2500 scripts/\n\u2502 \u2514\u2500\u2500\u2500 hooks.py\n\u251c\u2500\u2500 docs/\n\u2502 \u2514\u2500\u2500\u2500 index.md\n\u2502 \u2514\u2500\u2500 assets/\n\u2514\u2500\u2500 mkdocs.yml\n
Note that during development when you use mkdocs serve
and autoreload, you might not want to run this hook every time you make a change. You could use an environment variable inside your hook, for example something like if os.environ['disable_hook'] == 1: return None
.
You have different possible strategies to store and load your tables. This guide gives some examples.
"},{"location":"howto/project_structure/#one-table-per-markdown-file","title":"One table per markdown file","text":"If you only want to include an occasional table in a specific markdown file, just store it in the same directory as the markdown file:
project structurefolder/page.md.\n\u251c\u2500\u2500 docs/\n\u2502 \u251c\u2500\u2500 index.md\n\u2502 \u2514\u2500\u2500 folder/\n\u2502 \u2514\u2500\u2500 another_table.csv\n\u2502 \u2514\u2500\u2500 page.md\n\u2514\u2500\u2500 mkdocs.yml\n
Here is the table:\n\n\\{\\{ read_csv(\"another_table.csv\") \\}\\}\n
In page.md
, to read another_table.csv
, you can choose to use:
{{ read_csv(\"docs/folder/another_table.csv\") }}
(Path relative to mkdocs.yml){{ read_csv(\"folder/another_table.csv\") }}
(Path relative to docs/ directory){{ read_csv(\"another_table.csv\") }}
(Path relative to page source file)If you want to reuse tables in multiple markdown files, or have many tables, you'll want to store them in a central directory, like docs/assets/tables
. That way, if you restructure your navigation, the links to the tables won't break either. It's also great if you generate tables because the output directory will be the same.
Given the following project structure:
.\n\u251c\u2500\u2500 docs/\n\u2502 \u251c\u2500\u2500 index.md\n\u2502 \u2514\u2500\u2500 folder/\n\u2502 \u2514\u2500\u2500 page.md\n\u2502 \u2514\u2500\u2500 assets/\n\u2502 \u2514\u2500\u2500 tables/\n\u2502 \u2514\u2500\u2500 another_table.csv\n\u2514\u2500\u2500 mkdocs.yml\n
In page.md
, to read another_table.csv
, you can choose to use:
{{ read_csv(\"docs/assets/tables/another_table.csv\") }}
(Path relative to mkdocs.yml){{ read_csv(\"assets/tables/another_table.csv\") }}
(Path relative to docs/ directory){{ read_csv(\"../assets/tables/another_table.csv\") }}
(Path relative to page source file (note that ..
stands for \"one directory up\"))table-reader
supports mkdocs-macros-plugin
, which enables you to use jinja2 syntax inside markdown files (among other things).
To enable macros
, specify the plugin before table-reader
in your mkdocs.yml
file:
plugins:\n - macros\n - table-reader\n
Now you can do cool things like:
"},{"location":"howto/use_jinja2/#dynamically-load-a-list-of-tables","title":"Dynamically load a list of tables","text":"# index.md\n\n{% set table_names = [\"basic_table.csv\",\"basic_table2.csv\"] %}\n{% for table_name in table_names %}\n\n{ { read_csv(table_name) }}\n\n{% endfor %}\n
"},{"location":"howto/use_jinja2/#insert-tables-into-content-tabs","title":"Insert tables into content tabs","text":"If you inserted content has multiple lines, then indentation will be not be retained beyond the first line. This means things like content tabs will not work as expected.
To fix that, you can use the custom filter add_indendation
(a filter add to macros
by table-reader
plugin). For example:
{% set table_names = [\"basic_table.csv\",\"basic_table2.csv\"] %}\n{% for table_name in table_names %}\n\n=== \"{{ table_name }}\"\n\n { { read_csv(table_name) | add_indentation(spaces=4) }}\n\n{% endfor %}\n
site_name: test git_table_reader site\nuse_directory_urls: true\n\ntheme:\nname: material\n\nplugins:\n - search\n - macros\n - table-reader\n\nmarkdown_extensions:\n - pymdownx.superfences\n - pymdownx.tabbed:\n alternate_style: true\n
Note the space in { {
To avoid the tables being inserted into the code example, we replaced {{
with { {
. If you copy this example, make sure to fix.
mkdocs-macros-plugin
enables you to define additional functions (called macros) that you will be able to use within your markdown files. See their documentation on how to set this up. Here's an example with some functions to interact with the filesystem:
def define_env(env):\n \"\"\"\n Register additional mkdocs-macros-plugin functions that can be used as macros in markdown files.\n \"\"\" \n @env.macro\n def listdir(path):\n return os.listdir(path)\n\n @env.macro\n def path_exists(path):\n return Path(path).exists()\n\n @env.macro\n def is_file(path):\n return Path(path).is_file() \n
Now you could do something like:
# index.md\n\n{% for table_name in listdir('docs/assets/my_tables\") %}\n\n{ { read_csv(table_name) }}\n\n{% endfor %}\n
Note the space in { {
To avoid the tables being inserted into the code example, we replaced {{
with { {
. If you copy this example, make sure to fix.