Haraka fails to read docs/Plugins.md after installation on 3.0.4

[tyranron]

Describe the bug

npm install -g [email protected]
haraka -i /etc/haraka/

haraka -o -c /etc/haraka

Expected behavior

Something like:

loaded TLD files:
  1=1448
  2=8320
  3=3571
loaded 9672 Public Suffixes
loglevel: INFO
log format: DEFAULT

Hook: connect ------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
dnsbl                               connect_first                          0  30

Hook: helo ---------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
helo.checks                         proto_mismatch_smtp                    0  30
helo.checks                         init                                   0  30
helo.checks                         match_re                               0  30
helo.checks                         bare_ip                                0  30
helo.checks                         dynamic                                0  30
helo.checks                         big_company                            0  30
helo.checks                         valid_hostname                         0  30
helo.checks                         rdns_match                             0  30
helo.checks                         forward_dns                            0  30
helo.checks                         host_mismatch                          0  30
helo.checks                         emit_log                               0  30
helo.checks                         literal_mismatch                       0  30

Hook: ehlo ---------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
helo.checks                         proto_mismatch_esmtp                   0  30
helo.checks                         init                                   0  30
helo.checks                         match_re                               0  30
helo.checks                         bare_ip                                0  30
helo.checks                         dynamic                                0  30
helo.checks                         big_company                            0  30
helo.checks                         valid_hostname                         0  30
helo.checks                         rdns_match                             0  30
helo.checks                         forward_dns                            0  30
helo.checks                         host_mismatch                          0  30
helo.checks                         emit_log                               0  30
helo.checks                         literal_mismatch                       0  30

Hook: mail ---------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
mail_from.is_resolvable             hook_mail                              0  30
rcpt_to.in_host_list                hook_mail                              0  30

Hook: rcpt ---------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
rcpt_to.in_host_list                hook_rcpt                              0  30

Hook: data_post ----------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
headers                             duplicate_singular                     0  30
headers                             missing_required                       0  30
headers                             invalid_return_path                    0  30
headers                             invalid_date                           0  30
headers                             user_agent                             0  30
headers                             direct_to_mx                           0  30
headers                             from_match                             0  30
headers                             delivered_to                           0  30
headers                             mailing_list                           0  30
headers                             from_phish                             0  30

Hook: queue --------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
queue/smtp_forward                  queue_forward                          0  30

Hook: get_mx -------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
queue/smtp_forward                  get_mx                                 0  30

Observed behavior

loaded TLD files:
  1=1448
  2=8320
  3=3571
loaded 9672 Public Suffixes
loglevel: INFO
log format: DEFAULT

node:fs:561
  return binding.open(
                 ^

Error: ENOENT: no such file or directory, open 'docs/Plugins.md'
    at Object.openSync (node:fs:561:18)
    at Object.readFileSync (node:fs:445:35)
    at getHooks (/usr/local/lib/node_modules/Haraka/bin/haraka:231:8)
    at Object.<anonymous> (/usr/local/lib/node_modules/Haraka/bin/haraka:371:24)
    at Module._compile (node:internal/modules/cjs/loader:1546:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1691:10)
    at Module.load (node:internal/modules/cjs/loader:1317:32)
    at Module._load (node:internal/modules/cjs/loader:1127:12)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:217:24) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'docs/Plugins.md'
}

Node.js v22.7.0

Additional context

3.0.3 works OK.

[s-aga-r]

There are additional issues as well. For instance, in the hook_delivered params, the host is mistakenly set as the IP address. Additionally, the mailauth plugin's SPF check consistently fails.

[brutus]

Same here. If I manually copy docs/Plugin.md to the current working directory it no longer bails that way.

But fails like this:

loaded TLD files:
  1=1448
  2=8320
  3=3571
loaded 9672 Public Suffixes
loglevel: INFO
log format: DEFAULT

/opt/haraka/lib/node_modules/Haraka/bin/haraka:371
    for (const hook of getHooks()) {
                       ^

TypeError: getHooks is not a function or its return value is not iterable
    at Object.<anonymous> (/opt/haraka/lib/node_modules/Haraka/bin/haraka:371:24)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:12)
    at node:internal/main/run_main_module:28:49

I guess b/c the getHooks function is missing an return?

[lnedry]

Plugins.md needs to be installed in the docs folder, e.g. /etc/haraka/docs/

[msimerson]

Hey folks, @lnedry has a partial fix I merged in #3405 and this should be the other needful bit. Please confirm.

[lnedry]

I confirmed that Plugins.md is being installed in the docs folder and help is working as expected.

[tyranron]

@msimerson still not fixed for 3.0.5. Steps for reproducing are the same:

npm install -g [email protected]
haraka -i /etc/haraka/

haraka -o -c /etc/haraka

output

/ # npm install -g [email protected]
npm warn deprecated @npmcli/[email protected]: This functionality has been moved to @npmcli/fs
npm warn deprecated [email protected]: Glob versions prior to v9 are no longer supported
npm warn deprecated [email protected]: This package is no longer supported.
npm warn deprecated [email protected]: This package is no longer supported.
npm warn deprecated [email protected]: Rimraf versions prior to v4 are no longer supported
npm warn deprecated [email protected]: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm warn deprecated [email protected]: This package is no longer supported.
npm warn deprecated [email protected]: Glob versions prior to v9 are no longer supported

added 317 packages in 47s

26 packages are looking for funding
  run `npm fund` for details
/ # haraka -i /etc/haraka/
create: /etc/haraka
create: /etc/haraka/plugins
create: /etc/haraka/docs
create: /etc/haraka/config
create: /etc/haraka/docs/Plugins.md
create: /etc/haraka/config/smtp.ini
create: /etc/haraka/config/log.ini
create: /etc/haraka/config/plugins
/ # haraka -o -c /etc/haraka
loaded TLD files:
  1=1445
  2=8416
  3=3642
loaded 9773 Public Suffixes
loglevel: INFO
log format: DEFAULT

node:fs:561
  return binding.open(
                 ^

Error: ENOENT: no such file or directory, open 'docs/Plugins.md'
    at Object.openSync (node:fs:561:18)
    at Object.readFileSync (node:fs:445:35)
    at getHooks (/usr/local/lib/node_modules/Haraka/bin/haraka:231:15)
    at Object.<anonymous> (/usr/local/lib/node_modules/Haraka/bin/haraka:371:24)
    at Module._compile (node:internal/modules/cjs/loader:1546:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1691:10)
    at Module.load (node:internal/modules/cjs/loader:1317:32)
    at Module._load (node:internal/modules/cjs/loader:1127:12)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:217:24) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'docs/Plugins.md'
}

Node.js v22.9.0

full reproducing steps

docker run --rm -it node:22-alpine sh

apk update && apk upgrade
apk add python3 g++ make

npm install -g [email protected]
haraka -i /etc/haraka/

haraka -o -c /etc/haraka

[msimerson]

@tyranron , please try this patch against the haraka 'binary' in your path. It should resolve the issue, and if not it'll at least provide a useful message.

diff --git a/bin/haraka b/bin/haraka
index 170bfbe7..0db61428 100755
--- a/bin/haraka
+++ b/bin/haraka
@@ -228,7 +228,8 @@ const plugin_doc = [
 
 
 function getHooks () {   // see haraka/Haraka#3306
-    return fs.readFileSync('docs/Plugins.md').toString()
+    const pi_path = path.resolve(parsed.configs, 'docs', 'Plugins.md')
+    return fs.readFileSync(pi_path).toString()
         .split('## Available Hooks')[1]    // discard everything before this string
         .split('### rcpt')[0]              // discard everything after this string
         .match(/\*\s(\S+)/gm)              // grab the first word of lines starting with '* '

Reminder to self: paths should always be assembled using path.join or path.resolve, so that we don't needlessly harm Windows compatibility.

[msimerson]

Additionally, the mailauth plugin's SPF check consistently fails.

@s-aga-r, Please create separate issues for stuff like this. Preferably in the repo where the issue is. postalsys/haraka-plugin-mailauth#3 should fix that.

[lnedry]

I just installed in on a clean Debian 12 and it is working as expected.

# haraka -o -c /etc/haraka
loaded TLD files:
  1=1445
  2=8416
  3=3642
loaded 9773 Public Suffixes
loglevel: INFO
log format: DEFAULT

Hook: mail ---------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
mail_from.is_resolvable             hook_mail                              0  30
rcpt_to.in_host_list                hook_mail                              0  30

Hook: rcpt ---------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
rcpt_to.in_host_list                hook_rcpt                              0  30

Hook: queue --------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
queue/smtp_forward                  queue_forward                          0  30

Hook: get_mx -------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
queue/smtp_forward                  get_mx                                 0  30

[lnedry]

I also confirmed that @msimerson's patch also works.

[tyranron]

@msimerson I do confirm that the patch works.

repro

$ docker run --rm -it node:22-alpine3.20 sh 
/ # apk update && apk upgrade
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/community/x86_64/APKINDEX.tar.gz
v3.20.3-96-g1827497eea7 [https://dl-cdn.alpinelinux.org/alpine/v3.20/main]
v3.20.3-104-g27788c02ed4 [https://dl-cdn.alpinelinux.org/alpine/v3.20/community]
OK: 24163 distinct packages available
Upgrading critical system libraries and apk-tools:
(1/1) Upgrading apk-tools (2.14.4-r0 -> 2.14.4-r1)
Executing busybox-1.36.1-r29.trigger
Continuing the upgrade transaction with new apk-tools:
OK: 10 MiB in 16 packages
/ # apk add python3 g++ make
(1/29) Installing libstdc++-dev (13.2.1_git20240309-r0)
(2/29) Installing jansson (2.14-r4)
(3/29) Installing zstd-libs (1.5.6-r0)
(4/29) Installing binutils (2.42-r0)
(5/29) Installing libgomp (13.2.1_git20240309-r0)
(6/29) Installing libatomic (13.2.1_git20240309-r0)
(7/29) Installing gmp (6.3.0-r1)
(8/29) Installing isl26 (0.26-r1)
(9/29) Installing mpfr4 (4.2.1-r0)
(10/29) Installing mpc1 (1.3.1-r1)
(11/29) Installing gcc (13.2.1_git20240309-r0)
(12/29) Installing musl-dev (1.2.5-r0)
(13/29) Installing g++ (13.2.1_git20240309-r0)
(14/29) Installing make (4.4.1-r2)
(15/29) Installing libbz2 (1.0.8-r6)
(16/29) Installing libexpat (2.6.3-r0)
(17/29) Installing libffi (3.4.6-r0)
(18/29) Installing gdbm (1.23-r1)
(19/29) Installing xz-libs (5.6.2-r0)
(20/29) Installing mpdecimal (4.0.0-r0)
(21/29) Installing ncurses-terminfo-base (6.4_p20240420-r1)
(22/29) Installing libncursesw (6.4_p20240420-r1)
(23/29) Installing libpanelw (6.4_p20240420-r1)
(24/29) Installing readline (8.2.10-r0)
(25/29) Installing sqlite-libs (3.45.3-r1)
(26/29) Installing python3 (3.12.6-r0)
(27/29) Installing python3-pycache-pyc0 (3.12.6-r0)
(28/29) Installing pyc (3.12.6-r0)
(29/29) Installing python3-pyc (3.12.6-r0)
Executing busybox-1.36.1-r29.trigger
OK: 255 MiB in 45 packages
/ # npm install -g [email protected]
npm warn deprecated @npmcli/[email protected]: This functionality has been moved to @npmcli/fs
npm warn deprecated [email protected]: This package is no longer supported.
npm warn deprecated [email protected]: Glob versions prior to v9 are no longer supported
npm warn deprecated [email protected]: Rimraf versions prior to v4 are no longer supported
npm warn deprecated [email protected]: This package is no longer supported.
npm warn deprecated [email protected]: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm warn deprecated [email protected]: This package is no longer supported.
npm warn deprecated [email protected]: Glob versions prior to v9 are no longer supported

added 317 packages in 1m

26 packages are looking for funding
  run `npm fund` for details
/ # apk add patch
(1/1) Installing patch (2.7.6-r10)
Executing busybox-1.36.1-r29.trigger
OK: 255 MiB in 46 packages
/ # vi wtf.patch
/ # patch /usr/local/lib/node_modules/Haraka/bin/haraka < wtf.patch
patching file /usr/local/lib/node_modules/Haraka/bin/haraka
/ # haraka -i /etc/haraka/
create: /etc/haraka
create: /etc/haraka/plugins
create: /etc/haraka/docs
create: /etc/haraka/config
create: /etc/haraka/docs/Plugins.md
create: /etc/haraka/config/smtp.ini
create: /etc/haraka/config/log.ini
create: /etc/haraka/config/plugins
/ # haraka -o -c /etc/haraka
loaded TLD files:
  1=1445
  2=8416
  3=3642
loaded 9773 Public Suffixes
loglevel: INFO
log format: DEFAULT

Hook: mail ---------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
mail_from.is_resolvable             hook_mail                              0  30
rcpt_to.in_host_list                hook_mail                              0  30

Hook: rcpt ---------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
rcpt_to.in_host_list                hook_rcpt                              0  30

Hook: queue --------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
queue/smtp_forward                  queue_forward                          0  30

Hook: get_mx -------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
queue/smtp_forward                  get_mx                                 0  30