[need advice] no completion-at-point possible with gnus and notmuch?

[poulpoulsen]

hello,
i use your package about a year and it contains all that i need.
unfortunately since some time i can't complete at point with inside the message-buffer eg on the To: Header.
if i call (ebdb-complete-enable) a window pops up and i can select a candidate.
but i want completion-at-point.
i set ebdb-complete-mail to capf but without luck.

could you give an advice how to complete-at-point?
i use emacs 29 and your newest version on Linux Fedora 35.

inside using notmuch there is also no completion-at-point.

regards
poul

[girzel]

Hi there! If you just set ebdb-complete-mail to t, then compose a message, what is the value of message-completion-alist?

It should have an element in it like: ("^\\(Resent-\\)?\\(To\\|B?Cc\\|Reply-To\\|From\\|Mail-Followup-To\\|Mail-Copies-To\\):" . ebdb-complete-mail)

Then the key should run message-tab. Are any of those things not true?

[poulpoulsen]

hello there,
my ebdb-complete-mail usualy is set to capf but now i set it to t.
my message-completion-alist is
(("^\(Resent-\)?\(To\|B?Cc\|Reply-To\|From\|Mail-Followup-To\|Mail-Copies-To\):" . ebdb-complete-mail)

if i push tab a window opens with the address.
in the past i've got a completions-list of candidates at point.
like capf in emacs the usual way.
anything has changed...
perhaps something inside emacs?
i use emacs 29...

Regards
poul

[girzel]

When you say "a window opens with the address", do you mean it pops up an *EBDB* buffer? I'm still having trouble picturing what's actually happening. As far as I know, nothing has changed that would do this. You're sure is bound to message-tab, not ebdb-complete-message-tab? Are you calling ebdb-complete-enable anywhere?

[poulpoulsen]

you are right. i called ebdb-complee-enable.
after commented it out,
is bound to
TAB (translated from ) runs the command message-tab (found in
button-map), which is an interactive compiled Lisp function in
‘message.el’.

but unfortunately
ebdb in message-completion-alist is not set.
message-completion-alist is a variable defined in ‘message.el’.

Its value is
(("^\(Resent-\)?\(To\|B?Cc\|Reply-To\|From\|Mail-Followup-To\|Mail-Copies-To\):" . ebdb-complete-mail)
("^\(Newsgroups\|Followup-To\|Posted-To\|Gcc\):" . message-expand-group)
("^\([^ :]-\)?\(To\|B?Cc\|From\|Reply-to\|Mail-Followup-To\|Mail-Copies-To\):" . message-expand-name))
Original value was
(("^\(Newsgroups\|Followup-To\|Posted-To\|Gcc\):" . message-expand-group)
("^\([^ :]-\)?\(To\|B?Cc\|From\|Reply-to\|Mail-Followup-To\|Mail-Copies-To\):" . message-expand-name))

perhaps ebdb is not loaded.
how could i loaded correctly?
should it load in gnus.el ?

Regards
poul

[girzel]

That's correct! The first line in your message-completion-alist references ebdb-complete-mail, that's how it's supposed to work. Try restarting Emacs with these settings; I would expect EBDB completion to work normally now.

[poulpoulsen]

Hello,
unfortunately it is not working as expected.
I will explain:
Cursor is in the To: line and i type in info, then tab key.
Because i do have more then one adress that contains the string info, all candidates should displayed.
But only one candidate is displayed. In the past i've got a list with the candidates.
any idea?

[girzel]

Sorry I'm a bit behind on this...

The first thing to try is to confirm that (all-completions "info" ebdb-hashtable #'ebdb-completion-predicate) returns all the strings you'd expect.

Then, if possible, it would be great to have you edebug the source of ebdb-complete-mail, and see what is happening. The part you're looking for is down the function a little ways:

  (let* ((end (point))
         (done (unless beg 'nothing))
         (orig (and beg (buffer-substring-no-properties beg end)))
         (completion-ignore-case t)
         (completion (and orig
                          (try-completion orig ebdb-hashtable
                                          #'ebdb-completion-predicate)))
         all-completions dwim-completions one-record)

You want to be sure that 1) the orig variable holds the string you're expecting (the fragment you're trying to complete), and that later on, the call to all-completions continues to return all the expected strings, and then this part:

      (let ((records (delete-dups
                      (apply #'append (mapcar (lambda (compl)
					        (gethash compl ebdb-hashtable))
					      all-completions)))))

Actually returns records. Sorry that's a bunch of random recommendations, but I expect that something in there will show us what's going wrong.

[poulpoulsen]

hello,
no problem for the delay :-)
but sorry, i do not fully understand, what you explained and what i have to do.
i tried that:
new message buffer.
put cursor into TO:
call M-: (all-completions "info" ebdb-hashtable #'ebdb-completion-predicate)
("info" "[email protected]" "[email protected]")

this is strange, because these adresses are inside the ebdb-db but why are they selected???

can i help?

Regards
Poul

[poulpoulsen]

i do not understand the paragraph about edebug???

[girzel]

That's fine! Edebug is a debugging utility that lets you step through execution of elisp code, but if it's not something you're familiar with, this probably isn't the time to dive into it.

So it looks like EBDB thinks you have two email addresses that start with "info", and one... maybe record name? That is the exact string "info". Do you have an organization type record that is named "info"?

Either way, it should still be popping up a *Completions* buffer. Can you copy the function below into your *scratch* buffer, put point inside it somewhere, hit C-M-x, and the go back to a message composition buffer and try to complete the address again? Look in the *Messages* buffer for lines starting "EBDB log:", and paste them here.

(defun ebdb-complete-mail (&optional beg cycle-completion-buffer)
   (when (and (not beg)
             (<= (point)
                 (save-restriction	; `mail-header-end'
                   (widen)
                   (save-excursion
                     (rfc822-goto-eoh)
                     (point)))))
    (let ((end (point))
          start pnt state)
      (save-excursion
        ;; A header field name must appear at the beginning of a line,
        ;; and it must be terminated by a colon.
        (re-search-backward "^[^ \t\n:][^:]*:[ \t\n]+")
        (setq beg (match-end 0)
              start beg)
        (goto-char beg)
        ;; If we are inside a syntactically correct header field,
        ;; all continuation lines in between the field name and point
        ;; must begin with a white space character.
        (if (re-search-forward "\n[^ \t]" end t)
            ;; An invalid header is identified via BEG set to nil.
            (setq beg nil)
          ;; Parse field body up to END
          (with-syntax-table ebdb-quoted-string-syntax-table
            (while (setq pnt (re-search-forward ",[ \t\n]*" end t))
              (setq state (parse-partial-sexp start pnt nil nil state)
                    start pnt)
              (unless (nth 3 state) (setq beg pnt))))))))

  ;; Do we have a meaningful way to set BEG if we are not in a message header?
  (unless beg
    (message "Not a valid buffer position for mail completion")
    (sit-for 1))

  (let* ((end (point))
         (done (unless beg 'nothing))
         (orig (and beg (buffer-substring-no-properties beg end)))
         (completion-ignore-case t)
         (completion (and orig
                          (try-completion orig ebdb-hashtable
                                          #'ebdb-completion-predicate)))
         all-completions dwim-completions one-record)
    (message "EBDB log: %s" (format "string = %s" orig))
    (message "EBDB log: %s" (format "first completion = %s" completion))

    (unless done
      (if (and (stringp completion)
               (string-match "," completion))
          (setq completion (substring completion 0 (match-beginning 0))))

      (setq all-completions (all-completions orig ebdb-hashtable
                                             #'ebdb-completion-predicate))
      (message "EBDB log: %s" (format "all completions = %S" all-completions))

      ;; Resolve the records matching ORIG:
      ;; Multiple completions may match the same record
      (let ((records (delete-dups
                      (apply #'append (mapcar (lambda (compl)
					        (gethash compl ebdb-hashtable))
					      all-completions)))))
        ;; Is there only one matching record?
        (setq one-record (and (not (cdr records))
                              (car records))))

      ;; Clean up *Completions* buffer window, if it exists
      (let ((window (get-buffer-window "*Completions*")))
        (if (window-live-p window)
            (quit-window nil window)))

      (cond
       ;; Match for a single record
       (one-record
        (let ((completion-list (if (eq t ebdb-completion-list)
                                   '(name alt-names mail aka organization)
                                 ebdb-completion-list))
              (mails (ebdb-record-mail one-record))
              mail elt)
          (if (not mails)
              (progn
                (message "Matching record has no mail field")
                (sit-for 1)
                (setq done 'nothing))

            (if (try-completion
		 orig
                 (append
                  (if (memq 'name completion-list)
                      (list (or (ebdb-record-name-string one-record) "")))
                  (if (memq 'alt-names completion-list)
		      (or (ebdb-record-alt-names one-record) (list "")))
                  (if (memq 'organization completion-list)
                      (ebdb-record-organizations one-record))))
                (setq mail (car mails)))
            (unless mail
              (while (setq elt (pop mails))
                (if (try-completion orig (list (ebdb-string elt)))
                    (setq mail elt
                          mails nil))))
            ;; This error message indicates a bug!
            (unless mail (error "No match for %s" orig))

            (let ((dwim-mail (ebdb-dwim-mail one-record mail)))
              (if (string= dwim-mail orig)
                  ;; We get here if `ebdb-mail-avoid-redundancy' is 'mail-only
                  ;; and `ebdb-completion-list' includes 'mail.
                  (unless (and ebdb-complete-mail-allow-cycling
                               (< 1 (length (ebdb-record-mail one-record))))
                    (setq done 'unchanged))
                ;; Replace the text with the expansion
                (delete-region beg end)
                (insert dwim-mail)
                (ebdb-complete-mail-cleanup dwim-mail beg)
                (setq done 'unique))))))

       ;; Partial completion
       ((and (stringp completion)
             (not (ebdb-string= orig completion)))
        (delete-region beg end)
        (insert completion)
        (setq done 'partial))

       ;; Partial match not allowing further partial completion
       (completion
        (let ((completion-list (if (eq t ebdb-completion-list)
                                   '(name alt-names mail organization)
                                 ebdb-completion-list)))
          ;; Now collect all the dwim-addresses for each completion.
          ;; Add it if the mail is part of the completions
          (dolist (key all-completions)
            (dolist (record (gethash key ebdb-hashtable))
              (let ((mails (ebdb-record-mail record))
                    accept)
                (when mails
                  (dolist (field completion-list)
                    (cond ((eq field 'name)
                           (if (ebdb-string= key
					     (ebdb-record-name-string record))
                               (push (car mails) accept)))
                          ((eq field 'alt-names)
                           (if (member-ignore-case
				key (ebdb-record-alt-names record))
                               (push (car mails) accept)))
                          ((eq field 'organization)
                           (if (member-ignore-case key (ebdb-record-organizations
							record))
                               (push (car mails) accept)))
                          ((eq field 'primary)
                           (if (ebdb-string= key (ebdb-string
						  (car mails)))
                               (push (car mails) accept)))
                          ((eq field 'mail)
                           (dolist (mail mails)
                             (if (ebdb-string= key
					       (ebdb-string mail))
                                 (push mail accept))))))
                  (dolist (mail (delete-dups accept))
                    (push (ebdb-dwim-mail record mail) dwim-completions))))))

          (setq dwim-completions (sort (delete-dups dwim-completions)
                                       #'string-lessp))
          (cond ((not dwim-completions)
                 (message "Matching record has no mail field")
                 (sit-for 1)
                 (setq done 'nothing))
                ((eq 1 (length dwim-completions))
                 (delete-region beg end)
                 (insert (car dwim-completions))
                 (ebdb-complete-mail-cleanup (car dwim-completions) beg)
                 (setq done 'unique))
                (t (setq done 'choose)))))))

    (message "EBDB log: %s" (format "done is = %s" done))
    (message "EBDB log: %s" (format "allow cycling is = %s" ebdb-complete-mail-allow-cycling))

    (when (and (not done) ebdb-complete-mail-allow-cycling)
      ;; find the record we are working on.
      (let* ((address (ebdb-extract-address-components orig))
             (record (car (ebdb-message-search
                           (car address) (cadr address)))))
        (if (and record
                 (setq dwim-completions
                       (mapcar (lambda (m) (ebdb-dwim-mail record m))
                               (ebdb-record-mail record))))
            (cond ((and (= 1 (length dwim-completions))
                        (string= orig (car dwim-completions)))
                   (setq done 'unchanged))
                  (cycle-completion-buffer ; use completion buffer
                   (setq done 'cycle-choose))
		  ((let* ((canon (car (member-ignore-case
				       (nth 1 address)
				       (ebdb-record-mail-canon record))))
			  (dwim-mail (let ((case-fold-search t))
				       (seq-find (lambda (d)
						   (string-match-p canon d))
						 dwim-completions))))
		     (when (not (string= orig dwim-mail))
		       (delete-region beg end)
		       (insert dwim-mail)
		       (ebdb-complete-mail-cleanup dwim-mail beg)
		       (setq done 'reformat))
		     done))
                  (t
                   (let ((dwim-mail (or (nth 1 (member orig dwim-completions))
                                        (nth 0 dwim-completions))))
                     (delete-region beg end)
                     (insert dwim-mail)
                     (ebdb-complete-mail-cleanup dwim-mail beg)
                     (setq done 'cycle)))))))

    (when (member done '(choose cycle-choose))
      (message "EBDB log: %s" "should be popping up completion buffer now")
      (if (string< (substring emacs-version 0 4) "23.2")
          (message "*Completions* buffer requires at least GNU Emacs 23.2")
        (let ((status (not (eq (selected-window) (minibuffer-window))))
              (completion-base-position (list beg end))
              (completion-list-insert-choice-function
               `(lambda (beg end text)
                  ,(if (boundp 'completion-list-insert-choice-function)
		       `(funcall ',completion-list-insert-choice-function
				 beg end text))
		  (ebdb-complete-mail-cleanup text beg))))
          (if status (message "Making completion list..."))
          (with-output-to-temp-buffer "*Completions*"
            (display-completion-list dwim-completions))
          (if status (message "Making completion list...done")))))
    (unless (eq done 'nothing)
      done)))

[poulpoulsen]

hello,
hmmh it is not working as expected :-(
First
C-m-x is not defined here. I called eval-buffer and that works.
Second
if i complete names and push TAB it do not open Completion Buffer, its a EBDB-Message Buffer.
Perhaps because i activate ebdb with
(add-hook 'gnus-startup-hook #'ebdb-complete-enable)
.
But if i comment it out, i got no completion.
ebdb function is not bound to TAB.
so anything from ebdb should be loaded (with hook?) or should i set a anything special.

here are my settings:
(require 'ebdb)
(require 'ebdb-gnus)
(require 'ebdb-message)
(require 'ebdb-complete)
(require 'ebdb-format)
(require 'ebdb-org)
(require 'ebdb-snarf)
(require 'ebdb-vcard)
(require 'ebdb-i18n)
(require 'ebdb-html)

(setq ebdb-sources (locate-user-emacs-file "ebdb"))
(setq ebdb-permanent-ignores-file (locate-user-emacs-file "ebdb-permanent-ignores"))
(setq ebdb-mua-pop-up nil)
(setq ebdb-default-window-size 0.25)
(setq ebdb-mua-default-formatter ebdb-default-multiline-formatter)
(setq ebdb-mua-auto-update-p 'existing)
(setq ebdb-gnus-auto-update-p 'existing)
(setq ebdb-mua-reader-update-p 'existing)
(setq ebdb-mua-sender-update-p 'query)
(setq ebdb-message-auto-update-p 'query)
(setq ebdb-message-try-all-headers t)
(setq ebdb-message-headers
'((sender "From" "Resent-From" "Reply-To" "Sender")
(recipients "Resent-To" "Resent-Cc" "Resent-CC" "To" "Cc" "CC" "Bcc" "BCC")))
(setq ebdb-message-all-addresses t)
(setq ebdb-complete-mail 'capf)
(setq ebdb-mail-avoid-redundancy nil)
(setq ebdb-completion-display-record t)
(setq ebdb-complete-mail-allow-cycling t)
(setq ebdb-record-self nil)
(setq ebdb-user-name-address-re 'self) ; match the above
(setq ebdb-save-on-exit t)
(setq ebdb-use-diary nil)

(let ((map ebdb-mode-map))
(define-key map (kbd "D") #'ebdb-delete-field-or-record)
(define-key map (kbd "M") #'ebdb-email) ; disables ebdb-mail-each' (define-key map (kbd "m") #'ebdb-toggle-record-mark) (define-key map (kbd "t") #'ebdb-toggle-all-record-marks) (define-key map (kbd "T") #'ebdb-toggle-records-format) ; disables ebdb-toggle-all-records-format'
(define-key map (kbd "U") #'ebdb-unmark-all-records))

Hope it helps
Poul

[girzel]

I guess the first thing I would do here is remove all config that is setting variables to their default values! There's no sense in repeating those settings, and will shadow changes to the defaults in the future. For instance ebdb-record-self is nil by default, just leave that out unless you want to set it to the record that represents you.

Then the first thing would be to remove (require 'ebdb-complete), as you've stated you don't want to use that completion interface. And yes, definitely remove (add-hook 'gnus-startup-hook #'ebdb-complete-enable). Also remove the line (setq ebdb-complete-mail 'capf), let's just leave that at its default to try to get basic behavior working.

Then restart Emacs.

One thing to try here is to start composing a message, go to the "To:" header, enter a string that you know should complete to multiple records, and run "M-x ebdb-complete-mail". Do you get a completion buffer? That should tell us whether the problem is inside the EBDB machinery, or outside of it.

[poulpoulsen]

Am Mo, 2022-05-30, 11:39 -0700, Eric Abrahamsen ***@***.***> schrieb: hello, i just changed the things you mentioned. now the following occurs: ebdb-complete-mail shows definitely too less records. only the first 3 or four records sorted historically. ebdb-complete works as expected! shows all records. my tab key does nothing. i see, that it binds ebdb-complete-mail not ebdb-complete. can i change that? or is it hardcoded? regards, Jens

…

I guess the first thing I would do here is remove all config that is setting variables to their default values! There's no sense in repeating those settings, and will shadow changes to the defaults in the future. For instance `ebdb-record-self` is nil by default, just leave that out unless you want to set it to the record that represents you. Then the first thing would be to remove `(require 'ebdb-complete)`, as you've stated you don't want to use that completion interface. And yes, definitely remove `(add-hook 'gnus-startup-hook #'ebdb-complete-enable)`. Also remove the line `(setq ebdb-complete-mail 'capf)`, let's just leave that at its default to try to get basic behavior working. Then restart Emacs. One thing to try here is to start composing a message, go to the "To:" header, enter a string that you know should complete to multiple records, and run "M-x ebdb-complete-mail". Do you get a completion buffer? That should tell us whether the problem is inside the EBDB machinery, or outside of it.

-- Jens Reimer