racket-xp-mode: Do "semantic" highlighting of binding sites

This is mainly intended for, and by default only enabled for, the
major mode racket-hash-lang-mode.

This enables a "more colorful" approach, which is more similar to
"classic" racket-mode -- although it is not, and not intended to be,
exactly the same.

So the basic story here is: racket-hash-lang-mode does a basic level
of fontification applying 'face properties as usual, based on lexer
tokens. Then when racket-xp-mode is enabled, and this extra
highlighting is enabled, it will add `font-lock-face properties. Note
that 'face always wins over 'font-lock-face, so the minor mode can
never override the basic token coloring decisions, but it can enhance
text that lacks any 'face property at all. By configuring
hash-lang-mode not to apply any 'face for "symbol" tokens -- typically
the vast majority of a source -- this leaves the field free for a
minor mode like racket-xp-mode to augment, if a user prefers.

doc/generate.el

@@ -188,6 +188,7 @@
     racket-browse-url-function
     racket-xp-after-change-refresh-delay
     racket-xp-highlight-unused-regexp
+    racket-xp-binding-font-lock-face-modes
     racket-documentation-search-location
     "Hash lang variables"
     racket-hash-lang-token-face-alist
@@ -252,6 +253,8 @@
     racket-xp-unused-face
     racket-xp-tail-target-face
     racket-xp-tail-position-face
+    racket-xp-binding-def-font-lock-face
+    racket-xp-binding-use-font-lock-face
     racket-logger-config-face
     racket-logger-topic-face
     racket-logger-fatal-face

doc/racket-mode.texi

@@ -205,6 +205,7 @@ General variables
 * racket-browse-url-function::
 * racket-xp-after-change-refresh-delay::
 * racket-xp-highlight-unused-regexp::
+* racket-xp-binding-font-lock-face-modes::
 * racket-documentation-search-location::
 
 Hash lang variables
@@ -297,6 +298,8 @@ All
 * racket-xp-unused-face::
 * racket-xp-tail-target-face::
 * racket-xp-tail-position-face::
+* racket-xp-binding-def-font-lock-face::
+* racket-xp-binding-use-font-lock-face::
 * racket-logger-config-face::
 * racket-logger-topic-face::
 * racket-logger-fatal-face::
@@ -2843,6 +2846,7 @@ Delete the ``compiled'' directories made by @ref{racket-mode-start-faster}.
 * racket-browse-url-function::
 * racket-xp-after-change-refresh-delay::
 * racket-xp-highlight-unused-regexp::
+* racket-xp-binding-font-lock-face-modes::
 * racket-documentation-search-location::
 @end menu
 
@@ -2959,6 +2963,26 @@ Only give @ref{racket-xp-unused-face} to unused bindings that match this regexp.
 The default is to highlight identifiers that do not start with
 an underline, which is a common convention.
 
+@node racket-xp-binding-font-lock-face-modes
+@subsection racket-xp-binding-font-lock-face-modes
+
+Major modes where @ref{racket-xp-mode} will fontify binding identifier sites.
+
+A `font-lock-face property is added with the value:
+
+@itemize
+@item
+@ref{racket-xp-binding-def-font-lock-face} for each site that is
+a binding definition identifier.
+
+@item
+@ref{racket-xp-binding-use-font-lock-face} for each site that is
+a use of an @emph{imported} binding identifier.
+@end itemize
+
+This has a visible effect only when there is @emph{not} also a
+`face property applied by the major mode's fontification.
+
 @node racket-documentation-search-location
 @subsection racket-documentation-search-location
 
@@ -2991,18 +3015,34 @@ string should be a properly encoded URL.
 
 An association list from color-lexer token symbols to face symbols.
 
-Note: In many Racket languages, tokens for identifiers are lexed
-as ``symbol'', and in many programs, identifiers are a majority
-of the source. Using the Emacs face @code{default} is ``less noisy''
-and closer to the appearance in Dr Racket. However you could
-plausibly choose something like @code{font-lock-variable-face}.
+Note: In many Racket languages, the lexer classifies tokens for
+as `symbol. In many programs, a majority of the source
+consists of identifiers at binding definition and use sites.
+Therefore the appearance of ``symbol'' tokens is significant, and
+a matter of personal preference.
+
+@itemize
+@item
+If you prefer a ``plainer'' appearance, similar to Dr Racket:
+Add `symbol with the value `default. This gives an
+explicit `face property, which overrides any
+`font-lock-face property that a minor mode might apply to
+enhance the basic fontification.
+
+@item
+If you prefer a more ``colorful'' appearance, similar to
+``classic'' @ref{racket-mode}: Do @emph{not} map `symbol tokens in
+this list. Instead enable @ref{racket-xp-mode} and let it do
+``semantic'' highlighting of bindings; see the customization
+variable @ref{racket-xp-binding-font-lock-face-modes}.
+@end itemize
 
 Note: Some tokens are hardwired and not customizable by this
 list: Comment tokens use the face @code{font-lock-comment-face},
 sometimes blended with other faces. Parenthesis tokens use the
-face @code{paren} if defined. String tokens use
-@code{font-lock-string-face}. Text tokens, e.g. Scribble text, use the
-default face.
+face @code{parenthesis} if defined, as by the paren-face package.
+String tokens use @code{font-lock-string-face}. Text tokens, e.g.
+Scribble text, use the face @code{default}
 
 @node racket-hash-lang-module-language-hook
 @subsection racket-hash-lang-module-language-hook
@@ -3784,6 +3824,8 @@ A value for the variable @ref{racket-shell-or-terminal-function}.
 * racket-xp-unused-face::
 * racket-xp-tail-target-face::
 * racket-xp-tail-position-face::
+* racket-xp-binding-def-font-lock-face::
+* racket-xp-binding-use-font-lock-face::
 * racket-logger-config-face::
 * racket-logger-topic-face::
 * racket-logger-fatal-face::
@@ -3830,12 +3872,12 @@ Face for here strings.
 @node racket-xp-def-face
 @subsection racket-xp-def-face
 
-Face @ref{racket-xp-mode} uses to highlight definitions.
+Face @ref{racket-xp-mode} uses when point is on a definition.
 
 @node racket-xp-use-face
 @subsection racket-xp-use-face
 
-Face @ref{racket-xp-mode} uses to highlight uses.
+Face @ref{racket-xp-mode} uses when point is on a use.
 
 @node racket-xp-unused-face
 @subsection racket-xp-unused-face
@@ -3852,6 +3894,20 @@ Face @ref{racket-xp-mode} uses to highlight targets of a tail position.
 
 Face @ref{racket-xp-mode} uses to highlight expressions in a tail position.
 
+@node racket-xp-binding-def-font-lock-face
+@subsection racket-xp-binding-def-font-lock-face
+
+Value of the font-lock-face property @ref{racket-xp-mode} gives to definition sites.
+
+See the variable @ref{racket-xp-binding-font-lock-face-modes}.
+
+@node racket-xp-binding-use-font-lock-face
+@subsection racket-xp-binding-use-font-lock-face
+
+Value of the font-lock-face @ref{racket-xp-mode} gives to use sites.
+
+See the variable @ref{racket-xp-binding-font-lock-face-modes}.
+
 @node racket-logger-config-face
 @subsection racket-logger-config-face
 

racket-custom.el

@@ -146,6 +146,24 @@ an underline, which is a common convention."
   :safe #'stringp
   :group 'racket-xp)
 
+(defcustom racket-xp-binding-font-lock-face-modes '(racket-hash-lang-mode)
+  "Major modes where `racket-xp-mode' will fontify binding identifier sites.
+
+A \\='font-lock-face property is added with the value:
+
+  - `racket-xp-binding-def-font-lock-face' for each site that is
+    a binding definition identifier.
+
+  - `racket-xp-binding-use-font-lock-face' for each site that is
+    a use of an /imported/ binding identifier.
+
+This has a visible effect only when there is /not/ also a
+\\='face property applied by the major mode's fontification."
+  :tag "Racket Xp Mode Binding Font Lock Face Modes"
+  :type '(repeat symbol)
+  :safe #'listp
+  :group 'racket-xp)
+
 ;;; Hash Lang
 
 (defgroup racket-hash-lang nil
@@ -155,7 +173,6 @@ an underline, which is a common convention."
 
 (defcustom racket-hash-lang-token-face-alist
   `((constant           . font-lock-constant-face)
-    (symbol             . default)
     (error              . error)
     (other              . font-lock-doc-face)
     (keyword            . font-lock-keyword-face)
@@ -164,18 +181,30 @@ an underline, which is a common convention."
     (operator           . font-lock-variable-name-face))
   "An association list from color-lexer token symbols to face symbols.
 
-Note: In many Racket languages, tokens for identifiers are lexed
-as \"symbol\", and in many programs, identifiers are a majority
-of the source. Using the Emacs face `default' is \"less noisy\"
-and closer to the appearance in Dr Racket. However you could
-plausibly choose something like `font-lock-variable-face'.
+Note: In many Racket languages, the lexer classifies tokens for
+as \\='symbol. In many programs, a majority of the source
+consists of identifiers at binding definition and use sites.
+Therefore the appearance of \"symbol\" tokens is significant, and
+a matter of personal preference.
+
+  - If you prefer a \"plainer\" appearance, similar to Dr Racket:
+    Add \\='symbol with the value \\='default. This gives an
+    explicit \\='face property, which overrides any
+    \\='font-lock-face property that a minor mode might apply to
+    enhance the basic fontification.
+
+  - If you prefer a more \"colorful\" appearance, similar to
+    \"classic\" `racket-mode': Do /not/ map \\='symbol tokens in
+    this list. Instead enable `racket-xp-mode' and let it do
+    \"semantic\" highlighting of bindings; see the customization
+    variable `racket-xp-binding-font-lock-face-modes'.
 
 Note: Some tokens are hardwired and not customizable by this
 list: Comment tokens use the face `font-lock-comment-face',
 sometimes blended with other faces. Parenthesis tokens use the
-face `paren' if defined. String tokens use
-`font-lock-string-face'. Text tokens, e.g. Scribble text, use the
-default face."
+face `parenthesis' if defined, as by the paren-face package.
+String tokens use `font-lock-string-face'. Text tokens, e.g.
+Scribble text, use the face `default'"
   :tag "Hash Lang Token Face Alist"
   :type '(alist :key-type symbol :value-type face)
   :safe #'listp
@@ -550,14 +579,28 @@ ignore POS. Examples: `racket-show-echo-area' and
 
 (defface-racket racket-xp-def-face
   '((t (:inherit match :underline (:style line))))
-  "Face `racket-xp-mode' uses to highlight definitions."
+  "Face `racket-xp-mode' uses when point is on a definition."
   "Definition Face")
 
 (defface-racket racket-xp-use-face
   '((t (:inherit match)))
-  "Face `racket-xp-mode' uses to highlight uses."
+  "Face `racket-xp-mode' uses when point is on a use."
   "Use Face")
 
+(defface-racket racket-xp-binding-def-font-lock-face
+  '((t (:inherit font-lock-variable-name-face)))
+  "Value of the font-lock-face property `racket-xp-mode' gives to definition sites.
+
+See the variable `racket-xp-binding-font-lock-face-modes'."
+  "Binding Definition Font Lock Face")
+
+(defface-racket racket-xp-binding-use-font-lock-face
+  '((t (:inherit font-lock-keyword-face)))
+  "Value of the font-lock-face `racket-xp-mode' gives to use sites.
+
+See the variable `racket-xp-binding-font-lock-face-modes'."
+  "Binding Use Font Lock Face")
+
 (defface-racket racket-xp-error-face
   '((t (:underline (:color "red" :style wave))))
   "Face `racket-xp-mode' uses to highlight errors."