Note [Whitespace-sensitive operator parsing]

GHC/Parser/Lexer.x:637 compiler 1 ticket

In accord with GHC Proposal #229 https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0229-whitespace-bang-patterns.rst
we classify operator occurrences into four categories:

    a ! b   -- a loose infix occurrence
    a!b     -- a tight infix occurrence
    a !b    -- a prefix occurrence
    a! b    -- a suffix occurrence

The rules are a bit more elaborate than simply checking for whitespace, in
order to accommodate the following use cases:

    f (!a) = ...    -- prefix occurrence
    g (a !)         -- loose infix occurrence
    g (! a)         -- loose infix occurrence

The precise rules are as follows:

 * Identifiers, literals, and opening brackets (, (#, (|, [, [|, [||, [p|,
   [e|, [t|, {, ⟦, ⦇, are considered "opening tokens". The function
   followedByOpeningToken tests whether the next token is an opening token.

 * Identifiers, literals, and closing brackets ), #), |), ], |], }, ⟧, ⦈,
   are considered "closing tokens". The function precededByClosingToken tests
   whether the previous token is a closing token.

 * Whitespace, comments, separators, and other tokens, are considered
   neither opening nor closing.

 * Any unqualified operator occurrence is classified as prefix, suffix, or
   tight/loose infix, based on preceding and following tokens:

      precededByClosingToken | followedByOpeningToken | Occurrence
     ------------------------+------------------------+------------
      False                  | True                   | prefix
      True                   | False                  | suffix
      True                   | True                   | tight infix
      False                  | False                  | loose infix
     ------------------------+------------------------+------------

A loose infix occurrence is always considered an operator. Other types of
occurrences may be assigned a special per-operator meaning override:

  Operator |  Occurrence   | Token returned
 ----------+---------------+------------------------------------------
   !       |  prefix       | ITbang
           |               |   strictness annotation or bang pattern,
           |               |   e.g.  f !x = rhs, data T = MkT !a
           |  not prefix   | ITvarsym "!"
           |               |   ordinary operator or type operator,
           |               |   e.g.  xs ! 3, (! x), Int ! Bool
 ----------+---------------+------------------------------------------
   ~       |  prefix       | ITtilde
           |               |   laziness annotation or lazy pattern,
           |               |   e.g.  f ~x = rhs, data T = MkT ~a
           |  not prefix   | ITvarsym "~"
           |               |   ordinary operator or type operator,
           |               |   e.g.  xs ~ 3, (~ x), Int ~ Bool
 ----------+---------------+------------------------------------------
   .       |  prefix       | ITproj True
           |               |   field projection,
           |               |   e.g.  .x
           |  tight infix  | ITproj False
           |               |   field projection,
           |               |   e.g. r.x
           |  suffix       | ITdot
           |               |   function composition,
           |               |   e.g. f. g
           |  loose infix  | ITdot
           |               |   function composition,
           |               |   e.g.  f . g
 ----------+---------------+------------------------------------------
   $  $$   |  prefix       | ITdollar, ITdollardollar
           |               |   untyped or typed Template Haskell splice,
           |               |   e.g.  $(f x), $$(f x), $$"str"
           |  not prefix   | ITvarsym "$", ITvarsym "$$"
           |               |   ordinary operator or type operator,
           |               |   e.g.  f $ g x, a $$ b
 ----------+---------------+------------------------------------------
   @       |  prefix       | ITtypeApp
           |               |   type application, e.g.  fmap @Maybe
           |  tight infix  | ITat
           |               |   as-pattern, e.g.  f p@(a,b) = rhs
           |  suffix       | parse error
           |               |   e.g. f p@ x = rhs
           |  loose infix  | ITvarsym "@"
           |               |   ordinary operator or type operator,
           |               |   e.g.  f @ g, (f @)
 ----------+---------------+------------------------------------------

Also, some of these overrides are guarded behind language extensions.
According to the specification, we must determine the occurrence based on
surrounding *tokens* (see the proposal for the exact rules). However, in
the implementation we cheat a little and do the classification based on
characters, for reasons of both simplicity and efficiency (see
'followedByOpeningToken' and 'precededByClosingToken')

When an operator is subject to a meaning override, it is mapped to special
token: ITbang, ITtilde, ITat, ITdollar, ITdollardollar. Otherwise, it is
returned as ITvarsym.

For example, this is how we process the (!):

   precededByClosingToken | followedByOpeningToken | Token
  ------------------------+------------------------+-------------
   False                  | True                   | ITbang
   True                   | False                  | ITvarsym "!"
   True                   | True                   | ITvarsym "!"
   False                  | False                  | ITvarsym "!"
  ------------------------+------------------------+-------------

And this is how we process the (@):

   precededByClosingToken | followedByOpeningToken | Token
  ------------------------+------------------------+-------------
   False                  | True                   | ITtypeApp
   True                   | False                  | parse error
   True                   | True                   | ITat
   False                  | False                  | ITvarsym "@"
  ------------------------+------------------------+-------------

References 0

This Note does not link to any other.

Referenced by 14