Chapter 7

Core: the language

A typed lambda calculus small enough to fit on a page, with a type system strong enough to catch the optimiser's mistakes. Why GHC can be aggressive without being reckless.

Where this lives in the tree

Core is the reason GHC optimises well. Not because of any single transformation, but because a language this small makes transformations cheap to write and cheap to trust.

The whole expression language:

data Expr b
  = Var   Id
  | Lit   Literal
  | App   (Expr b) (Arg b)
  | Lam   b (Expr b)
  | Let   (Bind b) (Expr b)
  | Case  (Expr b) b Type [Alt b]
  | Cast  (Expr b) CoercionR
  | Tick  CoreTickish (Expr b)
  | Type  Type
  | Coercion Coercion

A transformation must handle nine cases. Compare that with the surface language, where every new extension adds syntax that every pass would have to know about. Haskell can grow; Core stays put.

It is explicitly typed

Core is not an untyped intermediate form. It is System FC (System F with coercions), and every expression carries enough information to reconstruct its type without inference. Type abstraction is a Lam binding a type variable; type application is an App whose argument is a Type.

This is why the dumps are so verbose, and it buys something specific: the optimiser’s output can be checked.

Note [GHC Formalism] GHC/Core/Lint.hs:127
This file implements the type-checking algorithm for System FC, the "official"
name of the Core language. Type safety of FC is heart of the claim that
executables produced by GHC do not have segmentation faults. Thus, it is
useful to be able to reason about System FC independently of reading the code.
To this purpose, there is a document core-spec.pdf built in docs/core-spec that
contains a formalism of the types and functions dealt with here. If you change
just about anything in this file or you change other types/functions throughout
the Core language (all signposted to this note), you should update that
formalism. See docs/core-spec/README for more info about how to do so.

Core Lint is a type checker for Core, run between optimisation passes with -dcore-lint. If a transformation produces something ill-typed, Lint says so, naming the pass. A whole category of optimiser bug (the kind that silently miscompiles) becomes a loud failure during development instead.

That Note is worth reading as a statement of engineering philosophy: the formalism exists so that “is this transformation correct?” has an answer that a machine can check, not merely an argument that a person can make.

Coercions

The Cast constructor and the Coercion type are what let Core keep its types honest across newtypes, GADTs and type families. A coercion is evidence that two types are equal, and it is a term: you can build one, name one, pass one around.

Cast e co says: e had one type, co proves that type equals another, so this expression has the other. Crucially, Cast has no runtime cost. It is erased before code generation. Newtypes are free precisely because their coercions are.

Two roles of equality matter. Nominal equality means the types are the same type. Representational equality means they have the same runtime representation, which is what a newtype gives you, and what Coercible exposes to programmers.

Invariants

Core’s smallness is only half the story; the other half is that its terms obey invariants the optimiser may assume. They are documented, and violating one is a bug even if the result still type-checks.

Note [Case expression invariants] GHC/Core.hs:562
Case expressions are one of the more complicated elements of the Core
language, and come with a number of invariants.  All of them should be
checked by Core Lint.

1. The list of alternatives may be empty;
   See Note [Empty case alternatives]

2. The 'DEFAULT' case alternative must be first in the list,
   if it occurs at all.  Checked in GHC.Core.Lint.checkCaseAlts.

3. The remaining cases are in order of (strictly) increasing
     tag  (for 'DataAlts') or
     lit  (for 'LitAlts').
   This makes finding the relevant constructor easy, and makes
   comparison easier too.   Checked in GHC.Core.Lint.checkCaseAlts.

4. The list of alternatives must be exhaustive. An /exhaustive/ case
   does not necessarily mention all constructors:

   @
        data Foo = Red | Green | Blue
        ... case x of
              Red   -> True
              other -> f (case x of
                              Green -> ...
                              Blue  -> ... ) ...
Show the rest of this Note (40 more lines)
   @

   The inner case does not need a @Red@ alternative, because @x@
   can't be @Red@ at that program point.

   This is not checked by Core Lint -- it's very hard to do so.
   E.g. suppose that inner case was floated out, thus:
         let a = case x of
                   Green -> ...
                   Blue  -> ... )
         case x of
           Red   -> True
           other -> f a
   Now it's really hard to see that the Green/Blue case is
   exhaustive.  But it is.

   If you have a case-expression that really /isn't/ exhaustive,
   we may generate seg-faults.  Consider the Green/Blue case
   above.  Since there are only two branches we may generate
   code that tests for Green, and if not Green simply /assumes/
   Blue (since, if the case is exhaustive, that's all that
   remains).  Of course, if it's not Blue and we start fetching
   fields that should be in a Blue constructor, we may die
   horribly. See also Note [Core Lint guarantee] in GHC.Core.Lint.

5. Floating-point values must not be scrutinised against literals.
   See #9238 and Note [Rules for floating-point comparisons]
   in GHC.Core.Opt.ConstantFold for rationale.  Checked in lintCaseExpr;
   see the call to isFloatingPrimTy.

6. The 'ty' field of (Case scrut bndr ty alts) is the type of the
   /entire/ case expression.  Checked in lintAltExpr.
   See also Note [Why does Case have a 'Type' field?].

7. The type of the scrutinee must be the same as the type
   of the case binder, obviously.  Checked in lintCaseExpr.

8. The multiplicity of the binders in constructor patterns must be the
   multiplicity of the corresponding field /scaled by the multiplicity of the
   case binder/. Checked in lintCoreAlt.

Join points are the most interesting of these. A join point is a let-bound function that is always tail-called: the Core representation of a label you jump to rather than a closure you allocate. Recognising them lets GHC compile a case-heavy loop into an actual loop:

Note [Invariants on join points] GHC/Core.hs:809
Join points must follow these invariants:

  1. All occurrences must be tail calls. Each of these tail calls must pass the
     same number of arguments, counting both types and values; we call this the
     "join arity" (to distinguish from regular arity, which only counts values).

     See Note [Join points are less general than the paper]

  2. For join arity n, the right-hand side must begin with at least n lambdas.
     No ticks, no casts, just lambdas!  C.f. GHC.Core.Utils.joinRhsArity.

     2a. Moreover, this same constraint applies to any unfolding of
         the binder.  Reason: if we want to push a continuation into
         the RHS we must push it into the unfolding as well.

     2b. The Arity (in the IdInfo) of a join point varies independently of the
         join-arity. For example, we could have
             j x = case x of { T -> \y.y; F -> \y.3 }
         Its join-arity is 1, but its idArity is 2; and we do not eta-expand
         join points: see Note [Do not eta-expand join points] in
                          GHC.Core.Opt.Simplify.Utils.

         Allowing the idArity to be bigger than the join-arity is
         important in arityType; see GHC.Core.Opt.Arity
         Note [Arity for recursive join bindings]
Show the rest of this Note (53 more lines)
         Historical note: see #17294.

  3. If the binding is recursive, then all other bindings in the recursive group
     must also be join points.

  4. The binding's type must not be polymorphic in its return type (as defined
     in Note [The polymorphism rule of join points]).

However, join points have simpler invariants in other ways

  5. A join point can have an unboxed type without the RHS being
     ok-for-speculation (i.e. drop the let-can-float invariant)
     e.g.  let j :: Int# = factorial x in ...

  6. The RHS of join point is not required to have a fixed runtime representation,
     e.g.  let j :: r :: TYPE l = fail (##) in ...
     This happened in an intermediate program #13394

Examples:

  join j1  x = 1 + x in jump j (jump j x)  -- Fails 1: non-tail call
  join j1' x = 1 + x in if even a
                          then jump j1 a
                          else jump j1 a b -- Fails 1: inconsistent calls
  join j2  x = flip (+) x in j2 1 2        -- Fails 2: not enough lambdas
  join j2' x = \y -> x + y in j3 1         -- Passes: extra lams ok
  join j @a (x :: a) = x                   -- Fails 4: polymorphic in ret type

Invariant 1 applies to left-hand sides of rewrite rules, so a rule for a join
point must have an exact call as its LHS.

Strictly speaking, invariant 3 is redundant, since a call from inside a lazy
binding isn't a tail call. Since a let-bound value can't invoke a free join
point, then, they can't be mutually recursive. (A Core binding group *can*
include spurious extra bindings if the occurrence analyser hasn't run, so
invariant 3 does still need to be checked.) For the rigorous definition of
"tail call", see Section 3 of the paper (Note [Join points]).

Invariant 4 is subtle; see Note [The polymorphism rule of join points].

Invariant 6 is to enable code like this:

  f = \(r :: RuntimeRep) (a :: TYPE r) (x :: T).
      join j :: a
           j = error @r @a "bloop"
      in case x of
           A -> j
           B -> j
           C -> error @r @a "blurp"

Core Lint will check these invariants, anticipating that any binder whose
OccInfo is marked AlwaysTailCalled will become a join point as soon as the
simplifier (or simpleOptPgm) runs.

And representation polymorphism has its own rule, because code generation must know how many bits a binder occupies:

Note [Representation polymorphism invariants] GHC/Core.hs:652
GHC allows us to abstract over calling conventions using **representation polymorphism**.
For example, we have:

  ($) :: forall (r :: RuntimeRep) (a :: Type) (b :: TYPE r). (a -> b) -> a -> b

In this example, the type `b` is representation-polymorphic: it has kind `TYPE r`,
where the type variable `r :: RuntimeRep` abstracts over the runtime representation
of values of type `b`.

To ensure that programs containing representation-polymorphism remain compilable,
we enforce the following representation-polymorphism invariants:

The paper "Levity Polymorphism" [PLDI'17] states the first two invariants:

  I1. The type of a bound variable must have a fixed runtime representation
      (except for join points: See Note [Invariants on join points])
  I2. The type of a function argument must have a fixed runtime representation.

Example of I1:

  \(r::RuntimeRep). \(a::TYPE r). \(x::a). e
Show the rest of this Note (35 more lines)
    This contravenes I1 because x's type has kind (TYPE r), which has 'r' free.
    We thus wouldn't know how to compile this lambda abstraction.

Example of I2:

  f (undefined :: (a :: TYPE r))

    This contravenes I2: we are applying the function `f` to a value
    with an unknown runtime representation.

Note that these two invariants require us to check other types than just the
types of bound variables and types of function arguments, due to transformations
that GHC performs. For example, the definition

  myCoerce :: forall {r} (a :: TYPE r) (b :: TYPE r). Coercible a b => a -> b
  myCoerce = coerce

is invalid, because `coerce` has no binding (see GHC.Types.Id.Make.coerceId).
So, before code-generation, GHC saturates the RHS of 'myCoerce' by performing
an eta-expansion (see GHC.CoreToStg.Prep.maybeSaturate):

  myCoerce = \ (x :: TYPE r) -> coerce x

However, this transformation would be invalid, because now the binding of x
in the lambda abstraction would violate I1.

See Note [Representation-polymorphism checking built-ins] in GHC.Tc.Utils.Concrete
and Note [Linting representation-polymorphic builtins] in GHC.Core.Lint for
more details.

Note that we currently require something slightly stronger than a fixed runtime
representation: we check whether bound variables and function arguments have a
/fixed RuntimeRep/ in the sense of Note [Fixed RuntimeRep] in GHC.Tc.Utils.Concrete.
See Note [Representation polymorphism checking] in GHC.Tc.Utils.Concrete
for an overview of how we enforce these invariants in the typechecker.

Reading the source yourself

  1. GHC/Core.hs: the type, then its Notes, which are among the best-written in the compiler. Budget real time for them; they are the specification.
  2. GHC/Core/Lint.hs next. Reading the checker is the fastest way to learn the invariants, because it is the invariants, executable.
  3. GHC/Core/Coercion.hs when you first hit a Cast you do not understand.
  4. docs/core-spec/ in the GHC tree contains a typeset formal specification of Core. If you like your semantics on paper, start there instead.

Run anything you are unsure about with -ddump-simpl -dsuppress-all for a readable view, then remove the suppression flags one at a time to see what was hidden. The handbook’s example explorer does exactly this with its detail toggle.