Note [Object determinism]
Object determinism means that GHC, for the same exact input, produces, deterministically, byte-for-byte identical objects (.o files, executables, libraries...) on separate multi-threaded runs. Deterministic objects are critical, for instance, for reproducible software packaging and distribution, or build systems with content-sensitive recompilation avoidance. The main cause of non-determinism in objects comes from the non-deterministic uniques leaking into the generated code. Apart from uniques previously affecting determinism both directly by showing up in symbol labels and indirectly, e.g. in the CLabel Ord instance, GHC already did a lot deterministically (modulo bugs) by the time we set out to achieve full object determinism: * The Simplifier is deterministic in the optimisations it applies (c.f. #25170) * Interface files are deterministic (which depends on the previous bullet) * The Cmm/NCG pipeline processes sections in a deterministic order, so the final object sections, closures, data, etc., are already always outputted in the same order for the same module. Beyond fixing small bugs in the above bullets and other smaller non-determinism leaks like the Ord instance of CLabels, we must ensure that/do the following to make GHC produce fully deterministic objects: * In STG -> Cmm, deterministically /rename/ all non-external uniques in the Cmm chunk, deterministically, before yielding. See Note [Renaming uniques deterministically] in GHC.Cmm.UniqueRenamer. This pass is necessary for object determinism but is currently guarded by -fobject-determinism. * Multiple Cmm passes work with non-deterministic @LabelMap@s -- that doesn't change since they are both important for performance and do not affect the determinism of the end result. As after the renaming pass the uniques are all produced deterministically, the orderings observable by the map are also going to be deterministic. In the brief period before a CmmGroup has been renamed, a list instead of LabelMap is used to preserve the ordering. See Note [DCmmGroup vs CmmGroup or: Deterministic Info Tables] in GHC.Cmm. * In the code generation pipeline from Cmm onwards, when new uniques need to be created for a given pass, use @UniqDSM@ instead of the previously used @UniqSM@. @UniqDSM@ supplies uniques iteratively, guaranteeing uniques produced by the backend are deterministic accross runs. See Note [Deterministic Uniques in the CG] in GHC.Types.Unique.DSM. Also, c.f. Note [Unique Determinism]
References 4
- Renaming uniques deterministically GHC.Cmm.UniqueRenamer
- DCmmGroup vs CmmGroup or: Deterministic Info Tables GHC.Cmm
- Deterministic Uniques in the CG GHC.Types.Unique.DSM
- Unique Determinism GHC.Types.Unique
Referenced by 9
- GHC.Cmm.CLabel call site ×2
- Deterministic Uniques in the CG GHC.Types.Unique.DSM ×2
- DCmmGroup vs CmmGroup or: Deterministic Info Tables GHC.Cmm
- Renaming uniques deterministically GHC.Cmm.UniqueRenamer
- GHC.StgToCmm call site
- GHC.StgToCmm.CgUtils call site
- Unique Determinism and code generation GHC.Types.Unique