\chapter{Syntax primitives}

 The rather generic framework of Isabelle/Isar syntax emerges from three main

 syntactic categories: \emph{commands} of the top-level Isar engine (covering

 theory and proof elements), \emph{methods} for general goal refinements

 (analogous to traditional ``tactics''), and \emph{attributes} for operations

 on facts (within a certain context).  Here we give a reference of basic

 syntactic entities underlying Isabelle/Isar syntax in a bottom-up manner.

 Concrete theory and proof language elements will be introduced later on.

 \medskip

 In order to get started with writing well-formed Isabelle/Isar documents, the

 most important aspect to be noted is the difference of \emph{inner} versus

 \emph{outer} syntax.  Inner syntax is that of Isabelle types and terms of the

 logic, while outer syntax is that of Isabelle/Isar theory sources (including

 proofs).  As a general rule, inner syntax entities may occur only as

 \emph{atomic entities} within outer syntax.  For example, the string

 \texttt{"x + y"} and identifier \texttt{z} are legal term specifications

 within a theory, while \texttt{x + y} is not.

 \begin{warn}

   Old-style Isabelle theories used to fake parts of the inner syntax of types,

   with rather complicated rules when quotes may be omitted.  Despite the minor

   drawback of requiring quotes more often, the syntax of Isabelle/Isar is

   somewhat simpler and more robust in that respect.

 \end{warn}

 Printed theory documents usually omit quotes to gain readability (this is a

 matter of {\LaTeX} macro setup, say via \verb,\isabellestyle,, see also

 \cite{isabelle-sys}).  Experienced users of Isabelle/Isar may easily

 reconstruct the lost technical information, while mere readers need not care

 about quotes at all.

 \medskip

 Isabelle/Isar input may contain any number of input termination characters

 ``\texttt{;}'' (semicolon) to separate commands explicitly.  This is

 particularly useful in interactive shell sessions to make clear where the

 current command is intended to end.  Otherwise, the interpreter loop will

 continue to issue a secondary prompt ``\verb,#,'' until an end-of-command is

 clearly recognized from the input syntax, e.g.\ encounter of the next command

 keyword.

 Advanced interfaces such as Proof~General \cite{proofgeneral} do not require

 explicit semicolons, the amount of input text is determined automatically by

 inspecting the present content of the Emacs text buffer.  In the printed

 presentation of Isabelle/Isar documents semicolons are omitted altogether for

 readability.

 \begin{warn}

   Proof~General requires certain syntax classification tables in order to

   achieve properly synchronized interaction with the Isabelle/Isar process.

   These tables need to be consistent with the Isabelle version and particular

   logic image to be used in a running session (common object-logics may well

   change the outer syntax).  The standard setup should work correctly with any

   of the ``official'' logic images derived from Isabelle/HOL (including HOLCF

   etc.).  Users of alternative logics may need to tell Proof~General

   explicitly, e.g.\ by giving an option \verb,-k ZF, (in conjunction with

   \verb,-l ZF, to specify the default logic image).

 \end{warn}

 \section{Lexical matters}\label{sec:lex-syntax}

 The Isabelle/Isar outer syntax provides token classes as presented below.

 Note that some of these coincide (by full intention) with the inner lexical

 syntax as presented in \cite{isabelle-ref}.

 \indexoutertoken{ident}\indexoutertoken{longident}\indexoutertoken{symident}

 \indexoutertoken{nat}\indexoutertoken{var}\indexoutertoken{typefree}

 \indexoutertoken{typevar}\indexoutertoken{string}\indexoutertoken{verbatim}

 \begin{matharray}{rcl}

   ident & = & letter~quasiletter^* \\

   longident & = & ident\verb,.,ident~\dots~ident \\

   symident & = & sym^+ ~|~ symbol \\

   nat & = & digit^+ \\

   var & = & \verb,?,ident ~|~ \verb,?,ident\verb,.,nat \\

   typefree & = & \verb,',ident \\

   typevar & = & \verb,?,typefree ~|~ \verb,?,typefree\verb,.,nat \\

   string & = & \verb,", ~\dots~ \verb,", \\

   verbatim & = & \verb,{*, ~\dots~ \verb,*}, \\

 \end{matharray}

 \begin{matharray}{rcl}

   letter & = & \verb,a, ~|~ \dots ~|~ \verb,z, ~|~ \verb,A, ~|~ \dots ~|~ \verb,Z, \\

   digit & = & \verb,0, ~|~ \dots ~|~ \verb,9, \\

   quasiletter & = & letter ~|~ digit ~|~ \verb,_, ~|~ \verb,', \\

   sym & = & \verb,!, ~|~ \verb,#, ~|~ \verb,$, ~|~ \verb,%, ~|~ \verb,&, ~|~  %$

    \verb,*, ~|~ \verb,+, ~|~ \verb,-, ~|~ \verb,/, ~|~ \verb,:, ~|~ \\

   & & \verb,<, ~|~ \verb,=, ~|~ \verb,>, ~|~ \verb,?, ~|~ \texttt{\at} ~|~

   \verb,^, ~|~ \verb,_, ~|~ \verb,`, ~|~ \verb,|, ~|~ \verb,~, \\

   symbol & = & {\forall} ~|~ {\exists} ~|~ {\land} ~|~ {\lor} ~|~ \dots

 \end{matharray}

 The syntax of $string$ admits any characters, including newlines; ``\verb|"|''

 (double-quote) and ``\verb|\|'' (backslash) need to be escaped by a backslash.

 Note that ML-style control characters are \emph{not} supported.  The body of

 $verbatim$ may consist of any text not containing ``\verb|*}|''; this allows

 convenient inclusion of quotes without further escapes.

 Comments take the form \texttt{(*~\dots~*)} and may in principle be nested,

 just as in ML.  Note that these are \emph{source} comments only, which are

 stripped after lexical analysis of the input.  The Isar document syntax also

 provides \emph{formal comments} that are considered as part of the text (see

 \S\ref{sec:comments}).

 \begin{warn}

   Proof~General does not handle nested comments properly; it is also unable to

   keep \verb,(*,\,/\,\verb,{*, and \verb,*),\,/\,\verb,*}, apart, despite

   their rather different meaning.  These are inherent problems of Emacs

   legacy.  Users should not be overly aggressive about nesting or alternating

   these delimiters.

 \end{warn}

 \medskip

 Mathematical symbols such as ``$\forall$'' are represented in plain ASCII as

 ``\verb,\<forall>,''.  Concerning Isabelle itself, any sequence of the form

 \verb,\<,$ident$\verb,>, (or \verb,\\<,$ident$\verb,>,) is a legal symbol.

 Display of appropriate glyphs is a matter of front-end tools, say the

 user-interface of Proof~General plus the X-Symbol package, or the {\LaTeX}

 macro setup of document output.  A list of predefined Isabelle symbols is

 given in \cite[appendix~A]{isabelle-sys}.

 \section{Common syntax entities}

 Subsequently, we introduce several basic syntactic entities, such as names,

 terms, and theorem specifications, which have been factored out of the actual

 Isar language elements to be described later.

 Note that some of the basic syntactic entities introduced below (e.g.\ 

 \railqtok{name}) act much like tokens rather than plain nonterminals (e.g.\ 

 \railnonterm{sort}), especially for the sake of error messages.  E.g.\ syntax

 elements like $\CONSTS$ referring to \railqtok{name} or \railqtok{type} would

 really report a missing name or type rather than any of the constituent

 primitive tokens such as \railtok{ident} or \railtok{string}.

 \subsection{Names}

 Entity \railqtok{name} usually refers to any name of types, constants,

 theorems etc.\ that are to be \emph{declared} or \emph{defined} (so qualified

 identifiers are excluded here).  Quoted strings provide an escape for

 non-identifier names or those ruled out by outer syntax keywords (e.g.\ 

 \verb|"let"|).  Already existing objects are usually referenced by

 \railqtok{nameref}.

 \indexoutertoken{name}\indexoutertoken{parname}\indexoutertoken{nameref}

 \indexoutertoken{int}

 \begin{rail}

   name: ident | symident | string | nat

   ;

   parname: '(' name ')'

   ;

   nameref: name | longident

   ;

   int: nat | '-' nat

   ;

 \end{rail}

 \subsection{Comments}\label{sec:comments}

 Large chunks of plain \railqtok{text} are usually given \railtok{verbatim},

 i.e.\ enclosed in \verb|{*|~\dots~\verb|*}|.  For convenience, any of the

 smaller text units conforming to \railqtok{nameref} are admitted as well.  A

 marginal \railnonterm{comment} is of the form \texttt{--} \railqtok{text}.

 Any number of these may occur within Isabelle/Isar commands.

 \indexoutertoken{text}\indexouternonterm{comment}

 \begin{rail}

   text: verbatim | nameref

   ;

   comment: '--' text

   ;

 \end{rail}

 \subsection{Type classes, sorts and arities}

 Classes are specified by plain names.  Sorts have a very simple inner syntax,

 which is either a single class name $c$ or a list $\{c@1, \dots, c@n\}$

 referring to the intersection of these classes.  The syntax of type arities is

 given directly at the outer level.

 \railalias{subseteq}{\isasymsubseteq}

 \railterm{subseteq}

 \indexouternonterm{sort}\indexouternonterm{arity}\indexouternonterm{simplearity}

 \indexouternonterm{classdecl}

 \begin{rail}

   classdecl: name (('<' | subseteq) (nameref + ','))?

   ;

   sort: nameref

   ;

   arity: ('(' (sort + ',') ')')? sort

   ;

   simplearity: ('(' (sort + ',') ')')? nameref

   ;

 \end{rail}

 \subsection{Types and terms}\label{sec:types-terms}

 The actual inner Isabelle syntax, that of types and terms of the logic, is far

 too sophisticated in order to be modelled explicitly at the outer theory

 level.  Basically, any such entity has to be quoted to turn it into a single

 token (the parsing and type-checking is performed internally later).  For

 convenience, a slightly more liberal convention is adopted: quotes may be

 omitted for any type or term that is already atomic at the outer level.  For

 example, one may just write \texttt{x} instead of \texttt{"x"}.  Note that

 symbolic identifiers (e.g.\ \texttt{++} or $\forall$) are available as well,

 provided these have not been superseded by commands or other keywords already

 (e.g.\ \texttt{=} or \texttt{+}).

 \indexoutertoken{type}\indexoutertoken{term}\indexoutertoken{prop}

 \begin{rail}

   type: nameref | typefree | typevar

   ;

   term: nameref | var

   ;

   prop: term

   ;

 \end{rail}

 Positional instantiations are indicated by giving a sequence of terms, or the

 placeholder ``$\_$'' (underscore), which means to skip a position.

 \indexoutertoken{inst}\indexoutertoken{insts}

 \begin{rail}

   inst: underscore | term

   ;

   insts: (inst *)

   ;

 \end{rail}

 Type declarations and definitions usually refer to \railnonterm{typespec} on

 the left-hand side.  This models basic type constructor application at the

 outer syntax level.  Note that only plain postfix notation is available here,

 but no infixes.

 \indexouternonterm{typespec}

 \begin{rail}

   typespec: (() | typefree | '(' ( typefree + ',' ) ')') name

   ;

 \end{rail}

 \subsection{Mixfix annotations}

 Mixfix annotations specify concrete \emph{inner} syntax of Isabelle types and

 terms.  Some commands such as $\TYPES$ (see \S\ref{sec:types-pure}) admit

 infixes only, while $\CONSTS$ (see \S\ref{sec:consts}) and

 $\isarkeyword{syntax}$ (see \S\ref{sec:syn-trans}) support the full range of

 general mixfixes and binders.

 \indexouternonterm{infix}\indexouternonterm{mixfix}\indexouternonterm{structmixfix}

 \begin{rail}

   infix: '(' ('infix' | 'infixl' | 'infixr') string? nat ')'

   ;

   mixfix: infix | '(' string prios? nat? ')' | '(' 'binder' string prios? nat ')'

   ;

   structmixfix: mixfix | '(' 'structure' ')'

   ;

   prios: '[' (nat + ',') ']'

   ;

 \end{rail}

 Here the \railtok{string} specifications refer to the actual mixfix template

 (see also \cite{isabelle-ref}), which may include literal text, spacing,

 blocks, and arguments (denoted by ``$_$''); the special symbol \verb,\<index>,

 (printed as ``\i'') represents an index argument that specifies an implicit

 structure reference (see also \S\ref{sec:locale}).  Infix and binder

 declarations provide common abbreviations for particular mixfix declarations.

 So in practice, mixfix templates mostly degenerate to literal text for

 concrete syntax, such as ``\verb,++,'' for an infix symbol, or ``\verb,++,\i''

 for an infix of an implicit structure.

 \subsection{Proof methods}\label{sec:syn-meth}

 Proof methods are either basic ones, or expressions composed of methods via

 ``\texttt{,}'' (sequential composition), ``\texttt{|}'' (alternative choices),

 ``\texttt{?}'' (try), ``\texttt{+}'' (repeat at least once).  In practice,

 proof methods are usually just a comma separated list of

 \railqtok{nameref}~\railnonterm{args} specifications.  Note that parentheses

 may be dropped for single method specifications (with no arguments).

 \indexouternonterm{method}

 \begin{rail}

   method: (nameref | '(' methods ')') (() | '?' | '+')

   ;

   methods: (nameref args | method) + (',' | '|')

   ;

 \end{rail}

 Proper use of Isar proof methods does \emph{not} involve goal addressing.

 Nevertheless, specifying goal ranges may occasionally come in handy in

 emulating tactic scripts.  Note that $[n-]$ refers to all goals, starting from

 $n$.  All goals may be specified by $[!]$, which is the same as $[1-]$.

 \indexouternonterm{goalspec}

 \begin{rail}

   goalspec: '[' (nat '-' nat | nat '-' | nat | '!' ) ']'

   ;

 \end{rail}

 \subsection{Attributes and theorems}\label{sec:syn-att}

 Attributes (and proof methods, see \S\ref{sec:syn-meth}) have their own

 ``semi-inner'' syntax, in the sense that input conforming to

 \railnonterm{args} below is parsed by the attribute a second time.  The

 attribute argument specifications may be any sequence of atomic entities

 (identifiers, strings etc.), or properly bracketed argument lists.  Below

 \railqtok{atom} refers to any atomic entity, including any \railtok{keyword}

 conforming to \railtok{symident}.

 \indexoutertoken{atom}\indexouternonterm{args}\indexouternonterm{attributes}

 \begin{rail}

   atom: nameref | typefree | typevar | var | nat | keyword

   ;

   arg: atom | '(' args ')' | '[' args ']'

   ;

   args: arg *

   ;

   attributes: '[' (nameref args * ',') ']'

   ;

 \end{rail}

 Theorem specifications come in several flavors: \railnonterm{axmdecl} and

 \railnonterm{thmdecl} usually refer to axioms, assumptions or results of goal

 statements, while \railnonterm{thmdef} collects lists of existing theorems.

 Existing theorems are given by \railnonterm{thmref} and \railnonterm{thmrefs},

 the former requires an actual singleton result.  Any of these theorem

 specifications may include lists of attributes both on the left and right hand

 sides; attributes are applied to any immediately preceding fact.  If names are

 omitted, the theorems are not stored within the theorem database of the theory

 or proof context; any given attributes are still applied, though.

 \indexouternonterm{thmdecl}\indexouternonterm{axmdecl}

 \indexouternonterm{thmdef}\indexouternonterm{thmrefs}

 \begin{rail}

   axmdecl: name attributes? ':'

   ;

   thmdecl: thmbind ':'

   ;

   thmdef: thmbind '='

   ;

   thmref: nameref attributes?

   ;

   thmrefs: thmref +

   ;

   thmbind: name attributes | name | attributes

   ;

 \end{rail}

 \subsection{Term patterns and declarations}\label{sec:term-decls}

 Wherever explicit propositions (or term fragments) occur in a proof text,

 casual binding of schematic term variables may be given specified via patterns

 of the form ``$\ISS{p@1\;\dots}{p@n}$''.  There are separate versions

 available for \railqtok{term}s and \railqtok{prop}s.  The latter provides a

 $\CONCLNAME$ part with patterns referring the (atomic) conclusion of a rule.

 \indexouternonterm{termpat}\indexouternonterm{proppat}

 \begin{rail}

   termpat: '(' ('is' term +) ')'

   ;

   proppat: '(' (('is' prop +) | 'concl' ('is' prop +) | ('is' prop +) 'concl' ('is' prop +)) ')'

   ;

 \end{rail}

 Declarations of local variables $x :: \tau$ and logical propositions $a :

 \phi$ represent different views on the same principle of introducing a local

 scope.  In practice, one may usually omit the typing of $vars$ (due to

 type-inference), and the naming of propositions (due to implicit references of

 current facts).  In any case, Isar proof elements usually admit to introduce

 multiple such items simultaneously.

 \indexouternonterm{vars}\indexouternonterm{props}

 \begin{rail}

   vars: (name+) ('::' type)?

   ;

   props: thmdecl? (prop proppat? +)

   ;

 \end{rail}

 The treatment of multiple declarations corresponds to the complementary focus

 of $vars$ versus $props$: in ``$x@1~\dots~x@n :: \tau$'' the typing refers to

 all variables, while in $a\colon \phi@1~\dots~\phi@n$ the naming refers to all

 propositions collectively.  Isar language elements that refer to $vars$ or

 $props$ typically admit separate typings or namings via another level of

 iteration, with explicit $\AND$ separators; e.g.\ see $\FIXNAME$ and

 $\ASSUMENAME$ in \S\ref{sec:proof-context}.

 \subsection{Antiquotations}\label{sec:antiq}

 \begin{matharray}{rcl}

   thm & : & \isarantiq \\

   prop & : & \isarantiq \\

   term & : & \isarantiq \\

   typ & : & \isarantiq \\

   text & : & \isarantiq \\

   goals & : & \isarantiq \\

   subgoals & : & \isarantiq \\

 \end{matharray}

 The text body of formal comments (see also \S\ref{sec:comments}) may contain

 antiquotations of logical entities, such as theorems, terms and types, which

 are to be presented in the final output produced by the Isabelle document

 preparation system (see also \S\ref{sec:document-prep}).

 Thus embedding of

 ``\texttt{{\at}{\ttlbrace}term~[show_types]~"f(x)~=~a~+~x"{\ttrbrace}}''

 within a text block would cause

 \isa{(f{\isasymColon}'a~{\isasymRightarrow}~'a)~(x{\isasymColon}'a)~=~(a{\isasymColon}'a)~+~x}

 to appear in the final {\LaTeX} document.  Also note that theorem

 antiquotations may involve attributes as well.  For example,

 \texttt{{\at}{\ttlbrace}thm~sym~[no_vars]{\ttrbrace}} would print the

 statement where all schematic variables have been replaced by fixed ones,

 which are easier to read.

 \indexisarant{thm}\indexisarant{prop}\indexisarant{term}

 \indexisarant{typ}\indexisarant{text}\indexisarant{goals}\indexisarant{subgoals}

 \begin{rail}

   atsign lbrace antiquotation rbrace

   ;

   antiquotation:

     'thm' options thmrefs |

     'prop' options prop |

     'term' options term |

     'typ' options type |

     'text' options name |

     'goals' options |

     'subgoals' options

   ;

   options: '[' (option * ',') ']'

   ;

   option: name | name '=' name

   ;

 \end{rail}

 Note that the syntax of antiquotations may \emph{not} include source comments

 \texttt{(*~\dots~*)} or verbatim text \verb|{*|~\dots~\verb|*}|.

 \begin{descr}

 \item [$\at\{thm~\vec a\}$] prints theorems $\vec a$. Note that attribute

   specifications may be included as well (see also \S\ref{sec:syn-att}); the

   $no_vars$ operation (see \S\ref{sec:misc-meth-att}) would be particularly

   useful to suppress printing of schematic variables.

 \item [$\at\{prop~\phi\}$] prints a well-typed proposition $\phi$.

 \item [$\at\{term~t\}$] prints a well-typed term $t$.

 \item [$\at\{typ~\tau\}$] prints a well-formed type $\tau$.

 \item [$\at\{text~s\}$] prints uninterpreted source text $s$.  This is

   particularly useful to print portions of text according to the Isabelle

   {\LaTeX} output style, without demanding well-formedness (e.g.\ small pieces

   of terms that should not be parsed or type-checked yet).

 \item [$\at\{goals\}$] prints the current \emph{dynamic} goal state.  This is

   mainly for support of tactic-emulation scripts within Isar --- presentation

   of goal states does not conform to actual human-readable proof documents.

   Please do not include goal states into document output unless you really

   know what you are doing!

 \item [$\at\{subgoals\}$] behaves almost like $goals$, except that it does not

   print the main goal.

 \end{descr}

 \medskip

 The following options are available to tune the output.  Note that most of

 these coincide with ML flags of the same names (see also \cite{isabelle-ref}).

 \begin{descr}

 \item[$show_types = bool$ and $show_sorts = bool$] control printing of

   explicit type and sort constraints.

 \item[$long_names = bool$] forces names of types and constants etc.\ to be

   printed in their fully qualified internal form.

 \item[$eta_contract = bool$] prints terms in $\eta$-contracted form.

 \item[$display = bool$] indicates if the text is to be output as multi-line

   ``display material'', rather than a small piece of text without line breaks

   (which is the default).

 \item[$quotes = bool$] indicates if the output should be enclosed in double

   quotes.

 \item[$mode = name$] adds $name$ to the print mode to be used for presentation

   (see also \cite{isabelle-ref}).  Note that the standard setup for {\LaTeX}

   output is already present by default, including the modes ``$latex$'',

   ``$xsymbols$'', ``$symbols$''.

 \item[$margin = nat$ and $indent = nat$] change the margin or indentation for

   pretty printing of display material.

 \item[$source = bool$] prints the source text of the antiquotation arguments,

   rather than the actual value.  Note that this does not affect

   well-formedness checks of $thm$, $term$, etc. (only the $text$ antiquotation

   admits arbitrary output).

 \item[$goals_limit = nat$] determines the maximum number of goals to be

   printed.

 \end{descr}

 For boolean flags, ``$name = true$'' may be abbreviated as ``$name$''.  All of

 the above flags are disabled by default, unless changed from ML.

 \medskip Note that antiquotations do not only spare the author from tedious

 typing of logical entities, but also achieve some degree of

 consistency-checking of informal explanations with formal developments:

 well-formedness of terms and types with respect to the current theory or proof

 context is ensured here.

 %%% Local Variables: 

 %%% mode: latex

 %%% TeX-master: "isar-ref"

 %%% End: