Abnf Form And Xml Form Grammars example essay topic

It's not my p [a per / I just find it in the internet / Abstract This document defines syntax for representing grammars for use in speech recognition so that developers can specify the words and patterns of words to be listened for by a speech recognizer. The syntax of the grammar format is presented in two forms, an Augmented BNF Form and an XML Form. The specification makes the two representations mappable to allow automatic transformations between the two forms. Status of this Document This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W 3 C publications and the latest revision of this technical report can be found in the W 3 C technical reports index at web document has been reviewed by W 3 C Members and other interested parties, and it has been endorsed by the Director as a W 3 C Recommendation.

W 3 C's role in making the Recommendation is to draw attention to the specification and to promote its widespread deployment. This enhances the functionality and interoperability of the Web. This specification is part of the W 3 C Speech Interface Framework and has been developed within the W 3 C Voice Browser Activity (activity statement) by participants in the Voice Browser Working Group (W 3 C Members only). The design of SRGS 1.0 has been widely reviewed (see the disposition of comments) and satisfies the Working Group's technical requirements. A list of implementations is included in the SRGS 1.0 implementation report, along with the associated test suite. Comments are welcome on - (archive).

See W 3 C mailing list and archive usage guidelines. The W 3 C maintains a list of any patent disclosures related to this work. Table of Contents 1. Introduction o 1.1 Grammar Processors and User Agents o 1.2 Scope o 1.3 Grammar Conversions o 1.4 Semantic Interpretation o 1.5 Embedded Grammars o 1.6 Terminology o 2. Rule Expansions o 2.1 Tokens o 2.2 Rule References 2.2. 1 Local References 2.2.

2 External Reference by URI 2.2. 3 Special Rules 2.2. 4 Referencing N-gram Documents o 2.3 Sequences and Encapsulation o 2.4 Alternatives 2.4. 1 Weights o 2.5 Repeats 2.5.

1 Repeat Probabilities o 2.6 Tags o 2.7 Language o 2.8 Precedence o 3. Rule Definitions o 3.1 Basic Rule Definition o 3.2 Scoping of Rule Definitions o 3.3 Example Phrases o 4. Grammar Documents o 4.1 Grammar Header Declarations o 4.2 ABNF Self-Identifying Header o 4.3 XML Form Prolog and Root Element o 4.4 Character Encoding o 4.5 Language o 4.6 Mode o 4.7 Root Rule o 4.8 Tag Format o 4.9 Base URI 4.9. 1 Resolving Relative URIs o 4.10 Pronunciation Lexicon o 4.11 Meta Data 4.11. 1 Meta and HTTP-Equiv 4.11. 2 XML Metadata (XML Only) o 4.12 Tag o 4.13 Comments o 4.14 Grammar Fetching o 4.15 ABNF Keywords o 5.

Conformance o 5.1 Conforming XML Form Grammar Fragments o 5.2 Conforming Stand-Alone XML Form Document o 5.3 Using XML Form Grammars with other Namespaces o 5.4 Conforming XML Form Grammar Processors o 5.5 Conforming Stand-Alone ABNF Form Grammar Documents o 5.6 Conforming ABNF Form Grammar Processors o 5.7 Conforming ABNF / XML Form Grammar Processors o 5.8 Conforming User Agents o 6. Acknowledgements o Appendix A. References o A. 1 Normative References o A. 2 Informative References o Appendix B. DTD for XML Form Grammars (Informative) o Appendix C. XML Schema Definition For XML Form Grammars (Normative) o Appendix D. Formal Syntax for Augmented BNF Form Grammars (Normative) o Appendix E. DTMF Grammars (Normative) o Appendix F. XSLT Style Sheet to Convert XML Form Grammars to the ABNF Form (Informative) o Appendix G. Media Types and File Suffix (Informative) o Appendix H. Logical Parse Structure (Informative) o H. 1 Terminology and Notation o H. 2 Parsing Rule References o H. 3 Recursion o Appendix I. Features under Consideration for Future Versions (Informative) o Appendix J. Example Grammars in ABNF Form and XML Form (Informative) o J. 1 Simple Examples (English) o J. 2 Cross-Reference Examples (English) o J. 3 Korean Examples o J. 4 Chinese Examples o J. 5 Swedish Examples 1. Introduction This document defines the syntax for grammar representation. The grammars are intended for use by speech recognizers and other grammar processors so that developers can specify the words and patterns of words to be listened for by a speech recognizer. The syntax of the grammar format is presented in two forms, an Augmented BNF (ABNF) Form and an XML Form. The specification ensures that the two representations are semantically mappable to allow automatic transformations between the two forms. o Augmented BNF syntax (ABNF): this is a plain-text (non-XML) representation which is similar to traditional BNF grammar and to many existing BNF-like representations commonly used in the field of speech recognition including the JSpeech Grammar Format [JSGF] from which this specification is derived.

Augmented BNF should not be confused with Extended BNF which is used in DTDs for XML and SGML. o XML: This syntax uses XML elements to represent the grammar constructs and adapts designs from the Pipe Beach grammar, TalkML [TALKML] and a research XML variant of the JSpeech Grammar Format [JSGF]. Both the ABNF Form and XML Form have the expressive power of a Context-Free Grammar (CFG). A grammar processor that does not support recursive grammars has the expressive power of a Finite State Machine (FSM) or regular expression language. For definitions of CFG, FSM, regular expressions and other formal computational language theory see, for example, [HU 79]. This form of language expression is sufficient for the vast majority of speech recognition applications. This W 3 C standard is known as the Speech Recognition Grammar Specification and is modelled on the JSpeech Grammar Format specification [JSGF], which is owned by Sun Microsystems, Inc., California, U.S.A. 1.1 Grammar Processors and User Agents grammar processor is any entity that accepts as input grammars as described in this specification.

A user agent is a grammar processor that accepts user input and matches that input against a grammar to produce a recognition result that represents the detected input. As the specification title implies, speech recognizers are an important class of grammar processor. Another class of grammar processor anticipated by this specification is a Dual-Tone Multi-Frequency (DTMF) detector. The type of input accepted by a user agent is determined by the mode or modes of grammars it can process: e.g. speech input for 'voice' mode grammars and DTMF input for ' mode grammars. For simplicity, throughout this document references to a speech recognizer apply to other types of grammar processor unless explicitly stated otherwise. A speech recognizer is a user agent with the following inputs and outputs: o Input: A grammar or multiple grammars as defined by this specification.

These grammars inform the recognizer of the words and patterns of words to listen for. o Input: An audio stream that may contain speech content that matches the grammar (s). o Output: Descriptions of results that indicate details about the speech content detected by the speech recognizer. The format and details of the content of the result are outside the scope of this specification. For informative purposes, most practical recognizers will include at least a transcription of any detected words. o Output: Error and other performance information may be provided to the host environment: e.g. to a voice browser that incorporates a grammar processor. The method of interaction with the host environment is outside the scope of this document. The specification does, however, require that a conform ant grammar processor inform the environment of errors in parsing and other processing of grammar documents. 1.2 Scope The primary use of a speech recognizer grammar is to permit a speech application to indicate to a recognizer what it should listen for, specifically: o Words that may be spoken, o Patterns in which those words may occur, o Spoken language of each word.

Speech recognizers may also support the Stochastic Language Models (N-Gram) Specification [NGRAM]. Both specifications define ways to set up a speech recognizer to detect spoken input but define the word and patterns of words by different and complementary means. Some recognizers permit cross-references between grammars in the two formats. The rule reference element of this specification describes how to reference an N-gram document.

The grammar specification does not address a number of other issues that affect speech recognition performance. Most of the following capabilities are addressed by the context in which a grammar is referenced or invoked: for example, through VoiceXML 2.0 [VXML 2] or through a speech recognizer API. o Speaker adaptation data: Some speech recognizers support the ability to dynamically adjust to the voice of a speaker and often the ability to store adaptation data for that voice for future use. The speaker data may also include lists of words more often spoken by the user. The grammar format does not explicitly address these capabilities. o Speech recognizer configuration: The grammar format does not incorporate features for setting recognizer features such as timeouts, recognition thresholds, search sizes or N-best result counts. o Lexicon: The grammar format does not address the loading of lexicons or the pronunciation of words referenced by the grammar. The W 3 C Voice Browser Working Group is considering the development of a standard lexicon format. If and when a format is developed appropriate updates will be made to this grammar specification. o Other speech processing capabilities: Speech processing technology exists for language identification, speaker verification (also known as voice printing), speaker recognition (also known as speaker identification) amongst many other capabilities.

Although these technologies may be associated with a speech recognizer they are outside the scope of this specification. 1.3 Grammar Conversions The ABNF Form and XML Form are specified to ensure that the two representations are semantically mappable. It should be possible to automatically convert an ABNF Form grammar to an XML Form grammar (or the reverse) so that the semantic performance of the grammars are identical. Equivalence of semantic performance implies that: 1. Both grammars accept the same language as input and reject the same language as input 2. Both grammars parse any input string identically The XSL Transformation document in Appendix F demonstrates automatic conversion from XML to ABNF.

The reverse conversion requires an ABNF parser and a transformational program. There are inherent limits to the automatic conversion to and From ABNF Form and XML Form. o Formatting white space cannot be preserved so a pretty-printable grammar in one Form cannot guarantee automatic conversion to a pretty-printable grammar in the other Form. Note: syntactically significant white space is preserved. o Some XML constructs have no equivalent in ABNF: XML Schema, DTD, character and entity declarations and references, processing instructions, name spaces. The XML parser in a conforming grammar processor should expand all character and entity references as defined in XML 1.0 [XML] prior to conversion to ABNF; other constructs are lost. RDF [RDF-SYNTAX] represents metadata as XML within XML Form grammar but could not be effectively utilized in ABNF Form grammars and so is not supported. o Comment ordering with respect to grammar constructs may be modified. 1.4 Semantic Interpretation speech recognizer is capable of matching audio input against a grammar to produce a raw text transcription (also known as literal text) of the detected input.

A recognizer may be capable of, but is not required to, perform subsequent processing of the raw text to produce a semantic interpretation of the input. For example, the natural language utterance 'I want to book a flight from Prague to Paris' could result in the following XML data structure. To perform this additional interpretation step requires semantic processing instructions that may be contained within a grammar that defines the legal spoken input or in an associated document. PragueParisThe Speech Recognition Grammar Specification provides syntactic support for limited semantic interpretation.

The tag construct and the tag-format and tag declarations provide a placeholder for instructions to a semantic processor. The W 3 C Voice Browser Working Group is presently developing the Semantic Interpretation for Speech Recognition specification [SEM]. That specification defines a language that can be embedded in tags within SRGS grammars to perform the interpretation process. The semantic processing is defined with respect to the logical parse structure for grammar processing (see Appendix H). Other tag formats could be used but are outside the scope of the W 3 C activities.

For examples of semantic interpretation in the latest working draft see [SEM]. The output of the semantic interpretation processor may be represented using the Natural Language Semantics Markup Language [NLS ML]. This XML representation of interpreted spoken input can be used to transmit the result, as input to VoiceXML 2.0 [VXML 2] processing or in other ways. The semantic interpretation carried out in the speech recognition process is typically characterized by: o Restricted context: the interpretation does not resolve deictic or anaphoric references or other language forms that span more than a single utterance. Example: if the utterance 'I want to book a flight from Prague to Paris' were followed later by 'I want to continue from there to London' the reference to 'there' could be resolved to 'Paris'. This requires analysis spanning more than one utterance and is typically outside the scope of the speech recognizer, but in scope for a dialog manager (e.g. a VoiceXML application). o Domain-specific: a speech recognition grammar is typically restricted to a narrow domain of input (e.g. collect flight booking data).

Within this domain semantic interpretation is an achievable task whereas semantic interpretation for an entire language is an extraordinarily complex task. o Language-specific: because each language has unique linguistic structures the process of converting from a raw text to a semantic result is necessarily language-specific. It is this restricted form of semantic interpretation that this approach is intended to support. A VoiceXML application that receives a speech result with semantic interpretation will typically process the user input to carry out a dialog. The application may also perform deeper semantic analysis, for example resolving deictic or anaphoric references.

1.5 Embedded Grammars The Speech Recognition Grammar Specification is designed to permit ABNF Form and XML Form grammars to be embedded into other documents. For example, VoiceXML 1.0 [VXML 1] and VoiceXML 2.0 [VXML 2] permit inline grammars [VXML 2 SS 3.1. 1.1] in which an ABNF Form grammar or XML Form grammar is contained within a VoiceXML document. Embedding an XML Form grammar within an XML document can be achieved with XML name spaces [XMLNS] or by incorporating the grammar XML Schema definition or DTD into to enclosing document's schema or DTD. An ABNF Form grammar may be embedded into any XML document as character data. ABNF grammars will often contain angle brackets which require special handling within XML.

A CDATA section [XML SS 2.7] or the escape sequences of ' may be required to create well-formed XML. Note: angle brackets (') are used in ABNF to delimit any URI, media type or repeat operator. 1.6 Terminology Requirements terms The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL', 'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED', 'MAY', and 'OPTIONAL' in this document are to be interpreted as described in [RFC 2119]. However, for readability, these words do not appear in all uppercase letters in this specification.

URI: Uniform Resource Identifier A URI is a unifying syntax for the expression of names and addresses of objects on the network as used in the World Wide Web. A URI is defined as any legal 'an yURI ' primitive as defined in XML Schema Part 2: Datatypes [SCHEMA 2 SS 3.2. 17]. The XML Schema definition follows [RFC 2396] and [RFC 2732].

The syntax representation of a URI differs between the ABNF Form and the XML Form. Any relative URI reference must be resolved according to the rules given in Section 4.9. 1. o ABNF URI: in the ABNF Form of this specification a URI is delimited by angle brackets For example, o XML URI: in the XML Form of this specification any URI is provided as an attribute to an element; for example the rule ref and lexicon elements. Media Type A media type (defined in [RFC 2045] and [RFC 2046]) specifies the nature of a linked resource. Media types are case insensitive. A list of registered media types is available for download [TYPES].

In places where a URI can be specified a media type may be provided to indicate the content type of URI. [See Appendix G for information on media types for the ABNF and XML Forms of the Speech Recognition Grammar Specification.] o ABNF URI with Media Type: in the ABNF Form a media type may be attached as a postfix to any URI. The media type is delimited by angle brackets (') and the URI and media type are separated by a tilde character without intervening white space. For example, ~ o XML URI with Media Type: in the XML Form any element that carries a URI attribute may carry a type attribute. Language identifier A language identifier labels information content as being of a particular human language variant. Following the XML specification for language identification [XML SS 2.12] a legal language identifier in ABNF Form grammars and XML Form grammars is identified by an RFC 3066 [RFC 3066] code.

A language code is required by RFC 3066. A country code or other sub tag identifier is optional by RFC 3066. A grammar's language declaration declares the language of a grammar. Additionally a legal rule expansion may be labeled by its language content. White space White space consists of one or more space (#x 20) characters, carriage returns, line feeds, or tabs. The normative definition for both ABNF and XML follows the XML white space definition [XML SS 2.3].

ABNF processors must also follow the end-of-line handling of XML 1.0 [XML SS 2.11]. DTMF DTMF (Dual Tone Multiple Frequency) is an ITU standard for telephony signaling. ITU Recommendation Q. 23 defines DTMF generation. ITU Recommendation Q. 24 [Q 24] defines DTMF reception. A grammar processor that accepts DTMF input should implement Q. 24.2. Rule Expansions 2.1 Tokens o 2.2 Rule References o 2.2.

1 Local References o 2.2. 2 External Reference by URI o 2.2. 3 Special Rules o 2.2. 4 Referencing N-gram Documents o 2.3 Sequences and Encapsulation o 2.4 Alternatives o 2.4. 1 Weights o 2.5 Repeats o 2.5.

1 Repeat Probabilities o 2.6 Tags o 2.7 Language o 2.8 Precedence A legal rule expansion is any legal token, rule reference, tag, or any logical combination of legal rule expansions as sequence, alternatives, repeated expansion or language-attributed expansion. A rule expansion is formally a regular expression (see, for example, [HU 79]). A rule definition associates a legal rule expansion with a rule name. 2.1 Tokens token (a. k. a. a terminal symbol) is the part of a grammar that defines words or other entities that may be spoken.

Any legal token is a legal expansion. For speech recognition, a token is typically an orthographic entity of the language being recognized. However, a token may be any string that the speech recognizer can convert to a phonetic representation. Token Content: In both the XML Form and ABNF Form any unmarked text within a rule definition, except example phrases (XML only) or tag content, is token content. The unmarked text is delimited by any syntactic construct of the grammar form (see below for details on the ABNF Form and XML Form). For each token content span in a grammar the grammar processor applies the following, white space normalization, token normalization and pronunciation lookup processes.

All token content in both the XML Form and ABNF Form is treated as Characters in [XML]. (informative: XML specifies Characters by reference to ISO / IEC 10646 [ISO / IEC 10646] and Unicode [Unicode].) Token ization behavior: Text spans containing token sequences are delimited as follows: o XML Form only: a element may contain character data only. The character data is treated as a single un normalized token. The character data must not contain any double quote characters. o Any token in ABNF Form or XML Form (except within element in XML Form) may be delimited by double quotes. The text contained within the double quotes is an un normalized token. The text must not contain any double quote characters. A token delimited by double quotes may contain white space. o Any token content not delimited by a element or double quotes is treated as a sequence of white-space-delimited tokens.

Each token contained in the token content is delimited at the start and at the end by any white space character or any syntactic construct that delimits a token content span. The syntactic constructs that delimit token content are different for the ABNF Form and XML Form. These tokens cannot contain white space characters. Token type Form Example Single unquoted token ABNF & XML hello Single unquoted token: non-alphabetic ABNF & XML 2 Single quoted token: including white space ABNF & XML 'San Francisco " Single quoted token: no white space ABNF & XML 'hello " Two tokens delimited by white space ABNF & XML bon voyage Four tokens delimited by white space ABNF & XML this is a test Single XML token in XML Only San Francisco White Space Normalization: White space must be normalized when contained in any token delimited by a elements or by double quotes.

Leading and trailing white space characters are stripped. Any token-internal white space character or sequence is collapsed to a single space character (#x 20). For example, the following are all normalized to the same string, 'San Francisco'.'s an Francisco's an Francisco 'San Francisco's an Francisco 'Because the presence of white space within a token is significant the following are distinct tokens.'s an Francisco'San Francisco'San Francisco " Token Normalization: Other normalization processes are applied to the white space normalized token according to the language and the capabilities of the speech recognizer. Grammar processors may assume Early Uniform Normalization as defined in the Character Model for the World Wide Web 1.0 [CHARMED SS 4]. Pronunciation Lookup: To match spoken (audio) input to a grammar a speech recognition must be capable of model ling the audio patterns of any token in a grammar.

Speech recognizers employ a diverse set of techniques for performing this key recognition process. The following is an informative description of techniques that a speech recognizer may apply based on conventional large vocabulary speech recognition technology. A large vocabulary speech recognizer converts each normalized token to a phoneme sequence or a set of possible phoneme sequences. Conversion of an orthographic form (token) to the spoken form (phonemes) is a highly language-specific process. In many cases the conversion is even specific to a national variant, regional dialect or other variant of the language. For example, for some tokens Parisian French, Quebec French and Swiss French will each convert to different pronunciations.

The text-to-phoneme conversion in a large vocabulary speech recognizer may involve some or all of the following sub-processes. o Pronunciation lexicon lookup: One of possibly many lexicons available to a recognizer can provide the phoneme sequence for a token. Both the ABNF Form and XML Form permit a grammar to specify one or more lexicon documents. Recognizers typically provide a built-in lexicon for each supported language though the coverage will vary between recognizers. The algorithm by which the lookup resolves a token to a pronunciation is defined by the lexicon format and / or the speech recognizer and may be language-specific. Case-insensitive string matching is recommended. o Morphological analysis: a recognizer may be capable of determining the transformation from a base token and phoneme string to a morphological variant and its pronunciation. For example given the pronunciation for 'Hyundai' a rule could infer the pronunciation for the pluralized form 'Hyundai's'. o Automatic text-to-phoneme conversion: for many, but not all, languages and scripts there are rules that automatically convert a token into a phoneme sequence.

For example, in English most but not all words ending with the letter sequence 'is' end with the phoneme sequence 'ai z'. A speech recognizer may use automated conversion to infer pronunciations for tokens that cannot be looked up in a lexicon. Any language is likely to have other specialized processes for determining a pronunciation for a token. For example, for Japanese special techniques are required for Kanji and each Kana form. For any language and recognizer there may be variation in coverage and completeness of the language's tokens.

When a grammar processor handles a grammar containing a token that it cannot convert to phonemic form or otherwise use in the speech recognition processing of audio it should inform the hosting environment. Limitations of token handling: the following is informative guidance to grammar developers. The Pronunciation Lexicon activity [LEX] of the W 3 C Voice Browser Working Group will provide guidance on the token-handling processes outlined above. Token handling will vary between recognizers and will vary between languages. Grammar authors can improve document portability by avoiding characters and forms in tokens that do not have obvious pronunciations in the language. For English, the following are ways to handle some orthographic forms: o Acronyms should be avoided.

Alphabetic characters should be widely available. For example, replace 'USA' by 'u's a'; replace 'W 3 C' by 'w three c'; replace 'I ' by 'i triple e'. o Abbreviations should be replaced by the unabbreviated form. For example, replace 'Dr. ' by 'drive' or 'doctor'. o Most punctuation should be expanded to a spelled form. For example replace '&' by 'ampersand' or 'and'; replace '+' by 'plus'; replace 'Note: the media type 'application / sr gs+xml' has been requested for XML Form grammars. See Appendix G for details on media types for grammars. 2.2. 3 Special Rules Several rule names are defined to have specific interpretation and processing by a speech recognizer.

A grammar must not redefine these rule names. In the ABNF Form a special rule reference is syntactically identical to a local rule reference. However, the names of the special rules are reserved to prevent a rule definition with the same name. In the XML Form a special rule name is represented with the special attribute on a rule ref element. It is illegal to provide both the special and the uri attributes. NULL Defines a rule that is automatically matched: that is, matched without the user speaking any word.

ABNF Form: $NULL XML Form: VOID Defines a rule that can never be spoken. Inserting VOID into a sequence automatically makes that sequence unspeakable. ABNF Form: $VOID XML Form: GARBAGE Defines a rule that may match any speech up until the next rule match, the next token or until the end of spoken input. A grammar processor must accept grammars that contain special references to GARBAGE. The behavior GARBAGE rule is implementation-specific. A user agent should be capable of matching arbitrary spoken input up to the next token but may treat GARBAGE as equivalent to NULL (match no spoken input).

ABNF Form: $GARBAGE XML Form: Informative example: given suitable definitions of US cities and states, a speech recognizer may implement the following ABNF and XML rule definitions to match 'Philadelphia in the great state of Pennsylvania' as well as simply 'Philadelphia Pennsylvania'. $location = $city $GARBAGE $state; 2.2. 4 Referencing N-gram Documents (Informative) The W 3 C Voice Browser Working Group has released a Working Draft for the Stochastic Language Models (N-Gram) Specification [NGRAM]. These two specifications represent different and complementary ways of informing a speech recognizer of which words and patterns of words to listen for. A speech recognizer may choose to support the Speech Recognition N-Gram Grammar Specification in addition to the speech recognition grammar defined in this document. If a speech recognizer supports both grammar representations it may optionally support references between the two formats. Grammars defined in the ABNF Form or XML Form may reference start symbols of N-Gram documents and vice versa.

The syntax for referencing an N-Gram is the same as referencing externally defined ABNF Form or XML Form grammar documents. A media type is recommended on a reference to an N-gram document. The Working Group has not yet applied for a type on N-gram documents so no example is given. The fragment identifier (a rule name when referencing ABNF Form and XML Form grammars) identifies a start symbol as defined by the N-Gram specification. If the start symbol is absent the N-Gram, as a whole, is referenced as defined in the N-Gram specification.

ABNF FormURI references to N-Gram documents follow the same syntax as references to other ABNF or XML Form grammar documents. The following are examples of references to an N-Gram document via an explicit rule reference and an implicit reference to the root rule. $$XML FormURI references to N-Gram documents follow the same syntax as reference to other ABNF Form and XML Form grammar documents. 2.3 Sequences and Encapsulation sequence of legal rule expansions is itself a legal rule expansion.

The sequence of rule expansions implies the temporal order in which the expansions must be detected by the user agent. This constraint applies to sequences of tokens, sequences of rule references, sequences of tags, parenthetical's and all combinations of these rule expansions. Both the ABNF Form and XML Form provide syntax for encapsulating any expansion. This is used, for example, to attach a repeat operator, a language identifier or to ensure correct precedence in parsing (ABNF only). ABNF FormA sequence of legal expansions separated by white space is a legal expansion. A legal expansion surrounded by parentheses (' (' and ') ') is a legal expansion. this is a test // sequence of tokens$action $object // sequence of rule references the $object is $color // sequence of tokens and rule references (fly to $city) // parentheses for encapsulation Special cases An empty parenthetical is legal as is a parenthetical containing only white space; e.g. ' ' or ' '.

Both forms are equivalent to $NULL and a grammar processor will behave as if the parenthetical were not present. // equivalent sequences phone home phone home XML FormA sequence of XML rule expansion elements (, , , ) and CDATA sections containing space separated tokens must be recognized in temporal sequence. (The only exception is where one or more 'item' elements appear within a one-of element.) An item element can surround any expansion to permit a repeat attribute or language identifier to be attached. The weight attribute of item is ignored unless the element appears within a one-of element. this is a test the is fly to Special cases An empty item element is legal as is an item element containing only white space. Both forms are equivalent to a NULL reference and a grammar processor will behave as if the item were not present. phone home phone home phone home phone home 2.4 Alternatives Any set of alternative legal rule expansions is itself a legal rule expansion.

For input to match a set of alternative rule expansions it must match one of the set of alternative expansions. A set of alternatives must contain one or more alternatives. Any set of alternatives may be labeled with a language attachment. In the XML Form an xml: lang attribute is present on the one-of element. In the ABNF Form to ensure correct precedence the set of alternatives must be delimited by parentheses with the ABNF language attachment immediately following. 2.4. 1 Weights weight may be optionally provided for any number of alternatives in an alternative expansion.

Weights are simple positive floating point values without exponential's. Legal formats are 'n', 'n. ', '. n' and 'n. n' where 'n' is a sequence of one or many digits. A weight is nominally a multiplying factor in the likelihood domain of a speech recognition search. A weight of 1.0 is equivalent to providing no weight at all.

A weight greater than '1.0' positively biases the alternative and a weight less than '1.0' negatively biases the alternative. [JEL 98] and [RAB 93] are informative references on the topic of speech recognition technology and the underlying statistical framework within which weights are applied. Grammar authors and speech recognizer developers should be aware of the following limitations upon the definition and application of weights as outlined above. o The application of weights to a speech recognition search is under the internal control of the recognizer. There is no normative or informative algorithm for applying weights.

Furthermore, speech recognition is a statistical process so consistent behavior cannot be guaranteed. o Appropriate weights are difficult to determine for any specific grammar and recognizer. Guessing weights does not always improve speech recognition performance. o Effective weights are best obtained by study of real speech input to a grammar. For example, a reasonable technique for developing portable weights is to use weights that are correlated with the occurrence counts of a set of alternatives. o Tuning weights for a particular recognizer does not guarantee improved recognition performance on other speech recognizers. ABNF FormA set of alternative choices is identified as a list of legal expansions separated by the vertical bar symbol.

If necessary, the set of alternative choices may be delimited by parentheses. Michael | Yuri ko | Mary | Duke | $other Names (1 | 2 | 3) A weight is surrounded by forward slashes and placed before each item in the alternatives list. /10/ small | /2/ medium | large/3.1415/ pie | /1.414/ root beer | /. 25/ cola Special Cases It is legal for an alternative to be a reference to $NULL, an empty parenthetical or a single tag. In each case the input is equivalent to matching $NULL and as a result the other alternatives are optional. // Legal$rule 1 = word | $NULL; $rule 2 = | word; $rule 3 = word | {TAG-CONTENT}; An empty alternative (white space only) is not legal.

// ILLEGAL$rule 1 = a | | b; $rule 2 = | b; $rule 3 = a |; The following construct is interpreted as a single weighted alternative. // Legal$rule 1 = /2/ word; $rule 2 = /2/ {TAG-CONTENT}; $rule 3 = /2/ $NULL; XML Form The one-of element identifies a set of alternative elements. Each alternative expansion is contained in a item element. There must be at least one item element contained within a one-of element. Weights are optionally indicated by the weight attribute on the item element. MichaelYurikoMaryDuke 1 2 3 beercolaSpecial cases one-of element containing a single item is legal and requires that input match the single item.

The single item may be optionally weighted. wordwordIs it legal for an alternative to be a reference to NULL, an empty item or a single tag. In each case the input is equivalent to matching NULL and as a result the other alternatives are optional. wordwordwordTAG-CONTENT 2.5 Repeats Any repeated legal rule expansion is itself a legal rule expansion. Operators are provided that define a legal rule expansion as being another sub-expansion that is optional, that is repeated zero or more times, that is repeated one or more times, or that is repeated some range of times. ABNF Form Example XML Form Example Behavior repeat = 'n'repeat = '6' The contained expansion is repeated exactly 'n' times. 'n' must be '0' or a positive integer. repeat = 'm-n'repeat = '4-6' The contained expansion is repeated between 'm' and 'n' times (inclusive). 'm' and 'n' must both be '0' or a positive integer and 'm' must be less than or equal to 'n'. repeat = 'm-'repeat = '3-' The contained expansion is repeated 'm' times or more (inclusive). 'm' must be '0' or a positive integer. For example, '3-' declares that the contained expansion can occur three, four, five or more times. [... ] repeat = '0-1' The contained expansion is optional. Common Repeats As indicated in the table above, an expansion that can occur 0-1 times is optional.

Because optionality is such a common form the ABNF syntax provides square brackets as a special operator for representing optionality. A repeat of '0-' indicates that an expansion can occur zero times, once or any number of multiple times. In regular expression languages this is often represented by the Kleen e star which is reserved but not used in ABNF. A repeat of '1-' indicates that an expansion can occur once or any number of multiple times. In regular expression languages this is often represented by the positive closure which is reserved but not used in ABNF. Although both ABNF and XML support a grammar that permits an unbounded number of input tokens it is not the case that users will speak indefinitely.

Speech recognition can perform more effectively if the author indicates a more limited range of repeat occurrences. Special Cases Where a number of possible repetitions (e.g. or (n 0) but not) is expressed on a construct whose only content is one or more tag elements, the behavior of the grammar processor is not defined and will be specific to individual implementations. Any number of non-optional repetitions (e. g., ; m 0) of VOID is equivalent to a single VOID. The behavior of a grammar processor in handling any number of repetitions of NULL is not defined and will be specific to individual implementations.

If the number of repetitions for any expansion can be only zero (i.e. or) then the expansion is equivalent to NULL. 2.5. 1 Repeat Probabilities Any repeat operator may specify an optional repeat probability. The value indicates the probability of successive repetition of the repeated expansion. A repeat probability value must be in the floating pointing range of '0.0' to '1.0' (inclusive).

Values outside this range are illegal. The floating point format is one of 'n', 'n. ', 'n. nnan', '. nnan' (with any number of digits after the dot). Note: repeat probabilities and weights are different logical entities and have a different impact upon a speech recognition search. Informative example: A simple example is an optional expansion (zero or one occurrences) with a probability - say '0.6'.

The grammar indicates that the chance that the expansion will be matched is 60% and that the chance that the expansion will not be present is 40%. When no maximum is specified in a range (m-) the probabilities decay exponentially. Grammar authors and speech recognizer developers should be aware of the following limitations upon the definition and application of repeat probabilities as outlined above. o The application of repeat probabilities to a speech recognition search is under the internal control of the recognizer. There is no specified algorithm for applying repeat probabilities in a speech recognition processor so consistent behavior cannot be guaranteed. o Appropriate repeat probabilities are often difficult to determine for any specific grammar and recognizer. Guessing repeat probabilities does not always improve speech recognition performance. o Appropriate repeat probabilities are best obtained by study of statistical patterns of real speech input.

Tuning repeat probabilities for a particular recognizer does not guarantee improved recognition performance on other speech recognizers. Useful references on statistical models of speech recognition include [JEL 98] and [RAB 93]. ABNF Form The following are postfix operators: . A postfix operator is logically attached to the preceding expansion. Postfix operators have high precedence and so are tightly bound to the immediately preceding expansion (see Section 2.8). Optional expansions may be delimited by square brackets: [expansion].

Alternatively, an optional expansion is indicated by the postfix operator '. The following symbols are reserved for future use in ABNF: ' ', '+', '?'. These symbols must not be used at any place in a grammar where the syntax currently permits a repeat operator. // the token 'very' is optional [very] very // the rule reference $digit can occur zero, one or many times$digit // the rule reference $digit can occur one or more times$digit // the rule reference $digit can occur four, five or six times$digit // the rule reference $digit can occur ten or more times$digit // Examples of the following expansion// 'pizza'// 'big pizza with pepperoni'// 'very big pizza with cheese and pepperoni' [ [very] big] pizza ( [with | and] $topping) Repeat probabilities are only supported in the range form. The probability is delimited by slash characters and contained within the angle brackets: and. // the token 'very' is optional and is 60% likely to occur// and 40% likely to be absent in input very // the rule reference $digit must occur two to four times // with 80% probability of recurrence$digit XML Form The item element has a repeat attribute that indicates the number of times the contained expansion may be repeated.

The following example illustrates the accepted values of the attribute. very very big pizzawithandThe repeat-prob on the item element carries the repeat probability. Repeat probabilities are supported on any item element but are ignored if the repeat attribute is not also specified. very 2.6 Tags tag is a legal rule expansion (a tag can also be declared in the grammar header - see S 4.1). A tag is an arbitrary string that may be included inline within any legal rule expansion. Any number of tags may be included inline within a rule expansion. Tags do not affect the legal word patterns defined by the grammars or the process of recognizing speech or other input given a grammar. Tags may contain content for semantic interpretation.

The semantic interpretation processes may affect the recognition result. Language attachments have no effect upon tags. The tag format declaration indicates the content type of all tags in a grammar. Special Cases It is legal to use a tag as a stand-alone expansion. For example, a rule may expand to a single tag and no tokens. $rule = {TAG-CONTENT}; TAG-CONTENT ABNF FormA tag is delimited by either a pair of opening and closing curly brackets - '{' and '}' - or by the following 3-character sequences which are considered very unlikely to occur within a tag - '{! {' and '}! }'.

A tag delimited by single curly brackets cannot contain the single closing curly bracket character A tag delimited by the 3-character sequence cannot contain the closing 3-character sequence ('}! }'). The tag content is all text between the opening and closing character sequences including leading and trailing white space. The contents of the tag are not parsed by the grammar processor. Tag precedence is the same as for rule references and tokens.

In the first example below there is a sequence of six space-separated expansions (3 tokens, a tag, a token and a tag). In the second example, the alternative is a choice between a sequence containing a token and a tag or a sequence containing a rule reference and a tag. $rule 1 = this is a {TAG-CONTENT-1} test {TAG-CONTENT-2}; $rule 2 = open {TAG-CONTENT-1} | $close {TAG-CONTENT-2}; $rule 3 = {! { a simple tag containing { and } needs no escaping }! }; XML FormA tag element can be a direct child of the item and rule elements. The content of tag is CDATA. this is a TAG-CONTENT-1 test TAG-CONTENT-2 open TAG-CONTENT-1 TAG-CONTENT-2 2.7 Language Any legal rule expansion that has an attached language identifier is itself a legal rule expansion. Both the ABNF Form and the XML Form permit a legal language identifier to be attached to any token, sequence or set of alternatives (Note that rule reference does not permit a language identifier to be attached). The syntax for the ABNF Form and for the XML Form are provided below.

The language declaration for a rule expansion affects only the contained content. Moreover, the language declaration affects only the handling of tokens in the contained content and does not affect tags or rule references. The application of language to token handling and particularly to pronunciation lookup is described in Section 2.1. By default a grammar is a single language document with a language identifier provided in the language declaration in the grammar header (see Section 4.5). All tokens within that grammar, unless otherwise declared, will be handled according to the grammar's language. In situations where applications target a multilingual user community, grammars that contain words in more than one language may be needed.

For example, in response to a prompt such as: 'Do you want to talk to Andr'e Pr " evo st?' (a combination of an English sentence with a French name), the response may be either 'yes' or 'oui'. The Speech Recognition Grammar Specification permits one grammar to collect input from more than one language. The specification also permits multiple grammars each with a separate single language to be used in parallel. The specification also permits a single input utterance to contain more than one language. Finally, the specification permits any combination of the above: for example, parallel grammars each with multi-lingual capability.

Not all user agents are required to support all languages, or indeed any or all of the multi-lingual capabilities. The conformance requirements regarding multi-lingual support for XML Form grammar processors and ABNF Form grammar processors are the same and are laid out in Section 5.4 and Section 5.6 respectively. There is a related challenge for multilingual applications that deal with proper names (people, streets, companies, etc.) that may be spoken with different pronunciations or accents depending upon the language of origin and the speaking language. It is often impossible to predict the language that users will use to pronounce certain tokens. In fact, users may actually use different languages for different words in the same sentence, and in unpredictable ways. For instance, the name 'Robert Jones' might be pronounced by a French-speaking user using the French pronunciation for 'Robert' but an English pronunciation for 'Jones', whereas a mono-lingual English speaker would use the English pronunciation for both words.

Language scoping: language declarations are scoped locally to a document and to a rule definition. In XML terminology, the language attribute is inherited down the document tree. Where a language change encompasses a reference to another grammar, the referenced rule and its containing grammar define the language of the reference expansion. The language in effect at the point of the rule reference does not have any effect upon the referenced rule.

Language and results: The language used in the recognition of a token is not considered a part of the speech result even in the case that a language declaration is associated with a token. ABNF Form In the ABNF Form a language identifier may be right-attached to any legal rule expansion except rule reference. The attachment is an exclamation point character ('!' ) followed by a legal language identifier without intervening white space. The language attachment has higher precedence than sequences or alternatives.

To attach a language to these rule expansion types the expansion should be delimited by parentheses (see Section 2.3). #ABNF 1.0 ISO-8859-1; // Default grammar language is US English language en-US; // Single language attachment to tokens// Note that 'fr-CA' (Canadian French) is applied to only// the word 'oui' because of precedence rules$yes = yes | oui! fr-CA; // Single language attachment to an expansion$people 1 = (Michel Tremblay | Andr'e Roy)! fr-CA; // Handling language-specific pronunciations of the same word// A capable speech recognizer will listen for Mexican Spanish and// US English pronunciations. $people 2 = Jose! en-US; | Jose! es-MX; / Multi-lingual input possible @example may I speak to Andr'e Roy @example may I speak to Jose / public $request = may I speak to ($people 1 | $people 2); XML Form XML 1.0 [XML SS 2.12] defines the xml: lang attribute for language identification. The attribute provides a single language identifier for the content of the element on which it appears. The xml: lang attribute may be attached to one-of, token and item. It applies the token handling of scoped tokens. yesouiMichel TremblayAndr'e RoyJoseJosemay I speak with Andr'e Roy may I speak with Jose may I speak with 2.8 Precedence This section defines the precedence of the ABNF rule expansion syntax. Because XML documents explicitly indicate structure there is no ambiguity and thus a precedence definition is not required.

The precedence definitions for the ABNF Form are intended to minimize the need for parentheses. ABNF Form The following is the ordering of precedence of rule expansions. Parentheses may be used to explicitly control rule structure. 1.

A rule reference, a quoted token, an unquoted token or a tag. 2. Parentheses (' (' and ') ') for grouping and square brackets (' [' and ']') for optional grouping. 3. Repeat operator (e.g. ') and language attachment (e.g. '! en-AU') apply to the tightest immediate preceding rule expansion. (To apply them to a sequence or to alternatives, use ' ' or ' ' for grouping.) 4.

Sequence of rule expansions. 5. Set of alternative rule expansions separated by vertical bars with optional weights. XML Form None required.

XML structure is explicit. 3. Rule Definitions 3.1 Basic Rule Definition o 3.2 Scoping of Rule Definitions o 3.3 Example Phrases A rule definition associates a legal rule expansion with a rule name. The rule definition is also responsible for defining the scope of the rule definition: whether it is local to the grammar in which it is defined or whether it may be referenced within other grammars. Finally, the rule definition may additionally include documentation comments and other pragmatics. The rule name for each rule definition must be unique within a grammar.

The same rule name may be used in multiple grammars. A rule definition is referenced by a URI in a rule reference with the rule name being represented as the fragment identifier. 3.1 Basic Rule Definition The core purpose of a rule definition is to associate a legal rule expansion with a rule name. A legal rule name in either the XML Form or ABNF Form is a character sequence that: o is an XML Name [XML SS 2.3], o and does not contain the following character '. ', ':', '-', o and is not the name of a special rule ('NULL', 'VOID', 'GARBAGE').

Defined rule names must be unique within a grammar. The schema enforces this by declaring the rule name as an XML ID. Rulename are case-sensitive in both XML and ABNF grammars. Exact string comparison is used to resolve rule name references. A legal rule name cannot be one of the special rules: specifically 'NULL', 'VOID' or 'GARBAGE'. ABNF Form The rule definition consists of an optional scoping declaration (explained in the next section) followed by a legal rule name, an equals sign, a legal rule expansion and a closing semicolon.

The rule definition has one of the following legal forms: $ruleName = rule Expansion; public $ruleName = rule Expansion; private $ruleName = rule Expansion; For example: $city = Boston | 'New York' | Madrid; $command = $action $object; Special Cases An empty rule definition is illegal. It is legal to define a rule that expands to empty parentheses or $NULL (equivalent forms). It is legal to define a rule that expands to a single tag. // Legal$rule = ; $rule = $NULL; $rule = {TAG-CONTENT}; // ILLEGAL$rule = ; XML FormA rule definition is represented by the rule element.

The id attribute of the element indicates the name of the rule and must be unique within the grammar (this is enforced by XML). The contents of the rule element may be any legal rule expansion defined in Section 2. The scope attribute is explained in the next section. Boston " San Francisco " Madrid Special Cases It is not legal to define an empty rule element or a rule element that contains only white space CDATA. It is legal to define a rule that expands to an empty item or to a single rule reference to NULL. It is legal to define a rule that expands to a single tag element.

TAG-CONTENT 3.2 Scoping of Rule Definitions Each defined rule has a scope. The scope is either 'private' or 'public'. If not explicitly declared in a rule definition then the scope defaults to 'private'. A public-scoped rule may be explicitly referenced (using the fragment identifier syntax of a URI) in the rule definitions of other grammars and in other non-grammar documents. A private-scoped rule cannot be so referenced and is directly accessible only within its containing grammar. A private rule may be explicitly referenced only by other rules within the same grammar.

Informative: grammar authors may consider the following guidance when scoping the rules of a grammar. o Grammar authoring shares many properties of programming. Establishing contracts of an API is analogous to defining a set of grammars and defining the public rules of a grammar each with defined language behavior. o Consistent design and implementation of public rules promotes grammar re-use and facilitates creation of grammar libraries. o Natural language grammars often require creation of many internal 'working' rules to create a smaller number of useful external rules. Hiding working rules with private scope allows revision of those rules without affecting other grammars that reference the grammar. Hiding working rules also prevents accidental mis-use of a working rule. o Grammar compilation resembles programming language compilation. Making rules private allows advanced grammar compilers to perform optimizations that cannot be applied when a rule is declared public.

ABNF FormA rule definition may be annotated with the keywords 'public' or 'private'. If no scope is provided, the default is 'private'. $town = Townsville | Bean town; private $city = Boston | 'New York' | Madrid; public $command = $action $object; XML Form The scope attribute of the rule element defines the scope of the rule definition. Defined values are public and private. If omitted, the default scope is private. TownsvilleBeantown Boston " San Francisco " Madrid 3.3 Example Phrases It is often desirable to include examples of phrases that match rule definitions along with the definition. Zero, one or many example phrases may be provided for any rule definition.

Because the examples are explicitly marked, automated tools can be used for regression testing and for generation of grammar documentation. ABNF FormA documentation comment is a C / C++/Java comment that starts with the sequence of characters / and which immediately precedes the relevant rule definition. Zero or more @example tags may be contained at the end of the documentation comment. The syntax follows the Tagged Paragraph of a documentation comment of the Java Programming Language [JAVA SS 18.4]. The of the example follows the and sequence rules defined in Section 2.1 and Section 2.3 respectively. / A simple directive to execute an action. @example open the window @example close the door / public $command = $action $object; XML Form Any number of 'example' elements may be provided as the initial content within a 'rule' element.

The of the example follows the and sequence rules defined in Section 2.1 and Section 2.3 respectively. open the window close the door 4. Grammar Documents conforming stand-alone grammar document consists of a legal header followed by a body consisting of a set of legal rule definitions. All rules defined within that grammar are scoped within the grammar's rule name name space and each rule name must be legal and unique. It is legal for a grammar to define no rules. The grammar cannot be used for processing input since it defines no patterns for matching user input. o 4.1 Grammar Header Declarations o 4.2 ABNF Self-Identifying Header o 4.3 XML Form Prolog and Root Element o 4.4 Character Encoding o 4.5 Language o 4.6 Mode o 4.7 Root Rule o 4.8 Tag Format o 4.9 Base URI o 4.9. 1 Resolving Relative URIs o 4.10 Pronunciation Lexicon o 4.11 Meta Data o 4.11.

1 Meta and HTTP-Equiv o 4.11. 2 Metadata (XML Only) o 4.12 Tag o 4.13 Comments o 4.14 Grammar Fetching o 4.15 ABNF Keywords 4.1 Grammar Header Declarations legal stand-alone grammar header consists of a number of required declarations and other optional declarations. In addition, the ABNF Form and XML Form each have additional requirements and capabilities of the header that are specific to each syntactic form. The ordering of header declarations is also specific to the two forms.

The table summarizes the information declared in a grammar header and the appropriate representation in the ABNF Form and XML Form. Declaration Status ABNF Form XML Form Grammar version Required SS 4.2: ABNF self-identifying header SS 4.3: version attribute on grammar element XML Namespace Required (XML only) Not applicable SS 4.3: xml ns attribute on grammar element Document type Recommended (XML only) Not applicable SS 4.3: XML DOCTYPE Character encoding Recommended SS 4.4: ABNF self-identifying header SS 4.4: encoding attribute in XML declaration Language Required in voice mode Ignored in DTMF mode SS 4.5: language declaration SS 4.5: xml: lang attribute on grammar element Mode Optional SS 4.6: mode declarationSS 4.6: mode attribute on grammar element Root rule Optional SS 4.7: root declarationSS 4.7: root attribute on grammar element Tag format Optional SS 4.8: tag-format declaration SS 4.8: tag-format attribute on grammar element Base URI Optional SS 4.9: base declarationSS 4.9: xml: base attribute on grammar element Pronunciation lexicon Optional. Multiple allowed. SS 4.10: lexicon declaration SS 4.10: lexicon element Metadata Optional.

Multiple allowed. SS 4.11. 1: meta and http-equiv declarations SS 4.11. 1: meta element XML metadata Optional. (XML Only) Not applicable SS 4.11. 2: metadata element Tag Optional.

Multiple allowed. SS 4.12: tag declarationSS 4.12: tag elementA grammar that complies to this specification must declare the version to be '1.0'. Note: the grammar version indicates the version of the specification implemented by the grammar and is not for versioning of the grammar content. A meta or metadata declaration may be used for content versioning. ABNF Form: Header SummaryA legal header for a stand-alone ABNF document consists of a required ABNF self-identifying header including the grammar version and optional character encoding followed by these declarations in any order: o Language o Mode o Root rule o Tag format o Base URI o Pronunciation lexicon (any number) o Meta and http-equiv (any number) o Tag (any number) ABNF comments may appear between the declarations in the ABNF header after the ABNF self-identifying header. The header declarations are followed by the rule definitions of the grammar.

The following are two examples of ABNF headers. Note that ordering of the declarations (except the ABNF self-identifying header) is unimportant. #ABNF 1.0 ISO-8859-1; language en; mode voice; root $my Rule; tag-format FORMAT-STRING; base; lexicon; lexicon ~; meta 'Author' is 'Stephanie Williams'; http-equiv 'Date' is 'Fri, 10 Feb 2002 17: 27: 21 GMT'; {var x = 1}; #ABNF 1.0; // A French Canadian grammar language fr-CA; // It's a speech grammar mode voice; // Here's the root rule root $Quebec Cities; XML Form: Header SummaryA legal stand-alone XML Form grammar document consists of: 1. Legal XML Prolog 2. Root grammar element with the following attributes o XML name space o XML Schema attributes o Version o Language o Mode o Root rule o Tag format o Base URI 3. grammar element containing any number of the following elements in any order: o Pronunciation lexicon (any number) o Meta and HTTP-Equiv (any number) o Metadata (any number) o Tag (any number) Rule definitions follow the lexicon, meta, metadata and tag declarations.

The following are examples of XML Form grammars headers each including all declarations permitted on the grammar element and one with the DOCTYPE declaration. 4.2 ABNF Self-Identifying Header The ABNF self-identifying header must be present in any legal stand-alone ABNF Form grammar document. The first character of an ABNF document must be the '#'s ymbol (x 23) unless preceded by an optional XML 1.0 byte order mark [XML SS 4.3. 3]. The ABNF byte order mark follows the XML definition and requirements.

For example, documents encoded in UTF-16 must begin with the byte order mark. The optional byte order mark and required '#'s ymbol must be followed immediately by the exact string 'ABNF' (x 41 x 42 x 4d x 46) or the appropriate equivalent for the document's encoding (e.g. for UTF-16 little-end ian: x 23 x 00 x 41 x 00 x 42 x 00 x 4d x 00 x 46 x 00). If the byte order mark is absent on a grammar encoded in UTF-16 then the grammar processor should perform auto-detection of character encoding in a manner analogous to auto-detection of character encoding in XML [XML SSF]. Next follows a single space character (x 20) and the required version number which is '1.0' for this specification (x 31 x 2 e x 30). Next follows an optional character encoding. Section 4.4 defines character encoding's in more detail.

If present, there must be a single space character (x 20) between the version number and the character encoding. The self-identifying header is finalized with a semicolon (x 3 b) followed immediately by a newline. The semicolon must be the first character following the version number or the character encoding if is present. For the remaining declarations of the ABNF header white space is not significant. 4.3 XML Form Prolog and Root ElementA legal stand-alone XML Form grammar document must have a legal XML Prolog [XML SS 2.8]. The XML pro log in an XML Form grammar comprises the XML declaration and an optional DOCTYPE declaration referencing the grammar DTD.

It is followed by the root grammar element. The XML pro log may also contain XML comments, processor instructions and other content permitted by XML in a pro log. The version number of the XML declaration indicates which version of XML is being used. The version number of the grammar element indicates which version of the grammar specification is being used - '1.0' for this specification. The grammar version is a required attribute. The grammar element must designate the grammar name space.

This can be achieved by declaring an xml ns attribute or an attribute with an 'xml ns' prefix. See [XMLNS] for details. Note that when the xml ns attribute is used alone, it sets the default name space for the element on which it appears and for any child elements. The name space for XML Form grammars is defined as web.