Metastatic.NL.AST (Metastatic v0.33.0)

View Source

M2: Meta-Model for Natural Language Abstract Syntax (NL-MetaAST).

Provides a universal, language-neutral abstract representation for natural language text, sentences, clauses, phrases, and tokens.

Meta-Modeling Hierarchy

  • M3: Elixir type system (@type, @spec)
  • M2: NL-MetaAST (this module) - defines what natural language AST nodes can be
  • M1: Language-specific concrete syntax trees (Tree-sitter English CST, Universal Dependencies, etc.)
  • M0: Concrete text utterances and written documents

Uniform 3-Tuple Representation

All NL-MetaAST nodes are uniform 3-element tuples:

{type_atom, keyword_meta, children_or_value}

Where:

  • type_atom - Node kind (e.g., :document, :sentence, :clause, :phrase, :token)
  • keyword_meta - Keyword list containing metadata (POS, lemma, voice, aspect, line, col, etc.)
  • children_or_value - Either a string/nil (for leaf nodes) or a list of child nodes

Stratification

  • M2.1 Core: Universal linguistic units present in all human languages: :document, :section, :paragraph, :sentence, :clause, :phrase, :token, :punctuation, :coordination, :subordination.
  • M2.2 Extended: Typological variations and semantic enrichment: Tense, Aspect, Mood, Voice (TAM), grammatical gender/number/case, hedging/weasel tags, semantic frames, and coreference indices.
  • M2.3 Native: Language-specific escape hatches via :native_nl_feature.

Summary

Types

aspect()

Grammatical aspect

clause_type()

Clause classification

hedging()

Hedging / weasel classification

nl_node()

An NL-MetaAST node 3-tuple

phrase_type()

Phrase syntactic types

pos_tag()

Universal POS (Part-of-Speech) tags

sentence_kind()

Sentence modality / kind

tense()

Grammatical tense

voice()

Grammatical voice

Functions

children(node)

Returns the children list or leaf value of an NL node.

clause(children, meta \\ [])

Constructs a :clause node.

clauses(node)

Extracts all :clause nodes from the tree.

conforms?(arg1)

Determines whether an arbitrary term conforms to the NL-MetaAST specification.

core_type?(type)

Returns whether a node type belongs to the Core (M2.1) layer.

document(children, meta \\ [])

Constructs a :document node.

extended_type?(type)

Returns whether a node type belongs to the Extended (M2.2) layer.

get_meta(node, key, default \\ nil)

Gets a metadata value by key, returning default if absent.

lemmas(node)

Extracts all distinct lemmas from the tokens.

meta(node)

Returns the metadata keyword list of an NL node.

native_type?(type)

Returns whether a node type belongs to the Native (M2.3) layer.

paragraph(children, meta \\ [])

Constructs a :paragraph node.

path(node, fun)

Path from a matching NL node up to the root, or nil.

phrase(type, children, meta \\ [])

Constructs a :phrase composite node.

phrases(node)

Extracts all :phrase nodes from the tree.

postwalk(node, fun)

Post-walk traversal over an NL-MetaAST.

prewalk(node, fun)

Pre-walk traversal over an NL-MetaAST.

prewalker(node)

Lazy depth-first pre-order enumerable over an NL-MetaAST.

prewalker(node, acc, fun)

Lazy pre-order enumerable that threads an accumulator.

punctuation(symbol, meta \\ [])

Constructs a :punctuation leaf node.

put_meta(node, key, value)

Sets or updates a metadata key.

section(children, meta \\ [])

Constructs a :section node.

sentence(children, meta \\ [])

Constructs a :sentence node.

sentences(node)

Extracts all :sentence nodes from the tree.

supported_types()

Returns the list of all supported NL-MetaAST node types.

text(node)

Reconstructs readable text from the leaf tokens and punctuation.

token(text, meta \\ [])

Constructs a :token leaf node.

tokens(node)

Extracts all :token nodes from the tree.

traverse(node, acc, pre, post)

Traverses an NL-MetaAST depth-first with pre-visit and post-visit functions.

type(node)

Returns the type atom of an NL node.

update_meta(node, updates)

Merges a keyword list of metadata updates into the node.

update_meta(node, key, default, fun)

Updates a metadata value with a function.

words(node)

Extracts all string words from word-like tokens (excluding punctuation).

Types

aspect()

@type aspect() :: :simple | :progressive | :perfect | :perfect_progressive

Grammatical aspect

clause_type()

@type clause_type() ::
  :main | :subordinate | :relative | :conditional | :coordinate | :adverbial

Clause classification

hedging()

@type hedging() :: :none | :weasel | :tentative | :certain

Hedging / weasel classification

nl_node()

@type nl_node() :: {atom(), keyword(), term()}

An NL-MetaAST node 3-tuple

phrase_type()

@type phrase_type() ::
  :noun_phrase
  | :verb_phrase
  | :prep_phrase
  | :adj_phrase
  | :adverb_phrase
  | :conjunction_phrase
  | :interjection_phrase

Phrase syntactic types

pos_tag()

@type pos_tag() ::
  :noun
  | :proper_noun
  | :verb
  | :adjective
  | :adverb
  | :pronoun
  | :determiner
  | :preposition
  | :postposition
  | :conjunction
  | :auxiliary
  | :numeral
  | :particle
  | :symbol
  | :punctuation
  | :interjection
  | :other

Universal POS (Part-of-Speech) tags

sentence_kind()

@type sentence_kind() :: :declarative | :interrogative | :imperative | :exclamative

Sentence modality / kind

tense()

@type tense() :: :present | :past | :future

Grammatical tense

voice()

@type voice() :: :active | :passive | :middle

Grammatical voice

Functions

children(node)

@spec children(nl_node()) :: term()

Returns the children list or leaf value of an NL node.

clause(children, meta \\ [])

@spec clause([nl_node()], keyword()) :: nl_node()

Constructs a :clause node.

clauses(node)

@spec clauses(nl_node()) :: [nl_node()]

Extracts all :clause nodes from the tree.

conforms?(arg1)

@spec conforms?(term()) :: boolean()

Determines whether an arbitrary term conforms to the NL-MetaAST specification.

core_type?(type)

@spec core_type?(atom()) :: boolean()

Returns whether a node type belongs to the Core (M2.1) layer.

document(children, meta \\ [])

@spec document([nl_node()], keyword()) :: nl_node()

Constructs a :document node.

extended_type?(type)

@spec extended_type?(atom()) :: boolean()

Returns whether a node type belongs to the Extended (M2.2) layer.

get_meta(node, key, default \\ nil)

@spec get_meta(nl_node(), atom(), term()) :: term()

Gets a metadata value by key, returning default if absent.

lemmas(node)

@spec lemmas(nl_node()) :: [String.t()]

Extracts all distinct lemmas from the tokens.

meta(node)

@spec meta(nl_node()) :: keyword()

Returns the metadata keyword list of an NL node.

native_type?(type)

@spec native_type?(atom()) :: boolean()

Returns whether a node type belongs to the Native (M2.3) layer.

paragraph(children, meta \\ [])

@spec paragraph([nl_node()], keyword()) :: nl_node()

Constructs a :paragraph node.

path(node, fun)

@spec path(nl_node(), (nl_node() -> as_boolean(term()))) :: [nl_node()] | nil

Path from a matching NL node up to the root, or nil.

phrase(type, children, meta \\ [])

@spec phrase(phrase_type(), [nl_node()], keyword()) :: nl_node()

Constructs a :phrase composite node.

phrases(node)

@spec phrases(nl_node()) :: [nl_node()]

Extracts all :phrase nodes from the tree.

postwalk(node, fun)

@spec postwalk(nl_node(), (nl_node() -> nl_node())) :: nl_node()

Post-walk traversal over an NL-MetaAST.

prewalk(node, fun)

@spec prewalk(nl_node(), (nl_node() -> nl_node())) :: nl_node()

Pre-walk traversal over an NL-MetaAST.

prewalker(node)

@spec prewalker(nl_node()) :: Enumerable.t()

Lazy depth-first pre-order enumerable over an NL-MetaAST.

prewalker(node, acc, fun)

@spec prewalker(nl_node(), acc, (nl_node(), acc -> {term(), acc})) :: Enumerable.t()
when acc: term()

Lazy pre-order enumerable that threads an accumulator.

punctuation(symbol, meta \\ [])

@spec punctuation(String.t(), keyword()) :: nl_node()

Constructs a :punctuation leaf node.

put_meta(node, key, value)

@spec put_meta(nl_node(), atom(), term()) :: nl_node()

Sets or updates a metadata key.

section(children, meta \\ [])

@spec section([nl_node()], keyword()) :: nl_node()

Constructs a :section node.

sentence(children, meta \\ [])

@spec sentence([nl_node()], keyword()) :: nl_node()

Constructs a :sentence node.

sentences(node)

@spec sentences(nl_node()) :: [nl_node()]

Extracts all :sentence nodes from the tree.

supported_types()

@spec supported_types() :: [atom()]

Returns the list of all supported NL-MetaAST node types.

text(node)

@spec text(nl_node()) :: String.t()

Reconstructs readable text from the leaf tokens and punctuation.

token(text, meta \\ [])

@spec token(String.t() | nil, keyword()) :: nl_node()

Constructs a :token leaf node.

tokens(node)

@spec tokens(nl_node()) :: [nl_node()]

Extracts all :token nodes from the tree.

traverse(node, acc, pre, post)

@spec traverse(nl_node(), acc, (nl_node(), acc -> {nl_node(), acc}), (nl_node(),
                                                                acc ->
                                                                  {nl_node(),
                                                                   acc})) ::
  {nl_node(), acc}
when acc: term()

Traverses an NL-MetaAST depth-first with pre-visit and post-visit functions.

Delegates to Metastatic.MetaAST.traverse/4, which also visits metadata values that contain nodes (e.g. a :coreference target).

type(node)

@spec type(nl_node()) :: atom()

Returns the type atom of an NL node.

update_meta(node, updates)

@spec update_meta(nl_node(), keyword()) :: nl_node()

Merges a keyword list of metadata updates into the node.

update_meta(node, key, default, fun)

@spec update_meta(nl_node(), atom(), term(), (term() -> term())) :: nl_node()

Updates a metadata value with a function.

words(node)

@spec words(nl_node()) :: [String.t()]

Extracts all string words from word-like tokens (excluding punctuation).