rapidyaml  0.11.0
parse and emit YAML, and do it fast
Classes | Public Types | List of all members
c4::yml::ParseEngine< EventHandler > Class Template Reference
Parse utilities

This is the main driver of parsing logic: it scans the YAML or JSON source for tokens, and emits the appropriate sequence of parsing events to its event handler. More...

#include <parse_engine.hpp>

Public Types

using handler_type = EventHandler
 

Public Member Functions

construction and assignment
 ParseEngine (EventHandler *evt_handler, ParserOptions opts={})
 
 ~ParseEngine ()
 
 ParseEngine (ParseEngine &&) noexcept
 
 ParseEngine (ParseEngine const &)
 
ParseEngine & operator= (ParseEngine &&) noexcept
 
ParseEngine & operator= (ParseEngine const &)
 
modifiers
void reserve_stack (id_type capacity)
 Reserve a certain capacity for the parsing stack. More...
 
void reserve_locations (size_t num_source_lines)
 Reserve a certain capacity for the array used to track node locations in the source buffer. More...
 
void reserve_filter_arena (size_t)
 
getters
ParserOptions const & options () const
 Get the options used to build this parser object. More...
 
Callbacks const & callbacks () const
 Get the current callbacks in the parser. More...
 
csubstr filename () const
 Get the name of the latest file parsed by this object. More...
 
csubstr source () const
 Get the latest YAML buffer parsed by this object. More...
 
Encoding_e encoding () const
 Get the encoding of the latest YAML buffer parsed by this object. More...
 
id_type stack_capacity () const
 
size_t locations_capacity () const
 
size_t filter_arena_capacity () const
 
parse methods
void parse_in_place_ev (csubstr filename, substr src)
 parse YAML in place, emitting events to the current handler More...
 
void parse_json_in_place_ev (csubstr filename, substr src)
 parse JSON in place, emitting events to the current handler More...
 
locations
csubstr location_contents (Location const &loc) const
 Get the string starting at a particular location, to the end of the parsed source buffer. More...
 
Location val_location (const char *val) const
 Given a pointer to a buffer position, get the location. More...
 
scalar filtering
FilterResult filter_scalar_plain (csubstr scalar, substr dst, size_t indentation)
 filter a plain scalar More...
 
FilterResult filter_scalar_plain_in_place (substr scalar, size_t cap, size_t indentation)
 filter a plain scalar in place More...
 
FilterResult filter_scalar_squoted (csubstr scalar, substr dst)
 filter a single-quoted scalar More...
 
FilterResult filter_scalar_squoted_in_place (substr scalar, size_t cap)
 filter a single-quoted scalar in place More...
 
FilterResult filter_scalar_dquoted (csubstr scalar, substr dst)
 filter a double-quoted scalar More...
 
FilterResultExtending filter_scalar_dquoted_in_place (substr scalar, size_t cap)
 filter a double-quoted scalar in place More...
 
FilterResult filter_scalar_block_literal (csubstr scalar, substr dst, size_t indentation, BlockChomp_e chomp)
 filter a block-literal scalar More...
 
FilterResult filter_scalar_block_literal_in_place (substr scalar, size_t cap, size_t indentation, BlockChomp_e chomp)
 filter a block-literal scalar in place More...
 
FilterResult filter_scalar_block_folded (csubstr scalar, substr dst, size_t indentation, BlockChomp_e chomp)
 filter a block-folded scalar More...
 
FilterResult filter_scalar_block_folded_in_place (substr scalar, size_t cap, size_t indentation, BlockChomp_e chomp)
 filter a block-folded scalar in place More...
 

Detailed Description

template<class EventHandler>
class c4::yml::ParseEngine< EventHandler >

This is the main driver of parsing logic: it scans the YAML or JSON source for tokens, and emits the appropriate sequence of parsing events to its event handler.

The parse engine itself has no special limitations, and can accomodate containers as keys; it is the event handler may introduce additional constraints.

There are two implemented handlers (see Event Handlers, which has important notes about the event model):

  • EventHandlerTree is the handler responsible for creating the ryml Tree
  • extra::EventHandlerTestSuite is a handler responsible for emitting standardized YAML test suite events, used (only) in the CI of this project. This is not part of the library and is not installed.
  • extra::EventHandlerInts is the handler responsible for emitting integer-coded events. It is intended for implementing fully-conformant parsing in other programming languages (integration is currently under work for YamlScript and go-yaml). It is not part of the library and is not installed.

Definition at line 264 of file parse_engine.hpp.

Member Typedef Documentation

◆ handler_type

template<class EventHandler >
using c4::yml::ParseEngine< EventHandler >::handler_type = EventHandler

Definition at line 268 of file parse_engine.hpp.

Constructor & Destructor Documentation

◆ ParseEngine() [1/3]

template<class EventHandler >
c4::yml::ParseEngine< EventHandler >::ParseEngine ( EventHandler *  evt_handler,
ParserOptions  opts = {} 
)

Definition at line 262 of file parse_engine.def.hpp.

263  : m_options(opts)
264  , m_file()
265  , m_buf()
266  , m_evt_handler(evt_handler)
267  , m_pending_anchors()
268  , m_pending_tags()
269  , m_was_inside_qmrk(false)
270  , m_doc_empty(false)
271  , m_prev_colon(npos)
272  , m_encoding(NOBOM)
273  , m_newline_offsets()
274  , m_newline_offsets_size(0)
275  , m_newline_offsets_capacity(0)
276  , m_newline_offsets_buf()
277 {
278  _RYML_CHECK_BASIC(evt_handler);
279 }
c4::yml::npos
@ npos
a null string position
Definition: common.hpp:258
c4::yml::NOBOM
@ NOBOM
No Byte Order Mark was found.
Definition: common.hpp:263

◆ ~ParseEngine()

template<class EventHandler >
c4::yml::ParseEngine< EventHandler >::~ParseEngine

Definition at line 255 of file parse_engine.def.hpp.

256 {
257  _free();
258  _clr();
259 }

◆ ParseEngine() [2/3]

template<class EventHandler >
c4::yml::ParseEngine< EventHandler >::ParseEngine ( ParseEngine< EventHandler > &&  that)
noexcept

Definition at line 282 of file parse_engine.def.hpp.

283  : m_options(that.m_options)
284  , m_file(that.m_file)
285  , m_buf(that.m_buf)
286  , m_evt_handler(that.m_evt_handler)
287  , m_pending_anchors(that.m_pending_anchors)
288  , m_pending_tags(that.m_pending_tags)
289  , m_was_inside_qmrk(false)
290  , m_doc_empty(false)
291  , m_prev_colon(npos)
292  , m_encoding(NOBOM)
293  , m_newline_offsets(that.m_newline_offsets)
294  , m_newline_offsets_size(that.m_newline_offsets_size)
295  , m_newline_offsets_capacity(that.m_newline_offsets_capacity)
296  , m_newline_offsets_buf(that.m_newline_offsets_buf)
297 {
298  that._clr();
299 }

◆ ParseEngine() [3/3]

template<class EventHandler >
c4::yml::ParseEngine< EventHandler >::ParseEngine ( ParseEngine< EventHandler > const &  that)

Definition at line 302 of file parse_engine.def.hpp.

303  : m_options(that.m_options)
304  , m_file(that.m_file)
305  , m_buf(that.m_buf)
306  , m_evt_handler(that.m_evt_handler)
307  , m_pending_anchors(that.m_pending_anchors)
308  , m_pending_tags(that.m_pending_tags)
309  , m_was_inside_qmrk(false)
310  , m_doc_empty(false)
311  , m_prev_colon(npos)
312  , m_encoding(NOBOM)
313  , m_newline_offsets()
314  , m_newline_offsets_size()
315  , m_newline_offsets_capacity()
316  , m_newline_offsets_buf()
317 {
318  if(that.m_newline_offsets_capacity)
319  {
320  _resize_locations(that.m_newline_offsets_capacity);
321  _RYML_CHECK_BASIC_(m_evt_handler->m_stack.m_callbacks, m_newline_offsets_capacity == that.m_newline_offsets_capacity);
322  memcpy(m_newline_offsets, that.m_newline_offsets, that.m_newline_offsets_size * sizeof(size_t));
323  m_newline_offsets_size = that.m_newline_offsets_size;
324  }
325 }

Member Function Documentation

◆ operator=() [1/2]

template<class EventHandler >
ParseEngine< EventHandler > & c4::yml::ParseEngine< EventHandler >::operator= ( ParseEngine< EventHandler > &&  that)
noexcept

Definition at line 328 of file parse_engine.def.hpp.

329 {
330  _free();
331  m_options = (that.m_options);
332  m_file = (that.m_file);
333  m_buf = (that.m_buf);
334  m_evt_handler = that.m_evt_handler;
335  m_pending_anchors = that.m_pending_anchors;
336  m_pending_tags = that.m_pending_tags;
337  m_was_inside_qmrk = that.m_was_inside_qmrk;
338  m_doc_empty = that.m_doc_empty;
339  m_prev_colon = that.m_prev_colon;
340  m_encoding = that.m_encoding;
341  m_newline_offsets = (that.m_newline_offsets);
342  m_newline_offsets_size = (that.m_newline_offsets_size);
343  m_newline_offsets_capacity = (that.m_newline_offsets_capacity);
344  m_newline_offsets_buf = (that.m_newline_offsets_buf);
345  that._clr();
346  return *this;
347 }

◆ operator=() [2/2]

template<class EventHandler >
ParseEngine< EventHandler > & c4::yml::ParseEngine< EventHandler >::operator= ( ParseEngine< EventHandler > const &  that)

Definition at line 350 of file parse_engine.def.hpp.

351 {
352  if(&that != this)
353  {
354  _free();
355  m_options = (that.m_options);
356  m_file = (that.m_file);
357  m_buf = (that.m_buf);
358  m_evt_handler = that.m_evt_handler;
359  m_pending_anchors = that.m_pending_anchors;
360  m_pending_tags = that.m_pending_tags;
361  m_was_inside_qmrk = that.m_was_inside_qmrk;
362  m_doc_empty = that.m_doc_empty;
363  m_prev_colon = that.m_prev_colon;
364  m_encoding = that.m_encoding;
365  if(that.m_newline_offsets_capacity > m_newline_offsets_capacity)
366  _resize_locations(that.m_newline_offsets_capacity);
367  _RYML_CHECK_BASIC_(m_evt_handler->m_stack.m_callbacks, m_newline_offsets_capacity >= that.m_newline_offsets_capacity);
368  _RYML_CHECK_BASIC_(m_evt_handler->m_stack.m_callbacks, m_newline_offsets_capacity >= that.m_newline_offsets_size);
369  memcpy(m_newline_offsets, that.m_newline_offsets, that.m_newline_offsets_size * sizeof(size_t));
370  m_newline_offsets_size = that.m_newline_offsets_size;
371  m_newline_offsets_buf = that.m_newline_offsets_buf;
372  }
373  return *this;
374 }

◆ reserve_stack()

template<class EventHandler >
void c4::yml::ParseEngine< EventHandler >::reserve_stack ( id_type  capacity)
inline

Reserve a certain capacity for the parsing stack.

This should be larger than the expected depth of the parsed YAML tree.

The parsing stack is the only (potential) heap memory used directly by the parser.

If the requested capacity is below the default stack size of 16, the memory is used directly in the parser object; otherwise it will be allocated from the heap.

Note
this reserves memory only for the parser itself; all the allocations for the parsed tree will go through the tree's allocator (when different).
for maximum efficiency, the tree and the arena can (and should) also be reserved.

Definition at line 307 of file parse_engine.hpp.

308  {
309  m_evt_handler->m_stack.reserve(capacity);
310  }

◆ reserve_locations()

template<class EventHandler >
void c4::yml::ParseEngine< EventHandler >::reserve_locations ( size_t  num_source_lines)
inline

Reserve a certain capacity for the array used to track node locations in the source buffer.

Definition at line 314 of file parse_engine.hpp.

315  {
316  _resize_locations(num_source_lines);
317  }

◆ reserve_filter_arena()

template<class EventHandler >
void c4::yml::ParseEngine< EventHandler >::reserve_filter_arena ( size_t  )
inline

Definition at line 320 of file parse_engine.hpp.

320 {}

◆ options()

template<class EventHandler >
ParserOptions const& c4::yml::ParseEngine< EventHandler >::options ( ) const
inline

Get the options used to build this parser object.

Definition at line 330 of file parse_engine.hpp.

330 { return m_options; }

◆ callbacks()

template<class EventHandler >
Callbacks const& c4::yml::ParseEngine< EventHandler >::callbacks ( ) const
inline

Get the current callbacks in the parser.

Definition at line 333 of file parse_engine.hpp.

333 { _RYML_ASSERT_BASIC(m_evt_handler); return m_evt_handler->m_stack.m_callbacks; }

◆ filename()

template<class EventHandler >
csubstr c4::yml::ParseEngine< EventHandler >::filename ( ) const
inline

Get the name of the latest file parsed by this object.

Definition at line 336 of file parse_engine.hpp.

336 { return m_file; }

◆ source()

template<class EventHandler >
csubstr c4::yml::ParseEngine< EventHandler >::source ( ) const
inline

Get the latest YAML buffer parsed by this object.

Definition at line 339 of file parse_engine.hpp.

339 { return m_buf; }

◆ encoding()

template<class EventHandler >
Encoding_e c4::yml::ParseEngine< EventHandler >::encoding ( ) const
inline

Get the encoding of the latest YAML buffer parsed by this object.

If no encoding was specified, UTF8 is assumed as per the YAML standard.

Definition at line 343 of file parse_engine.hpp.

343 { return m_encoding != NOBOM ? m_encoding : UTF8; }
c4::yml::UTF8
@ UTF8
UTF8.
Definition: common.hpp:264

References c4::yml::NOBOM, and c4::yml::UTF8.

◆ stack_capacity()

template<class EventHandler >
id_type c4::yml::ParseEngine< EventHandler >::stack_capacity ( ) const
inline

Definition at line 345 of file parse_engine.hpp.

345 { _RYML_ASSERT_BASIC(m_evt_handler); return m_evt_handler->m_stack.capacity(); }

◆ locations_capacity()

template<class EventHandler >
size_t c4::yml::ParseEngine< EventHandler >::locations_capacity ( ) const
inline

Definition at line 346 of file parse_engine.hpp.

346 { return m_newline_offsets_capacity; }

◆ filter_arena_capacity()

template<class EventHandler >
size_t c4::yml::ParseEngine< EventHandler >::filter_arena_capacity ( ) const
inline

Definition at line 349 of file parse_engine.hpp.

349 { return 0u; }

◆ parse_in_place_ev()

template<class EventHandler >
void c4::yml::ParseEngine< EventHandler >::parse_in_place_ev ( csubstr  filename,
substr  src 
)

parse YAML in place, emitting events to the current handler

◆ parse_json_in_place_ev()

template<class EventHandler >
void c4::yml::ParseEngine< EventHandler >::parse_json_in_place_ev ( csubstr  filename,
substr  src 
)

parse JSON in place, emitting events to the current handler

◆ location_contents()

template<class EventHandler >
csubstr c4::yml::ParseEngine< EventHandler >::location_contents ( Location const &  loc) const

Get the string starting at a particular location, to the end of the parsed source buffer.

◆ val_location()

template<class EventHandler >
Location c4::yml::ParseEngine< EventHandler >::val_location ( const char *  val) const

Given a pointer to a buffer position, get the location.

Parameters
[in]valmust be pointing to somewhere in the source buffer that was last parsed by this object.

◆ filter_scalar_plain()

template<class EventHandler >
FilterResult c4::yml::ParseEngine< EventHandler >::filter_scalar_plain ( csubstr  scalar,
substr  dst,
size_t  indentation 
)

filter a plain scalar

◆ filter_scalar_plain_in_place()

template<class EventHandler >
FilterResult c4::yml::ParseEngine< EventHandler >::filter_scalar_plain_in_place ( substr  scalar,
size_t  cap,
size_t  indentation 
)

filter a plain scalar in place

◆ filter_scalar_squoted()

template<class EventHandler >
FilterResult c4::yml::ParseEngine< EventHandler >::filter_scalar_squoted ( csubstr  scalar,
substr  dst 
)

filter a single-quoted scalar

◆ filter_scalar_squoted_in_place()

template<class EventHandler >
FilterResult c4::yml::ParseEngine< EventHandler >::filter_scalar_squoted_in_place ( substr  scalar,
size_t  cap 
)

filter a single-quoted scalar in place

◆ filter_scalar_dquoted()

template<class EventHandler >
FilterResult c4::yml::ParseEngine< EventHandler >::filter_scalar_dquoted ( csubstr  scalar,
substr  dst 
)

filter a double-quoted scalar

◆ filter_scalar_dquoted_in_place()

template<class EventHandler >
FilterResultExtending c4::yml::ParseEngine< EventHandler >::filter_scalar_dquoted_in_place ( substr  scalar,
size_t  cap 
)

filter a double-quoted scalar in place

◆ filter_scalar_block_literal()

template<class EventHandler >
FilterResult c4::yml::ParseEngine< EventHandler >::filter_scalar_block_literal ( csubstr  scalar,
substr  dst,
size_t  indentation,
BlockChomp_e  chomp 
)

filter a block-literal scalar

◆ filter_scalar_block_literal_in_place()

template<class EventHandler >
FilterResult c4::yml::ParseEngine< EventHandler >::filter_scalar_block_literal_in_place ( substr  scalar,
size_t  cap,
size_t  indentation,
BlockChomp_e  chomp 
)

filter a block-literal scalar in place

◆ filter_scalar_block_folded()

template<class EventHandler >
FilterResult c4::yml::ParseEngine< EventHandler >::filter_scalar_block_folded ( csubstr  scalar,
substr  dst,
size_t  indentation,
BlockChomp_e  chomp 
)

filter a block-folded scalar

◆ filter_scalar_block_folded_in_place()

template<class EventHandler >
FilterResult c4::yml::ParseEngine< EventHandler >::filter_scalar_block_folded_in_place ( substr  scalar,
size_t  cap,
size_t  indentation,
BlockChomp_e  chomp 
)

filter a block-folded scalar in place

The documentation for this class was generated from the following files: