|
rapidyaml 0.15.2
parse and emit YAML, and do it fast
|
A parser event handler that creates a compact representation of the YAML tree in a contiguous buffer of integers. More...
#include <event_handler_ints.hpp>
Public Types | |
types | |
| enum | { requires_strings_on_buffers = true } |
| using | value_type = ievt::evt_bits |
| using | state = EventHandlerIntsState |
Public Member Functions | |
construction and resetting | |
| EventHandlerInts (c4::yml::Callbacks const &cb) noexcept | |
| EventHandlerInts () noexcept | |
| void | reset (substr str, substr arena, ievt::evt_bits *dst, evt_size dst_size) |
| evt_size | required_size_events () const |
| get the size needed for the event buffer from the previous parse | |
| size_t | required_size_arena () const |
| get the size needed for the arena from the previous parse | |
| bool | fits_buffers () const |
| Predicate to test if the event and arena buffers successfully accomodated all the parse events. | |
| void | reserve_arena (int) |
| TagDirectives & | tag_directives () |
| TagCache & | tag_cache () |
parse events | |
| void | start_parse (const char *filename, substr src) |
| void | finish_parse () |
| void | cancel_parse () |
YAML stream events | |
| void | begin_stream () |
| void | end_stream () |
YAML document events | |
| void | begin_doc () |
| implicit doc start (without —) | |
| void | end_doc () |
| implicit doc end (without ...) | |
| void | begin_doc_expl () |
| explicit doc start, with — | |
| void | end_doc_expl () |
| explicit doc end, with ... | |
YAML map functions | |
| void | begin_map_key_flow () |
| void | begin_map_key_block () |
| void | begin_map_val_flow () |
| void | begin_map_val_block () |
| void | end_map_block () |
| void | end_map_flow (bool, type_bits=FLOW_ML1) |
YAML seq events | |
| void | begin_seq_key_flow () |
| void | begin_seq_key_block () |
| void | begin_seq_val_flow () |
| void | begin_seq_val_block () |
| void | end_seq_block () |
| void | end_seq_flow (bool, type_bits=FLOW_ML1) |
YAML structure events | |
| void | add_sibling () |
| void | actually_val_is_first_key_of_new_map_flow () |
| set the previous val as the first key of a new map, with flow style. | |
| void | actually_val_is_first_key_of_new_map_block () |
| like its flow counterpart, but this function can only be called after the end of a flow-val at root or doc level. | |
YAML scalar events | |
| void | set_key_scalar_plain_empty () |
| void | set_val_scalar_plain_empty () |
| void | set_key_scalar_plain (csubstr scalar) |
| void | set_val_scalar_plain (csubstr scalar) |
| void | set_key_scalar_dquoted (csubstr scalar) |
| void | set_val_scalar_dquoted (csubstr scalar) |
| void | set_key_scalar_squoted (csubstr scalar) |
| void | set_val_scalar_squoted (csubstr scalar) |
| void | set_key_scalar_literal (csubstr scalar) |
| void | set_val_scalar_literal (csubstr scalar) |
| void | set_key_scalar_folded (csubstr scalar) |
| void | set_val_scalar_folded (csubstr scalar) |
| void | mark_key_scalar_unfiltered () |
| void | mark_val_scalar_unfiltered () |
YAML anchor/reference events | |
| void | set_key_anchor (csubstr anchor) |
| void | set_val_anchor (csubstr anchor) |
| void | set_key_ref (csubstr ref) |
| void | set_val_ref (csubstr ref) |
YAML tag events | |
| void | set_key_tag (csubstr tag) |
| void | set_val_tag (csubstr tag) |
YAML directive events | |
| void | add_directive_yaml (csubstr yaml_version) |
| void | add_directive_tag (csubstr handle, csubstr prefix) |
arena events | |
| substr | arena () |
| substr | arena_rem () |
| substr | alloc_arena (size_t len) |
| this may fail, in which case an empty string is returned | |
implementation helpers | |
| void | _push () |
| push a new parent, add a child to the new parent, and set the child as the current node | |
| void | _pop () |
| end the current scope | |
| template<c4::yml::type_bits bits> | |
| void | enable_ () noexcept |
| template<c4::yml::type_bits bits> | |
| void | disable_ () noexcept |
| template<c4::yml::type_bits bits> | |
| bool | has_any_ () const noexcept |
| evt_size | _next (evt_size pos) const noexcept |
| evt_size | _prev (evt_size pos) const noexcept |
| bool | _is_sub_ (csubstr str) const noexcept |
| void | _send_flag_only_ (ievt::evt_bits flags) |
| void | _send_str_ (csubstr scalar, ievt::evt_bits flags) |
| void | _mark_parent_with_children_ () |
| csubstr | _get_latest_empty_scalar () const |
| evt_size | _find_last_bdoc (evt_size pos) const |
| evt_size | _find_matching_open (ievt::evt_bits open, ievt::evt_bits close, evt_size pos) const |
| evt_size | _extend_left_to_include_tag_and_or_anchor (evt_size pos) const |
Public Attributes | |
| detail::stack< state > | m_stack |
| state * | m_curr |
| current stack level: top of the stack. cached here for easier access. | |
| state * | m_parent |
| parent of the current stack level. | |
| substr | m_src |
Protected Member Functions | |
| void | _stack_start_parse (const char *filename, substr ymlsrc) |
| void | _stack_finish_parse () |
| void | _stack_reset_root () |
| void | _stack_reset_non_root () |
| void | _stack_push () |
| void | _stack_pop () |
| bool | _stack_should_push_on_begin_doc () const |
| bool | _stack_should_pop_on_end_doc () const |
A parser event handler that creates a compact representation of the YAML tree in a contiguous buffer of integers.
The integers are ievt::EventBits containing masks (to represent events), interleaved with offset+length (to represent strings in the source buffer).
This is meant for use by other programming languages, and supports container keys (unlike the ryml tree). It parses faster than the ryml tree parser, because the resulting data structure is much simpler.
The resulting integer buffer is a linear array of integers containing events (as a mask of ievt::EventBits), which in some cases (see ievt::WSTR) are followed by an encoded string (encoded as an offset and length to the parsed source buffer).
For example, parsing [a, bb, ccc] results in the following event buffer (grouped to highlight the event sequence structure):
Here is a sketch clarifying the meaning of this event sequence:
Note that the buffer contains both events and strings encoded as integer pairs. That is, events that have an associated string are immediately followed by two integers providing the offset and length of that string in the source buffer. (In the example above, this happens in the events for the strings a, bb, and ccc at positions 3, 6 and 9, respectively).
The flag ievt::PSTR and the mask ievt::WSTR are provided to enable easier iteration over the array: you can use them to test for presence of a string when iterating over the array.
The flag ievt::PSTR announces that an event is preceded by a string. That is, the previous event has a string, so that when this flag is found while iterating right-to-left, a jump of -3 should be used to get at the bitmask of the previous event. (In the example above, this flag is present for the events for bb and ccc, but not a because it is not preceded by a string).
Likewise, to signify that the current event is followed by a string, there is the mask ievt::WSTR, which is a mask of all the flags of events that have a string: ievt::SCLR, ievt::ALIA, ievt::ANCH and ievt::TAG_. While iterating left-to-right in the array, presence of any of the bits in the mask ievt::WSTR means that a jump of +3 should be employed to get at the bitmask of the next event.
Here's another example with the result of parsing a: bb
Typical code to iterate left-to-right over the array will look like this:
This handler must be initialized with the input source buffer, the output arena, and the output event buffer. This handler will not take ownership nor attempt to resize the output buffer. If the size required for the output buffer or arena are larger than their actual size, parsing goes all way to the end, determining the required buffer sizes without writing anything past the end of the respective buffer. After parsing is finished, the user must ensure that the buffer size was enough to accomodate all the data that needs to be written into it, or react accordingly (eg, throw an error, or resize the buffer then retry the parse).
A couple of functions will be helpful to do this. After parsing, EventHandlerInts::fits_buffers() must be used to verify that the output buffers were enough to accomodate the results. Then, EventHandlerInts::required_size_events() and EventHandlerInts::required_size_arena() can be used to retrieve to necessary information. To get an estimation of the number of events before parsing, see estimate_events_ints_size().
Typical code to parse YAML with this handler will look like this:
The result of estimate_events_ints_size() (click to see more info) must be an overprediction: it overpredicts for every single case among the many hundreds covered in the unit tests. This is deliberate, and aims at ensuring that a retry parse is not needed. But conceivably, it may underpredict in some instances not found in the out tests. What to do then?
First, open an issue to allow the estimation to be improved! Second, there are two ways to handle this situation in code:
1) throw an error (as sketched above)
2) grow the buffer to the required size (see EventHandlerInts::required_size_events()), and then parse again
If your code must be able to handle any case including where the prediction undershoots before the estimate function is fixed (after you open the issue), that is, if you are considering a parse retry, there is something important that needs attention. The YAML source buffer is mutated in-place during the parse, and cannot be used to parse again. So if you want to retry, you need to keep a pristine copy of the source, and use it for the retry:
When bringing this to other programming languages, the semantics will be very similar to this.
Definition at line 458 of file event_handler_ints.hpp.
Definition at line 464 of file event_handler_ints.hpp.
| using c4::yml::extra::EventHandlerInts::state = EventHandlerIntsState |
Definition at line 465 of file event_handler_ints.hpp.
| anonymous enum |
| Enumerator | |
|---|---|
| requires_strings_on_buffers | |
Definition at line 466 of file event_handler_ints.hpp.
|
inlinenoexcept |
Definition at line 494 of file event_handler_ints.hpp.
Referenced by EventHandlerInts().
|
inlinenoexcept |
Definition at line 499 of file event_handler_ints.hpp.
|
inline |
Definition at line 504 of file event_handler_ints.hpp.
Referenced by EventHandlerInts().
|
inline |
get the size needed for the event buffer from the previous parse
Definition at line 524 of file event_handler_ints.hpp.
Referenced by IntsResult::resize_buffers().
|
inline |
get the size needed for the arena from the previous parse
Definition at line 531 of file event_handler_ints.hpp.
Referenced by IntsResult::resize_buffers().
|
inline |
Predicate to test if the event and arena buffers successfully accomodated all the parse events.
Definition at line 540 of file event_handler_ints.hpp.
|
inline |
Definition at line 545 of file event_handler_ints.hpp.
|
inline |
Definition at line 550 of file event_handler_ints.hpp.
|
inline |
Definition at line 551 of file event_handler_ints.hpp.
|
inline |
Definition at line 560 of file event_handler_ints.hpp.
|
inline |
Definition at line 567 of file event_handler_ints.hpp.
|
inline |
Definition at line 572 of file event_handler_ints.hpp.
|
inline |
Definition at line 585 of file event_handler_ints.hpp.
|
inline |
Definition at line 590 of file event_handler_ints.hpp.
|
inline |
implicit doc start (without —)
Definition at line 603 of file event_handler_ints.hpp.
|
inline |
implicit doc end (without ...)
Definition at line 615 of file event_handler_ints.hpp.
|
inline |
explicit doc start, with —
Definition at line 628 of file event_handler_ints.hpp.
|
inline |
explicit doc end, with ...
Definition at line 637 of file event_handler_ints.hpp.
|
inline |
Definition at line 656 of file event_handler_ints.hpp.
|
inline |
Definition at line 664 of file event_handler_ints.hpp.
|
inline |
Definition at line 673 of file event_handler_ints.hpp.
|
inline |
Definition at line 681 of file event_handler_ints.hpp.
|
inline |
Definition at line 690 of file event_handler_ints.hpp.
Definition at line 696 of file event_handler_ints.hpp.
|
inline |
Definition at line 709 of file event_handler_ints.hpp.
|
inline |
Definition at line 717 of file event_handler_ints.hpp.
|
inline |
Definition at line 726 of file event_handler_ints.hpp.
|
inline |
Definition at line 734 of file event_handler_ints.hpp.
|
inline |
Definition at line 743 of file event_handler_ints.hpp.
Definition at line 749 of file event_handler_ints.hpp.
|
inline |
Definition at line 762 of file event_handler_ints.hpp.
|
inline |
Definition at line 776 of file event_handler_ints.hpp.
|
inline |
Definition at line 782 of file event_handler_ints.hpp.
|
inline |
Definition at line 790 of file event_handler_ints.hpp.
|
inline |
Definition at line 796 of file event_handler_ints.hpp.
|
inline |
Definition at line 804 of file event_handler_ints.hpp.
|
inline |
Definition at line 810 of file event_handler_ints.hpp.
|
inline |
Definition at line 818 of file event_handler_ints.hpp.
|
inline |
Definition at line 824 of file event_handler_ints.hpp.
|
inline |
Definition at line 832 of file event_handler_ints.hpp.
|
inline |
Definition at line 838 of file event_handler_ints.hpp.
|
inline |
Definition at line 846 of file event_handler_ints.hpp.
|
inline |
Definition at line 852 of file event_handler_ints.hpp.
|
inline |
Definition at line 860 of file event_handler_ints.hpp.
|
inline |
Definition at line 866 of file event_handler_ints.hpp.
|
inline |
Definition at line 902 of file event_handler_ints.hpp.
|
inline |
Definition at line 915 of file event_handler_ints.hpp.
|
inline |
Definition at line 929 of file event_handler_ints.hpp.
|
inline |
Definition at line 937 of file event_handler_ints.hpp.
|
inline |
Definition at line 953 of file event_handler_ints.hpp.
|
inline |
Definition at line 960 of file event_handler_ints.hpp.
|
inline |
Definition at line 975 of file event_handler_ints.hpp.
Definition at line 981 of file event_handler_ints.hpp.
|
inline |
set the previous val as the first key of a new map, with flow style.
See the documentation for Event Handlers, which has important notes about this event.
Definition at line 1002 of file event_handler_ints.hpp.
|
inline |
like its flow counterpart, but this function can only be called after the end of a flow-val at root or doc level.
See the documentation for Event Handlers, which has important notes about this event.
Definition at line 1096 of file event_handler_ints.hpp.
|
inline |
Definition at line 1136 of file event_handler_ints.hpp.
Referenced by reset().
|
inline |
Definition at line 1140 of file event_handler_ints.hpp.
Referenced by alloc_arena().
|
inline |
this may fail, in which case an empty string is returned
Definition at line 1145 of file event_handler_ints.hpp.
|
inline |
push a new parent, add a child to the new parent, and set the child as the current node
Definition at line 1165 of file event_handler_ints.hpp.
Referenced by actually_val_is_first_key_of_new_map_block(), actually_val_is_first_key_of_new_map_flow(), begin_doc(), begin_doc_expl(), begin_map_key_block(), begin_map_key_flow(), begin_map_val_block(), begin_map_val_flow(), begin_seq_key_block(), begin_seq_key_flow(), begin_seq_val_block(), and begin_seq_val_flow().
|
inline |
end the current scope
Definition at line 1172 of file event_handler_ints.hpp.
Referenced by cancel_parse(), end_doc(), end_doc_expl(), end_map_block(), end_map_flow(), end_seq_block(), and end_seq_flow().
|
inlinenoexcept |
Definition at line 1177 of file event_handler_ints.hpp.
|
inlinenoexcept |
Definition at line 1181 of file event_handler_ints.hpp.
|
inlinenoexcept |
Definition at line 1185 of file event_handler_ints.hpp.
Definition at line 1190 of file event_handler_ints.hpp.
Referenced by actually_val_is_first_key_of_new_map_flow().
Definition at line 1196 of file event_handler_ints.hpp.
Referenced by _extend_left_to_include_tag_and_or_anchor(), and _find_matching_open().
|
inlinenoexcept |
Definition at line 1202 of file event_handler_ints.hpp.
Referenced by set_key_tag(), and set_val_tag().
|
inline |
Definition at line 1207 of file event_handler_ints.hpp.
Referenced by begin_doc(), begin_doc_expl(), begin_map_key_block(), begin_map_key_flow(), begin_map_val_block(), begin_map_val_flow(), begin_seq_key_block(), begin_seq_key_flow(), begin_seq_val_block(), begin_seq_val_flow(), begin_stream(), end_doc(), end_doc_expl(), end_map_block(), end_map_flow(), end_seq_block(), end_seq_flow(), and end_stream().
|
inline |
Definition at line 1219 of file event_handler_ints.hpp.
Referenced by add_directive_tag(), add_directive_yaml(), set_key_ref(), set_key_scalar_dquoted(), set_key_scalar_folded(), set_key_scalar_literal(), set_key_scalar_plain(), set_key_scalar_plain_empty(), set_key_scalar_squoted(), set_key_tag(), set_val_ref(), set_val_scalar_dquoted(), set_val_scalar_folded(), set_val_scalar_literal(), set_val_scalar_plain(), set_val_scalar_plain_empty(), set_val_scalar_squoted(), and set_val_tag().
|
inline |
Definition at line 1232 of file event_handler_ints.hpp.
Referenced by begin_map_key_block(), begin_map_key_flow(), begin_map_val_block(), begin_map_val_flow(), begin_seq_key_block(), begin_seq_key_flow(), begin_seq_val_block(), and begin_seq_val_flow().
|
inline |
Definition at line 1238 of file event_handler_ints.hpp.
Referenced by set_key_scalar_plain_empty(), and set_val_scalar_plain_empty().
Definition at line 1246 of file event_handler_ints.hpp.
Referenced by actually_val_is_first_key_of_new_map_block().
|
inline |
Definition at line 1259 of file event_handler_ints.hpp.
Referenced by actually_val_is_first_key_of_new_map_flow().
|
inline |
Definition at line 1290 of file event_handler_ints.hpp.
Referenced by actually_val_is_first_key_of_new_map_flow().
|
inlineprotectedinherited |
Definition at line 58 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::start_parse().
|
inlineprotectedinherited |
Definition at line 65 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::finish_parse().
|
inlineprotectedinherited |
Definition at line 71 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::reset().
|
inlineprotectedinherited |
|
inlineprotectedinherited |
Definition at line 88 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::_push().
|
inlineprotectedinherited |
Definition at line 96 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::_pop().
|
inlineprotectedinherited |
Definition at line 121 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::begin_doc().
|
inlineprotectedinherited |
Definition at line 127 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::end_doc(), and c4::yml::extra::EventHandlerInts::end_doc_expl().
|
inherited |
Definition at line 46 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::_extend_left_to_include_tag_and_or_anchor(), c4::yml::extra::EventHandlerInts::_find_last_bdoc(), c4::yml::extra::EventHandlerInts::_find_matching_open(), c4::yml::extra::EventHandlerInts::_next(), c4::yml::extra::EventHandlerInts::_prev(), c4::yml::extra::EventHandlerInts::actually_val_is_first_key_of_new_map_block(), c4::yml::extra::EventHandlerInts::actually_val_is_first_key_of_new_map_flow(), c4::yml::extra::EventHandlerInts::add_directive_tag(), c4::yml::extra::EventHandlerInts::add_sibling(), c4::yml::extra::EventHandlerInts::cancel_parse(), c4::yml::extra::EventHandlerInts::set_key_anchor(), c4::yml::extra::EventHandlerInts::set_key_ref(), c4::yml::extra::EventHandlerInts::set_key_tag(), c4::yml::extra::EventHandlerInts::set_val_anchor(), c4::yml::extra::EventHandlerInts::set_val_ref(), c4::yml::extra::EventHandlerInts::set_val_tag(), and c4::yml::extra::EventHandlerInts::start_parse().
|
inherited |
current stack level: top of the stack. cached here for easier access.
Definition at line 47 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::_push(), c4::yml::extra::EventHandlerInts::_send_flag_only_(), c4::yml::extra::EventHandlerInts::_send_str_(), c4::yml::extra::EventHandlerInts::actually_val_is_first_key_of_new_map_block(), c4::yml::extra::EventHandlerInts::actually_val_is_first_key_of_new_map_flow(), c4::yml::extra::EventHandlerInts::add_directive_tag(), c4::yml::extra::EventHandlerInts::add_sibling(), c4::yml::extra::EventHandlerInts::disable_(), c4::yml::extra::EventHandlerInts::enable_(), c4::yml::extra::EventHandlerInts::has_any_(), c4::yml::extra::EventHandlerInts::reset(), c4::yml::extra::EventHandlerInts::set_key_ref(), and c4::yml::extra::EventHandlerInts::set_val_ref().
|
inherited |
parent of the current stack level.
Definition at line 48 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::_mark_parent_with_children_(), and c4::yml::extra::EventHandlerInts::add_sibling().
|
inherited |
Definition at line 49 of file event_handler_stack.hpp.
Referenced by c4::yml::extra::EventHandlerInts::_get_latest_empty_scalar(), c4::yml::extra::EventHandlerInts::_is_sub_(), c4::yml::extra::EventHandlerInts::reset(), c4::yml::extra::EventHandlerInts::set_key_scalar_dquoted(), c4::yml::extra::EventHandlerInts::set_key_scalar_folded(), c4::yml::extra::EventHandlerInts::set_key_scalar_literal(), c4::yml::extra::EventHandlerInts::set_key_scalar_plain(), c4::yml::extra::EventHandlerInts::set_key_scalar_squoted(), c4::yml::extra::EventHandlerInts::set_val_scalar_dquoted(), c4::yml::extra::EventHandlerInts::set_val_scalar_folded(), c4::yml::extra::EventHandlerInts::set_val_scalar_literal(), c4::yml::extra::EventHandlerInts::set_val_scalar_plain(), c4::yml::extra::EventHandlerInts::set_val_scalar_squoted(), and c4::yml::extra::EventHandlerInts::start_parse().