Class: ContextSpook::TOON::Encoder
- Inherits:
-
Object
- Object
- ContextSpook::TOON::Encoder
- Defined in:
- lib/context_spook/toon/encoder.rb
Overview
The ContextSpook::TOON::Encoder class encodes Ruby data structures into TOON (Token-Oriented Object Notation) v4.4 format. It is fully conformant to the v4.4 encode fixtures from the toon-format-spec repository.
It supports:
- Tabular arrays with nested field groups (v4.0)
- Keyed tabular form (v4.0)
- Strict JSON number grammar, shortest round-trip floats (v4.2)
- Delimiter-aware quoting, C0 escaping, BOM handling (v4.2)
- Structural-char/combining-mark quoting (v4.4)
- Streaming output via the
output:keyword argument
Defined Under Namespace
Classes: StringArray
Constant Summary collapse
- JSON_NUMBER =
/\A-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+-]?\d+)?\z/- SAFE_KEY =
/\A[a-zA-Z_][a-zA-Z0-9_.]*\z/- UNSAFE_STRING =
/ \A\# # comment prefix (§7.2) | \A\xEF\xBB\xBF[0-9.+-] # BOM + digit: decoders strip BOM (§12) | \A\s # leadingwhitespace (§7.1) | \s\z # trailing whitespace (§7.1) | [\x00-\x1F:\[\]{}"\\] # controls, reserved, structural (§7.1) | #{JSON_NUMBER.source} # full JSON number (§7.3) | \A(?:true|false|null)\z # JSON literal (§7.3) | \A[+-]?\d # number-like prefix (§7.3) | \A- # list item marker (§7.1) /x- ESCAPE_STRING =
{ ?\0 => '\\u0000'.freeze, ?\a => '\\u0007'.freeze, ?\b => '\\u0008'.freeze, ?\t => '\\t'.freeze, ?\n => '\\n'.freeze, ?\v => '\\u000b'.freeze, ?\f => '\\u000c'.freeze, ?\r => '\\r'.freeze, ?\e => '\\u001b'.freeze, ?\\ => '\\\\'.freeze, ?" => '\\"'.freeze, }
Instance Attribute Summary collapse
-
#delimiter ⇒ Object
readonly
Returns the value of attribute delimiter.
-
#indent_size ⇒ Object
readonly
Returns the value of attribute indent_size.
Class Method Summary collapse
-
.encode(data, indent: 2, delimiter: ',', output: nil) ⇒ String?
Encodes
datainto a TOON v4.4 document string (or streams it to an IO whenoutput:is given).
Instance Method Summary collapse
-
#build_field_spec(rows) ⇒ Array(String, Array<Array<String>>)
private
Builds the field-group header and per-field access paths for a set of row hashes.
-
#encode(value, depth) ⇒ Object
Encodes
value(Hash, Array, or scalar) at the given depth. -
#encode_array(array, depth) ⇒ Object
private
Encodes a root-level array, dispatching to empty, tabular, inline scalar, or list form based on the array's contents.
-
#encode_cell(cell) ⇒ String
private
Encodes a single tabular cell.
-
#encode_float(f) ⇒ String
private
Encodes a
Floatusing its shortest round-trippable decimal form. -
#encode_hash(hash, depth) ⇒ Object
private
Encodes a Hash, using keyed tabular form when eligible and otherwise emitting each field in turn.
-
#encode_hash_pair(key, value, depth) ⇒ Object
private
Encodes a single
key: valuepair of a non-keyed hash, dispatching the value to Hash, Array, or scalar encoding as appropriate. -
#encode_key(key) ⇒ String
private
Encodes a hash key per the TOON key-encoding rules.
-
#encode_keyed_tabular(hash, depth, key: nil, row_depth: nil) ⇒ Object
private
Encodes a hash in keyed tabular form: a header line
[n:]{fields}:followed by onekey: v1,v2row per entry. -
#encode_list(key, array, depth) ⇒ Object
private
Encodes a non-tabular array in list form: a
[n]:header line followed by one list item per element. -
#encode_list_array(array, depth) ⇒ Object
private
Encodes an anonymous array that is itself a list item, prefixed with a hyphen.
-
#encode_list_hash(hash, depth) ⇒ Object
private
Encodes a Hash that is a list item: the first field is emitted on the bullet line and the remaining fields follow at the sibling indent.
-
#encode_list_item(item, depth) ⇒ Object
private
Encodes a single list item, dispatching on its type: Hashes expand into a field block, Arrays into a nested list, scalars into a plain bullet value.
-
#encode_scalar(value) ⇒ String
private
Encodes a single scalar value (nil, boolean, number, or string) into its TOON representation.
-
#encode_string(str) ⇒ String
private
Encodes a scalar string value.
-
#encode_tabular(key, array, depth, row_depth: depth + 1) ⇒ Object
private
Encodes an array of uniform Hashes in tabular form: a header line
[n]{fields}:followed by one delimited row per element. -
#escape_string(str) ⇒ String
private
Escapes a string for safe inclusion inside a double-quoted TOON value.
-
#hash_has_array?(hash) ⇒ Boolean
private
Recursively reports whether any value inside
hash(or nested hashes) is an Array. -
#initialize(indent: 2, delimiter: ',', output: nil) ⇒ Encoder
constructor
Creates a new encoder.
-
#keyed_tabular?(hash) ⇒ Boolean
private
Reports whether
hashis eligible for keyed tabular form: at least two entries, all values non-empty uniform Hashes, none containing Array values. -
#length_header(n, keyed: false) ⇒ String
private
Builds a TOON length header such as
[3]or[2:]. -
#primitive_array?(array) ⇒ Boolean
private
Reports whether every element of
arrayis a TOON scalar (nil, String, Numeric, or boolean), i.e. -
#push(depth, content) ⇒ ContextSpook::TOON::Encoder
private
Writes one output line at the given indent depth, managing line separation.
-
#push_list_field(key, value, depth, hyphen:) ⇒ Object
private
Encodes one field of a list item.
-
#safe_unquoted?(str) ⇒ Boolean
private
Reports whether
strcan be emitted as an unquoted TOON value. -
#tabular_array?(array) ⇒ Boolean
private
Reports whether
arrayis an array of non-empty, key-uniform Hashes whose fields can be flattened into tabular columns. -
#tabular_leaves?(hashes) ⇒ Boolean
private
Recursively reports whether each field's values across
hashesare either all scalars or all uniform nested Hashes without Array values, so they can be flattened into tabular columns. -
#to_s ⇒ String
Returns the accumulated TOON output as a single string.
-
#uniform_hashes?(hashes) ⇒ Boolean
private
Reports whether every hash in
hasheshas the same set of keys (order-insensitive), a prerequisite for tabular encoding.
Constructor Details
#initialize(indent: 2, delimiter: ',', output: nil) ⇒ Encoder
Creates a new encoder.
64 65 66 67 68 69 |
# File 'lib/context_spook/toon/encoder.rb', line 64 def initialize(indent: 2, delimiter: ',', output: nil) @indent_size = indent @delimiter = delimiter @output = output || StringArray.new @started = false if output end |
Instance Attribute Details
#delimiter ⇒ Object (readonly)
Returns the value of attribute delimiter.
37 38 39 |
# File 'lib/context_spook/toon/encoder.rb', line 37 def delimiter @delimiter end |
#indent_size ⇒ Object (readonly)
Returns the value of attribute indent_size.
37 38 39 |
# File 'lib/context_spook/toon/encoder.rb', line 37 def indent_size @indent_size end |
Class Method Details
.encode(data, indent: 2, delimiter: ',', output: nil) ⇒ String?
Encodes data into a TOON v4.4 document string (or streams it to an
IO when output: is given).
80 81 82 83 84 |
# File 'lib/context_spook/toon/encoder.rb', line 80 def self.encode(data, indent: 2, delimiter: ',', output: nil) encoder = new(indent:, delimiter:, output:) encoder.encode(data, 0) encoder.to_s unless output end |
Instance Method Details
#build_field_spec(rows) ⇒ Array(String, Array<Array<String>>) (private)
Builds the field-group header and per-field access paths for a set of
row hashes. Nested uniform Hash fields collapse into {name{...}}
groups; the returned paths are key chains used to fetch each cell.
384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 |
# File 'lib/context_spook/toon/encoder.rb', line 384 def build_field_spec(rows) keys = rows.first.keys parts = [] paths = [] keys.each do |key| sub = rows.map { |r| r[key] } ek = encode_key(key) if sub.all? { |v| v.is_a?(Hash) && !v.empty? } && uniform_hashes?(sub) sub_header, sub_paths = build_field_spec(sub) parts.push("#{ek}{#{sub_header}}") sub_paths.each { |p| paths.push([key] + p) } else parts.push(ek) paths.push([key]) end end [parts.join(@delimiter), paths] end |
#encode(value, depth) ⇒ Object
Encodes value (Hash, Array, or scalar) at the given depth. This is
the internal entry point used by ContextSpook::TOON::Encoder.selfself.encode.
91 92 93 94 95 96 97 |
# File 'lib/context_spook/toon/encoder.rb', line 91 def encode(value, depth) case value when Hash then encode_hash(value, depth) when Array then encode_array(value, depth) else push(depth, encode_scalar(value)) end end |
#encode_array(array, depth) ⇒ Object (private)
Encodes a root-level array, dispatching to empty, tabular, inline scalar, or list form based on the array's contents.
199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 |
# File 'lib/context_spook/toon/encoder.rb', line 199 def encode_array(array, depth) if array.empty? push(depth, '[]') return end if tabular_array?(array) encode_tabular(nil, array, depth) elsif primitive_array?(array) line = array.map { |v| encode_scalar(v) }.join(@delimiter) push(depth, "#{length_header(array.size)}: #{line}") else encode_list(nil, array, depth) end end |
#encode_cell(cell) ⇒ String (private)
Encodes a single tabular cell. Nested Hash/Array cells are JSON-
encoded and quoted; nil becomes "null"; everything else goes
through the scalar encoder.
413 414 415 416 417 418 419 420 421 422 |
# File 'lib/context_spook/toon/encoder.rb', line 413 def encode_cell(cell) case cell when Hash, Array "\"#{JSON.generate(cell)}\"" when nil 'null' else encode_scalar(cell) end end |
#encode_float(f) ⇒ String (private)
Encodes a Float using its shortest round-trippable decimal form.
Whole-number floats become integers, zero becomes "0", and exponent
notation is expanded via BigDecimal to a plain decimal.
534 535 536 537 538 539 540 541 |
# File 'lib/context_spook/toon/encoder.rb', line 534 def encode_float(f) return '0' if f.zero? fi = f.to_i return fi.to_s if f == fi && f.abs < 1e15 fs = f.to_s return fs unless fs.match?(/e/i) BigDecimal(fs).to_s('F') end |
#encode_hash(hash, depth) ⇒ Object (private)
Encodes a Hash, using keyed tabular form when eligible and otherwise emitting each field in turn.
150 151 152 153 154 155 156 |
# File 'lib/context_spook/toon/encoder.rb', line 150 def encode_hash(hash, depth) if keyed_tabular?(hash) encode_keyed_tabular(hash, depth) else hash.each { |key, value| encode_hash_pair(key, value, depth) } end end |
#encode_hash_pair(key, value, depth) ⇒ Object (private)
Encodes a single key: value pair of a non-keyed hash, dispatching
the value to Hash, Array, or scalar encoding as appropriate.
164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 |
# File 'lib/context_spook/toon/encoder.rb', line 164 def encode_hash_pair(key, value, depth) k = encode_key(key) case value when Hash if value.empty? push(depth, "#{k}:") elsif keyed_tabular?(value) encode_keyed_tabular(value, depth, key: k) else push(depth, "#{k}:") encode_hash(value, depth + 1) end when Array if value.empty? push(depth, "#{k}: []") elsif tabular_array?(value) encode_tabular(k, value, depth) elsif primitive_array?(value) push(depth, "#{k}#{length_header(value.size)}: " \ "#{value.map { |v| encode_scalar(v) }.join(@delimiter)}") else encode_list(k, value, depth) end else push(depth, "#{k}: #{encode_scalar(value)}") end end |
#encode_key(key) ⇒ String (private)
Encodes a hash key per the TOON key-encoding rules. Bare word-like keys are emitted as-is; anything resembling a number or literal, or containing structural characters, is quoted and escaped.
566 567 568 569 570 571 572 573 574 575 |
# File 'lib/context_spook/toon/encoder.rb', line 566 def encode_key(key) key = key.to_s key = key.scrub unless key.empty? || key.valid_encoding? if key.match?(SAFE_KEY) && !key.match?(JSON_NUMBER) key else "\"#{escape_string(key)}\"" end end |
#encode_keyed_tabular(hash, depth, key: nil, row_depth: nil) ⇒ Object (private)
Encodes a hash in keyed tabular form: a header line
[n:]{fields}: followed by one key: v1,v2 row per entry.
360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 |
# File 'lib/context_spook/toon/encoder.rb', line 360 def encode_keyed_tabular(hash, depth, key: nil, row_depth: nil) rd = row_depth || depth + 1 values = hash.values header, paths = build_field_spec(values) lh = length_header(hash.size, keyed: true) label = key ? "#{key}#{lh}{#{header}}:" : "#{lh}{#{header}}:" push(depth, label) hash.each do |k, row| values_line = paths.map do |path| cell = row.dig(*path) encode_cell(cell) end push(rd, "#{encode_key(k)}: #{values_line.join(@delimiter)}") end end |
#encode_list(key, array, depth) ⇒ Object (private)
Encodes a non-tabular array in list form: a [n]: header line
followed by one list item per element.
221 222 223 224 225 226 |
# File 'lib/context_spook/toon/encoder.rb', line 221 def encode_list(key, array, depth) label = key ? "#{key}#{length_header(array.size)}:" : \ "#{length_header(array.size)}:" push(depth, label) array.each { |item| encode_list_item(item, depth + 1) } end |
#encode_list_array(array, depth) ⇒ Object (private)
Encodes an anonymous array that is itself a list item, prefixed with a hyphen. Scalar arrays use inline form; anything else expands into nested list items.
319 320 321 322 323 324 325 326 327 328 329 |
# File 'lib/context_spook/toon/encoder.rb', line 319 def encode_list_array(array, depth) if array.empty? push(depth, "- #{length_header(0)}:") elsif primitive_array?(array) line = array.map { |v| encode_scalar(v) }.join(@delimiter) push(depth, "- #{length_header(array.size)}: #{line}") else push(depth, "- #{length_header(array.size)}:") array.each { |item| encode_list_item(item, depth + 1) } end end |
#encode_list_hash(hash, depth) ⇒ Object (private)
Encodes a Hash that is a list item: the first field is emitted on the bullet line and the remaining fields follow at the sibling indent.
254 255 256 257 258 259 260 261 262 263 264 |
# File 'lib/context_spook/toon/encoder.rb', line 254 def encode_list_hash(hash, depth) first = true hash.each do |k, v| if first push_list_field(k, v, depth, hyphen: true) first = false else push_list_field(k, v, depth + 1, hyphen: false) end end end |
#encode_list_item(item, depth) ⇒ Object (private)
Encodes a single list item, dispatching on its type: Hashes expand into a field block, Arrays into a nested list, scalars into a plain bullet value.
234 235 236 237 238 239 240 241 242 243 244 245 246 247 |
# File 'lib/context_spook/toon/encoder.rb', line 234 def encode_list_item(item, depth) case item when Hash if item.empty? push(depth, '-') else encode_list_hash(item, depth) end when Array encode_list_array(item, depth) else push(depth, "- #{encode_scalar(item)}") end end |
#encode_scalar(value) ⇒ String (private)
Encodes a single scalar value (nil, boolean, number, or string) into its TOON representation.
517 518 519 520 521 522 523 524 525 526 |
# File 'lib/context_spook/toon/encoder.rb', line 517 def encode_scalar(value) case value when nil then 'null' when true then 'true' when false then 'false' when Integer then value.to_s when Float then encode_float(value) else encode_string(value.to_s) end end |
#encode_string(str) ⇒ String (private)
Encodes a scalar string value. The string is UTF-8-scrubbed, then emitted bare when safe or double-quoted with escaping otherwise.
548 549 550 551 552 553 554 555 556 557 558 |
# File 'lib/context_spook/toon/encoder.rb', line 548 def encode_string(str) return '""' if str.empty? str = str.scrub unless str.valid_encoding? if safe_unquoted?(str) str else "\"#{escape_string(str)}\"" end end |
#encode_tabular(key, array, depth, row_depth: depth + 1) ⇒ Object (private)
Encodes an array of uniform Hashes in tabular form: a header line
[n]{fields}: followed by one delimited row per element.
338 339 340 341 342 343 344 345 346 347 348 349 350 351 |
# File 'lib/context_spook/toon/encoder.rb', line 338 def encode_tabular(key, array, depth, row_depth: depth + 1) header, paths = build_field_spec(array) label = key ? "#{key}#{length_header(array.size)}{#{header}}:" : \ "#{length_header(array.size)}{#{header}}:" push(depth, label) array.each do |row| values = paths.map do |path| cell = row.dig(*path) encode_cell(cell) end push(row_depth, values.join(@delimiter)) end end |
#escape_string(str) ⇒ String (private)
Escapes a string for safe inclusion inside a double-quoted TOON value. Backslashes, double quotes, and C0 control characters are escaped; remaining control characters are emitted as \uXXXX.
625 626 627 |
# File 'lib/context_spook/toon/encoder.rb', line 625 def escape_string(str) str.gsub(/[\\"\n\r\t\x00-\x08\x0B\x0C\x0E-\x1F]/, ESCAPE_STRING) end |
#hash_has_array?(hash) ⇒ Boolean (private)
Recursively reports whether any value inside hash (or nested
hashes) is an Array.
443 444 445 446 447 448 449 450 451 |
# File 'lib/context_spook/toon/encoder.rb', line 443 def hash_has_array?(hash) hash.values.any? do |v| case v when Array then true when Hash then hash_has_array?(v) else false end end end |
#keyed_tabular?(hash) ⇒ Boolean (private)
Reports whether hash is eligible for keyed tabular form: at least
two entries, all values non-empty uniform Hashes, none containing
Array values.
430 431 432 433 434 435 436 |
# File 'lib/context_spook/toon/encoder.rb', line 430 def keyed_tabular?(hash) return false if hash.size < 2 values = hash.values return false unless values.all? { |v| v.is_a?(Hash) && !v.empty? } return false unless uniform_hashes?(values) !values.any? { |v| hash_has_array?(v) } end |
#length_header(n, keyed: false) ⇒ String (private)
Builds a TOON length header such as [3] or [2:]. When the active
delimiter is not the default comma it is echoed inside the brackets
(e.g. [3|]) to signal the separator in use.
140 141 142 143 |
# File 'lib/context_spook/toon/encoder.rb', line 140 def length_header(n, keyed: false) d = @delimiter == ',' ? '' : @delimiter keyed ? "[#{n}:#{d}]" : "[#{n}#{d}]" end |
#primitive_array?(array) ⇒ Boolean (private)
Reports whether every element of array is a TOON scalar (nil,
String, Numeric, or boolean), i.e. it can use inline form.
505 506 507 508 509 510 |
# File 'lib/context_spook/toon/encoder.rb', line 505 def primitive_array?(array) array.all? do |v| v.nil? || v.is_a?(String) || v.is_a?(Numeric) || v.is_a?(TrueClass) || v.is_a?(FalseClass) end end |
#push(depth, content) ⇒ ContextSpook::TOON::Encoder (private)
Writes one output line at the given indent depth, managing line separation.
This method appends a line to the internal output buffer, prefixing it with indentation based on the provided depth and indent size. It handles the state of the buffer to ensure newlines are correctly inserted between lines, except after the very first line which starts the buffer.
121 122 123 124 125 126 127 128 129 130 131 |
# File 'lib/context_spook/toon/encoder.rb', line 121 def push(depth, content) if @started.nil? @output << (' ' * (depth * @indent_size)).concat(content) elsif @started == true @output << (+?\n).concat(' ' * (depth * @indent_size)).concat(content) else @output << (' ' * (depth * @indent_size)).concat(content) @started = true end self end |
#push_list_field(key, value, depth, hyphen:) ⇒ Object (private)
Encodes one field of a list item. The hyphen: flag controls whether
the field sits on the "- " bullet line and whether its nested content
is indented one extra level to clear the bullet.
274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 |
# File 'lib/context_spook/toon/encoder.rb', line 274 def push_list_field(key, value, depth, hyphen:) k = encode_key(key) prefix = hyphen ? '- ' : '' # Nested content under a hyphen line gets one extra indent level # because "- " occupies one indent step. child_depth = hyphen ? depth + 2 : depth + 1 case value when Hash if value.empty? push(depth, "#{prefix}#{k}:") elsif keyed_tabular?(value) rd = hyphen ? depth + 2 : depth + 1 encode_keyed_tabular(value, depth, key: "#{prefix}#{k}", row_depth: rd) else push(depth, "#{prefix}#{k}:") encode_hash(value, child_depth) end when Array if value.empty? push(depth, "#{prefix}#{k}: []") elsif tabular_array?(value) row_depth = hyphen ? depth + 2 : depth + 1 encode_tabular("#{prefix}#{k}", value, depth, row_depth:) elsif primitive_array?(value) line = value.map { |v| encode_scalar(v) }.join(@delimiter) push(depth, "#{prefix}#{k}#{length_header(value.size)}: #{line}") else push(depth, "#{prefix}#{k}#{length_header(value.size)}:") value.each { |item| encode_list_item(item, child_depth) } end else push(depth, "#{prefix}#{k}: #{encode_scalar(value)}") end end |
#safe_unquoted?(str) ⇒ Boolean (private)
Reports whether str can be emitted as an unquoted TOON value.
The UNSAFE_STRING regex covers all static structural conflicts;
the active delimiter is checked separately (instance state).
596 597 598 599 600 |
# File 'lib/context_spook/toon/encoder.rb', line 596 def safe_unquoted?(str) return false if str.match?(UNSAFE_STRING) return false if str.include?(@delimiter) true end |
#tabular_array?(array) ⇒ Boolean (private)
Reports whether array is an array of non-empty, key-uniform Hashes
whose fields can be flattened into tabular columns.
458 459 460 461 462 463 |
# File 'lib/context_spook/toon/encoder.rb', line 458 def tabular_array?(array) !array.empty? && array.all? { |v| v.is_a?(Hash) && !v.empty? } && uniform_hashes?(array) && tabular_leaves?(array) end |
#tabular_leaves?(hashes) ⇒ Boolean (private)
Recursively reports whether each field's values across hashes are
either all scalars or all uniform nested Hashes without Array
values, so they can be flattened into tabular columns.
471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 |
# File 'lib/context_spook/toon/encoder.rb', line 471 def tabular_leaves?(hashes) keys = hashes.first.keys keys.all? do |key| values = hashes.map { |h| h[key] } if values.all? { |v| v.nil? || v.is_a?(String) || v.is_a?(Numeric) || v.is_a?(TrueClass) || v.is_a?(FalseClass) } true elsif values.all? { |v| v.is_a?(Hash) && !v.empty? } && uniform_hashes?(values) && !values.any? { |v| v.values.any? { |x| x.is_a?(Array) } } tabular_leaves?(values) else false end end end |
#to_s ⇒ String
Returns the accumulated TOON output as a single string.
102 103 104 |
# File 'lib/context_spook/toon/encoder.rb', line 102 def to_s @output.string end |
#uniform_hashes?(hashes) ⇒ Boolean (private)
Reports whether every hash in hashes has the same set of keys
(order-insensitive), a prerequisite for tabular encoding.
493 494 495 496 497 498 |
# File 'lib/context_spook/toon/encoder.rb', line 493 def uniform_hashes?(hashes) first = hashes.first keys = first.keys size = first.size hashes.all? { |h| h.size == size && keys.all? { |k| h.key?(k) } } end |