Module: OllamaChat::Compaction

Included in:
Chat
Defined in:
lib/ollama_chat/compaction.rb

Overview

Provides compaction support for chat sessions. Mixed into Chat.

Supplies configuration resolvers that derive concrete token budgets from the compaction: config block (ratios of num_ctx with absolute floors), the LLM-based summarization pipeline (+summarize_for_compaction+), and the tool_summary_line resolver for per-tool summary templates.

See Also:

  • for the `compaction:` block structure (`reserve`, `keep_recent`, `summary`).

Defined Under Namespace

Classes: Result

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.tool_summary_line(tool_name, result) ⇒ String

Generates a one-line summary for a tool call result.

Looks up the registered tool class and delegates to its summary_template(result:) class method. Falls back to a generic sentence if the tool is not registered or the template raises.

Parameters:

  • tool_name (String) —

    the registered name, e.g. 'read_file'.

  • result (String) —

    the raw JSON result string from execute.

Returns:

  • (String) —

    a short natural-language description of the call.



22
23
24
25
26
27
28
 
# File 'lib/ollama_chat/compaction.rb', line 22

def tool_summary_line(tool_name, result)
  default_summary = "was called."
  klass           = OllamaChat::Tools.registered_class(tool_name)
  klass&.summary_template(result:) || default_summary
rescue StandardError
  default_summary
end

Instance Method Details

#assemble_summary(narrative, tool_entries, old_tool_entries = []) ⇒ Array(String, Array<Hash>) (private)

Assembles the final summary content and merged tool entries.

Parameters:

  • narrative (String) —

    the LLM-generated narrative.

  • tool_entries (Array<Hash>) —

    new tool-call entries from this pass.

  • old_tool_entries (Array<Hash>) (defaults to: []) —

    entries carried over from a prior compaction.

Returns:

  • (Array(String, Array<Hash>)) —

    the content string and the merged entries array.



263
264
265
266
267
268
269
270
271
 
# File 'lib/ollama_chat/compaction.rb', line 263

def assemble_summary(narrative, tool_entries, old_tool_entries = [])
  all_tools    = old_tool_entries + tool_entries
  tool_section = all_tools.map(&:to_json) * ?\n

  template = prompt(:assemble, context: 'compaction').to_s
  content  = template.named_placeholders_interpolate({ narrative:, tool_section: })

  [content, all_tools]
end

#build_tool_entries(messages) ⇒ Array<Hash> (private)

Builds the deterministic tool-call entries from tool result messages.

Parameters:

Returns:

  • (Array<Hash>) —

    tool-call entry hashes, empty if no tool results.



199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
 
# File 'lib/ollama_chat/compaction.rb', line 199

def build_tool_entries(messages)
  messages.filter_map do |msg|
    next unless tool_name = msg.tool_name.full?
    next if %w[ runtime_information trigger_inject ].include?(tool_name)
    uuid = msg.group_uuid.to_s[-8..]
    summary = OllamaChat::Compaction.tool_summary_line(
      tool_name, msg.content.to_s
    )
    time = msg.group_time&.strftime('%Y-%m-%d %H:%M')
    {
      'type'    => 'tool',
      'name'    => tool_name,
      'summary' => summary,
      'uuid'    => uuid,
      'time'    => time,
    }
  end
end

#call_summarizer(groups:, previous_summary:) ⇒ String (private)

Calls the LLM to generate the narrative portion of the summary.

Parameters:

  • groups (String) —

    the serialized groups block.

  • previous_summary (String, nil) —

    the prior summary for iteration.

Returns:

  • (String) —

    the LLM-generated narrative text.



223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
 
# File 'lib/ollama_chat/compaction.rb', line 223

def call_summarizer(groups:, previous_summary:)
  previous = previous_summary.to_s

  prompt = prompt(:summarize, context: 'compaction').to_s.
    named_placeholders_interpolate({ previous:, groups: })
  system = prompt(:compaction, context: 'system').to_s

  es = OllamaChat::TokenEstimator.estimate(prompt)
  log(:info, "Compaction: sending prompt " \
    "#{es.tokens_formatted} (#{es.bytes_formatted}) to #{@model}")

  response = Infobar.busy(
    label: 'Compacting context…',
    frames: :braille7,
    output: STDOUT,
  ) do
    generate(system:, prompt:, think: true).strip
  end

  if response.empty?
    log(:error, 'Compaction: LLM returned empty response')
    raise OllamaChat::CompactionError,
          'Summarization failed: model returned empty response'
  end

  res_es = OllamaChat::TokenEstimator.estimate(response)
  log(:info, "Compaction: got response " \
    "#{res_es.tokens_formatted} (#{res_es.bytes_formatted})")

  response
end

#compact_min_tokens(name) ⇒ Integer (private)

Returns the absolute minimum token floor for a given concern.

Parameters:

  • name (Symbol) —

    the concern name, e.g. :reserve, :keep_recent, or :summary.

Returns:

  • (Integer) —

    the configured min_tokens floor.



294
295
296
 
# File 'lib/ollama_chat/compaction.rb', line 294

def compact_min_tokens(name)
  config.compaction.attribute_get!(name).min_tokens
end

#compact_ratio(name) ⇒ Float (private)

Returns the fractional ratio (portion of num_ctx) for a given concern.

Parameters:

  • name (Symbol) —

    the concern name, e.g. :reserve, :keep_recent, or :summary.

Returns:

  • (Float) —

    the configured ratio value.



303
304
305
 
# File 'lib/ollama_chat/compaction.rb', line 303

def compact_ratio(name)
  config.compaction.attribute_get!(name).ratio
end

#compact_ratio_tokens(name, tokens) ⇒ Integer

Resolves the effective token budget for a concern given a context size.

Computes max(ratio * tokens, min_tokens).floor, so the floor acts as a guaranteed minimum even for very small context windows.

Parameters:

  • name (Symbol) —

    the concern name, e.g. :reserve, :keep_recent, or :summary.

  • tokens (Integer) —

    the context window size (num_ctx) to derive the budget from.

Returns:

  • (Integer) —

    the resolved token budget.



98
99
100
101
102
103
 
# File 'lib/ollama_chat/compaction.rb', line 98

def compact_ratio_tokens(name, tokens)
  [
    compact_ratio(name).to_f * tokens,
    compact_min_tokens(name).to_f,
  ].max.floor
end

#compact_with_retry ⇒ Boolean

Attempts compaction, prompting the user to retry on failure.

compact! is idempotent on failure: the LLM call (and thus any CompactionError) happens before the message list is mutated, so a retry always starts from a clean state.

Returns:

  • (Boolean) —

    true if compaction succeeded, false if the user declined to retry.



113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
 
# File 'lib/ollama_chat/compaction.rb', line 113

def compact_with_retry
  start  = Time.now
  result = messages.compact!
  session_sync
  if result
    report_compaction(result, duration: Time.now - start)
  else
    feedback('Nothing to compact.', type: :info)
  end
  return true
rescue OllamaChat::CompactionError => e
  feedback("Compaction failed: #{e.message}", type: :warn)
  log(:error, "Compaction failed", data: {
    model: @model,
    ctx:   current_context_length,
    size:  messages.size,
  })
  retry if confirm?(
    prompt: '🔔 Retry compaction? (y/n) ',
    yes: /\Ay/i
  )
  false
end

#extract_narrative(content) ⇒ String (private)

Extracts the narrative portion from a summary message's content.

Parameters:

  • content (String, nil) —

    the summary content.

Returns:

  • (String) —

    the narrative text.



277
278
279
 
# File 'lib/ollama_chat/compaction.rb', line 277

def extract_narrative(content)
  content.to_s.sub(/\ntool_calls:\n.*\z/m, '').strip
end

#extract_old_tool_calls(summary_msg) ⇒ Array<Hash> (private)

Returns the tool-call entries from a previous summary message.

Parameters:

Returns:

  • (Array<Hash>) —

    the tool-call entries, empty if none.



285
286
287
 
# File 'lib/ollama_chat/compaction.rb', line 285

def extract_old_tool_calls(summary_msg)
  summary_msg.tool_calls || []
end

#format_summary_line(msg) ⇒ String (private)

Formats a single message as a summary line.

Parameters:

Returns:

  • (String) —

    a single line describing the message.



181
182
183
184
185
186
187
188
189
190
191
192
193
 
# File 'lib/ollama_chat/compaction.rb', line 181

def format_summary_line(msg)
  tool_name = msg.tool_name.full?
  parts  = +"[#{msg.role}]"
  parts << " (#{tool_name})" if tool_name
  parts << " (g:#{msg.group_uuid.to_s[-8..]})" if msg.group_uuid
  if tool_name
    line = OllamaChat::Compaction.tool_summary_line(tool_name, msg.content.to_s)
    parts << " #{line}"
  elsif msg.content.present?
    parts << " #{msg.content.to_s.strip}"
  end
  parts
end

#report_compaction(result, duration:) ⇒ Object (private)

Displays a compact compaction report to STDOUT after the Infobar.busy spinner has finished.

Replaces the former bare "Conversation compacted." line with before/after context metrics, candidate volume, and summary size.

Parameters:

  • result (Result) —

    the compaction result from compact!.

  • duration (Float) —

    elapsed seconds for the compaction operation, used in the voice announcement.



316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
 
# File 'lib/ollama_chat/compaction.rb', line 316

def report_compaction(result, duration:)
  feedback(<<~EOT, type: :success)
    Conversation compacted:

    Summarized:   #{result.candidates} messages (#{result.candidate_size})
    Summary:      #{result.summary_size}
    Context:      #{result.context_before} → #{result.context_after}
    Stored total: #{result.stored_total}
  EOT
  duration        = Tins::Duration.new(duration)
  duration_string = duration.format(
    duration.minutes? ? '%M minutes and %u seconds' : '%u seconds'
  )
  speak(
    'Compressed conversation context from %.0f percent to %.0f percent in %s.' % [
      100 * result.context_filled_before,
      100 * result.context_filled_after,
      duration_string,
    ]
  )
end

#serialize_groups(messages) ⇒ String (private)

Serializes non-system messages into a readable block for the prompt.

Parameters:

Returns:

  • (String) —

    a formatted block of message lines.



170
171
172
173
174
175
 
# File 'lib/ollama_chat/compaction.rb', line 170

def serialize_groups(messages)
  messages
    .reject { _1.role == 'system' }
    .map { |m| format_summary_line(m) }
    .join("\n")
end

#show_compaction_summary ⇒ nil

Displays the most recent compaction summary in the pager.

Shows the narrative and the tool-call lookup index (the lookup_group entries with short UUIDs) carried in the summary message, prefixed with size metadata.

Returns:

  • (nil) —

    always returns nil (command block handles :next).



144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
 
# File 'lib/ollama_chat/compaction.rb', line 144

def show_compaction_summary
  unless msg = messages.find_summary
    feedback('No compaction summary yet.', type: :info)
    return
  end
  es = OllamaChat::TokenEstimator.estimate(msg.content.to_s)
  use_pager do |output|
    output.puts <<~EOT
      #{bold{'Compaction Summary'}}
      #{'─' * Tins::Terminal.columns}

      #{kramdown_ansi_parse(msg.content.to_s)}

      #{'─' * Tins::Terminal.columns}
      Size:   👾#{es.bytes_formatted} 🧩#{es.tokens_formatted}
      Tools:  #{msg.tool_calls&.size || 0} lookup_group entries
    EOT
  end
end

#summarize_for_compaction(messages:, previous_summary: nil) ⇒ Array(String, Array<Hash>)

Generates a context summary for the given messages via LLM.

Serializes non-system messages into a structured prompt, calls the model via generate, and assembles the final summary with deterministic tool-call entries stored out-of-band.

Parameters:

  • messages (Array<OllamaChat::Message>) —

    the messages to summarize.

  • previous_summary (OllamaChat::Message, nil) (defaults to: nil) —

    the previous summary message for iterative re-compaction.

Returns:

  • (Array(String, Array<Hash>)) —

    the assembled content and the merged tool-call entries array.



69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
 
# File 'lib/ollama_chat/compaction.rb', line 69

def summarize_for_compaction(messages:, previous_summary: nil)
  groups       = serialize_groups(messages)
  tool_entries = build_tool_entries(messages)

  es = OllamaChat::TokenEstimator.estimate(groups)
  log(:info, "Compaction: #{messages.size} messages, " \
    "#{es.tokens_formatted} (#{es.bytes_formatted}) groups")

  narrative_prev = ''
  old_tool_calls = []
  if previous_summary
    narrative_prev = extract_narrative(previous_summary.content)
    old_tool_calls = extract_old_tool_calls(previous_summary)
  end

  narrative = call_summarizer(groups:, previous_summary: narrative_prev)
  assemble_summary(narrative, tool_entries, old_tool_calls)
end