Module: OllamaChat::Compaction

Included in:
Chat
Defined in:
lib/ollama_chat/compaction.rb

Overview

Provides compaction support for chat sessions. Mixed into Chat.

Supplies configuration resolvers that derive concrete token budgets from the compaction: config block (ratios of num_ctx with absolute floors), the LLM-based summarization pipeline (+summarize_for_compaction+), and the tool_summary_line resolver for per-tool summary templates.

See Also:

  • for the `compaction:` block structure (`reserve`, `keep_recent`, `summary`).

Defined Under Namespace

Classes: Result

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.tool_summary_line(tool_name, result) ⇒ String

Generates a one-line summary for a tool call result.

Looks up the registered tool class and delegates to its summary_template(result:) class method. Falls back to a generic sentence if the tool is not registered or the template raises.

Parameters:

  • tool_name (String) —

    the registered name, e.g. 'read_file'.

  • result (String) —

    the raw JSON result string from execute.

Returns:

  • (String) —

    a short natural-language description of the call.



22
23
24
25
26
27
28
 
# File 'lib/ollama_chat/compaction.rb', line 22

def tool_summary_line(tool_name, result)
  default_summary = "was called."
  klass           = OllamaChat::Tools.registered[tool_name.to_s]&.class
  klass&.summary_template(result:) || default_summary
rescue StandardError
  default_summary
end

Instance Method Details

#assemble_summary(narrative, tool_entries, old_tool_entries = []) ⇒ Array(String, Array<Hash>) (private)

Assembles the final summary content and merged tool entries.

Parameters:

  • narrative (String) —

    the LLM-generated narrative.

  • tool_entries (Array<Hash>) —

    new tool-call entries from this pass.

  • old_tool_entries (Array<Hash>) (defaults to: []) —

    entries carried over from a prior compaction.

Returns:

  • (Array(String, Array<Hash>)) —

    the content string and the merged entries array.



229
230
231
232
233
234
235
236
237
 
# File 'lib/ollama_chat/compaction.rb', line 229

def assemble_summary(narrative, tool_entries, old_tool_entries = [])
  all_tools    = old_tool_entries + tool_entries
  tool_section = all_tools.map(&:to_json) * ?\n

  template = prompt(:assemble, context: 'compaction').to_s
  content  = template % { narrative:, tool_section: }

  [content, all_tools]
end

#build_tool_entries(messages) ⇒ Array<Hash> (private)

Builds the deterministic tool-call entries from tool result messages.

Parameters:

Returns:

  • (Array<Hash>) —

    tool-call entry hashes, empty if no tool results.



164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
 
# File 'lib/ollama_chat/compaction.rb', line 164

def build_tool_entries(messages)
  messages.filter_map do |msg|
    next unless tool_name = msg.tool_name.full?
    next if tool_name == 'runtime_information'
    uuid = msg.group_uuid.to_s[-8..]
    summary = OllamaChat::Compaction.tool_summary_line(
      tool_name, msg.content.to_s
    )
    time = msg.group_time&.strftime('%Y-%m-%d %H:%M')
    {
      'type'    => 'tool',
      'name'    => tool_name,
      'summary' => summary,
      'uuid'    => uuid,
      'time'    => time,
    }
  end
end

#call_summarizer(groups:, previous_summary:) ⇒ String (private)

Calls the LLM to generate the narrative portion of the summary.

Parameters:

  • groups (String) —

    the serialized groups block.

  • previous_summary (String, nil) —

    the prior summary for iteration.

Returns:

  • (String) —

    the LLM-generated narrative text.



188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
 
# File 'lib/ollama_chat/compaction.rb', line 188

def call_summarizer(groups:, previous_summary:)
  previous = previous_summary.to_s

  prompt = prompt(:summarize, context: 'compaction').to_s % {
    previous:, groups:
  }
  system = prompt(:compaction, context: 'system').to_s

  es = OllamaChat::TokenEstimator.estimate(prompt)
  log(:info, "Compaction: sending prompt " \
    "#{es.tokens_formatted} (#{es.bytes_formatted}) to #{@model}")

  response = Infobar.busy(
    label: 'Compacting context…',
    frames: :braille7,
    output: STDOUT,
  ) do
    generate(system:, prompt:, think: true).strip
  end

  if response.empty?
    log(:error, 'Compaction: LLM returned empty response')
    raise OllamaChat::CompactionError,
          'Summarization failed: model returned empty response'
  end

  res_es = OllamaChat::TokenEstimator.estimate(response)
  log(:info, "Compaction: got response " \
    "#{res_es.tokens_formatted} (#{res_es.bytes_formatted})")

  response
end

#compact_min_tokens(name) ⇒ Integer (private)

Returns the absolute minimum token floor for a given concern.

Parameters:

  • name (Symbol) —

    the concern name, e.g. :reserve, :keep_recent, or :summary.

Returns:

  • (Integer) —

    the configured min_tokens floor.



260
261
262
 
# File 'lib/ollama_chat/compaction.rb', line 260

def compact_min_tokens(name)
  config.compaction.attribute_get!(name).min_tokens
end

#compact_ratio(name) ⇒ Float (private)

Returns the fractional ratio (portion of num_ctx) for a given concern.

Parameters:

  • name (Symbol) —

    the concern name, e.g. :reserve, :keep_recent, or :summary.

Returns:

  • (Float) —

    the configured ratio value.



269
270
271
 
# File 'lib/ollama_chat/compaction.rb', line 269

def compact_ratio(name)
  config.compaction.attribute_get!(name).ratio
end

#compact_ratio_tokens(name, tokens) ⇒ Integer

Resolves the effective token budget for a concern given a context size.

Computes max(ratio * tokens, min_tokens).floor, so the floor acts as a guaranteed minimum even for very small context windows.

Parameters:

  • name (Symbol) —

    the concern name, e.g. :reserve, :keep_recent, or :summary.

  • tokens (Integer) —

    the context window size (num_ctx) to derive the budget from.

Returns:

  • (Integer) —

    the resolved token budget.



91
92
93
94
95
96
 
# File 'lib/ollama_chat/compaction.rb', line 91

def compact_ratio_tokens(name, tokens)
  [
    compact_ratio(name).to_f * tokens,
    compact_min_tokens(name).to_f,
  ].max.floor
end

#compact_with_retry ⇒ Boolean

Attempts compaction, prompting the user to retry on failure.

compact! is idempotent on failure: the LLM call (and thus any CompactionError) happens before the message list is mutated, so a retry always starts from a clean state.

Returns:

  • (Boolean) —

    true if compaction succeeded, false if the user declined to retry.



106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
 
# File 'lib/ollama_chat/compaction.rb', line 106

def compact_with_retry
  result   = messages.compact!
  session_sync
  if result
    report_compaction(result)
  else
    STDOUT.puts('Nothing to compact.')
  end
  return true
rescue OllamaChat::CompactionError => e
  STDERR.puts "⚠️  Compaction failed: #{e.message}"
  log(:error, "Compaction failed", data: {
    model: @model,
    ctx:   current_context_length,
    size:  messages.size,
  })
  retry if confirm?(
    prompt: '🔔 Retry compaction? (y/n) ',
    yes: /\Ay/i
  )
  false
end

#extract_narrative(content) ⇒ String (private)

Extracts the narrative portion from a summary message's content.

Parameters:

  • content (String, nil) —

    the summary content.

Returns:

  • (String) —

    the narrative text.



243
244
245
 
# File 'lib/ollama_chat/compaction.rb', line 243

def extract_narrative(content)
  content.to_s.sub(/\ntool_calls:\n.*\z/m, '').strip
end

#extract_old_tool_calls(summary_msg) ⇒ Array<Hash> (private)

Returns the tool-call entries from a previous summary message.

Parameters:

Returns:

  • (Array<Hash>) —

    the tool-call entries, empty if none.



251
252
253
 
# File 'lib/ollama_chat/compaction.rb', line 251

def extract_old_tool_calls(summary_msg)
  summary_msg.tool_calls || []
end

#format_summary_line(msg) ⇒ String (private)

Formats a single message as a summary line.

Parameters:

Returns:

  • (String) —

    a single line describing the message.



146
147
148
149
150
151
152
153
154
155
156
157
158
 
# File 'lib/ollama_chat/compaction.rb', line 146

def format_summary_line(msg)
  tool_name = msg.tool_name.full?
  parts  = +"[#{msg.role}]"
  parts << " (#{tool_name})" if tool_name
  parts << " (g:#{msg.group_uuid.to_s[-8..]})" if msg.group_uuid
  if tool_name
    line = OllamaChat::Compaction.tool_summary_line(tool_name, msg.content.to_s)
    parts << " #{line}"
  elsif msg.content.present?
    parts << " #{msg.content.to_s.strip}"
  end
  parts
end

#report_compaction(result) ⇒ Object (private)

Displays a compact compaction report to STDOUT after the Infobar.busy spinner has finished.

Replaces the former bare "Conversation compacted." line with before/after context metrics, candidate volume, and summary size.

Parameters:

  • result (Result) —

    the compaction result from compact!.



280
281
282
283
284
285
286
287
288
 
# File 'lib/ollama_chat/compaction.rb', line 280

def report_compaction(result)
  STDOUT.puts(<<~EOT)
    ✅ Conversation compacted.
       Summarized:   #{result.candidates} messages (#{result.candidate_size})
       Summary:      #{result.summary_size}
       Context:      #{result.context_before} → #{result.context_after}
       Stored total: #{result.stored_total}
  EOT
end

#serialize_groups(messages) ⇒ String (private)

Serializes non-system messages into a readable block for the prompt.

Parameters:

Returns:

  • (String) —

    a formatted block of message lines.



135
136
137
138
139
140
 
# File 'lib/ollama_chat/compaction.rb', line 135

def serialize_groups(messages)
  messages
    .reject { _1.role == 'system' }
    .map { |m| format_summary_line(m) }
    .join("\n")
end

#summarize_for_compaction(messages:, previous_summary: nil) ⇒ Array(String, Array<Hash>)

Generates a context summary for the given messages via LLM.

Serializes non-system messages into a structured prompt, calls the model via generate, and assembles the final summary with deterministic tool-call entries stored out-of-band.

Parameters:

  • messages (Array<OllamaChat::Message>) —

    the messages to summarize.

  • previous_summary (OllamaChat::Message, nil) (defaults to: nil) —

    the previous summary message for iterative re-compaction.

Returns:

  • (Array(String, Array<Hash>)) —

    the assembled content and the merged tool-call entries array.



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
 
# File 'lib/ollama_chat/compaction.rb', line 62

def summarize_for_compaction(messages:, previous_summary: nil)
  groups       = serialize_groups(messages)
  tool_entries = build_tool_entries(messages)

  es = OllamaChat::TokenEstimator.estimate(groups)
  log(:info, "Compaction: #{messages.size} messages, " \
    "#{es.tokens_formatted} (#{es.bytes_formatted}) groups")

  narrative_prev = ''
  old_tool_calls = []
  if previous_summary
    narrative_prev = extract_narrative(previous_summary.content)
    old_tool_calls = extract_old_tool_calls(previous_summary)
  end

  narrative = call_summarizer(groups:, previous_summary: narrative_prev)
  assemble_summary(narrative, tool_entries, old_tool_calls)
end