Class: OllamaChat::TTS

Inherits:
Object
  • Object
show all
Includes:
Ollama::Handlers::Concern, KramdownANSI, Speaker, Utils::UTF8Converter, Utils::ValueFormatter
Defined in:
lib/ollama_chat/tts.rb

Overview

Text-to-Speech handler for streaming audio playback.

This class handles converting streamed text responses into spoken audio using a remote TTS server. It manages parallel TTS requests for text blocks while maintaining strict ordering of audio chunks through an OrderedQueue. A coordinator thread ensures chunks are played in the correct sequence, even when multiple TTS requests complete out of order.

Examples:

tts = OllamaChat::TTS.new(chat: chat, voice: 'alloy')
tts.call(response)

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Speaker

#cancel_speaking, #speak, #wait_for_speaker

Methods included from KramdownANSI

#configure_kramdown_ansi_styles, #kramdown_ansi_parse, #kramdown_markdown_remove, #table_column_budgets, #wrap_table_rows

Methods included from Utils::ValueFormatter

#format_bytes, #format_tokens

Methods included from Utils::UTF8Converter

#convert_to_utf8

Constructor Details

#initialize(chat:, voice: nil) ⇒ OllamaChat::TTS

Initializes a new TTS handler.

Sets up the audio player, ordered queue for managing audio chunks, and starts a coordinator thread that processes chunks in order.

Parameters:

  • chat (OllamaChat::Chat) —

    the chat instance for logging

  • voice (String, nil) (defaults to: nil) —

    the voice ID to use for TTS (must be in the list returned by self.voices; falls back to nil if invalid or not provided)



58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/ollama_chat/tts.rb', line 58

def initialize(chat:, voice: nil)
  if voice
    if model = @voice_model = chat.config.voice.model?
      self.class.voices(model:, chat:).member?(voice) or voice = nil
    else
      self.class.voices(chat:).member?(voice) or voice = nil
      @voice_model = 'tts-1'
    end
  end
  @chat              = chat
  @voice             = voice
  @buffer            = +''
  @audio_player      = OllamaChat::Utils::AudioPlayer.new
  @ordered_queue     = OllamaChat::Utils::OrderedQueue.new
  @id_mutex          = Mutex.new
  @next_id           = 0
  @current_thread_id = 1
  @enqueued_count    = 0
  @finished_count    = 0
  @worker_threads    = []
  @coordinator       = Thread.new { run_coordinator }

  super(output: Tins::NULL)
end

Instance Attribute Details

#voice ⇒ String? (readonly)

The voice attribute reader returns the voice associated with the object.

Returns:

  • (String, nil) —

    the voice ID or nil if not set



87
88
89
 
# File 'lib/ollama_chat/tts.rb', line 87

def voice
  @voice
end

#voice_model ⇒ String? (readonly)

The model attribute reader returns the TTS model ID from config.

Returns:

  • (String, nil) —

    the model ID or nil if not configured



92
93
94
 
# File 'lib/ollama_chat/tts.rb', line 92

def voice_model
  @voice_model
end

Class Method Details

.voices(model: nil, chat:) ⇒ Array<String>

Returns a sorted list of available TTS voice IDs.

When model is nil the generic /v1/voices endpoint is queried and a hash array ([{"id": "…"}, …]) is expected. When a model name is given, /v1/audio/voices?model=<id> is queried and a string array (["…", …]) is expected (audio.cpp convention). If the server is unreachable or returns an error, an empty array is returned.

Parameters:

  • model (String, nil) (defaults to: nil) —

    the TTS model ID for per-model queries

  • chat (OllamaChat::Chat) —

    the chat instance (for HTTP middleware)

Returns:

  • (Array<String>) —

    a sorted list of unique voice identifiers



33
34
35
36
37
38
39
40
41
42
43
44
45
 
# File 'lib/ollama_chat/tts.rb', line 33

def self.voices(model: nil, chat:)
  url = model ? OC::OLLAMA::CHAT::TTS_URL + "/v1/audio/voices?model=#{model}"
              : OC::OLLAMA::CHAT::TTS_URL + '/v1/voices'
  result = nil
  chat.request_url_response(:get, url) do |response|
    data   = JSON.parse(response.body)
    result = data['voices']
    result = result.map { |v| v['id'] } unless model
  end
  result&.sort || []
rescue Excon::Error, JSON::ParserError
  []
end

Instance Method Details

#call(response) ⇒ self

Processes a streaming response chunk for TTS conversion.

Buffers incoming text content and extracts complete blocks (separated by blank lines) for immediate TTS processing. When the response is done, finalizes playback by waiting for all pending TTS requests to complete.

Parameters:

  • response (Ollama::Response) —

    the streaming response object containing text content and a done flag

Returns:

  • (self) —

    returns self for chaining



104
105
106
107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/ollama_chat/tts.rb', line 104

def call(response)
  if content = response.response || response.message&.content

    @buffer << content

    process_pending_blocks # Process and "read" completed blocks immediately! 🏎️💨
  end

  if response.done
    finalize
  end

  self
end

#enqueue_tts(block) ⇒ Thread (private)

Enqueues a block of text for TTS conversion.

Spawns a background thread that fetches audio chunks from the TTS server and pushes them into the ordered queue along with a :finished sentinel. Each enqueue operation increments the pending thread counter.

Parameters:

  • block (String) —

    the text to convert to speech

Returns:

  • (Thread) —

    the spawned TTS fetch thread



148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
 
# File 'lib/ollama_chat/tts.rb', line 148

def enqueue_tts(block)
  block = kramdown_markdown_remove(block, markdown: @chat.markdown.on?)
  thread_id = @id_mutex.synchronize { @next_id += 1 }
  @chat.log(:info, "TTS: Enqueueing block for synthesis", data: { thread_id:, block: })
  @id_mutex.synchronize { @enqueued_count += 1 }

  @worker_threads << Thread.new do
    chunk_id = 0
    data = {
      model:           @voice_model,
      input:           block,
      response_format: 'pcm', # This is currently ignored by audio.cpp 😿
    }
    if stream = @chat.config.voice.stream?&.enabled
      data |= {
        stream:        true,
        stream_format: @chat.config.voice.stream.format,
      }
    else
      data[:stream] = false
    end
    @voice and data[:voice] = @voice

    @ordered_queue.push([thread_id, chunk_id += 1], :started)

    fetch_tts(data) do |chunk|
      @ordered_queue.push([thread_id, chunk_id += 1], chunk)
    end

    @ordered_queue.push([thread_id, chunk_id += 1], :finished)
  end
end

#fetch_tts(data) {|chunk| ... } ⇒ Object (private)

Fetches TTS audio chunks from the remote server.

Makes a POST request to the TTS API with the given data, streaming audio chunks back via the provided block. Handles connection timeouts and error responses gracefully.

Parameters:

  • data (Hash) —

    the TTS request payload including:

    • :model [String] the TTS model to use (default: "tts-1")
    • :input [String] the text to convert
    • :response_format [String] the audio format (default: "pcm")
    • :stream [Boolean] whether to stream (default: true)
    • :voice [String, nil] optional voice ID

Yields:

  • (chunk) —

    yields each audio chunk (raw PCM bytes)

Yield Parameters:

  • chunk (String) —

    the raw audio data

Raises:

  • (Excon::Error) —

    on HTTP failure (logged and rescued)



294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'lib/ollama_chat/tts.rb', line 294

def fetch_tts(data, &block)
  @chat.log(:info, 'TTS: Requesting audio synthesis', data:)
  body = JSON.dump(data)
  url = OC::OLLAMA::CHAT::TTS_URL + '/v1/audio/speech'
  first_chunk = true
  response_block = -> chunk, _remaining, _total do
    if first_chunk
      first_chunk = false
      # audio.cpp wraps non-streaming responses in a 44-byte RIFF/WAV
      # header regardless of the requested response_format; strip it
      # so the player receives raw PCM.
      chunk = chunk[44..] if chunk.start_with?('RIFF')
    end
    block.call(chunk) unless chunk.empty?
  end
  response = @chat.request_url_response(
    :post, url,
    body:            ,
    headers:         { 'Content-Type' => 'application/json' },
    expects:         200,
    response_block:  ,
    logger:          @chat.debug ? OllamaChat::Utils::ExconLogger.new(@chat) : nil,
  )
rescue => e
  data = {}
  if response
    result = JSON.parse(response.body) rescue nil
    data[:response] = {
      status: response.status,
      result: ,
    }
  end
  @chat.log(:error, e, data:)
end

#finalize ⇒ Object (private)

Finalizes TTS playback.

Flushes any remaining buffered text, waits for all enqueued TTS threads to complete, stops the audio player, and waits for playback to finish. This ensures all audio is played before the handler shuts down.



186
187
188
189
190
191
192
193
194
195
196
197
198
199
 
# File 'lib/ollama_chat/tts.rb', line 186

def finalize
  @audio_player.start

  unless @buffer.blank?
    enqueue_tts(@buffer.dup)
    @buffer.clear
  end

  # Wait for all enqueued TTS threads to finish
  sleep 0.1 while @id_mutex.synchronize { @finished_count < @enqueued_count }

  @audio_player.stop
  sleep 0.1 while @audio_player.playing?
end

#join_workers ⇒ Object (private)

Joins all worker threads spawned by enqueue_tts.

Useful for tests that need to ensure all background fetch threads have completed before teardown.



205
206
207
 
# File 'lib/ollama_chat/tts.rb', line 205

def join_workers
  @worker_threads.each(&:join)
end

#process_pending_blocks ⇒ Object (private)

Note:

Blocks are identified by double-newline separators, ignoring leading numbered lists like "1."

Processes pending blocks from the buffer.

Extracts complete text blocks (terminated by blank lines \n\n) from the buffer and enqueues them for TTS conversion. This allows for near- real-time speech synthesis as text arrives incrementally.



129
130
131
132
133
134
135
136
137
 
# File 'lib/ollama_chat/tts.rb', line 129

def process_pending_blocks
  loop do
    match = @buffer.match(/\A(.{160,})\n\n/m) || @buffer.match(/\A(.*?)\n\n/m) or break
    size = match[0].size

    chunk = @buffer.slice!(0, size).full?(:strip) or next
    enqueue_tts(chunk)
  end
end

#run_coordinator ⇒ Object (private)

Note:

This runs in a background thread started during initialization

The coordinator loop that processes audio chunks in order.

Continuously monitors the ordered queue, popping chunks only when they belong to the current thread (maintaining strict ordering within each TTS request). When a :finished sentinel is encountered, switches to the next thread's chunks if available.



217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
 
# File 'lib/ollama_chat/tts.rb', line 217

def run_coordinator
  loop do
    item = @ordered_queue.peek

    if item.nil?
      sleep 0.01
      next
    end

    (thread_id, _chunk_id), _payload = item

    if @chat.debug
      @chat.log(
        :debug,
        'TTS: Coordinator peeking queue',
        data: {
          id: [ thread_id, _chunk_id ],
          payload: _payload.is_a?(Symbol) ? _payload : :audio_data,
        }
      )
    end

    @current_thread_id ||= thread_id

    if thread_id == @current_thread_id
      _id, payload = @ordered_queue.pop

      case payload
      when :started
        @chat.log(:info, 'TTS: Block synthesis scheduled', data: {
          id: _id,
          finished: @finished_count,
          pending: (@enqueued_count - @finished_count),
        })
      when :finished
        @id_mutex.synchronize do
          @finished_count += 1
          @chat.log(:info, 'TTS: Block synthesis completed', data: {
            id: _id,
            finished: @finished_count,
            pending: (@enqueued_count - @finished_count),
          })
        end
        next_item = @ordered_queue.peek
        @current_thread_id = next_item ? next_item[0][0] : nil
      else
        if @chat.debug
          @chat.log(
            :debug, 'TTS: Feeding audio chunk to player',
            data: { id: _id, size: format_bytes(payload.bytesize) }
          )
        end
        @audio_player << payload
      end
    else
      sleep 0.01
    end
  end
end