Class: OllamaChat::Tools::RunTests

Inherits:
Object
  • Object
show all
Includes:
Concern, Utils::PathValidator
Defined in:
lib/ollama_chat/tools/run_tests.rb

Overview

Tool for executing RSpec / Test‑Unit test suites.

The tool is registered under the name run_tests and exposes a single function that accepts a path (file or directory) and an optional coverage flag. The implementation simply spawns the configured test runner (RSpec or Minitest) and streams its output back to the caller.

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Utils::PathValidator

#assert_valid_path

Methods included from Concern

#name, #to_hash, #valid_json?

Methods included from Utils::Backup

#perform_backup

Methods included from Utils::ValueFormatter

#format_bytes, #format_tokens

Class Method Details

.register_name ⇒ String

Returns the registered name for this tool.

Returns:

  • (String) —

    the registered name for this tool



14
 
# File 'lib/ollama_chat/tools/run_tests.rb', line 14

def self.register_name = 'run_tests'

Instance Method Details

#check_path(path, config) ⇒ Pathname (private)

The check_path method determines the appropriate test directory path based on existing directories and validates it against a whitelist.

Parameters:

  • path (String) —

    the initial path to be checked

  • config (Object) —

    configuration object containing tool function settings

Returns:

  • (Pathname) —

    the expanded, validated and existing path



94
95
96
97
98
99
100
101
102
103
104
105
106
107
 
# File 'lib/ollama_chat/tools/run_tests.rb', line 94

def check_path(path, config)
  if path.blank?
    if File.exist?('./spec')
      path = './spec'
    elsif File.exist?('./test')
      path = './test'
    elsif File.exist?('./tests')
      path = './tests'
    else
      raise ArgumentError, 'path could not be determined'
    end
  end
  assert_valid_path(path, config.tools.functions.run_tests.allowed?, check: true)
end

#execute(tool_call, **opts) ⇒ String

Execute the tool with the provided arguments.

Parameters:

  • tool_call (ToolCall) —

    the tool invocation containing arguments

  • opts (Hash) —

    additional options (currently unused)

Returns:

  • (String) —

    JSON containing either result metrics (success, path, output, status) or error details (error, message).



55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
 
# File 'lib/ollama_chat/tools/run_tests.rb', line 55

def execute(tool_call, **opts)
  chat     = opts[:chat]
  config   = chat.config
  path     = tool_call.function.arguments.path
  coverage = tool_call.function.arguments.coverage || false
  path     = check_path(path, config)
  output, success = run_tests(path, coverage)
  result = JSON.parse(output.lines.last) rescue nil
  chat.log(:info, "Tests executed", data: {
    tool: name, path: path.to_s, success:, result:
  })

  message =
    if success
      "✨ All tests passed successfully in #{path.to_s.inspect}!"
    else
      "❌ Some tests failed in #{path.to_s.inspect}. Please check the error messages above."
    end

  {
    success: ,
    path:    path.to_s,
    output:  ,
    status:  success ? 'passed' : 'failed',
    message: ,
  }.to_json
rescue => e
  chat.log(:error, e, data: { tool: name, path: path.to_s })
  { error: e.class, message: e.message }.to_json
end

#execute_test_command(env, cmd) ⇒ String, Boolean (private)

Executes the test command in a separate process, streaming output to STDOUT.

Parameters:

  • env (Hash) —

    environment variables for the process

  • cmd (String) —

    the command to execute

Returns:

  • (String, Boolean) —

    the combined output from stdout and stderr and a success flag



139
140
141
142
143
144
145
146
147
148
149
150
 
# File 'lib/ollama_chat/tools/run_tests.rb', line 139

def execute_test_command(env, cmd)
  output  = +''
  success = false
  Open3.popen2e(env, cmd) do |_,io,waiter|
    while line = io.gets
      STDOUT.puts line
      output << line
    end
    success = waiter.value.success?
  end
  return output, success
end

#run_tests(path, coverage) ⇒ String, Boolean (private)

Run the test suite using the configured test runner.

Parameters:

  • path (String) —

    file or directory to test

  • coverage (Boolean) —

    whether to enable SimpleCov

Returns:

  • (String, Boolean) —

    the captured output and a success flag



114
115
116
117
118
119
120
121
122
123
 
# File 'lib/ollama_chat/tools/run_tests.rb', line 114

def run_tests(path, coverage)
  env = ENV.to_h | { 'START_SIMPLECOV' => coverage ? '1' : '0' }
  runner = test_runner
  unless runner.include?('%{path}')
    runner = runner.gsub('%', '%%') % ' %{path}'
  end
  cmd = runner % { path: Shellwords.escape(path) }
  output, success = execute_test_command(env, cmd)
  return output, success
end

#test_runner ⇒ String (private)

Resolve the test runner executable from configuration.

Returns:

  • (String) —

    the command to invoke the test runner



128
129
130
 
# File 'lib/ollama_chat/tools/run_tests.rb', line 128

def test_runner
  OC::OLLAMA::CHAT::TOOLS::TEST_RUNNER
end

#tool ⇒ Tool

Build the OpenAI function schema for the tool.

Returns:

  • (Tool) —

    a tool definition for running tests



18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
 
# File 'lib/ollama_chat/tools/run_tests.rb', line 18

def tool
  Tool.new(
    type: 'function',
    function: Tool::Function.new(
      name:,
      description: <<~EOT,
         Test Runner - Runs all tests/specs under *path* the path were the
         tests/specs are located. `coverage=false` by default; set to true
         for a coverage report. Returns JSON with the raw output and
         status. After execution, provide a concise overview of the test
         run, summarizing examples, failures, and coverage percentage
         if available.
      EOT
      parameters: Tool::Function::Parameters.new(
        type: 'object',
        properties: {
          path: Tool::Function::Parameters::Property.new(
            type: 'string',
            description: 'Path to file or directory to run tests for (path has to be allowed!)'
          ),
          coverage: Tool::Function::Parameters::Property.new(
            type: 'boolean',
            description: 'True if coverage data should be created, (default: false)'
          )
        },
        required: []
      )
    )
  )
end