mirror of
https://github.com/yusufipk/dikte.git
synced 2026-09-11 10:56:10 +00:00
Merge pull request #11 from yusufipk/local-models
Transcribe and clean up on this machine, fetching both programs
This commit is contained in:
Yusuf İpek
@@ -1,9 +1,9 @@
|
||||
# Dikte
|
||||
|
||||
Press `Ctrl+Space`, talk, press again. The recording goes to OpenAI or OpenRouter
|
||||
for transcription, a model cleans it up (dropping the *uh*s, the
|
||||
restarts, the missing punctuation), and the result lands in your clipboard and
|
||||
is pasted into whatever window you were typing in.
|
||||
Press `Ctrl+Space`, talk, press again. The recording is transcribed on this
|
||||
machine by default, a model cleans it up (dropping the *uh*s, the restarts, the
|
||||
missing punctuation), and the result lands in your clipboard and is pasted into
|
||||
whatever window you were typing in.
|
||||
|
||||
Built for KDE Plasma 6 on Wayland. No dependencies beyond system packages:
|
||||
just the Python standard library and PyQt6.
|
||||
@@ -41,10 +41,13 @@ two global shortcuts, whose keys are its two arguments. `./update.sh` pulls and
|
||||
puts all of that back, keeping the keys you chose; `./uninstall.sh` takes it away
|
||||
again and leaves your settings and dictations alone unless you pass `--purge`.
|
||||
|
||||
Three keys go in the settings window: **OpenAI**, **Groq** and **OpenRouter**.
|
||||
Speech to text runs on any of them (`gpt-4o-transcribe` by default), cleanup
|
||||
on OpenRouter (`google/gemini-3.5-flash-lite`) or, when either is installed, on
|
||||
Claude Code or Codex instead, so a single OpenRouter key can cover both. They fall back to `OPENAI_API_KEY`, `GROQ_API_KEY` and
|
||||
Speech to text and cleanup each pick a provider in the settings window, and both
|
||||
can run here, on whisper.cpp and llama.cpp: the program and the model are
|
||||
downloaded from that window, checksummed, so nothing has to be installed first
|
||||
and nothing leaves the machine. Otherwise speech to text runs on **OpenAI**,
|
||||
**Groq** or **OpenRouter** (`gpt-4o-transcribe` by default) and cleanup on
|
||||
OpenRouter (`google/gemini-3.5-flash-lite`) or, when either is installed, on
|
||||
Claude Code or Codex. The keys fall back to `OPENAI_API_KEY`, `GROQ_API_KEY` and
|
||||
`OPENROUTER_API_KEY`, and are
|
||||
stored in `~/.config/dikte/config.json`, mode 600. Cleanup can be switched off,
|
||||
in which case the raw transcript is pasted, and a thinking model's effort can be
|
||||
@@ -147,8 +150,10 @@ ipc.py one request and one reply over the local socket
|
||||
audio.py PCM capture: pw-record for dictation, ffmpeg for a meeting
|
||||
meeting.py channel split, speaker labelling, cleanup, minutes
|
||||
assistant.py running a dictation through Claude Code, Codex or OpenRouter
|
||||
api.py transcription on either provider, OpenRouter cleanup (stdlib only)
|
||||
cleanup.py who rewrites the transcript: OpenRouter, Claude Code or Codex
|
||||
api.py transcription and cleanup requests (stdlib only)
|
||||
cleanup.py who rewrites the transcript: OpenRouter, here, Claude or Codex
|
||||
ggml.py whisper.cpp and llama.cpp here: fetch, verify, keep serving
|
||||
hub.py what GitHub and Hugging Face have on offer today
|
||||
worker.py transcribe → clean up → clipboard → paste
|
||||
vad.py deciding whether a recording holds speech at all
|
||||
filetranscribe.py file transcription: ffmpeg, chunking, timestamps
|
||||
|
||||
+16
-12
@@ -1,9 +1,8 @@
|
||||
# Dikte
|
||||
|
||||
`Ctrl+Space`'e bas, konuş, tekrar bas. Ses OpenAI'ye ya da OpenRouter'a gidip
|
||||
yazıya çevrilir, bir model transkripti temizler (ıı'lar,
|
||||
tekrarlar, eksik noktalama), sonuç panoya kopyalanır ve o an yazdığın pencereye
|
||||
yapıştırılır.
|
||||
`Ctrl+Space`'e bas, konuş, tekrar bas. Ses varsayılan olarak bu makinede yazıya
|
||||
çevrilir, bir model transkripti temizler (ıı'lar, tekrarlar, eksik noktalama),
|
||||
sonuç panoya kopyalanır ve o an yazdığın pencereye yapıştırılır.
|
||||
|
||||
KDE Plasma 6 / Wayland için yazıldı. Sistem paketleri dışında bağımlılığı yok:
|
||||
sadece Python standart kütüphanesi ve PyQt6.
|
||||
@@ -42,12 +41,15 @@ son sürümü çeker ve bunları senin seçtiğin tuşlarla yerine koyar;
|
||||
`./uninstall.sh` hepsini geri alır, `--purge` demedikçe ayarlarına ve
|
||||
diktelerine dokunmaz.
|
||||
|
||||
Ayarlar penceresinde üç anahtar istenir: **OpenAI**, **Groq** ve **OpenRouter**.
|
||||
Sesi yazıya çevirme üçünden birinde çalışır (varsayılan `gpt-4o-transcribe`),
|
||||
temizleme OpenRouter'da (`google/gemini-3.5-flash-lite`) ya da kuruluysa Claude
|
||||
Code veya Codex'te, yani tek bir OpenRouter anahtarı ikisine de yeter. Boş bırakırsan `OPENAI_API_KEY`,
|
||||
`GROQ_API_KEY` ve `OPENROUTER_API_KEY` kullanılır;
|
||||
anahtarlar `~/.config/dikte/config.json`
|
||||
Sesi yazıya çevirme ve temizleme, ayarlar penceresinde ayrı ayrı sağlayıcı
|
||||
seçer; ikisi de burada çalışabilir, whisper.cpp ve llama.cpp üzerinde: program da
|
||||
model de o pencereden, sha256 doğrulamasıyla indirilir, yani önceden hiçbir şey
|
||||
kurman gerekmez ve makineden hiçbir şey çıkmaz. Bulutu seçersen sesi yazıya
|
||||
çevirme **OpenAI**, **Groq** ya da **OpenRouter**'da (varsayılan
|
||||
`gpt-4o-transcribe`), temizleme OpenRouter'da
|
||||
(`google/gemini-3.5-flash-lite`) ya da kuruluysa Claude Code veya Codex'te
|
||||
çalışır. Anahtarları boş bırakırsan `OPENAI_API_KEY`, `GROQ_API_KEY` ve
|
||||
`OPENROUTER_API_KEY` kullanılır; anahtarlar `~/.config/dikte/config.json`
|
||||
içinde, izinler 600. Temizlemeyi tamamen kapatabilirsin, o zaman ham transkript
|
||||
yapıştırılır; modelin yanındaki kutudan düşünme seviyesini de seçebilirsin.
|
||||
|
||||
@@ -146,8 +148,10 @@ ipc.py yerel sokette bir istek, bir cevap
|
||||
audio.py PCM kaydı: diktede pw-record, toplantıda ffmpeg
|
||||
meeting.py kanal ayırma, konuşmacı etiketi, temizleme, tutanak
|
||||
assistant.py dikteyi Claude Code, Codex ya da OpenRouter'dan geçirme
|
||||
api.py iki sağlayıcıda transkript + OpenRouter temizleme (yalnız stdlib)
|
||||
cleanup.py transkripti kim temizler: OpenRouter, Claude Code ya da Codex
|
||||
api.py transkript ve temizleme istekleri (yalnız stdlib)
|
||||
cleanup.py transkripti kim temizler: OpenRouter, burası, Claude ya da Codex
|
||||
ggml.py whisper.cpp ve llama.cpp'yi indirip burada çalıştırma
|
||||
hub.py GitHub ve Hugging Face'te bugün ne olduğu
|
||||
worker.py transkript → temizleme → pano → yapıştırma
|
||||
vad.py kayıtta gerçekten konuşma var mı kararı
|
||||
filetranscribe.py dosyadan transkript: ffmpeg, parçalama, zaman damgaları
|
||||
|
||||
@@ -1,9 +1,13 @@
|
||||
"""OpenAI, Groq and OpenRouter calls, stdlib only.
|
||||
"""OpenAI, Groq, OpenRouter and this machine, stdlib only.
|
||||
|
||||
Transcription runs on any of the three: Groq and OpenRouter both mirror OpenAI's
|
||||
/audio/transcriptions endpoint field for field, so one multipart request serves
|
||||
all of them and only the key, the base URL and the model id change. Cleanup is
|
||||
always OpenRouter.
|
||||
Transcription runs on any of the four: Groq and OpenRouter both mirror OpenAI's
|
||||
/audio/transcriptions endpoint field for field, and ggml.py starts whisper.cpp
|
||||
on that same path, so one multipart request serves all of them and only the key,
|
||||
the base URL and the model id change. llama.cpp answers /chat/completions the way
|
||||
OpenRouter does, so cleanup here is the same request too.
|
||||
|
||||
What is on this machine has no key, and its base URL is not known until a server
|
||||
is up, which is the one thing this module has to fill in for it.
|
||||
"""
|
||||
|
||||
import collections
|
||||
@@ -14,6 +18,7 @@ import secrets
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
import ggml
|
||||
from i18n import t
|
||||
|
||||
APP_URL = "https://github.com/yusufipk/dikte"
|
||||
@@ -22,6 +27,12 @@ OPENAI_URL = "https://api.openai.com/v1"
|
||||
GROQ_URL = "https://api.groq.com/openai/v1"
|
||||
OPENROUTER_URL = "https://openrouter.ai/api/v1"
|
||||
|
||||
# The floor for a local request. The timeouts elsewhere are sized for a hosted
|
||||
# API, where a slow answer is a bill running; here the only thing being spent is
|
||||
# time, and a long recording on a machine without a graphics card takes a good
|
||||
# deal of it. Cutting that off would throw the work away for nothing.
|
||||
LOCAL_TIMEOUT = 3600
|
||||
|
||||
# Where a transcription request goes; built by config.Config.transcribe_target().
|
||||
# `service` is the name the user sees in an error, `provider` the one the code
|
||||
# branches on.
|
||||
@@ -33,9 +44,11 @@ def timestamp_model(provider, selected=""):
|
||||
|
||||
OpenAI keeps them to whisper-1 and OpenRouter namespaces that id. Everything
|
||||
Groq transcribes with is a whisper, so the model already chosen does it and
|
||||
the fallback is only for a provider left on its default.
|
||||
the fallback is only for a provider left on its default. So is everything the
|
||||
local server runs, whatever the file is called, and there asking for another
|
||||
model would name one it has never heard of.
|
||||
"""
|
||||
if provider == "groq":
|
||||
if provider in ("groq", "local"):
|
||||
return selected or "whisper-large-v3-turbo"
|
||||
return "openai/whisper-1" if provider == "openrouter" else "whisper-1"
|
||||
|
||||
@@ -114,7 +127,11 @@ def _multipart(fields, file_field, file_path):
|
||||
|
||||
|
||||
def _headers(provider, api_key, content_type=None):
|
||||
headers = {"Authorization": f"Bearer {api_key}", "User-Agent": USER_AGENT}
|
||||
headers = {"User-Agent": USER_AGENT}
|
||||
# A server on this machine has nothing to authorise, and sending it a
|
||||
# bearer token would only be a made-up one.
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
if content_type:
|
||||
headers["Content-Type"] = content_type
|
||||
if provider == "openrouter":
|
||||
@@ -124,16 +141,45 @@ def _headers(provider, api_key, content_type=None):
|
||||
return headers
|
||||
|
||||
|
||||
def serving(server):
|
||||
"""The base URL of a local server, started if it is not up yet.
|
||||
|
||||
It picks its own port, so this is the first moment its address exists.
|
||||
serve() is idempotent: once it is running this costs nothing.
|
||||
"""
|
||||
try:
|
||||
return server.serve()
|
||||
except ggml.LocalError as exc:
|
||||
raise ApiError(str(exc)) from None
|
||||
|
||||
|
||||
def local_failure(service, server, exc):
|
||||
"""A server that died mid-request, explained by its own output.
|
||||
|
||||
Without this the message is that the connection dropped, when the reason for
|
||||
it was printed by the process at the other end.
|
||||
"""
|
||||
detail = server.error()
|
||||
return ApiError(f"{service}: {exc}" + (f" ({detail})" if detail else ""),
|
||||
exc.status)
|
||||
|
||||
|
||||
def _transcribe_request(target, wav_path, language, prompt, response_format,
|
||||
granularity=None, timeout=300):
|
||||
if not target.api_key:
|
||||
if target.provider == "local":
|
||||
# The timeouts here are sized for a hosted API, where a slow answer is a
|
||||
# bill running. Locally the only thing being spent is time.
|
||||
target = target._replace(base_url=serving(ggml.whisper))
|
||||
timeout = max(timeout, LOCAL_TIMEOUT)
|
||||
elif not target.api_key:
|
||||
raise ApiError(t("{service} API key is empty. Add it in Settings.",
|
||||
service=target.service))
|
||||
fields = [("model", target.model), ("response_format", response_format)]
|
||||
if language and language != "auto":
|
||||
fields.append(("language", language))
|
||||
# OpenRouter takes the hint field and throws it away, so spare it the bytes.
|
||||
# The same words still reach the cleanup model as a glossary.
|
||||
# The same words still reach the cleanup model as a glossary. whisper.cpp
|
||||
# takes it as the initial prompt, the way OpenAI does.
|
||||
if prompt and target.provider != "openrouter":
|
||||
fields.append(("prompt", prompt))
|
||||
if granularity:
|
||||
@@ -145,14 +191,58 @@ def _transcribe_request(target, wav_path, language, prompt, response_format,
|
||||
_headers(target.provider, target.api_key, ctype), timeout=timeout,
|
||||
)
|
||||
except ApiError as exc:
|
||||
if target.provider == "local":
|
||||
raise local_failure(target.service, ggml.whisper, exc) from None
|
||||
raise explain(exc, target.service) from None
|
||||
|
||||
|
||||
# Whisper marks the start of a word with a leading space, so a piece of text
|
||||
# that does not begin with one continues the word before it rather than starting
|
||||
# a new one. Both helpers below turn on that.
|
||||
def _continues_a_word(previous, following):
|
||||
return bool(previous) and not previous[-1:].isspace() and not following[:1].isspace()
|
||||
|
||||
|
||||
def _local_text(text):
|
||||
"""whisper.cpp's segments, joined back into the flowing line OpenAI returns.
|
||||
|
||||
Its plain text puts one segment per line, and a segment boundary falls
|
||||
wherever the tokens fell, which in Turkish lands inside a word about as
|
||||
often as between two. Nothing takes the line break's place: whisper's own
|
||||
leading spaces are what separate the words, and a break inside "değ|iller"
|
||||
has nothing on either side of it worth keeping.
|
||||
"""
|
||||
return "".join(text.split("\n"))
|
||||
|
||||
|
||||
def _merge_word_splits(segments):
|
||||
"""Fold a segment that begins mid-word into the one it continues.
|
||||
|
||||
The hosted whisper-1 hands back segments cut on sentences; whisper.cpp cuts
|
||||
them on tokens, and a subtitle cue reading "değ" is not a cue. The times are
|
||||
joined along with the text, so the merged segment still covers the whole
|
||||
word.
|
||||
"""
|
||||
merged = []
|
||||
for seg in segments:
|
||||
text = seg.get("text") or ""
|
||||
if merged and _continues_a_word(merged[-1]["text"], text):
|
||||
merged[-1]["text"] += text
|
||||
merged[-1]["end"] = seg.get("end") or merged[-1]["end"]
|
||||
continue
|
||||
merged.append({"text": text, "start": seg.get("start") or 0.0,
|
||||
"end": seg.get("end") or 0.0})
|
||||
return merged
|
||||
|
||||
|
||||
def transcribe(target, wav_path, language="", prompt="", timeout=300):
|
||||
data = _transcribe_request(
|
||||
target, wav_path, language, prompt, "json", timeout=timeout
|
||||
)
|
||||
text = (data.get("text") or "").strip()
|
||||
text = data.get("text") or ""
|
||||
if target.provider == "local":
|
||||
text = _local_text(text)
|
||||
text = text.strip()
|
||||
if not text:
|
||||
raise ApiError(t("Transcript came back empty."))
|
||||
return text
|
||||
@@ -166,6 +256,8 @@ def transcribe_segments(target, wav_path, language="", prompt="", timeout=300):
|
||||
granularity="segment", timeout=timeout,
|
||||
)
|
||||
segments = data.get("segments") or []
|
||||
if target.provider == "local":
|
||||
segments = _merge_word_splits(segments)
|
||||
out = []
|
||||
for seg in segments:
|
||||
text = (seg.get("text") or "").strip()
|
||||
@@ -174,18 +266,55 @@ def transcribe_segments(target, wav_path, language="", prompt="", timeout=300):
|
||||
end = float(seg.get("end") or 0.0)
|
||||
out.append((start, max(end, start), text))
|
||||
if not out:
|
||||
text = (data.get("text") or "").strip()
|
||||
text = data.get("text") or ""
|
||||
if target.provider == "local":
|
||||
text = _local_text(text)
|
||||
text = text.strip()
|
||||
if not text:
|
||||
raise ApiError(t("Transcript came back empty."))
|
||||
out = [(0.0, 0.0, text)]
|
||||
return out
|
||||
|
||||
|
||||
def _thinking(payload, provider, reasoning):
|
||||
"""Ask for as much thinking as this provider understands, or for none.
|
||||
|
||||
An empty level means "whatever the model does on its own", so nothing is
|
||||
sent. The two mean opposite things by that, which is why the setting is kept
|
||||
per provider: OpenRouter's cleanup models answer straight away, while a local
|
||||
model that was trained to think will think, and cleanup is punctuation rather
|
||||
than a job worth thinking about.
|
||||
"""
|
||||
if not reasoning:
|
||||
return
|
||||
if provider == "local-llm":
|
||||
# What llama.cpp passes to the chat template. The models that think read
|
||||
# it; the ones that do not ignore it.
|
||||
payload["chat_template_kwargs"] = {"enable_thinking": reasoning != "none"}
|
||||
elif reasoning != "none":
|
||||
# The thinking itself is never shown, so ask for it to be left out.
|
||||
payload["reasoning"] = {"effort": reasoning, "exclude": True}
|
||||
|
||||
|
||||
def local_ceiling(text):
|
||||
"""How much of a reply is worth waiting for from a model on this machine.
|
||||
|
||||
Cleanup gives back what it was given, near enough, so a reply several times
|
||||
the length of the transcript is a model that has lost the thread rather than
|
||||
one doing the job. A small one will happily repeat the transcript until the
|
||||
context is full, and every one of those tokens is a second of somebody
|
||||
waiting. A hosted model is left alone: there the same runaway is rare, and a
|
||||
ceiling would cut the minutes short instead.
|
||||
"""
|
||||
return max(512, len(text))
|
||||
|
||||
|
||||
def cleanup(text, api_key, model, system_prompt, reasoning="",
|
||||
base_url=OPENROUTER_URL, timeout=180):
|
||||
if not api_key:
|
||||
base_url=OPENROUTER_URL, timeout=180, provider="openrouter",
|
||||
service="OpenRouter"):
|
||||
if not api_key and provider != "local-llm":
|
||||
raise ApiError(t("{service} API key is empty. Add it in Settings.",
|
||||
service="OpenRouter"))
|
||||
service=service))
|
||||
payload = {
|
||||
"model": model,
|
||||
"temperature": 0,
|
||||
@@ -194,25 +323,30 @@ def cleanup(text, api_key, model, system_prompt, reasoning="",
|
||||
{"role": "user", "content": f"<transcript>\n{text}\n</transcript>"},
|
||||
],
|
||||
}
|
||||
# An empty level means "whatever the model does on its own"; anything else is
|
||||
# one of OpenRouter's efforts. The thinking itself is never shown, so ask for
|
||||
# it to be left out of the reply.
|
||||
if reasoning:
|
||||
payload["reasoning"] = {"effort": reasoning, "exclude": True}
|
||||
if provider == "local-llm":
|
||||
payload["max_tokens"] = local_ceiling(text)
|
||||
_thinking(payload, provider, reasoning)
|
||||
try:
|
||||
data = _request(
|
||||
f"{base_url.rstrip('/')}/chat/completions",
|
||||
json.dumps(payload).encode("utf-8"),
|
||||
_headers("openrouter", api_key, "application/json"),
|
||||
_headers(provider, api_key, "application/json"),
|
||||
timeout=timeout,
|
||||
)
|
||||
except ApiError as exc:
|
||||
raise explain(exc, "OpenRouter") from None
|
||||
raise explain(exc, service) from None
|
||||
choices = data.get("choices") or []
|
||||
if not choices:
|
||||
raise ApiError(_extract_error(json.dumps(data)))
|
||||
content = ((choices[0].get("message") or {}).get("content") or "").strip()
|
||||
message = choices[0].get("message") or {}
|
||||
content = (message.get("content") or "").strip()
|
||||
if not content:
|
||||
# A thinking model can spend the whole reply on the thinking and leave
|
||||
# nothing to paste. Worth naming, because the fix is a setting rather
|
||||
# than a retry: cleanup is not a job that wants thinking.
|
||||
if message.get("reasoning_content") or message.get("reasoning"):
|
||||
raise ApiError(t("The cleanup model spent its whole reply on "
|
||||
"thinking. Set Thinking to \u201cOff\u201d."))
|
||||
raise ApiError(t("The cleanup model returned an empty reply."))
|
||||
return content
|
||||
|
||||
|
||||
+28
-1
@@ -20,9 +20,10 @@ import tempfile
|
||||
|
||||
import api
|
||||
import assistant
|
||||
import ggml
|
||||
from i18n import t
|
||||
|
||||
PROVIDERS = ("openrouter", "claude", "codex")
|
||||
PROVIDERS = ("openrouter", "local", "claude", "codex")
|
||||
|
||||
|
||||
class CleanupError(api.ApiError):
|
||||
@@ -47,6 +48,8 @@ def executable(name):
|
||||
def model(conf):
|
||||
"""Which model does the cleaning, for the history and the settings window."""
|
||||
name = provider(conf)
|
||||
if name == "local":
|
||||
return conf["local_llm_model"]
|
||||
if name == "claude":
|
||||
return conf["cleanup_claude_model"].strip() or "haiku"
|
||||
if name == "codex":
|
||||
@@ -65,10 +68,34 @@ def run(text, conf, system_prompt, timeout=180):
|
||||
reasoning=conf["cleanup_reasoning"],
|
||||
base_url=conf["openrouter_base_url"], timeout=timeout,
|
||||
)
|
||||
if name == "local":
|
||||
return _local(text, conf, system_prompt, timeout)
|
||||
runner = _claude if name == "claude" else _codex
|
||||
return runner(text, conf, system_prompt, timeout)
|
||||
|
||||
|
||||
def _local(text, conf, system_prompt, timeout):
|
||||
"""llama.cpp, on this machine, answering the request OpenRouter answers.
|
||||
|
||||
No key and no bill, and the address does not exist until the server is up,
|
||||
which is what starting it here is for. The timeout is the hosted one raised:
|
||||
the only thing being spent is time.
|
||||
"""
|
||||
service = t("Local model")
|
||||
try:
|
||||
return api.cleanup(
|
||||
text, "", conf["local_llm_model"], system_prompt,
|
||||
reasoning=conf["local_llm_reasoning"],
|
||||
base_url=api.serving(ggml.llm),
|
||||
timeout=max(timeout, api.LOCAL_TIMEOUT),
|
||||
provider="local-llm", service=service,
|
||||
)
|
||||
except api.ApiError as exc:
|
||||
# A server that died mid-request would otherwise report only that the
|
||||
# connection dropped, when the reason is in its own output.
|
||||
raise api.local_failure(service, ggml.llm, exc) from None
|
||||
|
||||
|
||||
def _wrap(text):
|
||||
"""The same fence the OpenRouter call puts around it: this is the material,
|
||||
not the instruction, however much of it reads like one."""
|
||||
|
||||
@@ -7,7 +7,9 @@ import os
|
||||
import pathlib
|
||||
|
||||
import api
|
||||
import ggml
|
||||
import i18n
|
||||
from i18n import t
|
||||
|
||||
|
||||
def _xdg(var, default):
|
||||
@@ -368,18 +370,49 @@ DEFAULTS = {
|
||||
"groq_base_url": "https://api.groq.com/openai/v1",
|
||||
"openrouter_api_key": "",
|
||||
"openrouter_base_url": "https://openrouter.ai/api/v1",
|
||||
"transcribe_provider": "openai", # a key of TRANSCRIBERS
|
||||
"transcribe_provider": "local", # "local", or a key of TRANSCRIBERS
|
||||
"transcribe_model": "gpt-4o-transcribe", # used when provider is openai
|
||||
"groq_transcribe_model": "whisper-large-v3-turbo",
|
||||
"openrouter_transcribe_model": "openai/gpt-4o-transcribe",
|
||||
"language": "tr",
|
||||
"transcribe_prompt": "",
|
||||
|
||||
# --- whisper.cpp, on this machine ---------------------------------------
|
||||
# The program and the model are both fetched from Settings; empty means
|
||||
# nothing has been downloaded yet, which is what opens Settings on a first
|
||||
# run.
|
||||
# Pointed at the suggestion rather than at nothing, so the settings window
|
||||
# opens with the Download button already on the right model.
|
||||
"local_model": ggml.SUGGESTED_WHISPER,
|
||||
"local_threads": 0, # 0 -> whisper.cpp picks
|
||||
"local_gpu": True,
|
||||
"local_preload": True, # load the model while Dikte starts, rather
|
||||
# than on the first dictation
|
||||
"local_binary": "", # empty -> whichever copy ggml.py finds
|
||||
|
||||
"cleanup_enabled": True,
|
||||
"cleanup_provider": "openrouter", # openrouter | claude | codex
|
||||
"cleanup_provider": "openrouter", # a name in cleanup.PROVIDERS
|
||||
"cleanup_model": "google/gemini-3.5-flash-lite",
|
||||
"cleanup_claude_model": "haiku", # Claude Code: an alias, or a full model id
|
||||
"cleanup_codex_model": "", # empty -> whatever Codex is set to
|
||||
"cleanup_reasoning": "", # empty -> whatever the model does by default
|
||||
|
||||
# --- llama.cpp, on this machine -----------------------------------------
|
||||
# Kept apart from the meeting settings on purpose. Cleanup is punctuation
|
||||
# and filler words, which a small model does in a moment; the minutes are a
|
||||
# summary of an hour, which it does not.
|
||||
"local_llm_model": "", # a file name, e.g. gemma-3-4b-it-Q4_K_M.gguf
|
||||
# Where the model list is read from; the settings window offers the
|
||||
# publishers ggml.py knows of and takes any other one that is typed in.
|
||||
"local_llm_repo": ggml.SUGGESTED_LLM[0],
|
||||
"local_llm_threads": 0,
|
||||
"local_llm_gpu": True,
|
||||
"local_llm_context": 8192,
|
||||
"local_llm_binary": "",
|
||||
"local_llm_preload": False, # heavier than whisper, so only when asked
|
||||
# Off rather than empty: a model trained to think will, and 300 tokens of
|
||||
# reasoning about a comma is 300 tokens of waiting.
|
||||
"local_llm_reasoning": "none",
|
||||
"cleanup_prompt": "", # empty -> language-specific default
|
||||
"auto_paste": True,
|
||||
"paste_shortcut": "ctrl+v",
|
||||
@@ -528,14 +561,63 @@ class Config:
|
||||
return self.api_key("openrouter_api_key")
|
||||
|
||||
def transcribe_target(self):
|
||||
"""Key, endpoint and model for whichever provider does speech to text."""
|
||||
"""Key, endpoint and model for whichever provider does speech to text.
|
||||
|
||||
The local one is not in the table and leaves its base URL empty on
|
||||
purpose: the server picks a port when it starts, and reading a setting
|
||||
must not be what launches a process. api.py fills the address in when it
|
||||
is about to send the request, which is the moment the server is needed
|
||||
anyway.
|
||||
"""
|
||||
name = self["transcribe_provider"]
|
||||
if name == "local":
|
||||
return api.Target("local", t("Local whisper"), "", "",
|
||||
self["local_model"])
|
||||
if name not in TRANSCRIBERS:
|
||||
name = DEFAULTS["transcribe_provider"]
|
||||
# A config written by a fork, or by a version that dropped one. The
|
||||
# shipped default is not in the table, so this names the hosted one
|
||||
# to land on rather than reading it from there.
|
||||
name = "openai"
|
||||
who = TRANSCRIBERS[name]
|
||||
return api.Target(name, who.service, self.api_key(who.key),
|
||||
self[who.url], self[who.model])
|
||||
|
||||
def transcribe_ready(self):
|
||||
"""Whether speech to text could run right now, without opening Settings."""
|
||||
if self["transcribe_provider"] == "local":
|
||||
return self.local_whisper_ready()
|
||||
return bool(self.transcribe_target().api_key)
|
||||
|
||||
def local_whisper_ready(self):
|
||||
return bool(ggml.program_path(ggml.WHISPER, self["local_binary"])
|
||||
and self["local_model"]
|
||||
and ggml.have_model(ggml.whisper_model_path(self["local_model"])))
|
||||
|
||||
def local_llm_ready(self):
|
||||
return bool(ggml.program_path(ggml.LLAMA, self["local_llm_binary"])
|
||||
and self["local_llm_model"]
|
||||
and ggml.have_model(ggml.llm_model_path(self["local_llm_model"])))
|
||||
|
||||
def apply_local(self):
|
||||
"""Hand the local settings to the servers, restarting what they change."""
|
||||
ggml.whisper.configure(
|
||||
model=self["local_model"],
|
||||
threads=int(self["local_threads"]),
|
||||
gpu=bool(self["local_gpu"]),
|
||||
binary=self["local_binary"],
|
||||
)
|
||||
ggml.llm.configure(
|
||||
model=self["local_llm_model"],
|
||||
threads=int(self["local_llm_threads"]),
|
||||
gpu=bool(self["local_llm_gpu"]),
|
||||
binary=self["local_llm_binary"],
|
||||
context=int(self["local_llm_context"]),
|
||||
)
|
||||
|
||||
def uses_local_llm(self):
|
||||
"""Whether anything is set to run the local cleanup model."""
|
||||
return self["cleanup_provider"] == "local"
|
||||
|
||||
def cleanup_prompt(self, with_timestamps=False, with_speakers=False,
|
||||
subtitles=False):
|
||||
turkish = i18n.language() == "tr"
|
||||
|
||||
@@ -7,16 +7,20 @@ terminal talks to. Every verb it answers is in cli.py, which is also what runs
|
||||
command line says "there is no instance to talk to, so be one".
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
import socket
|
||||
import sys
|
||||
import threading
|
||||
|
||||
# A Wayland client cannot place a window in a screen corner, so the indicator
|
||||
# is drawn through XWayland.
|
||||
if os.environ.get("XDG_SESSION_TYPE") == "wayland" and os.environ.get("DISPLAY"):
|
||||
os.environ.setdefault("QT_QPA_PLATFORM", "xcb")
|
||||
|
||||
from PyQt6.QtCore import QTimer, QElapsedTimer # noqa: E402
|
||||
from PyQt6.QtCore import QTimer, QElapsedTimer, QSocketNotifier # noqa: E402
|
||||
from PyQt6.QtGui import QAction, QIcon # noqa: E402
|
||||
from PyQt6.QtNetwork import QLocalServer, QLocalSocket # noqa: E402
|
||||
from PyQt6.QtWidgets import QApplication, QMenu, QSystemTrayIcon # noqa: E402
|
||||
@@ -25,6 +29,7 @@ import assistant # noqa: E402
|
||||
import audio # noqa: E402
|
||||
import cli # noqa: E402
|
||||
import config as cfg # noqa: E402
|
||||
import ggml # noqa: E402
|
||||
import hotkey # noqa: E402
|
||||
import i18n # noqa: E402
|
||||
import ipc # noqa: E402
|
||||
@@ -92,6 +97,9 @@ class Dikte:
|
||||
self.meeting_recorder = audio.MeetingRecorder()
|
||||
self.meetings = MeetingPipeline(self.conf)
|
||||
self.evdev = hotkey.EvdevHotkey()
|
||||
# Before anything of ours is started: a server from a Dikte that was
|
||||
# killed outright is still holding a model in memory.
|
||||
ggml.sweep()
|
||||
|
||||
self.recorder.level.connect(self._on_level)
|
||||
self.recorder.stopped.connect(self._on_recorded)
|
||||
@@ -805,9 +813,44 @@ class Dikte:
|
||||
# Don't drop the object while its own signal is still being delivered.
|
||||
QTimer.singleShot(0, lambda: setattr(self, "settings_window", None))
|
||||
|
||||
def _apply_local(self):
|
||||
"""Pass the local settings on, and hold the models ready if asked to.
|
||||
|
||||
Loading a model takes a second or two for whisper and longer for an LLM.
|
||||
Doing it while Dikte starts rather than on the first dictation is the
|
||||
whole reason a server is kept alive instead of running the program once
|
||||
per recording; the checkboxes are there for the machine whose memory is
|
||||
wanted elsewhere.
|
||||
"""
|
||||
self.conf.apply_local()
|
||||
wanted = []
|
||||
if self.conf["transcribe_provider"] == "local":
|
||||
if self.conf["local_preload"] and self.conf.local_whisper_ready():
|
||||
wanted.append((ggml.whisper, "whisper"))
|
||||
else:
|
||||
ggml.whisper.stop() # give the memory back when it is not in use
|
||||
if self.conf.uses_local_llm():
|
||||
if self.conf["local_llm_preload"] and self.conf.local_llm_ready():
|
||||
wanted.append((ggml.llm, "llama"))
|
||||
else:
|
||||
ggml.llm.stop()
|
||||
|
||||
def warm():
|
||||
for server, name in wanted:
|
||||
try:
|
||||
server.serve()
|
||||
except ggml.LocalError as exc:
|
||||
# Not worth an indicator: the first dictation raises the
|
||||
# same thing where the user can act on it.
|
||||
print(f"dikte: {name}: {exc}", file=sys.stderr)
|
||||
|
||||
if wanted:
|
||||
threading.Thread(target=warm, daemon=True).start()
|
||||
|
||||
def _apply_settings(self):
|
||||
self.overlay.corner = self.conf["overlay_corner"]
|
||||
self.ask_overlay.corner = self.conf["overlay_corner"]
|
||||
self._apply_local()
|
||||
self._build_tray()
|
||||
self._refresh_tray()
|
||||
if self.conf["evdev_hotkey"]:
|
||||
@@ -836,6 +879,9 @@ class Dikte:
|
||||
self.meeting_recorder.stop()
|
||||
self.overlay.dismiss()
|
||||
self.ask_overlay.dismiss()
|
||||
# Also on the restart path, which replaces the process without ever
|
||||
# reaching atexit and would otherwise leave the models in memory.
|
||||
ggml.stop_all()
|
||||
self.tray.hide()
|
||||
|
||||
|
||||
@@ -861,6 +907,41 @@ def main():
|
||||
return run_app([arg for arg in argv if arg != "--gui"])
|
||||
|
||||
|
||||
def install_signal_handlers(app):
|
||||
"""Quit properly on the signals a session sends, rather than dying where we stand.
|
||||
|
||||
Qt spends its time blocked inside C, and a Python signal handler only runs
|
||||
between bytecodes, so on its own it would not run until the next event
|
||||
arrived, which for an idle tray icon may be never. set_wakeup_fd writes the
|
||||
signal number to a socket instead, and a notifier turns that into an event
|
||||
Qt does deliver.
|
||||
|
||||
Worth the trouble because of what shutdown() does: a logout sends SIGTERM,
|
||||
and without this a whisper.cpp or llama.cpp server outlives the session
|
||||
holding its model in memory. SIGKILL cannot be caught at all, which is what
|
||||
ggml.sweep() is for.
|
||||
|
||||
Returns the objects it made; they have to stay alive to keep working.
|
||||
"""
|
||||
reader, writer = socket.socketpair()
|
||||
reader.setblocking(False)
|
||||
writer.setblocking(False)
|
||||
signal.set_wakeup_fd(writer.fileno())
|
||||
notifier = QSocketNotifier(reader.fileno(), QSocketNotifier.Type.Read)
|
||||
|
||||
def woken():
|
||||
with contextlib.suppress(OSError):
|
||||
reader.recv(64)
|
||||
app.quit() # aboutToQuit runs shutdown()
|
||||
|
||||
notifier.activated.connect(woken)
|
||||
for sig in (signal.SIGINT, signal.SIGTERM, signal.SIGHUP):
|
||||
# A handler that does nothing, so that the default action, stopping the
|
||||
# process where it stands, is replaced by the wakeup above.
|
||||
signal.signal(sig, lambda *_: None)
|
||||
return reader, writer, notifier
|
||||
|
||||
|
||||
def run_app(args):
|
||||
command = args[0] if args else ""
|
||||
|
||||
@@ -868,6 +949,12 @@ def run_app(args):
|
||||
app.setApplicationName("Dikte")
|
||||
app.setDesktopFileName("dikte")
|
||||
app.setQuitOnLastWindowClosed(False)
|
||||
# Before Dikte is built, because building it is what may start a server, and
|
||||
# a signal arriving in the middle of that would otherwise take the default
|
||||
# action and leave the server behind. A signal this early lands in the
|
||||
# socket and is delivered as soon as the event loop starts. Held in a name
|
||||
# so that the notifier and its socket outlive this function.
|
||||
signal_plumbing = install_signal_handlers(app) # noqa: F841
|
||||
|
||||
if not QSystemTrayIcon.isSystemTrayAvailable():
|
||||
print("dikte: no system tray found, running anyway")
|
||||
@@ -916,7 +1003,10 @@ def run_app(args):
|
||||
|
||||
# No key for the chosen transcription provider means nothing can work yet,
|
||||
# so the settings window is the only useful thing to open.
|
||||
if command == "settings" or not dikte.conf.transcribe_target().api_key:
|
||||
# A transcription provider that cannot run yet, whether that is a missing
|
||||
# API key or a model nobody has downloaded, means nothing can work, so the
|
||||
# settings window is the only useful thing to open.
|
||||
if command == "settings" or not dikte.conf.transcribe_ready():
|
||||
dikte.open_settings()
|
||||
elif command == "toggle":
|
||||
QTimer.singleShot(0, dikte.toggle)
|
||||
|
||||
@@ -0,0 +1,792 @@
|
||||
"""Speech to text and cleanup on this machine: whisper.cpp and llama.cpp.
|
||||
|
||||
Two programs, one treatment. Fetch a release from GitHub, unpack it under the
|
||||
data directory, fetch a model from Hugging Face, then keep one server alive on a
|
||||
port of its own. Both of them speak the shape api.py already sends to the hosted
|
||||
providers, so what the rest of Dikte sees is a base URL and nothing else:
|
||||
whisper-server is started on `--inference-path /v1/audio/transcriptions`, the
|
||||
exact path api.py builds, and llama-server answers /v1/chat/completions the way
|
||||
OpenRouter does.
|
||||
|
||||
A server rather than a one-shot run, because the model is the slow part. Loading
|
||||
a large whisper model takes a second or two while transcribing a few seconds of
|
||||
speech takes a fraction of one, and an LLM is worse: a server pays that once and
|
||||
a run per dictation pays it every time.
|
||||
|
||||
Nothing downloaded is trusted for having arrived. Every file is checked against
|
||||
the sha256 its index published, and the bytes go to a `.part` that is only
|
||||
renamed once the whole thing is there, so an interrupted download can never be
|
||||
mistaken for a working one.
|
||||
|
||||
This module imports hub and the string table, and nothing else of Dikte's: it
|
||||
knows how to fetch a file and how to run a process, and nothing about dictation.
|
||||
Its errors leave as LocalError and api.py turns them into the ApiError the
|
||||
interface already knows how to show.
|
||||
"""
|
||||
|
||||
import atexit
|
||||
import collections
|
||||
import ctypes.util
|
||||
import hashlib
|
||||
import http.client
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import platform
|
||||
import shutil
|
||||
import signal
|
||||
import socket
|
||||
import subprocess
|
||||
import tarfile
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
import hub
|
||||
from i18n import t
|
||||
|
||||
HOST = "127.0.0.1"
|
||||
# The path api.py asks for, so its URL and the server's line up.
|
||||
INFERENCE_PATH = "/v1/audio/transcriptions"
|
||||
|
||||
DATA_DIR = (pathlib.Path(os.environ.get("XDG_DATA_HOME")
|
||||
or os.path.expanduser("~/.local/share")) / "dikte")
|
||||
BIN_DIR = DATA_DIR / "bin"
|
||||
MODELS_DIR = DATA_DIR / "models"
|
||||
|
||||
# Loading a large model onto a GPU is the slow part of a start, and on a cold
|
||||
# page cache a large LLM read from a spinning disk is slower still.
|
||||
STARTUP_TIMEOUT = 180.0
|
||||
DOWNLOAD_CHUNK = 1 << 20
|
||||
|
||||
# `health` is the path that answers only once the model is in memory. whisper
|
||||
# does not have one and does not need one: it binds its port after the model is
|
||||
# loaded, so the port opening is the signal.
|
||||
Program = collections.namedtuple("Program", "name repo binary health")
|
||||
|
||||
WHISPER = Program("whisper", "ggml-org/whisper.cpp", "whisper-server", "")
|
||||
LLAMA = Program("llama", "ggml-org/llama.cpp", "llama-server", "/health")
|
||||
|
||||
# Where the models are listed. Neither list is written into Dikte: a catalogue
|
||||
# in the source means a release of Dikte for every model somebody else
|
||||
# publishes.
|
||||
WHISPER_MODELS_REPO = "ggerganov/whisper.cpp"
|
||||
LLM_AUTHOR = "ggml-org"
|
||||
|
||||
# What the whisper repository holds besides models: Core ML encoders for Apple
|
||||
# hardware and the odd loose file.
|
||||
WHISPER_PREFIX = "ggml-"
|
||||
WHISPER_SUFFIX = ".bin"
|
||||
|
||||
# What a GGUF repository holds besides the model: mmproj is the vision half of a
|
||||
# multimodal model, mtp a draft head for speculative decoding. Neither is a model
|
||||
# a server can be started on, and offering them is offering a failure.
|
||||
GGUF_SKIP = ("mmproj", "mtp-")
|
||||
# Big enough for a 12B at Q4 and far past anything cleanup wants; the point is
|
||||
# to keep a 400 GB frontier model out of a list somebody might click.
|
||||
GGUF_MAX_BYTES = 16 << 30
|
||||
|
||||
# Suggestions, not a catalogue: the list itself is fetched, and these are only
|
||||
# the rows that float to the top of it. Small instruction-following models,
|
||||
# because cleanup is punctuation and filler words rather than anything that
|
||||
# wants thinking about.
|
||||
SUGGESTED_LLM = (
|
||||
"ggml-org/gemma-3-4b-it-GGUF",
|
||||
"ggml-org/gemma-4-E2B-it-GGUF",
|
||||
"ggml-org/gemma-4-E4B-it-GGUF",
|
||||
"ggml-org/SmolLM3-3B-GGUF",
|
||||
)
|
||||
# Turbo at q5_0 is smaller than `small` and better than it, which makes the
|
||||
# usual "start small" advice point at the same file as "start good".
|
||||
SUGGESTED_WHISPER = "ggml-large-v3-turbo-q5_0.bin"
|
||||
|
||||
|
||||
class LocalError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def human_size(count):
|
||||
for unit in ("B", "KB", "MB", "GB"):
|
||||
if count < 1024 or unit == "GB":
|
||||
return f"{count:.0f} {unit}" if unit == "B" else f"{count:.1f} {unit}"
|
||||
count /= 1024.0
|
||||
return f"{count:.1f} GB"
|
||||
|
||||
|
||||
# --- fetching -------------------------------------------------------------
|
||||
|
||||
|
||||
def download(item, target, on_progress=None, should_stop=None, require_hash=True):
|
||||
"""Fetch one hub.Item to `target`. True when it landed, False when stopped.
|
||||
|
||||
The bytes go to a `.part` that is renamed only after both the length and the
|
||||
hash agree with what the index said. A truncated file would otherwise sit
|
||||
there looking installed and fail much later, inside a server, as a corrupt
|
||||
model; a file that is the right length but the wrong content is worse, and
|
||||
this is a program as often as it is a model.
|
||||
|
||||
A file whose index published no hash is refused rather than taken on trust.
|
||||
Everything fetched here is either run or parsed by something written in C++,
|
||||
and GitHub did not always publish a digest: a release old enough to predate
|
||||
that would otherwise install unchecked, which is the one case where this
|
||||
would matter most and say least.
|
||||
"""
|
||||
target = pathlib.Path(target)
|
||||
if require_hash and not item.sha256:
|
||||
raise LocalError(t("{name} is published without a checksum, so there is "
|
||||
"no way to tell what arrived. Nothing was installed.",
|
||||
name=item.name))
|
||||
part = target.with_name(target.name + ".part")
|
||||
try:
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
except OSError as exc:
|
||||
raise LocalError(t("Could not create {path}: {error}",
|
||||
path=target.parent, error=exc)) from exc
|
||||
|
||||
request = urllib.request.Request(item.url, headers={"User-Agent": hub.USER_AGENT})
|
||||
digest = hashlib.sha256()
|
||||
done = 0
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=60) as response:
|
||||
total = int(response.headers.get("Content-Length") or item.size or 0)
|
||||
with open(part, "wb") as out:
|
||||
while True:
|
||||
if should_stop is not None and should_stop():
|
||||
part.unlink(missing_ok=True)
|
||||
return False
|
||||
block = response.read(DOWNLOAD_CHUNK)
|
||||
if not block:
|
||||
break
|
||||
out.write(block)
|
||||
digest.update(block)
|
||||
done += len(block)
|
||||
# More than was announced: a body that does not end is the
|
||||
# one way this loop could run until the disk is full.
|
||||
if total and done > total:
|
||||
part.unlink(missing_ok=True)
|
||||
raise LocalError(t("{name} is longer than it said it "
|
||||
"would be.", name=item.name))
|
||||
if on_progress is not None:
|
||||
on_progress(done, total)
|
||||
# A proxy notice or an error page that came back as 200 would otherwise
|
||||
# be renamed into place and only fail when something tries to read it.
|
||||
if total and done != total:
|
||||
part.unlink(missing_ok=True)
|
||||
raise LocalError(t("The download stopped early ({done} of {total}).",
|
||||
done=human_size(done), total=human_size(total)))
|
||||
if item.sha256 and digest.hexdigest() != item.sha256:
|
||||
part.unlink(missing_ok=True)
|
||||
raise LocalError(t("{name} does not match its published checksum. "
|
||||
"Nothing was installed.", name=item.name))
|
||||
part.replace(target)
|
||||
return True
|
||||
except urllib.error.HTTPError as exc:
|
||||
part.unlink(missing_ok=True)
|
||||
exc.close() # it holds the response body open until it is collected
|
||||
raise LocalError(t("Could not download {name}: HTTP {code}",
|
||||
name=item.name, code=exc.code)) from exc
|
||||
except urllib.error.URLError as exc:
|
||||
part.unlink(missing_ok=True)
|
||||
raise LocalError(t("Could not download {name}: {error}",
|
||||
name=item.name, error=exc.reason)) from exc
|
||||
except OSError as exc:
|
||||
# A connection cut mid-body arrives here too, and gigabytes in is
|
||||
# exactly where that happens.
|
||||
part.unlink(missing_ok=True)
|
||||
raise LocalError(t("Could not write {name}: {error}",
|
||||
name=item.name, error=exc)) from exc
|
||||
|
||||
|
||||
# --- the programs ---------------------------------------------------------
|
||||
|
||||
|
||||
def _arch():
|
||||
machine = platform.machine().lower()
|
||||
if machine in ("aarch64", "arm64"):
|
||||
return "arm64"
|
||||
return "x64"
|
||||
|
||||
|
||||
def _has_vulkan():
|
||||
"""Whether a Vulkan loader is installed, which decides which build to fetch.
|
||||
|
||||
llama.cpp publishes no CUDA build for Linux, so Vulkan is what a graphics
|
||||
card gets here. The build without it is smaller and runs on the CPU, and
|
||||
fetching the Vulkan one for a machine that cannot load it would only make
|
||||
the download bigger.
|
||||
"""
|
||||
return bool(ctypes.util.find_library("vulkan"))
|
||||
|
||||
|
||||
def _wanted_assets(program):
|
||||
"""Asset name endings to accept, best first."""
|
||||
arch = _arch()
|
||||
if program is LLAMA and _has_vulkan():
|
||||
return (f"bin-ubuntu-vulkan-{arch}.tar.gz", f"bin-ubuntu-{arch}.tar.gz")
|
||||
return (f"bin-ubuntu-{arch}.tar.gz",)
|
||||
|
||||
|
||||
def _install_record(program):
|
||||
return BIN_DIR / program.name / "installed.json"
|
||||
|
||||
|
||||
def installed_program(program):
|
||||
"""The binary Dikte downloaded, or "" when there is none that still runs."""
|
||||
try:
|
||||
record = json.loads(_install_record(program).read_text(encoding="utf-8"))
|
||||
path = record.get("binary") or ""
|
||||
except (OSError, ValueError):
|
||||
return ""
|
||||
return path if os.path.isfile(path) and os.access(path, os.X_OK) else ""
|
||||
|
||||
|
||||
def installed_version(program):
|
||||
try:
|
||||
record = json.loads(_install_record(program).read_text(encoding="utf-8"))
|
||||
return record.get("tag") or ""
|
||||
except (OSError, ValueError):
|
||||
return ""
|
||||
|
||||
|
||||
def program_path(program, custom=""):
|
||||
"""Which copy of the program to run, or "" when there is none.
|
||||
|
||||
A system one wins over a downloaded one. The distribution package is built
|
||||
against whatever the machine has, which on this platform means it may reach
|
||||
the graphics card, while the release binaries carry CPU backends only.
|
||||
"""
|
||||
custom = (custom or "").strip()
|
||||
if custom:
|
||||
return custom if os.path.isfile(custom) and os.access(custom, os.X_OK) else ""
|
||||
return shutil.which(program.binary) or installed_program(program)
|
||||
|
||||
|
||||
def system_program(program):
|
||||
"""Whether the program came from the system rather than from Dikte."""
|
||||
return bool(shutil.which(program.binary))
|
||||
|
||||
|
||||
def _find_binary(root, name):
|
||||
for path in sorted(pathlib.Path(root).rglob(name)):
|
||||
if path.is_file():
|
||||
return path
|
||||
return None
|
||||
|
||||
|
||||
def _extract(archive, into):
|
||||
"""Unpack a release tarball, refusing anything that reaches outside `into`.
|
||||
|
||||
The archives lay their libraries next to their binaries and are linked with
|
||||
an $ORIGIN runpath, so a whole directory is what has to survive the trip and
|
||||
the binary cannot be lifted out of it.
|
||||
"""
|
||||
try:
|
||||
with tarfile.open(archive, "r:gz") as tar:
|
||||
try:
|
||||
tar.extractall(into, filter="data")
|
||||
except TypeError: # Python without the extraction filters
|
||||
tar.extractall(into)
|
||||
except (tarfile.TarError, OSError) as exc:
|
||||
raise LocalError(t("Could not unpack {name}: {error}",
|
||||
name=os.path.basename(str(archive)), error=exc)) from exc
|
||||
|
||||
|
||||
def install_program(program, tag="", on_progress=None, should_stop=None,
|
||||
refresh=False):
|
||||
"""Fetch and unpack a release. The path to the binary, or "" when stopped.
|
||||
|
||||
`tag` is empty for whatever the project released last, which is the point:
|
||||
a version pinned in Dikte's source would mean a release of Dikte every time
|
||||
whisper.cpp has one.
|
||||
"""
|
||||
try:
|
||||
tag, assets = hub.release(program.repo, tag or "latest", refresh=refresh)
|
||||
except hub.HubError as exc:
|
||||
raise LocalError(str(exc)) from exc
|
||||
|
||||
item = None
|
||||
for ending in _wanted_assets(program):
|
||||
item = next((a for a in assets if a.name.endswith(ending)), None)
|
||||
if item:
|
||||
break
|
||||
if item is None:
|
||||
raise LocalError(t("{repo} {tag} has no build for this machine.",
|
||||
repo=program.repo, tag=tag))
|
||||
|
||||
into = BIN_DIR / program.name / tag
|
||||
shutil.rmtree(into, ignore_errors=True)
|
||||
archive = BIN_DIR / program.name / item.name
|
||||
try:
|
||||
if not download(item, archive, on_progress, should_stop):
|
||||
return ""
|
||||
_extract(archive, into)
|
||||
binary = _find_binary(into, program.binary)
|
||||
if binary is None:
|
||||
raise LocalError(t("{name} was not in the download.",
|
||||
name=program.binary))
|
||||
binary.chmod(binary.stat().st_mode | 0o111)
|
||||
_install_record(program).write_text(
|
||||
json.dumps({"tag": tag, "binary": str(binary)}), encoding="utf-8")
|
||||
except OSError as exc:
|
||||
raise LocalError(t("Could not install {name}: {error}",
|
||||
name=program.name, error=exc)) from exc
|
||||
finally:
|
||||
try:
|
||||
archive.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
_drop_old_versions(program, keep=tag)
|
||||
return str(binary)
|
||||
|
||||
|
||||
def _drop_old_versions(program, keep):
|
||||
"""Leave one unpacked release behind, not one per update."""
|
||||
root = BIN_DIR / program.name
|
||||
try:
|
||||
for path in root.iterdir():
|
||||
if path.is_dir() and path.name != keep:
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
# --- the models -----------------------------------------------------------
|
||||
|
||||
|
||||
def whisper_models(refresh=False):
|
||||
"""[hub.Item] for every whisper model on offer, smallest first."""
|
||||
try:
|
||||
files = hub.files(WHISPER_MODELS_REPO, refresh=refresh)
|
||||
except hub.HubError as exc:
|
||||
raise LocalError(str(exc)) from exc
|
||||
models = [f for f in files
|
||||
if f.name.startswith(WHISPER_PREFIX) and f.name.endswith(WHISPER_SUFFIX)
|
||||
and f.size > 0]
|
||||
return sorted(models, key=lambda f: f.size)
|
||||
|
||||
|
||||
def llm_repos(refresh=False):
|
||||
"""Repository ids for the GGUF models on offer, suggestions first."""
|
||||
try:
|
||||
found = [r.id for r in hub.repos(author=LLM_AUTHOR, refresh=refresh)]
|
||||
except hub.HubError:
|
||||
# A menu rather than a catalogue: with nothing to show, the suggestions
|
||||
# are still worth showing, and whatever is wrong with the network will
|
||||
# say so where it matters, when a download is asked for.
|
||||
found = []
|
||||
if not found:
|
||||
return list(SUGGESTED_LLM)
|
||||
first = [r for r in SUGGESTED_LLM if r in found]
|
||||
return first + [r for r in found if r not in first]
|
||||
|
||||
|
||||
def llm_quants(repo, refresh=False):
|
||||
"""[hub.Item] for the model files in one GGUF repository, smallest first."""
|
||||
try:
|
||||
files = hub.files(repo, refresh=refresh)
|
||||
except hub.HubError as exc:
|
||||
raise LocalError(str(exc)) from exc
|
||||
out = []
|
||||
for item in files:
|
||||
name = item.name.rsplit("/", 1)[-1]
|
||||
if not name.endswith(".gguf") or name.startswith(GGUF_SKIP):
|
||||
continue
|
||||
# A model split across files needs all of them and a different command
|
||||
# line; anything cleanup wants fits in one.
|
||||
if "-of-000" in name or not 0 < item.size <= GGUF_MAX_BYTES:
|
||||
continue
|
||||
out.append(item)
|
||||
return sorted(out, key=lambda f: f.size)
|
||||
|
||||
|
||||
def whisper_model_path(name):
|
||||
return MODELS_DIR / "whisper" / name
|
||||
|
||||
|
||||
def llm_model_path(name):
|
||||
return MODELS_DIR / "llm" / name.rsplit("/", 1)[-1]
|
||||
|
||||
|
||||
def have_model(path):
|
||||
path = pathlib.Path(path)
|
||||
return path.is_file() and path.stat().st_size > 0
|
||||
|
||||
|
||||
def installed_whisper_models():
|
||||
return sorted(p.name for p in (MODELS_DIR / "whisper").glob("*.bin"))
|
||||
|
||||
|
||||
def installed_llm_models():
|
||||
return sorted(p.name for p in (MODELS_DIR / "llm").glob("*.gguf"))
|
||||
|
||||
|
||||
def delete_model(path):
|
||||
try:
|
||||
pathlib.Path(path).unlink()
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
except OSError as exc:
|
||||
raise LocalError(t("Could not delete the model: {error}", error=exc)) from exc
|
||||
|
||||
|
||||
# --- one server -----------------------------------------------------------
|
||||
|
||||
|
||||
def _free_port():
|
||||
"""A port nothing is listening on, handed straight to the server.
|
||||
|
||||
Between closing this socket and the server binding it, something else could
|
||||
take it; that is why a start retries rather than trusting the number.
|
||||
"""
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
sock.bind((HOST, 0))
|
||||
return sock.getsockname()[1]
|
||||
|
||||
|
||||
def _listening(port):
|
||||
try:
|
||||
with socket.create_connection((HOST, port), timeout=0.5):
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def _healthy(port, path):
|
||||
"""Whether the model is in memory, for a server that says so.
|
||||
|
||||
Spoken over http.client rather than urllib because this never leaves the
|
||||
machine: it is the same question as _listening, one layer up.
|
||||
"""
|
||||
connection = http.client.HTTPConnection(HOST, port, timeout=2)
|
||||
try:
|
||||
connection.request("GET", path)
|
||||
# 503 for as long as the model is still being read in.
|
||||
return connection.getresponse().status == 200
|
||||
except (http.client.HTTPException, OSError):
|
||||
return False
|
||||
finally:
|
||||
connection.close()
|
||||
|
||||
|
||||
def _tail(path, lines=3):
|
||||
try:
|
||||
with open(path, encoding="utf-8", errors="replace") as fh:
|
||||
found = [line.strip() for line in fh if line.strip()]
|
||||
except OSError:
|
||||
return ""
|
||||
return " | ".join(found[-lines:])
|
||||
|
||||
|
||||
class Server:
|
||||
"""One process, started when something needs it and stopped when nothing does.
|
||||
|
||||
`build` turns the settings into a command line; everything else about
|
||||
running a server is the same for both programs.
|
||||
"""
|
||||
|
||||
def __init__(self, program, build, defaults):
|
||||
self.program = program
|
||||
self._build = build
|
||||
self._settings = dict(defaults)
|
||||
# Two locks on purpose. `_lock` is held for the length of a dictionary
|
||||
# lookup, so the interface can ask what is running while a model is
|
||||
# being loaded; `_starting` is held across the start itself, which can
|
||||
# take a minute and which two threads must not both do.
|
||||
self._lock = threading.Lock()
|
||||
self._starting = threading.Lock()
|
||||
self._proc = None
|
||||
self._port = 0
|
||||
self._log = ""
|
||||
self._key = None
|
||||
|
||||
# ---- settings --------------------------------------------------------
|
||||
|
||||
def configure(self, **changes):
|
||||
"""Apply settings. A server started on the old ones is stopped."""
|
||||
with self._lock:
|
||||
for key, value in changes.items():
|
||||
if value is not None and key in self._settings:
|
||||
self._settings[key] = value
|
||||
stale = self._proc is not None and self._key != self._settings_key()
|
||||
if stale:
|
||||
self.stop()
|
||||
|
||||
def settings(self):
|
||||
with self._lock:
|
||||
return dict(self._settings)
|
||||
|
||||
def _settings_key(self):
|
||||
"""What a running server would have to be restarted for."""
|
||||
return json.dumps(self._settings, sort_keys=True, default=str)
|
||||
|
||||
# ---- process ---------------------------------------------------------
|
||||
|
||||
@property
|
||||
def running(self):
|
||||
with self._lock:
|
||||
return self._proc is not None and self._proc.poll() is None
|
||||
|
||||
def base_url(self):
|
||||
with self._lock:
|
||||
return f"http://{HOST}:{self._port}/v1" if self._port else ""
|
||||
|
||||
def error(self):
|
||||
"""The last thing the server printed, for a failure after it started."""
|
||||
with self._lock:
|
||||
log = self._log
|
||||
return _tail(log) if log else ""
|
||||
|
||||
def serve(self):
|
||||
"""The base URL of a server that is up and running the current settings."""
|
||||
ready = self._current_url()
|
||||
if ready:
|
||||
return ready
|
||||
with self._starting:
|
||||
# Somebody may have started it while this thread waited its turn.
|
||||
ready = self._current_url()
|
||||
if ready:
|
||||
return ready
|
||||
self.stop()
|
||||
with self._lock:
|
||||
settings, key = dict(self._settings), self._settings_key()
|
||||
proc, port, log = self._launch(settings)
|
||||
with self._lock:
|
||||
self._proc, self._port, self._log, self._key = proc, port, log, key
|
||||
return self.base_url()
|
||||
|
||||
def _current_url(self):
|
||||
with self._lock:
|
||||
up = self._proc is not None and self._proc.poll() is None
|
||||
return (f"http://{HOST}:{self._port}/v1"
|
||||
if up and self._key == self._settings_key() else "")
|
||||
|
||||
def _launch(self, settings):
|
||||
args = self._build(settings) # raises LocalError when unusable
|
||||
last = ""
|
||||
for _ in range(3):
|
||||
port = _free_port()
|
||||
log = DATA_DIR / f"{self.program.name}-server.log"
|
||||
try:
|
||||
log.parent.mkdir(parents=True, exist_ok=True)
|
||||
sink = open(log, "wb")
|
||||
except OSError as exc:
|
||||
raise LocalError(t("Could not start {name}: {error}",
|
||||
name=self.program.name, error=exc)) from exc
|
||||
try:
|
||||
with sink:
|
||||
proc = subprocess.Popen(
|
||||
args + ["--host", HOST, "--port", str(port)],
|
||||
stdout=sink, stderr=subprocess.STDOUT,
|
||||
stdin=subprocess.DEVNULL,
|
||||
)
|
||||
except OSError as exc:
|
||||
raise LocalError(t("Could not start {name}: {error}",
|
||||
name=self.program.name, error=exc)) from exc
|
||||
|
||||
# Written before it is ready rather than after, so that a kill
|
||||
# during the model load leaves something for the sweep to find.
|
||||
self._remember(proc.pid)
|
||||
try:
|
||||
ready = self._wait_ready(proc, port)
|
||||
except BaseException:
|
||||
# Whatever went wrong while waiting, the process is ours and
|
||||
# nothing else is left holding a reference to it. Leaving it
|
||||
# running would leak a loaded model with nobody to ask it
|
||||
# anything, which is the whole failure this class is careful
|
||||
# about elsewhere.
|
||||
self._kill(proc)
|
||||
self._forget()
|
||||
raise
|
||||
if ready:
|
||||
return proc, port, str(log)
|
||||
last = _tail(log)
|
||||
self._forget()
|
||||
# A port taken between the probe and the bind is the one failure
|
||||
# worth another go; anything else will fail the same way again.
|
||||
if "address" not in last.lower() and "bind" not in last.lower():
|
||||
break
|
||||
raise LocalError(t("{name} did not start: {error}",
|
||||
name=self.program.binary, error=last or t("no output")))
|
||||
|
||||
def _wait_ready(self, proc, port):
|
||||
deadline = time.monotonic() + STARTUP_TIMEOUT
|
||||
while time.monotonic() < deadline:
|
||||
if proc.poll() is not None:
|
||||
return False
|
||||
if _listening(port):
|
||||
# whisper binds after the model is loaded, so the open port is
|
||||
# the answer. llama binds first and answers /health with 503
|
||||
# until it is ready.
|
||||
if not self.program.health or _healthy(port, self.program.health):
|
||||
return True
|
||||
time.sleep(0.1)
|
||||
self._kill(proc)
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def _kill(proc, gently=False):
|
||||
"""Stop a process of ours, and wait for it rather than assume."""
|
||||
if proc is None or proc.poll() is not None:
|
||||
return
|
||||
if gently:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=5)
|
||||
return
|
||||
except subprocess.TimeoutExpired:
|
||||
pass
|
||||
proc.kill()
|
||||
try:
|
||||
proc.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
pass
|
||||
|
||||
def stop(self):
|
||||
with self._lock:
|
||||
proc, self._proc = self._proc, None
|
||||
self._port, self._log, self._key = 0, "", None
|
||||
self._kill(proc, gently=True)
|
||||
if proc is not None:
|
||||
self._forget()
|
||||
|
||||
# ---- servers a killed Dikte left behind -------------------------------
|
||||
|
||||
def _pid_file(self):
|
||||
return DATA_DIR / f"{self.program.name}-server.pid"
|
||||
|
||||
def _remember(self, pid):
|
||||
try:
|
||||
path = self._pid_file()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(str(pid))
|
||||
except OSError:
|
||||
pass # the sweep is a safety net, not something to fail a run over
|
||||
|
||||
def _forget(self):
|
||||
try:
|
||||
self._pid_file().unlink()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def _is_ours(self, pid):
|
||||
"""Whether that pid is still the server this Dikte started.
|
||||
|
||||
Asked because pids are handed out again: by the time anyone looks, the
|
||||
number could belong to something else entirely, and killing it would be
|
||||
a good deal worse than the leak being cleaned up. The program name alone
|
||||
could be somebody else's copy; the name together with Dikte's own data
|
||||
directory on the command line could not.
|
||||
"""
|
||||
try:
|
||||
blob = pathlib.Path(f"/proc/{pid}/cmdline").read_bytes()
|
||||
except OSError:
|
||||
return False
|
||||
return (self.program.binary.encode() in blob
|
||||
and str(DATA_DIR).encode() in blob)
|
||||
|
||||
def sweep(self):
|
||||
"""Kill a server a previous Dikte left behind. True when one was found.
|
||||
|
||||
stop() and atexit cover every exit that gets to run code. A SIGKILL does
|
||||
not, and neither does a session torn down from under it, and the server
|
||||
would then sit there holding the model with nothing left alive to ask it
|
||||
anything.
|
||||
"""
|
||||
try:
|
||||
pid = int(self._pid_file().read_text().strip())
|
||||
except (OSError, ValueError):
|
||||
return False
|
||||
self._forget()
|
||||
if not self._is_ours(pid):
|
||||
return False
|
||||
try:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
except OSError:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
# --- the two of them ------------------------------------------------------
|
||||
|
||||
|
||||
def _whisper_args(settings):
|
||||
binary = program_path(WHISPER, settings["binary"])
|
||||
if not binary:
|
||||
raise LocalError(t("whisper.cpp is not installed. Settings → API and "
|
||||
"models → Download."))
|
||||
model = whisper_model_path(settings["model"])
|
||||
if not settings["model"] or not have_model(model):
|
||||
raise LocalError(t("No whisper model has been downloaded yet. "
|
||||
"Settings → API and models → Download."))
|
||||
args = [
|
||||
binary, "-m", str(model),
|
||||
"--inference-path", INFERENCE_PATH,
|
||||
# Whatever language the request does not name. api.py leaves the field
|
||||
# out when the language is "auto", and the server's own default is
|
||||
# English rather than detection.
|
||||
"-l", "auto",
|
||||
# Stock phrases invented for near-silence come from non-speech tokens,
|
||||
# and verbose_json otherwise pays for a language probability sweep
|
||||
# nothing here reads.
|
||||
"-sns", "-nlp",
|
||||
]
|
||||
if int(settings["threads"]) > 0:
|
||||
args += ["-t", str(int(settings["threads"]))]
|
||||
if not settings["gpu"]:
|
||||
args.append("-ng")
|
||||
return args
|
||||
|
||||
|
||||
def _llm_args(settings):
|
||||
binary = program_path(LLAMA, settings["binary"])
|
||||
if not binary:
|
||||
raise LocalError(t("llama.cpp is not installed. Settings → API and "
|
||||
"models → Download."))
|
||||
model = llm_model_path(settings["model"])
|
||||
if not settings["model"] or not have_model(model):
|
||||
raise LocalError(t("No local cleanup model has been downloaded yet. "
|
||||
"Settings → API and models → Download."))
|
||||
args = [binary, "-m", str(model), "-c", str(int(settings["context"]))]
|
||||
# All of them, or as many as fit: llama.cpp stops offloading when the card
|
||||
# is full rather than failing, and a build with no GPU backend ignores it.
|
||||
args += ["-ngl", "99" if settings["gpu"] else "0"]
|
||||
if int(settings["threads"]) > 0:
|
||||
args += ["-t", str(int(settings["threads"]))]
|
||||
return args
|
||||
|
||||
|
||||
whisper = Server(WHISPER, _whisper_args, {
|
||||
"model": "",
|
||||
"threads": 0,
|
||||
"gpu": True,
|
||||
"binary": "",
|
||||
})
|
||||
|
||||
llm = Server(LLAMA, _llm_args, {
|
||||
"model": "",
|
||||
"threads": 0,
|
||||
"gpu": True,
|
||||
"binary": "",
|
||||
# A dictation and its prompt are short. This is sized for the longest
|
||||
# cleanup block rather than for a conversation, and it is what the model
|
||||
# costs in memory beyond its own weights.
|
||||
"context": 8192,
|
||||
})
|
||||
|
||||
SERVERS = (whisper, llm)
|
||||
|
||||
|
||||
def sweep():
|
||||
"""Clean up after a Dikte that was killed outright. True when one was found."""
|
||||
return any([server.sweep() for server in SERVERS])
|
||||
|
||||
|
||||
def stop_all():
|
||||
for server in SERVERS:
|
||||
server.stop()
|
||||
|
||||
|
||||
# Dikte stops the servers itself on quit and on restart; this catches the paths
|
||||
# that skip that, such as an unhandled exception on the way out.
|
||||
atexit.register(stop_all)
|
||||
@@ -0,0 +1,186 @@
|
||||
"""Where the programs and the models come from: GitHub releases and Hugging Face.
|
||||
|
||||
Both answer plain JSON over HTTPS without a key, and both publish a sha256 for
|
||||
every file they hand out: GitHub as the asset digest, Hugging Face as the LFS
|
||||
object id. Nothing that lands on disk is trusted for having arrived, which
|
||||
matters more here than it usually would, because half of what is fetched is a
|
||||
program Dikte then runs.
|
||||
|
||||
The lists are read rather than kept. A model catalogue written into the source
|
||||
means a release of Dikte for every new model, and a pinned whisper.cpp version
|
||||
means one for every whisper.cpp release; both of those are somebody else's news,
|
||||
not Dikte's. Answers are cached for a few hours, and a cache that has gone stale
|
||||
is still a better answer than none when the network is down.
|
||||
|
||||
Nothing here imports the rest of Dikte apart from the string table: this module
|
||||
knows two websites and nothing about dictation.
|
||||
"""
|
||||
|
||||
import collections
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
from i18n import t
|
||||
|
||||
GITHUB_API = "https://api.github.com"
|
||||
HF_API = "https://huggingface.co/api"
|
||||
HF_FILES = "https://huggingface.co"
|
||||
USER_AGENT = "dikte/1.0 (+https://github.com/yusufipk/dikte)"
|
||||
|
||||
CACHE_DIR = (pathlib.Path(os.environ.get("XDG_CACHE_HOME")
|
||||
or os.path.expanduser("~/.cache")) / "dikte")
|
||||
# Long enough that opening the settings window twice in an evening asks nobody
|
||||
# anything, short enough that a model published this morning is offered today.
|
||||
CACHE_TTL = 6 * 3600
|
||||
|
||||
# `sha256` is empty for the few files neither side stores in LFS; those are the
|
||||
# small ones, and a checksum is only worth having where there is something to
|
||||
# check.
|
||||
Item = collections.namedtuple("Item", "name url size sha256")
|
||||
Repo = collections.namedtuple("Repo", "id downloads updated")
|
||||
|
||||
|
||||
class HubError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _get(url, timeout=20):
|
||||
request = urllib.request.Request(url, headers={
|
||||
"User-Agent": USER_AGENT,
|
||||
"Accept": "application/json",
|
||||
})
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||
return json.loads(response.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as exc:
|
||||
exc.close() # it holds the response body open until it is collected
|
||||
raise HubError(t("{url} answered HTTP {code}.",
|
||||
url=urllib.parse.urlsplit(url).netloc, code=exc.code)) from exc
|
||||
except urllib.error.URLError as exc:
|
||||
raise HubError(t("Could not reach {url}: {error}",
|
||||
url=urllib.parse.urlsplit(url).netloc,
|
||||
error=exc.reason)) from exc
|
||||
except (ValueError, OSError) as exc:
|
||||
raise HubError(t("Could not read the answer from {url}: {error}",
|
||||
url=urllib.parse.urlsplit(url).netloc, error=exc)) from exc
|
||||
|
||||
|
||||
def _cache_file(key):
|
||||
safe = "".join(c if c.isalnum() or c in "-._" else "-" for c in key)
|
||||
return CACHE_DIR / f"{safe}.json"
|
||||
|
||||
|
||||
def _read_cache(key, ttl):
|
||||
"""What was stored under this key, or None. `ttl` of 0 ignores the age."""
|
||||
path = _cache_file(key)
|
||||
try:
|
||||
age = time.time() - path.stat().st_mtime
|
||||
if ttl and age > ttl:
|
||||
return None
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _write_cache(key, payload):
|
||||
try:
|
||||
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
_cache_file(key).write_text(json.dumps(payload), encoding="utf-8")
|
||||
except OSError:
|
||||
pass # a cache that cannot be written is not a failed lookup
|
||||
|
||||
|
||||
def _fetch(key, url, ttl=CACHE_TTL, refresh=False):
|
||||
"""The JSON at `url`, from the cache when it is fresh enough.
|
||||
|
||||
A lookup that fails falls back to the cache however old it is: an offline
|
||||
settings window that shows yesterday's list is worth a great deal more than
|
||||
one that shows an error.
|
||||
"""
|
||||
if not refresh:
|
||||
cached = _read_cache(key, ttl)
|
||||
if cached is not None:
|
||||
return cached
|
||||
try:
|
||||
payload = _get(url)
|
||||
except HubError:
|
||||
stale = _read_cache(key, 0)
|
||||
if stale is not None:
|
||||
return stale
|
||||
raise
|
||||
_write_cache(key, payload)
|
||||
return payload
|
||||
|
||||
|
||||
def _digest(value):
|
||||
"""GitHub writes its digests as "sha256:…"; Hugging Face writes the hash."""
|
||||
value = (value or "").strip()
|
||||
return value.split(":", 1)[1] if value.startswith("sha256:") else value
|
||||
|
||||
|
||||
def release(repo, tag="latest", refresh=False):
|
||||
"""(tag, [Item]) for one GitHub release, newest when no tag is given."""
|
||||
where = "latest" if tag in ("", "latest") else f"tags/{tag}"
|
||||
data = _fetch(f"gh-{repo}-{tag or 'latest'}",
|
||||
f"{GITHUB_API}/repos/{repo}/releases/{where}", refresh=refresh)
|
||||
if not isinstance(data, dict) or not data.get("assets"):
|
||||
raise HubError(t("{repo} has no downloadable release.", repo=repo))
|
||||
assets = [Item(a.get("name") or "", a.get("browser_download_url") or "",
|
||||
int(a.get("size") or 0), _digest(a.get("digest")))
|
||||
for a in data["assets"] if a.get("browser_download_url")]
|
||||
return data.get("tag_name") or tag, assets
|
||||
|
||||
|
||||
def files(repo, revision="main", refresh=False):
|
||||
"""[Item] for every file in a Hugging Face repository.
|
||||
|
||||
The size is there whether or not the file is in LFS; the hash is only there
|
||||
when it is, which for anything worth downloading it always is.
|
||||
"""
|
||||
data = _fetch(f"hf-tree-{repo}-{revision}",
|
||||
f"{HF_API}/models/{repo}/tree/{revision}?recursive=true",
|
||||
refresh=refresh)
|
||||
if not isinstance(data, list):
|
||||
raise HubError(t("{repo} did not return a file list.", repo=repo))
|
||||
out = []
|
||||
for entry in data:
|
||||
if entry.get("type") != "file":
|
||||
continue
|
||||
path = entry.get("path") or ""
|
||||
lfs = entry.get("lfs") or {}
|
||||
out.append(Item(
|
||||
path,
|
||||
f"{HF_FILES}/{repo}/resolve/{revision}/{urllib.parse.quote(path)}",
|
||||
int(lfs.get("size") or entry.get("size") or 0),
|
||||
_digest(lfs.get("oid") or lfs.get("sha256")),
|
||||
))
|
||||
return out
|
||||
|
||||
|
||||
def repos(author="", search="", limit=40, refresh=False):
|
||||
"""[Repo] of GGUF repositories, newest first.
|
||||
|
||||
Filtered by author on purpose. Hugging Face's own trending list is open to
|
||||
everyone and reads like it: asking it for the popular GGUF today answers
|
||||
with a wall of roleplay merges, which is not what a dictation transcript
|
||||
wants cleaning up. An author is a small enough thing to trust and a large
|
||||
enough one to keep the list current without Dikte being updated.
|
||||
"""
|
||||
query = {"filter": "gguf", "sort": "lastModified", "direction": "-1",
|
||||
"limit": str(limit)}
|
||||
if author:
|
||||
query["author"] = author
|
||||
if search:
|
||||
query["search"] = search
|
||||
url = f"{HF_API}/models?{urllib.parse.urlencode(query)}"
|
||||
data = _fetch(f"hf-models-{author}-{search}-{limit}", url, refresh=refresh)
|
||||
if not isinstance(data, list):
|
||||
raise HubError(t("Hugging Face did not return a model list."))
|
||||
return [Repo(m.get("id") or "", int(m.get("downloads") or 0),
|
||||
m.get("lastModified") or "")
|
||||
for m in data if m.get("id")]
|
||||
+2
-1
@@ -132,5 +132,6 @@ fi
|
||||
|
||||
echo
|
||||
ok "Done. Start it with: dikte"
|
||||
say "The settings window opens on first run; add an OpenAI, Groq or OpenRouter key."
|
||||
say "The settings window opens on first run: download a speech model, or add"
|
||||
say "an OpenAI or OpenRouter key instead."
|
||||
echo
|
||||
|
||||
+450
-18
@@ -19,6 +19,7 @@ import audio
|
||||
import cleanup
|
||||
import config as cfg
|
||||
import filetranscribe
|
||||
import ggml
|
||||
import hotkey
|
||||
import ipc
|
||||
import meeting
|
||||
@@ -31,8 +32,10 @@ LANGUAGES = [
|
||||
("German", "de"), ("French", "fr"), ("Spanish", "es"), ("Arabic", "ar"),
|
||||
]
|
||||
CORNERS = ["bottom-left", "bottom-right", "top-left", "top-right"]
|
||||
# The provider box offers what config knows how to reach, in that order.
|
||||
TRANSCRIBE_PROVIDERS = [(who.service, name) for name, who in cfg.TRANSCRIBERS.items()]
|
||||
# The provider box offers what config knows how to reach, this machine first.
|
||||
TRANSCRIBE_PROVIDERS = ([("This machine (whisper.cpp)", "local")]
|
||||
+ [(who.service, name)
|
||||
for name, who in cfg.TRANSCRIBERS.items()])
|
||||
# Starting points for the model box; "Fetch model list" replaces them with
|
||||
# whatever the provider offers today.
|
||||
TRANSCRIBE_MODELS = {
|
||||
@@ -50,11 +53,12 @@ CLEANUP_MODELS = [
|
||||
"google/gemini-2.5-flash-lite", "anthropic/claude-haiku-4.5",
|
||||
"openai/gpt-5-mini", "meta-llama/llama-3.3-70b-instruct",
|
||||
]
|
||||
# The same two CLIs the agent can run on, doing the smaller job instead. They
|
||||
# are offered second: a request to OpenRouter is over in a second, and a CLI
|
||||
# opens a session first.
|
||||
# In the order they answer in. A request to OpenRouter is over in a second, a
|
||||
# model here takes a little longer and costs nothing, and the two CLIs the agent
|
||||
# can run on open a whole session to do the smaller job.
|
||||
CLEANUP_PROVIDERS = [
|
||||
("OpenRouter", "openrouter"), ("Claude Code", "claude"), ("Codex", "codex"),
|
||||
("OpenRouter", "openrouter"), ("This machine (llama.cpp)", "local"),
|
||||
("Claude Code", "claude"), ("Codex", "codex"),
|
||||
]
|
||||
# Cleaning up a sentence is the lightest thing either of them will ever be
|
||||
# asked, so the small model comes first.
|
||||
@@ -116,6 +120,338 @@ AUDIO_FILTER = ("*.mp3 *.wav *.m4a *.ogg *.opus *.flac *.aac *.wma "
|
||||
"*.mp4 *.mkv *.webm *.mov *.avi")
|
||||
|
||||
|
||||
class LocalModelBox(QGroupBox):
|
||||
"""The program, the model, and the two downloads that put them there.
|
||||
|
||||
One class for whisper.cpp and llama.cpp, because the job is the same one
|
||||
twice: say whether the program is here, offer the models somebody publishes,
|
||||
fetch the chosen one, and stay usable while a gigabyte arrives. Nothing is
|
||||
listed in the source; `repos` and `models` are asked at the moment the box is
|
||||
opened, so a model published this morning is in the list this afternoon.
|
||||
"""
|
||||
|
||||
_listed = pyqtSignal(list, str)
|
||||
_quants = pyqtSignal(list, str)
|
||||
# qint64 rather than int, which is C++'s 32-bit one: a 2.3 GB model is more
|
||||
# than fits in it, and the count comes out the far side negative.
|
||||
_progress = pyqtSignal("qint64", "qint64")
|
||||
_finished = pyqtSignal(str, str)
|
||||
_installed = pyqtSignal(str, str)
|
||||
|
||||
changed = pyqtSignal()
|
||||
|
||||
def __init__(self, program, title, models, model_path, repos=None, parent=None):
|
||||
super().__init__(title, parent)
|
||||
self.program = program
|
||||
self._models = models # () -> [hub.Item], or (repo) -> [hub.Item]
|
||||
self._model_path = model_path # (name) -> Path
|
||||
self._repos = repos # None, or () -> [repo id]
|
||||
self._downloading = False
|
||||
self._pending = False
|
||||
self._stop = False
|
||||
self._wanted = "" # the model to select once a list arrives
|
||||
|
||||
form = QFormLayout(self)
|
||||
|
||||
self.program_label = QLabel("")
|
||||
self.program_label.setWordWrap(True)
|
||||
self.install_button = QPushButton(t("Download"))
|
||||
self.install_button.clicked.connect(self._install_program)
|
||||
form.addRow(t("Program"), self._side_by_side(self.program_label,
|
||||
self.install_button))
|
||||
|
||||
if self._repos is not None:
|
||||
self.repo = QComboBox()
|
||||
self.repo.setEditable(True)
|
||||
self.repo.setToolTip(t("A Hugging Face repository of GGUF files. The "
|
||||
"list is fetched; any other one can be typed in."))
|
||||
self.repo.currentTextChanged.connect(self._repo_changed)
|
||||
form.addRow(t("Publisher"), self.repo)
|
||||
|
||||
self.model = QComboBox()
|
||||
self.download_button = QPushButton(t("Download"))
|
||||
self.download_button.clicked.connect(self._download)
|
||||
self.delete_button = QPushButton(t("Delete"))
|
||||
self.delete_button.clicked.connect(self._delete)
|
||||
form.addRow(t("Model"), self._side_by_side(self.model,
|
||||
self.download_button,
|
||||
self.delete_button))
|
||||
self.model.currentIndexChanged.connect(self._model_changed)
|
||||
|
||||
self.status = QLabel("")
|
||||
self.status.setWordWrap(True)
|
||||
form.addRow(self.status)
|
||||
|
||||
self._listed.connect(self._on_listed)
|
||||
self._quants.connect(self._on_listed)
|
||||
self._progress.connect(self._on_progress)
|
||||
self._finished.connect(self._on_finished)
|
||||
self._installed.connect(self._on_installed)
|
||||
|
||||
@staticmethod
|
||||
def _fit_popup(combo):
|
||||
"""Let the list that drops down be as wide as its longest row.
|
||||
|
||||
A combo box hands its own width to the list under it and elides
|
||||
whatever does not fit, which lands in the middle of the name:
|
||||
`ggml-org/Qwen....7B-Base-GGUF` is not a model anybody can choose
|
||||
between. The box itself stays the width the form gave it.
|
||||
"""
|
||||
view = combo.view()
|
||||
view.setTextElideMode(Qt.TextElideMode.ElideNone)
|
||||
metrics = combo.fontMetrics()
|
||||
widest = max((metrics.horizontalAdvance(combo.itemText(row))
|
||||
for row in range(combo.count())), default=0)
|
||||
# Room for the frame and for a scroll bar, which a long list will have.
|
||||
view.setMinimumWidth(widest + view.verticalScrollBar().sizeHint().width() + 24)
|
||||
|
||||
@staticmethod
|
||||
def _side_by_side(*widgets):
|
||||
layout = QHBoxLayout()
|
||||
layout.setContentsMargins(0, 0, 0, 0)
|
||||
for index, widget in enumerate(widgets):
|
||||
layout.addWidget(widget, 1 if index == 0 else 0)
|
||||
holder = QWidget()
|
||||
holder.setLayout(layout)
|
||||
return holder
|
||||
|
||||
# ---- what is here ----------------------------------------------------
|
||||
|
||||
def selected(self):
|
||||
return self.model.currentData() or ""
|
||||
|
||||
def repository(self):
|
||||
return self.repo.currentText().strip() if self._repos is not None else ""
|
||||
|
||||
def load(self, model, repo=""):
|
||||
"""Show what is stored. What else is on offer is asked for on the way up.
|
||||
|
||||
Nothing is fetched here: building the settings window is not the same as
|
||||
opening it, and a list nobody is looking at is not worth a request. What
|
||||
is already on this disk is shown straight away either way.
|
||||
"""
|
||||
self._wanted = model
|
||||
self._pending = True
|
||||
self._show_program()
|
||||
if self._repos is not None:
|
||||
self.repo.blockSignals(True)
|
||||
self.repo.clear()
|
||||
self.repo.addItems(list(ggml.SUGGESTED_LLM))
|
||||
self.repo.setCurrentText(repo or ggml.SUGGESTED_LLM[0])
|
||||
self.repo.blockSignals(False)
|
||||
self._fit_popup(self.repo)
|
||||
self._fill_models([])
|
||||
|
||||
def showEvent(self, event):
|
||||
super().showEvent(event)
|
||||
if self._pending:
|
||||
self._pending = False
|
||||
if self._repos is not None:
|
||||
self._fill_repos(self.repository())
|
||||
self._fetch_models(self.repository())
|
||||
|
||||
def _show_program(self):
|
||||
path = ggml.program_path(self.program)
|
||||
if not path:
|
||||
self.program_label.setText(t("Not installed."))
|
||||
self.install_button.setVisible(True)
|
||||
return
|
||||
self.install_button.setVisible(not ggml.installed_program(self.program)
|
||||
and not ggml.system_program(self.program))
|
||||
if ggml.system_program(self.program):
|
||||
# Worth saying which one is running: a distribution package is built
|
||||
# for this machine and may reach the graphics card, while the
|
||||
# released binaries carry processor backends only.
|
||||
self.program_label.setText(t("Installed on the system: {path}", path=path))
|
||||
else:
|
||||
self.program_label.setText(
|
||||
t("Downloaded, version {version}.",
|
||||
version=ggml.installed_version(self.program) or "?"))
|
||||
|
||||
# ---- the lists -------------------------------------------------------
|
||||
|
||||
def _fill_repos(self, current):
|
||||
def work():
|
||||
self._listed.emit([("repos", ggml.llm_repos())], "")
|
||||
|
||||
threading.Thread(target=work, daemon=True).start()
|
||||
|
||||
def _repo_changed(self):
|
||||
if not self._downloading:
|
||||
self._fetch_models(self.repository())
|
||||
|
||||
def _fetch_models(self, repo=""):
|
||||
self.status.setText(t("Fetching the model list…"))
|
||||
|
||||
def work():
|
||||
try:
|
||||
found = self._models(repo) if self._repos is not None else self._models()
|
||||
self._quants.emit([("models", found)], "")
|
||||
except ggml.LocalError as exc:
|
||||
self._quants.emit([], str(exc))
|
||||
|
||||
threading.Thread(target=work, daemon=True).start()
|
||||
|
||||
def _on_listed(self, payload, error):
|
||||
if error:
|
||||
self.status.setText(error)
|
||||
self._refresh_buttons()
|
||||
return
|
||||
kind, found = payload[0]
|
||||
if kind == "repos":
|
||||
current = self.repo.currentText()
|
||||
self.repo.blockSignals(True)
|
||||
self.repo.clear()
|
||||
self.repo.addItems(found)
|
||||
self.repo.setCurrentText(current)
|
||||
self.repo.blockSignals(False)
|
||||
self._fit_popup(self.repo)
|
||||
return
|
||||
self._fill_models(found)
|
||||
|
||||
def _fill_models(self, items):
|
||||
"""One row per model, saying what it weighs and whether it is here."""
|
||||
wanted = self._wanted or self.selected()
|
||||
here = [name for name in (self._model_path(i.name).name for i in items)]
|
||||
self.model.blockSignals(True)
|
||||
self.model.clear()
|
||||
for item, name in zip(items, here):
|
||||
mark = (t("downloaded") if ggml.have_model(self._model_path(item.name))
|
||||
else ggml.human_size(item.size))
|
||||
self.model.addItem(f"{name} ({mark})", name)
|
||||
self.model.setItemData(self.model.count() - 1, item, Qt.ItemDataRole.UserRole + 1)
|
||||
# A model that was downloaded and then dropped from the list upstream is
|
||||
# still on this disk and still works, so it stays on offer.
|
||||
for name in self._on_disk():
|
||||
if self.model.findData(name) < 0:
|
||||
self.model.addItem(f"{name} ({t('downloaded')})", name)
|
||||
# And one that is chosen but not here, because the file was deleted from
|
||||
# underneath or the settings came from another machine, stays chosen:
|
||||
# Save reads this box, and a row missing here would quietly empty the
|
||||
# setting rather than showing that the model needs downloading again.
|
||||
if wanted and self.model.findData(wanted) < 0:
|
||||
self.model.addItem(f"{wanted} ({t('not downloaded')})", wanted)
|
||||
index = self.model.findData(wanted)
|
||||
self.model.setCurrentIndex(max(index, 0))
|
||||
self.model.blockSignals(False)
|
||||
self._fit_popup(self.model)
|
||||
self._wanted = ""
|
||||
self._model_changed()
|
||||
|
||||
def _on_disk(self):
|
||||
return (ggml.installed_whisper_models() if self.program is ggml.WHISPER
|
||||
else ggml.installed_llm_models())
|
||||
|
||||
# ---- fetching --------------------------------------------------------
|
||||
|
||||
def _install_program(self):
|
||||
self.install_button.setEnabled(False)
|
||||
self.program_label.setText(t("Downloading…"))
|
||||
|
||||
def work():
|
||||
try:
|
||||
ggml.install_program(self.program, on_progress=self._report)
|
||||
self._installed.emit("", "")
|
||||
except ggml.LocalError as exc:
|
||||
self._installed.emit("", str(exc))
|
||||
|
||||
threading.Thread(target=work, daemon=True).start()
|
||||
|
||||
def _on_installed(self, _, error):
|
||||
self.install_button.setEnabled(True)
|
||||
self._show_program()
|
||||
if error:
|
||||
self.program_label.setText(error)
|
||||
self.changed.emit()
|
||||
|
||||
def _current_item(self):
|
||||
return self.model.currentData(Qt.ItemDataRole.UserRole + 1)
|
||||
|
||||
def _download(self):
|
||||
if self._downloading:
|
||||
self._stop = True
|
||||
return
|
||||
item = self._current_item()
|
||||
if item is None:
|
||||
return
|
||||
self._downloading, self._stop = True, False
|
||||
self._refresh_buttons()
|
||||
|
||||
def work():
|
||||
try:
|
||||
landed = ggml.download(item, self._model_path(item.name),
|
||||
on_progress=self._report,
|
||||
should_stop=lambda: self._stop)
|
||||
self._finished.emit(item.name if landed else "", "")
|
||||
except ggml.LocalError as exc:
|
||||
self._finished.emit("", str(exc))
|
||||
|
||||
threading.Thread(target=work, daemon=True).start()
|
||||
|
||||
def _report(self, done, total):
|
||||
self._progress.emit(done, total)
|
||||
|
||||
def _on_progress(self, done, total):
|
||||
share = f" ({done * 100 // total}%)" if total else ""
|
||||
text = t("Downloading: {done} of {total}{share}",
|
||||
done=ggml.human_size(done), total=ggml.human_size(total or done),
|
||||
share=share)
|
||||
if self._downloading:
|
||||
self.status.setText(text)
|
||||
else:
|
||||
self.program_label.setText(text)
|
||||
|
||||
def _on_finished(self, name, error):
|
||||
self._downloading = False
|
||||
if error:
|
||||
self.status.setText(error)
|
||||
elif not name:
|
||||
self.status.setText(t("Download stopped."))
|
||||
self._fill_models_from_current()
|
||||
self.changed.emit()
|
||||
|
||||
def _fill_models_from_current(self):
|
||||
"""Redraw the rows without asking anybody anything again."""
|
||||
items = [self.model.itemData(i, Qt.ItemDataRole.UserRole + 1)
|
||||
for i in range(self.model.count())]
|
||||
self._wanted = self.selected()
|
||||
self._fill_models([i for i in items if i is not None])
|
||||
|
||||
def _delete(self):
|
||||
name = self.selected()
|
||||
if not name or not ggml.have_model(self._model_path(name)):
|
||||
return
|
||||
if QMessageBox.question(self, t("Delete model"),
|
||||
t("Delete {name} from this machine?", name=name)) \
|
||||
!= QMessageBox.StandardButton.Yes:
|
||||
return
|
||||
try:
|
||||
ggml.delete_model(self._model_path(name))
|
||||
except ggml.LocalError as exc:
|
||||
self.status.setText(str(exc))
|
||||
self._fill_models_from_current()
|
||||
self.changed.emit()
|
||||
|
||||
def _model_changed(self):
|
||||
self._refresh_buttons()
|
||||
self.changed.emit()
|
||||
|
||||
def _refresh_buttons(self):
|
||||
name = self.selected()
|
||||
here = bool(name) and ggml.have_model(self._model_path(name))
|
||||
self.delete_button.setEnabled(here and not self._downloading)
|
||||
self.download_button.setText(t("Stop") if self._downloading else t("Download"))
|
||||
self.download_button.setEnabled(self._downloading or (bool(name) and not here))
|
||||
if self._downloading:
|
||||
return
|
||||
if not name:
|
||||
self.status.setText(t("Nothing downloaded yet."))
|
||||
elif here:
|
||||
self.status.setText(t("Ready: {name}.", name=name))
|
||||
else:
|
||||
self.status.setText(t("{name} has not been downloaded yet.", name=name))
|
||||
|
||||
|
||||
class SettingsWindow(QDialog):
|
||||
applied = pyqtSignal()
|
||||
|
||||
@@ -143,9 +479,9 @@ class SettingsWindow(QDialog):
|
||||
self.setWindowTitle(t("Dikte Settings"))
|
||||
self.resize(680, 640)
|
||||
|
||||
tabs = QTabWidget(self)
|
||||
tabs = self.tabs = QTabWidget(self)
|
||||
tabs.addTab(self._general_tab(), t("General"))
|
||||
tabs.addTab(self._api_tab(), t("API and models"))
|
||||
self.api_tab_index = tabs.addTab(self._api_tab(), t("API and models"))
|
||||
tabs.addTab(self._prompt_tab(), t("Cleanup rules"))
|
||||
tabs.addTab(self._assistant_tab(), t("Agent"))
|
||||
tabs.addTab(self._meeting_tab(), t("Meeting"))
|
||||
@@ -176,6 +512,10 @@ class SettingsWindow(QDialog):
|
||||
self.meetings.finished.connect(self._on_minutes_finished)
|
||||
self.meetings.failed.connect(self._on_minutes_failed)
|
||||
self._load()
|
||||
# On a machine where nothing can transcribe yet, this window was opened
|
||||
# because of that, so open it on the tab that fixes it.
|
||||
if not conf.transcribe_ready():
|
||||
self.tabs.setCurrentIndex(self.api_tab_index)
|
||||
|
||||
# ---- tabs ----------------------------------------------------------
|
||||
|
||||
@@ -274,20 +614,52 @@ class SettingsWindow(QDialog):
|
||||
stt_form = QFormLayout(stt)
|
||||
self.transcribe_provider = QComboBox()
|
||||
for label, value in TRANSCRIBE_PROVIDERS:
|
||||
self.transcribe_provider.addItem(label, value)
|
||||
self.transcribe_provider.addItem(t(label), value)
|
||||
stt_form.addRow(t("Provider"), self.transcribe_provider)
|
||||
|
||||
# A hosted provider takes any model id that is typed at it; the local
|
||||
# one offers what has been published. One row each, and only the rows of
|
||||
# whoever is chosen are on screen.
|
||||
self.stt_form = stt_form
|
||||
self.transcribe_model = QComboBox()
|
||||
self.transcribe_model.setEditable(True)
|
||||
self.refresh_transcribe_models = QPushButton(t("Fetch model list"))
|
||||
self.refresh_transcribe_models.clicked.connect(self._load_transcribe_models)
|
||||
stt_form.addRow(t("Model"),
|
||||
self._row(self.transcribe_model, self.refresh_transcribe_models))
|
||||
self.transcribe_model_row = self._row(self.transcribe_model,
|
||||
self.refresh_transcribe_models)
|
||||
stt_form.addRow(t("Model"), self.transcribe_model_row)
|
||||
# A spanning row: in the narrow field column a wrapped label gets a
|
||||
# height that fits one line, and the rest of the text is cut off.
|
||||
self.transcribe_status = QLabel("")
|
||||
self.transcribe_status.setWordWrap(True)
|
||||
stt_form.addRow(self.transcribe_status)
|
||||
|
||||
self.local_whisper = LocalModelBox(
|
||||
ggml.WHISPER, t("On this machine"),
|
||||
ggml.whisper_models, ggml.whisper_model_path)
|
||||
stt_form.addRow(self.local_whisper)
|
||||
|
||||
self.local_gpu = QCheckBox(t("Use the graphics card"))
|
||||
self.local_gpu.setToolTip(
|
||||
t("whisper.cpp reaches the card through CUDA, ROCm or Vulkan when the "
|
||||
"build it is running was made with one. A build without any of them "
|
||||
"runs on the processor whatever this says."))
|
||||
self.local_preload = QCheckBox(t("Load the model when Dikte starts"))
|
||||
self.local_preload.setToolTip(
|
||||
t("A large model takes a second or two to load. Loading it up front "
|
||||
"spends that once instead of on the first dictation, at the cost of "
|
||||
"the memory it sits in."))
|
||||
self.local_threads = QSpinBox()
|
||||
self.local_threads.setRange(0, 64)
|
||||
self.local_threads.setSpecialValueText(t("Automatic"))
|
||||
self.local_options = QWidget()
|
||||
options_form = QFormLayout(self.local_options)
|
||||
options_form.setContentsMargins(0, 0, 0, 0)
|
||||
options_form.addRow("", self.local_gpu)
|
||||
options_form.addRow("", self.local_preload)
|
||||
options_form.addRow(t("Threads"), self.local_threads)
|
||||
stt_form.addRow(self.local_options)
|
||||
|
||||
self.transcribe_provider.currentIndexChanged.connect(self._provider_changed)
|
||||
outer.addWidget(stt)
|
||||
|
||||
@@ -301,9 +673,10 @@ class SettingsWindow(QDialog):
|
||||
self.cleanup_provider.addItem(t(label), value)
|
||||
self.cleanup_provider.setToolTip(t(
|
||||
"OpenRouter is the quickest and the only one that needs nothing "
|
||||
"installed. Claude Code and Codex clean up on the subscription you "
|
||||
"already have, without a second key, and take a few seconds longer "
|
||||
"because each one opens a session to do it."
|
||||
"installed. llama.cpp runs here, on a model downloaded below. Claude "
|
||||
"Code and Codex clean up on the subscription you already have, "
|
||||
"without a second key, and take a few seconds longer because each "
|
||||
"one opens a session to do it."
|
||||
))
|
||||
self.cleanup_provider.currentIndexChanged.connect(self._cleanup_provider_changed)
|
||||
orr_form.addRow(t("Runs on"), self.cleanup_provider)
|
||||
@@ -342,6 +715,33 @@ class SettingsWindow(QDialog):
|
||||
self.models_label = QLabel(t("Runs on OpenRouter."))
|
||||
self.models_label.setWordWrap(True)
|
||||
orr_form.addRow(self.models_label)
|
||||
|
||||
self.local_llm = LocalModelBox(
|
||||
ggml.LLAMA, t("On this machine"),
|
||||
ggml.llm_quants, ggml.llm_model_path, repos=ggml.llm_repos)
|
||||
orr_form.addRow(self.local_llm)
|
||||
|
||||
self.local_llm_gpu = QCheckBox(t("Use the graphics card"))
|
||||
self.local_llm_preload = QCheckBox(t("Load the model when Dikte starts"))
|
||||
self.local_llm_preload.setToolTip(
|
||||
t("An LLM is slower to load than a whisper model and sits in more "
|
||||
"memory. Off means it is loaded on the first cleanup instead."))
|
||||
self.local_llm_reasoning = QComboBox()
|
||||
for label, value in REASONING_LEVELS:
|
||||
self.local_llm_reasoning.addItem(t(label), value)
|
||||
self.local_llm_reasoning.setToolTip(
|
||||
t("A model trained to think will think unless it is told not to, and "
|
||||
"spending 300 tokens of reasoning on a comma is 300 tokens of "
|
||||
"waiting. Off is what cleanup wants."))
|
||||
self.local_llm_options = QWidget()
|
||||
llm_form = QFormLayout(self.local_llm_options)
|
||||
|
||||
llm_form.setContentsMargins(0, 0, 0, 0)
|
||||
llm_form.addRow("", self.local_llm_gpu)
|
||||
llm_form.addRow("", self.local_llm_preload)
|
||||
llm_form.addRow(t("Thinking"), self.local_llm_reasoning)
|
||||
orr_form.addRow(self.local_llm_options)
|
||||
|
||||
outer.addWidget(orr)
|
||||
outer.addStretch(1)
|
||||
return page
|
||||
@@ -985,6 +1385,11 @@ class SettingsWindow(QDialog):
|
||||
self._shown_provider = ""
|
||||
self._select_data(self.transcribe_provider, conf["transcribe_provider"])
|
||||
self._provider_changed() # selecting index 0 fires no signal
|
||||
self.local_gpu.setChecked(conf["local_gpu"])
|
||||
self.local_preload.setChecked(conf["local_preload"])
|
||||
self.local_threads.setValue(int(conf["local_threads"]))
|
||||
self.local_whisper.load(conf["local_model"])
|
||||
|
||||
self.cleanup_enabled.setChecked(conf["cleanup_enabled"])
|
||||
self.cleanup_model.setCurrentText(conf["cleanup_model"])
|
||||
self.cleanup_claude_model.setCurrentText(conf["cleanup_claude_model"])
|
||||
@@ -994,6 +1399,10 @@ class SettingsWindow(QDialog):
|
||||
self._select_data(self.cleanup_provider, conf["cleanup_provider"])
|
||||
self._cleanup_provider_changed() # selecting index 0 fires no signal
|
||||
self._select_data(self.cleanup_reasoning, conf["cleanup_reasoning"])
|
||||
self.local_llm_gpu.setChecked(conf["local_llm_gpu"])
|
||||
self.local_llm_preload.setChecked(conf["local_llm_preload"])
|
||||
self._select_data(self.local_llm_reasoning, conf["local_llm_reasoning"])
|
||||
self.local_llm.load(conf["local_llm_model"], conf["local_llm_repo"])
|
||||
self.cleanup_prompt.setPlainText(conf["cleanup_prompt"] or cfg.default_cleanup_prompt())
|
||||
self.file_cleanup_prompt.setPlainText(
|
||||
conf["file_cleanup_prompt"] or cfg.default_file_cleanup_prompt()
|
||||
@@ -1063,12 +1472,17 @@ class SettingsWindow(QDialog):
|
||||
conf["filter_hallucinations"] = self.filter_hallucinations.isChecked()
|
||||
conf["keep_audio"] = self.keep_audio.isChecked()
|
||||
|
||||
provider = self.transcribe_provider.currentData() or "openai"
|
||||
provider = self.transcribe_provider.currentData() or "local"
|
||||
if provider in self._models:
|
||||
self._models[provider] = self.transcribe_model.currentText().strip()
|
||||
conf["transcribe_provider"] = provider
|
||||
for name, who in cfg.TRANSCRIBERS.items():
|
||||
conf[who.key] = self._key_fields[name].text().strip()
|
||||
conf[who.model] = self._models[name].strip() or cfg.DEFAULTS[who.model]
|
||||
conf["local_model"] = self.local_whisper.selected()
|
||||
conf["local_gpu"] = self.local_gpu.isChecked()
|
||||
conf["local_preload"] = self.local_preload.isChecked()
|
||||
conf["local_threads"] = self.local_threads.value()
|
||||
|
||||
conf["cleanup_enabled"] = self.cleanup_enabled.isChecked()
|
||||
conf["cleanup_provider"] = self.cleanup_provider.currentData() or "openrouter"
|
||||
@@ -1080,6 +1494,11 @@ class SettingsWindow(QDialog):
|
||||
"" if codex_cleanup_model == t("Codex's own default") else codex_cleanup_model
|
||||
)
|
||||
conf["cleanup_reasoning"] = self.cleanup_reasoning.currentData() or ""
|
||||
conf["local_llm_model"] = self.local_llm.selected()
|
||||
conf["local_llm_repo"] = self.local_llm.repository()
|
||||
conf["local_llm_gpu"] = self.local_llm_gpu.isChecked()
|
||||
conf["local_llm_preload"] = self.local_llm_preload.isChecked()
|
||||
conf["local_llm_reasoning"] = self.local_llm_reasoning.currentData() or ""
|
||||
|
||||
# Store an empty prompt when it matches the default, so switching the
|
||||
# interface language also switches the prompt language.
|
||||
@@ -1163,10 +1582,17 @@ class SettingsWindow(QDialog):
|
||||
|
||||
def _provider_changed(self):
|
||||
"""Swap the model box over to the newly chosen provider's own model."""
|
||||
if self._shown_provider:
|
||||
if self._shown_provider in TRANSCRIBE_MODELS:
|
||||
self._models[self._shown_provider] = self.transcribe_model.currentText().strip()
|
||||
provider = self.transcribe_provider.currentData() or "openai"
|
||||
provider = self.transcribe_provider.currentData() or "local"
|
||||
self._shown_provider = provider
|
||||
local = provider == "local"
|
||||
self.stt_form.setRowVisible(self.transcribe_model_row, not local)
|
||||
self.stt_form.setRowVisible(self.transcribe_status, not local)
|
||||
self.stt_form.setRowVisible(self.local_whisper, local)
|
||||
self.stt_form.setRowVisible(self.local_options, local)
|
||||
if local:
|
||||
return
|
||||
self.transcribe_model.clear()
|
||||
self.transcribe_model.addItems(TRANSCRIBE_MODELS[provider])
|
||||
self.transcribe_model.setCurrentText(self._models[provider])
|
||||
@@ -1404,9 +1830,15 @@ class SettingsWindow(QDialog):
|
||||
provider == "claude")
|
||||
self.cleanup_form.setRowVisible(self.cleanup_codex_model,
|
||||
provider == "codex")
|
||||
self.cleanup_form.setRowVisible(self.cleanup_reasoning,
|
||||
provider != "local")
|
||||
self.cleanup_form.setRowVisible(self.local_llm, provider == "local")
|
||||
self.cleanup_form.setRowVisible(self.local_llm_options, provider == "local")
|
||||
binary = cleanup.executable(provider)
|
||||
found = shutil.which(binary) if binary else ""
|
||||
if not binary:
|
||||
if provider == "local":
|
||||
self.models_label.setText(t("Runs on this machine, on llama.cpp."))
|
||||
elif not binary:
|
||||
self.models_label.setText(t("Runs on OpenRouter."))
|
||||
elif found:
|
||||
self.models_label.setText(t("Found: {path}", path=found))
|
||||
|
||||
@@ -18,6 +18,7 @@ import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import wave
|
||||
from unittest import mock
|
||||
|
||||
@@ -40,6 +41,12 @@ linux_only = unittest.skipUnless(
|
||||
)
|
||||
|
||||
|
||||
def _no_network(*args, **kwargs):
|
||||
raise AssertionError(
|
||||
"a test reached the network; wrap the call in support.fake_urlopen"
|
||||
)
|
||||
|
||||
|
||||
def _no_exec(*args, **kwargs):
|
||||
raise AssertionError(
|
||||
"a test reached os.execv, which would replace the test process with the "
|
||||
@@ -78,6 +85,11 @@ class DikteTest(unittest.TestCase):
|
||||
# with it and hang, so it fails loudly here instead.
|
||||
self.patch_attr(os, "execv", _no_exec)
|
||||
|
||||
# Every way out of here goes through urllib, so closing it is enough to
|
||||
# keep the suite offline. A test that means to answer a request patches
|
||||
# this again through fake_urlopen.
|
||||
self.patch_attr(urllib.request, "urlopen", _no_network)
|
||||
|
||||
# ---- helpers ---------------------------------------------------------
|
||||
|
||||
def path(self, *parts):
|
||||
|
||||
@@ -10,6 +10,7 @@ import os
|
||||
import unittest
|
||||
|
||||
import api
|
||||
import ggml
|
||||
from tests.support import (
|
||||
DikteTest,
|
||||
fake_urlopen,
|
||||
@@ -481,3 +482,99 @@ class ModelLists(DikteTest):
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
|
||||
class FakeServer:
|
||||
"""A ggml.Server as far as api.py is concerned."""
|
||||
|
||||
def __init__(self, url="http://127.0.0.1:9999/v1", fails="", log=""):
|
||||
self.url = url
|
||||
self.fails = fails
|
||||
self.log = log
|
||||
self.starts = 0
|
||||
|
||||
def serve(self):
|
||||
self.starts += 1
|
||||
if self.fails:
|
||||
raise ggml.LocalError(self.fails)
|
||||
return self.url
|
||||
|
||||
def error(self):
|
||||
return self.log
|
||||
|
||||
|
||||
LOCAL = api.Target("local", "Local whisper", "", "", "ggml-base.bin")
|
||||
|
||||
|
||||
class TranscribeHere(DikteTest):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.wav = str(self.path("clip.wav"))
|
||||
os.makedirs(self.root, exist_ok=True)
|
||||
with open(self.wav, "wb") as fh:
|
||||
fh.write(b"RIFFfake")
|
||||
self.server = FakeServer()
|
||||
self.patch_attr(ggml, "whisper", self.server)
|
||||
|
||||
def test_the_address_comes_from_the_server_it_starts(self):
|
||||
with fake_urlopen({"text": "hello"}) as calls:
|
||||
api.transcribe(LOCAL, self.wav)
|
||||
self.assertEqual(self.server.starts, 1)
|
||||
self.assertEqual(calls[0].full_url,
|
||||
"http://127.0.0.1:9999/v1/audio/transcriptions")
|
||||
|
||||
def test_nothing_local_is_authorised(self):
|
||||
with fake_urlopen({"text": "hello"}) as calls:
|
||||
api.transcribe(LOCAL, self.wav)
|
||||
self.assertNotIn("Authorization", calls[0].headers)
|
||||
|
||||
def test_a_server_that_will_not_start_is_the_error_shown(self):
|
||||
self.patch_attr(ggml, "whisper", FakeServer(fails="no model downloaded"))
|
||||
with self.assertRaises(api.ApiError) as caught:
|
||||
api.transcribe(LOCAL, self.wav)
|
||||
self.assertIn("no model downloaded", str(caught.exception))
|
||||
|
||||
def test_a_server_that_dies_mid_request_says_what_it_printed(self):
|
||||
self.patch_attr(ggml, "whisper", FakeServer(log="out of memory"))
|
||||
with fake_urlopen(url_error("connection reset")):
|
||||
with self.assertRaises(api.ApiError) as caught:
|
||||
api.transcribe(LOCAL, self.wav)
|
||||
self.assertIn("out of memory", str(caught.exception))
|
||||
|
||||
def test_the_hint_reaches_whisper_as_its_initial_prompt(self):
|
||||
with fake_urlopen({"text": "hi"}) as calls:
|
||||
api.transcribe(LOCAL, self.wav, prompt="Dikte, Paraşüt")
|
||||
self.assertEqual(multipart_fields(calls[0])["prompt"], "Dikte, Paraşüt")
|
||||
|
||||
def test_a_word_broken_over_two_lines_is_put_back_together(self):
|
||||
# whisper.cpp cuts on tokens and writes one segment per line, which in
|
||||
# Turkish lands inside a word about as often as between two.
|
||||
with fake_urlopen({"text": "Onlar akraba değ\niller. Ve\n devamı."}):
|
||||
# The line break inside a word leaves nothing in its place; the
|
||||
# one between two words is where whisper's own leading space is.
|
||||
self.assertEqual(api.transcribe(LOCAL, self.wav),
|
||||
"Onlar akraba değiller. Ve devamı.")
|
||||
|
||||
def test_a_local_timeout_is_not_a_hosted_one(self):
|
||||
# Nothing is being spent but time, and a long file on a machine without
|
||||
# a graphics card takes a good deal of it.
|
||||
with fake_urlopen({"text": "hi"}):
|
||||
api.transcribe(LOCAL, self.wav, timeout=300)
|
||||
self.assertGreaterEqual(api.LOCAL_TIMEOUT, 600)
|
||||
|
||||
def test_segments_that_continue_a_word_are_merged(self):
|
||||
reply = {"segments": [
|
||||
{"start": 0.0, "end": 1.0, "text": " Onlar akraba değ"},
|
||||
{"start": 1.0, "end": 1.4, "text": "iller."},
|
||||
{"start": 2.0, "end": 3.0, "text": " Başka bir cümle."},
|
||||
]}
|
||||
with fake_urlopen(reply):
|
||||
out = api.transcribe_segments(LOCAL, self.wav)
|
||||
self.assertEqual([text for _, _, text in out],
|
||||
["Onlar akraba değiller.", "Başka bir cümle."])
|
||||
self.assertEqual(out[0][1], 1.4) # the merged cue covers the whole word
|
||||
|
||||
def test_the_loaded_model_is_the_one_asked_for_again(self):
|
||||
with fake_urlopen({"segments": [{"start": 0, "end": 1, "text": " hi"}]}) as calls:
|
||||
api.transcribe_segments(LOCAL, self.wav)
|
||||
self.assertEqual(multipart_fields(calls[0])["model"], "ggml-base.bin")
|
||||
|
||||
+76
-1
@@ -13,7 +13,9 @@ from unittest import mock
|
||||
|
||||
import api
|
||||
import cleanup
|
||||
from tests.support import DikteTest
|
||||
import ggml
|
||||
from tests.support import DikteTest, fake_urlopen, sent_json, url_error
|
||||
from tests.test_api import FakeServer, chat_reply
|
||||
|
||||
|
||||
def fake_run(stdout="", code=0, stderr="", last_message=""):
|
||||
@@ -209,3 +211,76 @@ class Codex(DikteTest):
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
|
||||
class Here(DikteTest):
|
||||
"""llama.cpp, answering the request OpenRouter answers."""
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.conf = self.config(cleanup_provider="local",
|
||||
local_llm_model="gemma.gguf")
|
||||
self.server = FakeServer()
|
||||
self.patch_attr(ggml, "llm", self.server)
|
||||
|
||||
def test_the_address_comes_from_the_server_it_starts(self):
|
||||
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||
self.assertEqual(cleanup.run("uh, done", self.conf, "the rules"),
|
||||
"Done.")
|
||||
self.assertEqual(self.server.starts, 1)
|
||||
self.assertEqual(calls[0].full_url,
|
||||
"http://127.0.0.1:9999/v1/chat/completions")
|
||||
|
||||
def test_no_key_is_wanted_and_none_is_sent(self):
|
||||
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||
cleanup.run("uh, done", self.conf, "the rules")
|
||||
self.assertNotIn("Authorization", calls[0].headers)
|
||||
|
||||
def test_thinking_is_turned_off_in_the_words_llama_cpp_uses(self):
|
||||
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||
cleanup.run("uh, done", self.conf, "the rules")
|
||||
self.assertEqual(sent_json(calls[0])["chat_template_kwargs"],
|
||||
{"enable_thinking": False})
|
||||
|
||||
def test_the_models_own_default_asks_for_nothing(self):
|
||||
self.conf["local_llm_reasoning"] = ""
|
||||
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||
cleanup.run("uh, done", self.conf, "the rules")
|
||||
self.assertNotIn("chat_template_kwargs", sent_json(calls[0]))
|
||||
|
||||
def test_a_reply_longer_than_the_transcript_is_cut_off(self):
|
||||
# A small model will repeat the transcript until the context is full,
|
||||
# and every one of those tokens is a second of somebody waiting.
|
||||
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||
cleanup.run("x" * 4000, self.conf, "the rules")
|
||||
self.assertEqual(sent_json(calls[0])["max_tokens"], 4000)
|
||||
|
||||
def test_a_short_dictation_still_gets_room_to_answer(self):
|
||||
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||
cleanup.run("uh, done", self.conf, "the rules")
|
||||
self.assertEqual(sent_json(calls[0])["max_tokens"], 512)
|
||||
|
||||
def test_a_reply_that_was_all_thinking_names_the_setting_that_fixes_it(self):
|
||||
reply = {"choices": [{"message": {"content": "", "reasoning": "hmm"}}]}
|
||||
with fake_urlopen(reply), self.assertRaises(api.ApiError) as caught:
|
||||
cleanup.run("uh, done", self.conf, "the rules")
|
||||
self.assertIn("Thinking", str(caught.exception))
|
||||
|
||||
def test_a_server_that_will_not_start_is_the_error_shown(self):
|
||||
self.patch_attr(ggml, "llm", FakeServer(fails="llama.cpp is not installed"))
|
||||
with self.assertRaises(api.ApiError) as caught:
|
||||
cleanup.run("uh, done", self.conf, "the rules")
|
||||
self.assertIn("llama.cpp", str(caught.exception))
|
||||
|
||||
def test_a_server_that_dies_mid_request_says_what_it_printed(self):
|
||||
self.patch_attr(ggml, "llm", FakeServer(log="out of memory"))
|
||||
with fake_urlopen(url_error("connection reset")):
|
||||
with self.assertRaises(api.ApiError) as caught:
|
||||
cleanup.run("uh, done", self.conf, "the rules")
|
||||
self.assertIn("out of memory", str(caught.exception))
|
||||
|
||||
def test_no_cli_is_started_for_it(self):
|
||||
patcher, calls = fake_run(stdout="never")
|
||||
with patcher, fake_urlopen(chat_reply("Done.")):
|
||||
cleanup.run("uh, done", self.conf, "the rules")
|
||||
self.assertEqual(calls, [])
|
||||
|
||||
+81
-3
@@ -12,7 +12,9 @@ import unittest
|
||||
from unittest import mock
|
||||
|
||||
import api
|
||||
import cleanup
|
||||
import config as cfg
|
||||
import ggml
|
||||
import i18n
|
||||
from tests.support import DikteTest
|
||||
|
||||
@@ -131,8 +133,17 @@ class Keys(DikteTest):
|
||||
|
||||
|
||||
class TranscribeTarget(DikteTest):
|
||||
def test_openai_by_default(self):
|
||||
target = self.config(openai_api_key="sk-test").transcribe_target()
|
||||
def test_this_machine_by_default(self):
|
||||
target = cfg.Config().transcribe_target()
|
||||
self.assertEqual(target.provider, "local")
|
||||
self.assertEqual(target.api_key, "")
|
||||
# Empty on purpose: the server picks a port when it starts, and reading
|
||||
# a setting must not be what starts it.
|
||||
self.assertEqual(target.base_url, "")
|
||||
|
||||
def test_openai_when_it_is_picked(self):
|
||||
target = self.config(transcribe_provider="openai",
|
||||
openai_api_key="sk-test").transcribe_target()
|
||||
self.assertEqual(target.provider, "openai")
|
||||
self.assertEqual(target.service, "OpenAI")
|
||||
self.assertEqual(target.api_key, "sk-test")
|
||||
@@ -165,7 +176,8 @@ class TranscribeTarget(DikteTest):
|
||||
self.assertEqual(target.provider, "openai")
|
||||
|
||||
def test_a_self_hosted_endpoint(self):
|
||||
conf = self.config(openai_base_url="http://localhost:8080/v1")
|
||||
conf = self.config(transcribe_provider="openai",
|
||||
openai_base_url="http://localhost:8080/v1")
|
||||
self.assertEqual(conf.transcribe_target().base_url, "http://localhost:8080/v1")
|
||||
|
||||
|
||||
@@ -444,3 +456,69 @@ class Defaults(unittest.TestCase):
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
|
||||
class LocalCleanup(DikteTest):
|
||||
def test_the_local_model_is_what_the_history_records(self):
|
||||
conf = self.config(cleanup_provider="local",
|
||||
local_llm_model="gemma-3-4b-it-Q4_K_M.gguf")
|
||||
self.assertEqual(cleanup.provider(conf), "local")
|
||||
self.assertEqual(cleanup.model(conf), "gemma-3-4b-it-Q4_K_M.gguf")
|
||||
|
||||
def test_it_needs_no_program_on_the_path(self):
|
||||
# whisper.cpp and llama.cpp are fetched rather than installed, so unlike
|
||||
# Claude Code and Codex there is no executable to look for.
|
||||
self.assertEqual(cleanup.executable("local"), "")
|
||||
|
||||
def test_the_minutes_do_not_follow_the_cleanup_provider(self):
|
||||
# A 4B model here will strip the filler words out of a dictation and
|
||||
# will not write up an hour long meeting.
|
||||
conf = self.config(cleanup_provider="local")
|
||||
self.assertEqual(conf["meeting_model"], cfg.DEFAULTS["meeting_model"])
|
||||
|
||||
def test_only_the_cleanup_setting_asks_for_the_local_model(self):
|
||||
self.assertFalse(cfg.Config().uses_local_llm())
|
||||
self.assertTrue(self.config(cleanup_provider="local").uses_local_llm())
|
||||
|
||||
|
||||
class ReadyToRun(DikteTest):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.patch_attr(ggml, "MODELS_DIR", self.path("models"))
|
||||
|
||||
def install(self, name):
|
||||
path = ggml.whisper_model_path(name)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_bytes(b"model")
|
||||
|
||||
def test_a_missing_program_is_not_ready(self):
|
||||
with mock.patch("shutil.which", return_value=None):
|
||||
self.install("ggml-base.bin")
|
||||
conf = self.config(local_model="ggml-base.bin")
|
||||
self.assertFalse(conf.transcribe_ready())
|
||||
|
||||
def test_a_missing_model_is_not_ready_either(self):
|
||||
with mock.patch("shutil.which", return_value="/usr/bin/whisper-server"):
|
||||
conf = self.config(local_model="ggml-base.bin")
|
||||
self.assertFalse(conf.transcribe_ready())
|
||||
|
||||
def test_both_halves_in_place(self):
|
||||
with mock.patch("shutil.which", return_value="/usr/bin/whisper-server"):
|
||||
self.install("ggml-base.bin")
|
||||
conf = self.config(local_model="ggml-base.bin")
|
||||
self.assertTrue(conf.transcribe_ready())
|
||||
|
||||
def test_a_hosted_provider_is_ready_when_it_has_a_key(self):
|
||||
conf = self.config(transcribe_provider="openai", openai_api_key="sk-test")
|
||||
self.assertTrue(conf.transcribe_ready())
|
||||
|
||||
def test_the_settings_reach_the_servers(self):
|
||||
conf = self.config(local_model="ggml-base.bin", local_threads=4,
|
||||
local_gpu=False, local_llm_model="gemma.gguf",
|
||||
local_llm_context=4096)
|
||||
conf.apply_local()
|
||||
self.addCleanup(ggml.whisper.configure, model="", threads=0, gpu=True)
|
||||
self.assertEqual(ggml.whisper.settings()["model"], "ggml-base.bin")
|
||||
self.assertEqual(ggml.whisper.settings()["threads"], 4)
|
||||
self.assertFalse(ggml.whisper.settings()["gpu"])
|
||||
self.assertEqual(ggml.llm.settings()["context"], 4096)
|
||||
|
||||
@@ -0,0 +1,664 @@
|
||||
"""Fetching a program and a model, and keeping a server alive on them.
|
||||
|
||||
No network and no whisper.cpp: the downloads are answered from memory, and the
|
||||
servers are stand-in scripts that take the same arguments and open their port
|
||||
when they are told to, which is the only thing the code waits on.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import hashlib
|
||||
import io
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import tarfile
|
||||
import textwrap
|
||||
import threading
|
||||
import time
|
||||
from unittest import mock
|
||||
|
||||
import ggml
|
||||
import hub
|
||||
from tests.support import (DikteTest, fake_urlopen, http_error, json_body,
|
||||
linux_only, url_error)
|
||||
|
||||
|
||||
def body(data, length=None):
|
||||
"""What urlopen hands back for a download: a reader with a length header."""
|
||||
class Body:
|
||||
def __init__(self):
|
||||
self._buf = io.BytesIO(data)
|
||||
self.headers = {"Content-Length":
|
||||
str(len(data) if length is None else length)}
|
||||
|
||||
def read(self, count=-1):
|
||||
return self._buf.read(count)
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *_):
|
||||
return False
|
||||
return Body()
|
||||
|
||||
|
||||
def item(name, data, url="https://example.invalid/f", sha=True):
|
||||
return hub.Item(name, url, len(data),
|
||||
hashlib.sha256(data).hexdigest() if sha else "")
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def serving(release, archive):
|
||||
"""Answer by what is being asked for rather than by what came before.
|
||||
|
||||
An install asks GitHub what the release is and then asks for one file out of
|
||||
it, and the first of those two comes from the cache the second time around.
|
||||
Answering in order would then hand the archive request the release listing.
|
||||
"""
|
||||
def opener(request, timeout=None):
|
||||
url = request.full_url
|
||||
if "api.github.com" in url:
|
||||
return json_body(release)
|
||||
return body(archive)
|
||||
|
||||
with mock.patch("urllib.request.urlopen", side_effect=opener) as calls:
|
||||
yield calls
|
||||
|
||||
|
||||
def tarball(entries):
|
||||
"""A .tar.gz laid out the way the releases are: one directory of files."""
|
||||
buf = io.BytesIO()
|
||||
with tarfile.open(fileobj=buf, mode="w:gz") as tar:
|
||||
for name, content in entries.items():
|
||||
info = tarfile.TarInfo(name)
|
||||
info.size = len(content)
|
||||
info.mode = 0o755
|
||||
tar.addfile(info, io.BytesIO(content))
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
class Local(DikteTest):
|
||||
"""A test with its own bin, models and cache directories."""
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.patch_attr(ggml, "DATA_DIR", self.path("data"))
|
||||
self.patch_attr(ggml, "BIN_DIR", self.path("data", "bin"))
|
||||
self.patch_attr(ggml, "MODELS_DIR", self.path("data", "models"))
|
||||
self.patch_attr(hub, "CACHE_DIR", self.path("cache"))
|
||||
|
||||
|
||||
# --- downloading ----------------------------------------------------------
|
||||
|
||||
|
||||
class Download(Local):
|
||||
def test_it_lands_and_the_part_file_is_gone(self):
|
||||
data = b"a model, more or less" * 100
|
||||
target = self.path("data", "models", "m.bin")
|
||||
with fake_urlopen(body(data)):
|
||||
self.assertTrue(ggml.download(item("m.bin", data), target))
|
||||
self.assertEqual(target.read_bytes(), data)
|
||||
self.assertFalse(target.with_name("m.bin.part").exists())
|
||||
|
||||
def test_a_wrong_checksum_installs_nothing(self):
|
||||
data = b"the bytes that arrived"
|
||||
wrong = hub.Item("m.bin", "https://example.invalid/f", len(data), "f" * 64)
|
||||
target = self.path("data", "models", "m.bin")
|
||||
with fake_urlopen(body(data)):
|
||||
with self.assertRaises(ggml.LocalError) as caught:
|
||||
ggml.download(wrong, target)
|
||||
self.assertIn("checksum", str(caught.exception))
|
||||
self.assertFalse(target.exists())
|
||||
self.assertFalse(target.with_name("m.bin.part").exists())
|
||||
|
||||
def test_a_body_shorter_than_its_header_installs_nothing(self):
|
||||
data = b"half of it"
|
||||
target = self.path("data", "models", "m.bin")
|
||||
with fake_urlopen(body(data, length=len(data) * 2)):
|
||||
with self.assertRaises(ggml.LocalError):
|
||||
ggml.download(item("m.bin", data), target)
|
||||
self.assertFalse(target.exists())
|
||||
|
||||
def test_a_file_with_no_published_checksum_is_refused(self):
|
||||
# Everything fetched here is run or parsed by something written in C++,
|
||||
# and GitHub did not always publish a digest.
|
||||
data = b"a program, say"
|
||||
target = self.path("data", "models", "m.bin")
|
||||
with fake_urlopen(body(data)):
|
||||
with self.assertRaises(ggml.LocalError) as caught:
|
||||
ggml.download(item("m.bin", data, sha=False), target)
|
||||
self.assertIn("checksum", str(caught.exception))
|
||||
self.assertFalse(target.exists())
|
||||
|
||||
def test_nothing_is_asked_for_before_it_is_refused(self):
|
||||
# The refusal is not worth a gigabyte of somebody's bandwidth first.
|
||||
with fake_urlopen(body(b"never read")) as calls:
|
||||
with self.assertRaises(ggml.LocalError):
|
||||
ggml.download(item("m.bin", b"x", sha=False), self.path("m.bin"))
|
||||
self.assertEqual(calls, [])
|
||||
|
||||
def test_stopping_leaves_nothing_behind(self):
|
||||
data = b"x" * (ggml.DOWNLOAD_CHUNK * 3)
|
||||
target = self.path("data", "models", "m.bin")
|
||||
with fake_urlopen(body(data)):
|
||||
landed = ggml.download(item("m.bin", data), target,
|
||||
should_stop=lambda: True)
|
||||
self.assertFalse(landed)
|
||||
self.assertFalse(target.exists())
|
||||
self.assertFalse(target.with_name("m.bin.part").exists())
|
||||
|
||||
def test_progress_is_reported_against_the_total(self):
|
||||
data = b"y" * (ggml.DOWNLOAD_CHUNK + 5)
|
||||
seen = []
|
||||
with fake_urlopen(body(data)):
|
||||
ggml.download(item("m.bin", data), self.path("data", "m.bin"),
|
||||
on_progress=lambda done, total: seen.append((done, total)))
|
||||
self.assertEqual(seen[-1], (len(data), len(data)))
|
||||
self.assertGreater(len(seen), 1)
|
||||
|
||||
def test_a_refused_connection_says_which_file(self):
|
||||
with fake_urlopen(url_error("no route to host")):
|
||||
with self.assertRaises(ggml.LocalError) as caught:
|
||||
ggml.download(item("m.bin", b"x"), self.path("data", "m.bin"))
|
||||
self.assertIn("m.bin", str(caught.exception))
|
||||
|
||||
def test_an_http_error_is_not_written_to_disk(self):
|
||||
target = self.path("data", "m.bin")
|
||||
with fake_urlopen(http_error(404)):
|
||||
with self.assertRaises(ggml.LocalError):
|
||||
ggml.download(item("m.bin", b"x"), target)
|
||||
self.assertFalse(target.exists())
|
||||
|
||||
|
||||
# --- installing a program -------------------------------------------------
|
||||
|
||||
|
||||
class InstallProgram(Local):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
# Built once, because the release listing has to publish its checksum
|
||||
# and a tarball is not the same bytes twice.
|
||||
self.archive = tarball({
|
||||
"whisper-bin-ubuntu-x64/whisper-server": b"#!/bin/sh\nexit 0\n",
|
||||
"whisper-bin-ubuntu-x64/libwhisper.so": b"not really a library",
|
||||
})
|
||||
|
||||
def release(self, *names, archive=None):
|
||||
digest = hashlib.sha256(self.archive if archive is None else archive)
|
||||
return {"tag_name": "v1.9.1", "assets": [
|
||||
{"name": name, "browser_download_url": f"https://example.invalid/{name}",
|
||||
"size": 10, "digest": "sha256:" + digest.hexdigest()}
|
||||
for name in names]}
|
||||
|
||||
def install(self, *names, archive=None):
|
||||
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||
blob = self.archive if archive is None else archive
|
||||
with serving(self.release(*names, archive=blob), blob) as calls:
|
||||
path = ggml.install_program(ggml.WHISPER)
|
||||
return path, [call.args[0].full_url for call in calls.call_args_list]
|
||||
|
||||
def test_the_binary_and_its_libraries_land_together(self):
|
||||
path, _ = self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||
self.assertTrue(os.path.isfile(path))
|
||||
self.assertTrue(os.access(path, os.X_OK))
|
||||
self.assertTrue(os.path.isfile(os.path.join(os.path.dirname(path),
|
||||
"libwhisper.so")))
|
||||
|
||||
def test_the_build_for_this_machine_is_the_one_fetched(self):
|
||||
_, urls = self.install("whisper-bin-x64.zip", "whisper-bin-ubuntu-arm64.tar.gz",
|
||||
"whisper-bin-ubuntu-x64.tar.gz")
|
||||
self.assertTrue(urls[1].endswith("whisper-bin-ubuntu-x64.tar.gz"))
|
||||
|
||||
def test_a_release_with_nothing_for_this_machine_says_so(self):
|
||||
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||
with fake_urlopen(self.release("whisper-bin-Win32.zip")):
|
||||
with self.assertRaises(ggml.LocalError) as caught:
|
||||
ggml.install_program(ggml.WHISPER)
|
||||
self.assertIn("this machine", str(caught.exception))
|
||||
|
||||
def test_what_was_installed_is_remembered(self):
|
||||
path, _ = self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||
self.assertEqual(ggml.installed_program(ggml.WHISPER), path)
|
||||
self.assertEqual(ggml.installed_version(ggml.WHISPER), "v1.9.1")
|
||||
|
||||
def test_a_record_pointing_at_a_deleted_binary_counts_for_nothing(self):
|
||||
path, _ = self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||
os.unlink(path)
|
||||
self.assertEqual(ggml.installed_program(ggml.WHISPER), "")
|
||||
|
||||
def test_the_archive_is_not_kept(self):
|
||||
self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||
left = list((self.path("data", "bin", "whisper")).glob("*.tar.gz"))
|
||||
self.assertEqual(left, [])
|
||||
|
||||
def test_the_previous_version_is_swept_up(self):
|
||||
self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||
old = self.path("data", "bin", "whisper", "v1.9.0")
|
||||
old.mkdir(parents=True)
|
||||
(old / "whisper-server").write_bytes(b"older")
|
||||
self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||
self.assertFalse(old.exists())
|
||||
|
||||
def test_an_archive_without_the_binary_is_refused(self):
|
||||
empty = tarball({"whisper-bin-ubuntu-x64/README": b"nothing here"})
|
||||
with self.assertRaises(ggml.LocalError) as caught:
|
||||
self.install("whisper-bin-ubuntu-x64.tar.gz", archive=empty)
|
||||
self.assertIn("whisper-server", str(caught.exception))
|
||||
|
||||
|
||||
def test_a_release_without_a_published_checksum_is_refused(self):
|
||||
# GitHub did not always publish one, and whisper.cpp v1.8.0 and older
|
||||
# still have none.
|
||||
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||
listing = {"tag_name": "v1.8.0", "assets": [
|
||||
{"name": "whisper-bin-ubuntu-x64.tar.gz",
|
||||
"browser_download_url": "https://example.invalid/w.tar.gz",
|
||||
"size": 10}]}
|
||||
with serving(listing, self.archive):
|
||||
with self.assertRaises(ggml.LocalError) as caught:
|
||||
ggml.install_program(ggml.WHISPER)
|
||||
self.assertIn("checksum", str(caught.exception))
|
||||
self.assertEqual(ggml.installed_program(ggml.WHISPER), "")
|
||||
|
||||
def test_an_archive_that_is_not_what_was_promised_installs_nothing(self):
|
||||
listing = self.release("whisper-bin-ubuntu-x64.tar.gz")
|
||||
other = tarball({"whisper-bin-ubuntu-x64/whisper-server": b"#!/bin/sh\nrm -rf\n"})
|
||||
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||
with serving(listing, other):
|
||||
with self.assertRaises(ggml.LocalError) as caught:
|
||||
ggml.install_program(ggml.WHISPER)
|
||||
self.assertIn("checksum", str(caught.exception))
|
||||
self.assertEqual(ggml.installed_program(ggml.WHISPER), "")
|
||||
|
||||
def test_an_archive_cannot_write_outside_the_directory_it_is_opened_in(self):
|
||||
# An archive is not a trusted thing to unpack: a member named ../../ is
|
||||
# how one writes over a file it was never given.
|
||||
escape = tarball({"../../../escaped": b"should not land"})
|
||||
path = self.path("data", "bin", "whisper", "v1.9.1")
|
||||
with self.assertRaises(ggml.LocalError):
|
||||
self.install("whisper-bin-ubuntu-x64.tar.gz", archive=escape)
|
||||
self.assertFalse(self.path("escaped").exists())
|
||||
self.assertFalse((path.parent.parent / "escaped").exists())
|
||||
|
||||
def test_a_symlink_out_of_the_directory_does_not_survive_either(self):
|
||||
buf = io.BytesIO()
|
||||
with tarfile.open(fileobj=buf, mode="w:gz") as tar:
|
||||
info = tarfile.TarInfo("whisper-bin-ubuntu-x64/whisper-server")
|
||||
info.type, info.linkname = tarfile.SYMTYPE, "/etc/passwd"
|
||||
tar.addfile(info)
|
||||
with self.assertRaises(ggml.LocalError):
|
||||
self.install("whisper-bin-ubuntu-x64.tar.gz", archive=buf.getvalue())
|
||||
|
||||
def test_everything_is_asked_for_over_tls(self):
|
||||
for url in (hub.GITHUB_API, hub.HF_API, hub.HF_FILES):
|
||||
with self.subTest(url=url):
|
||||
self.assertTrue(url.startswith("https://"))
|
||||
|
||||
def test_llama_takes_the_vulkan_build_when_there_is_a_loader(self):
|
||||
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||
self.patch_attr(ggml, "_has_vulkan", lambda: True)
|
||||
self.assertEqual(ggml._wanted_assets(ggml.LLAMA)[0],
|
||||
"bin-ubuntu-vulkan-x64.tar.gz")
|
||||
|
||||
def test_llama_falls_back_to_the_plain_build_without_one(self):
|
||||
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||
self.patch_attr(ggml, "_has_vulkan", lambda: False)
|
||||
self.assertEqual(ggml._wanted_assets(ggml.LLAMA), ("bin-ubuntu-x64.tar.gz",))
|
||||
|
||||
|
||||
class WhichCopyRuns(Local):
|
||||
def test_a_system_build_wins_over_a_downloaded_one(self):
|
||||
self.patch_attr(ggml, "installed_program", lambda program: "/data/whisper-server")
|
||||
with mock.patch("shutil.which", return_value="/usr/bin/whisper-server"):
|
||||
self.assertEqual(ggml.program_path(ggml.WHISPER), "/usr/bin/whisper-server")
|
||||
|
||||
def test_the_downloaded_one_is_used_when_there_is_no_system_build(self):
|
||||
self.patch_attr(ggml, "installed_program", lambda program: "/data/whisper-server")
|
||||
with mock.patch("shutil.which", return_value=None):
|
||||
self.assertEqual(ggml.program_path(ggml.WHISPER), "/data/whisper-server")
|
||||
|
||||
def test_a_setting_pointing_at_nothing_is_no_program(self):
|
||||
self.assertEqual(ggml.program_path(ggml.WHISPER, "/nowhere/whisper-server"), "")
|
||||
|
||||
def test_a_setting_pointing_at_a_program_wins(self):
|
||||
mine = self.path("mine")
|
||||
mine.write_text("#!/bin/sh\n")
|
||||
mine.chmod(0o755)
|
||||
with mock.patch("shutil.which", return_value="/usr/bin/whisper-server"):
|
||||
self.assertEqual(ggml.program_path(ggml.WHISPER, str(mine)), str(mine))
|
||||
|
||||
|
||||
# --- the lists ------------------------------------------------------------
|
||||
|
||||
|
||||
WHISPER_TREE = [
|
||||
{"type": "file", "path": "ggml-base.bin", "size": 147951465,
|
||||
"lfs": {"oid": "a" * 64}},
|
||||
{"type": "file", "path": "ggml-large-v3-turbo-q5_0.bin", "size": 574041195,
|
||||
"lfs": {"oid": "b" * 64}},
|
||||
{"type": "file", "path": "ggml-base-encoder.mlmodelc.zip", "size": 37922638,
|
||||
"lfs": {"oid": "c" * 64}},
|
||||
{"type": "file", "path": "README.md", "size": 3196},
|
||||
]
|
||||
|
||||
GGUF_TREE = [
|
||||
{"type": "file", "path": "gemma-3-4b-it-Q4_K_M.gguf", "size": 2489000000,
|
||||
"lfs": {"oid": "a" * 64}},
|
||||
{"type": "file", "path": "gemma-3-4b-it-Q8_0.gguf", "size": 4130000000,
|
||||
"lfs": {"oid": "b" * 64}},
|
||||
{"type": "file", "path": "mmproj-model-f16.gguf", "size": 851000000,
|
||||
"lfs": {"oid": "c" * 64}},
|
||||
{"type": "file", "path": "mtp-gemma-4-E4B-it-Q4_0.gguf", "size": 59000000,
|
||||
"lfs": {"oid": "d" * 64}},
|
||||
{"type": "file", "path": "huge-00001-of-00009.gguf", "size": 40000000000,
|
||||
"lfs": {"oid": "e" * 64}},
|
||||
{"type": "file", "path": "README.md", "size": 100},
|
||||
]
|
||||
|
||||
|
||||
class Catalogue(Local):
|
||||
def test_only_models_are_offered_and_the_small_ones_first(self):
|
||||
with fake_urlopen(WHISPER_TREE):
|
||||
models = ggml.whisper_models()
|
||||
self.assertEqual([m.name for m in models],
|
||||
["ggml-base.bin", "ggml-large-v3-turbo-q5_0.bin"])
|
||||
|
||||
def test_the_core_ml_encoders_are_not_models(self):
|
||||
with fake_urlopen(WHISPER_TREE):
|
||||
names = [m.name for m in ggml.whisper_models()]
|
||||
self.assertNotIn("ggml-base-encoder.mlmodelc.zip", names)
|
||||
|
||||
def test_the_projector_and_the_draft_head_are_not_models(self):
|
||||
with fake_urlopen(GGUF_TREE):
|
||||
names = [q.name for q in ggml.llm_quants("ggml-org/gemma-3-4b-it-GGUF")]
|
||||
self.assertEqual(names,
|
||||
["gemma-3-4b-it-Q4_K_M.gguf", "gemma-3-4b-it-Q8_0.gguf"])
|
||||
|
||||
def test_a_model_split_across_files_is_left_out(self):
|
||||
with fake_urlopen(GGUF_TREE):
|
||||
names = [q.name for q in ggml.llm_quants("ggml-org/gemma-3-4b-it-GGUF")]
|
||||
self.assertNotIn("huge-00001-of-00009.gguf", names)
|
||||
|
||||
def test_the_suggestions_come_first_and_the_rest_follow(self):
|
||||
listing = [{"id": "ggml-org/something-new-GGUF"},
|
||||
{"id": ggml.SUGGESTED_LLM[0]}]
|
||||
with fake_urlopen(listing):
|
||||
found = ggml.llm_repos()
|
||||
self.assertEqual(found[0], ggml.SUGGESTED_LLM[0])
|
||||
self.assertIn("ggml-org/something-new-GGUF", found)
|
||||
|
||||
def test_an_unreachable_list_still_offers_the_suggestions(self):
|
||||
with fake_urlopen(url_error()):
|
||||
self.assertEqual(ggml.llm_repos(), list(ggml.SUGGESTED_LLM))
|
||||
|
||||
def test_an_unreachable_whisper_list_is_an_error_worth_showing(self):
|
||||
with fake_urlopen(url_error()):
|
||||
with self.assertRaises(ggml.LocalError):
|
||||
ggml.whisper_models()
|
||||
|
||||
def test_what_is_on_disk_is_read_from_disk(self):
|
||||
self.assertEqual(ggml.installed_whisper_models(), [])
|
||||
path = ggml.whisper_model_path("ggml-base.bin")
|
||||
path.parent.mkdir(parents=True)
|
||||
path.write_bytes(b"model")
|
||||
self.assertEqual(ggml.installed_whisper_models(), ["ggml-base.bin"])
|
||||
self.assertTrue(ggml.have_model(path))
|
||||
|
||||
def test_an_empty_file_is_not_a_model(self):
|
||||
path = ggml.llm_model_path("ggml-org/x-GGUF/model.gguf")
|
||||
path.parent.mkdir(parents=True)
|
||||
path.write_bytes(b"")
|
||||
self.assertFalse(ggml.have_model(path))
|
||||
|
||||
def test_a_model_is_named_by_its_file_not_its_repository(self):
|
||||
self.assertEqual(ggml.llm_model_path("ggml-org/x-GGUF/model.gguf").name,
|
||||
"model.gguf")
|
||||
|
||||
|
||||
# --- keeping a server alive -----------------------------------------------
|
||||
|
||||
|
||||
STAND_IN = textwrap.dedent("""
|
||||
import http.server, sys, threading, time
|
||||
|
||||
args = sys.argv[1:]
|
||||
|
||||
def opt(name, default=""):
|
||||
return args[args.index(name) + 1] if name in args else default
|
||||
|
||||
if "--die" in args:
|
||||
print("could not load model: no such file")
|
||||
sys.exit(2)
|
||||
|
||||
time.sleep(float(opt("--wait", "0")))
|
||||
|
||||
started = time.monotonic()
|
||||
healthy_after = float(opt("--healthy-after", "0"))
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
ok = time.monotonic() - started >= healthy_after
|
||||
self.send_response(200 if ok else 503)
|
||||
self.end_headers()
|
||||
self.wfile.write(b"{}")
|
||||
|
||||
def log_message(self, *a):
|
||||
pass
|
||||
|
||||
server = http.server.HTTPServer((opt("--host"), int(opt("--port"))), Handler)
|
||||
print("listening on " + opt("--port"), flush=True)
|
||||
server.serve_forever()
|
||||
""")
|
||||
|
||||
|
||||
class Servers(Local):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.path("data").mkdir(parents=True, exist_ok=True)
|
||||
# Named for the program and kept inside the data directory, because that
|
||||
# is what the sweep looks for on a command line.
|
||||
self.script = self.path("data", "whisper-server.py")
|
||||
self.script.write_text(STAND_IN)
|
||||
self.addCleanup(ggml.stop_all)
|
||||
self.servers = []
|
||||
|
||||
def server(self, program=ggml.WHISPER, **settings):
|
||||
defaults = {"extra": []}
|
||||
defaults.update(settings)
|
||||
made = ggml.Server(
|
||||
program,
|
||||
lambda values: [sys.executable, str(self.script)] + list(values["extra"]),
|
||||
defaults,
|
||||
)
|
||||
self.servers.append(made)
|
||||
self.addCleanup(made.stop)
|
||||
return made
|
||||
|
||||
def test_a_started_server_hands_back_its_address(self):
|
||||
server = self.server()
|
||||
url = server.serve()
|
||||
self.assertRegex(url, r"^http://127\.0\.0\.1:\d+/v1$")
|
||||
self.assertTrue(server.running)
|
||||
|
||||
def test_the_second_call_does_not_start_a_second_one(self):
|
||||
server = self.server()
|
||||
first = server.serve()
|
||||
self.assertEqual(server.serve(), first)
|
||||
|
||||
def test_a_settings_change_stops_what_was_running(self):
|
||||
server = self.server()
|
||||
server.serve()
|
||||
server.configure(extra=["--wait", "0"])
|
||||
self.assertFalse(server.running)
|
||||
|
||||
def test_the_new_settings_are_what_the_next_start_uses(self):
|
||||
server = self.server()
|
||||
server.serve()
|
||||
server.configure(extra=["--healthy-after", "0"])
|
||||
second = server.serve()
|
||||
self.assertTrue(server.running)
|
||||
self.assertTrue(second)
|
||||
|
||||
def test_a_program_that_dies_reports_what_it_printed(self):
|
||||
server = self.server(extra=["--die"])
|
||||
with self.assertRaises(ggml.LocalError) as caught:
|
||||
server.serve()
|
||||
self.assertIn("no such file", str(caught.exception))
|
||||
self.assertFalse(server.running)
|
||||
|
||||
def test_a_model_that_is_still_loading_is_not_ready_yet(self):
|
||||
# llama binds its port first and answers /health with 503 until the
|
||||
# model is in memory, so the open port on its own is not the signal.
|
||||
server = self.server(program=ggml.LLAMA, extra=["--healthy-after", "0.4"])
|
||||
started = time.monotonic()
|
||||
server.serve()
|
||||
self.assertGreaterEqual(time.monotonic() - started, 0.4)
|
||||
|
||||
def test_a_start_that_never_becomes_ready_gives_up(self):
|
||||
self.patch_attr(ggml, "STARTUP_TIMEOUT", 0.5)
|
||||
server = self.server(program=ggml.LLAMA, extra=["--healthy-after", "30"])
|
||||
with self.assertRaises(ggml.LocalError):
|
||||
server.serve()
|
||||
|
||||
def test_stopping_leaves_nothing_running(self):
|
||||
server = self.server()
|
||||
server.serve()
|
||||
server.stop()
|
||||
self.assertFalse(server.running)
|
||||
self.assertEqual(server.base_url(), "")
|
||||
|
||||
def test_the_last_thing_it_printed_is_available(self):
|
||||
server = self.server()
|
||||
server.serve()
|
||||
self.assertIn("listening", server.error())
|
||||
|
||||
def test_asking_what_is_running_does_not_wait_for_a_start(self):
|
||||
"""A model being loaded must not freeze the settings window.
|
||||
|
||||
The interface asks a running server what it is doing while a start is in
|
||||
flight, and a lock held across the whole start would stop it dead.
|
||||
"""
|
||||
server = self.server(extra=["--wait", "0.6"])
|
||||
answers = []
|
||||
|
||||
def start():
|
||||
server.serve()
|
||||
|
||||
thread = __import__("threading").Thread(target=start)
|
||||
thread.start()
|
||||
try:
|
||||
time.sleep(0.15)
|
||||
began = time.monotonic()
|
||||
answers.append(server.settings())
|
||||
answers.append(server.running)
|
||||
self.assertLess(time.monotonic() - began, 0.2)
|
||||
finally:
|
||||
thread.join(timeout=10)
|
||||
|
||||
@linux_only
|
||||
def test_a_server_a_killed_dikte_left_behind_is_swept_up(self):
|
||||
server = self.server()
|
||||
server.serve()
|
||||
# What a SIGKILL of Dikte leaves: the child still running, the pid file
|
||||
# still on disk, and nothing left that knows about either.
|
||||
proc, server._proc = server._proc, None
|
||||
self.assertTrue(server.sweep())
|
||||
self.assertEqual(proc.wait(timeout=5), -signal.SIGTERM)
|
||||
|
||||
@linux_only
|
||||
def test_a_pid_that_belongs_to_something_else_is_left_alone(self):
|
||||
server = self.server()
|
||||
server._remember(os.getpid()) # this test runner, not a server
|
||||
self.assertFalse(server.sweep())
|
||||
|
||||
def test_no_pid_file_is_nothing_to_sweep(self):
|
||||
self.assertFalse(self.server().sweep())
|
||||
|
||||
def test_a_start_that_goes_wrong_takes_its_process_with_it(self):
|
||||
started = []
|
||||
|
||||
def explode(inner, proc, port):
|
||||
started.append(proc)
|
||||
raise RuntimeError("something in the wait went wrong")
|
||||
|
||||
self.patch_attr(ggml.Server, "_wait_ready", explode)
|
||||
server = self.server()
|
||||
with self.assertRaises(RuntimeError):
|
||||
server.serve()
|
||||
# Nothing else holds a reference to it, so leaving it running would leak
|
||||
# a loaded model with nobody left to ask it anything.
|
||||
self.assertIsNotNone(started[0].poll())
|
||||
self.assertFalse(server.sweep()) # and the pid file went with it
|
||||
|
||||
|
||||
class Arguments(Local):
|
||||
"""What the two command lines say, since neither program is here to say it."""
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.binary = self.path("whisper-server")
|
||||
self.binary.write_text("#!/bin/sh\n")
|
||||
self.binary.chmod(0o755)
|
||||
|
||||
def whisper_model(self, name="ggml-base.bin"):
|
||||
path = ggml.whisper_model_path(name)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_bytes(b"model")
|
||||
return name
|
||||
|
||||
def test_the_inference_path_is_the_one_api_py_builds(self):
|
||||
args = ggml._whisper_args({"binary": str(self.binary), "gpu": True,
|
||||
"threads": 0, "model": self.whisper_model()})
|
||||
self.assertIn("--inference-path", args)
|
||||
self.assertEqual(args[args.index("--inference-path") + 1],
|
||||
"/v1/audio/transcriptions")
|
||||
|
||||
def test_detection_rather_than_english_when_nothing_is_asked_for(self):
|
||||
args = ggml._whisper_args({"binary": str(self.binary), "gpu": True,
|
||||
"threads": 0, "model": self.whisper_model()})
|
||||
self.assertEqual(args[args.index("-l") + 1], "auto")
|
||||
|
||||
def test_the_graphics_card_is_turned_off_rather_than_asked_for(self):
|
||||
settings = {"binary": str(self.binary), "gpu": False, "threads": 2,
|
||||
"model": self.whisper_model()}
|
||||
args = ggml._whisper_args(settings)
|
||||
self.assertIn("-ng", args)
|
||||
self.assertEqual(args[args.index("-t") + 1], "2")
|
||||
|
||||
def test_a_missing_model_is_a_message_about_settings(self):
|
||||
with self.assertRaises(ggml.LocalError) as caught:
|
||||
ggml._whisper_args({"binary": str(self.binary), "gpu": True,
|
||||
"threads": 0, "model": "ggml-nothing.bin"})
|
||||
self.assertIn("Settings", str(caught.exception))
|
||||
|
||||
def test_a_missing_program_says_so_before_a_missing_model(self):
|
||||
with mock.patch("shutil.which", return_value=None):
|
||||
with self.assertRaises(ggml.LocalError) as caught:
|
||||
ggml._whisper_args({"binary": "", "gpu": True, "threads": 0,
|
||||
"model": self.whisper_model()})
|
||||
self.assertIn("whisper.cpp", str(caught.exception))
|
||||
|
||||
def test_the_layers_go_to_the_card_when_there_is_one(self):
|
||||
model = ggml.llm_model_path("m.gguf")
|
||||
model.parent.mkdir(parents=True, exist_ok=True)
|
||||
model.write_bytes(b"gguf")
|
||||
args = ggml._llm_args({"binary": str(self.binary), "gpu": True,
|
||||
"threads": 0, "model": "m.gguf", "context": 4096})
|
||||
self.assertEqual(args[args.index("-ngl") + 1], "99")
|
||||
self.assertEqual(args[args.index("-c") + 1], "4096")
|
||||
|
||||
def test_no_card_means_no_layers_offloaded(self):
|
||||
model = ggml.llm_model_path("m.gguf")
|
||||
model.parent.mkdir(parents=True, exist_ok=True)
|
||||
model.write_bytes(b"gguf")
|
||||
args = ggml._llm_args({"binary": str(self.binary), "gpu": False,
|
||||
"threads": 0, "model": "m.gguf", "context": 4096})
|
||||
self.assertEqual(args[args.index("-ngl") + 1], "0")
|
||||
|
||||
|
||||
class Sizes(DikteTest):
|
||||
def test_bytes_are_written_the_way_a_download_is_talked_about(self):
|
||||
self.assertEqual(ggml.human_size(512), "512 B")
|
||||
self.assertEqual(ggml.human_size(574041195), "547.4 MB")
|
||||
self.assertEqual(ggml.human_size(3_095_033_483), "2.9 GB")
|
||||
|
||||
@@ -0,0 +1,184 @@
|
||||
"""What GitHub and Hugging Face are asked, and what is believed of the answer."""
|
||||
|
||||
import json
|
||||
|
||||
import hub
|
||||
from tests.support import DikteTest, fake_urlopen, http_error, url_error
|
||||
|
||||
RELEASE = {
|
||||
"tag_name": "v1.9.1",
|
||||
"assets": [
|
||||
{"name": "whisper-bin-ubuntu-x64.tar.gz",
|
||||
"browser_download_url": "https://example.invalid/ubuntu-x64.tar.gz",
|
||||
"size": 9379235, "digest": "sha256:" + "a" * 64},
|
||||
{"name": "whisper-bin-x64.zip",
|
||||
"browser_download_url": "https://example.invalid/win.zip",
|
||||
"size": 100, "digest": "sha256:" + "b" * 64},
|
||||
{"name": "no-url-here.zip", "size": 1},
|
||||
],
|
||||
}
|
||||
|
||||
TREE = [
|
||||
{"type": "file", "path": ".gitattributes", "size": 1477},
|
||||
{"type": "file", "path": "ggml-base.bin", "size": 147951465,
|
||||
"lfs": {"oid": "c" * 64, "size": 147951465}},
|
||||
{"type": "directory", "path": "extra"},
|
||||
{"type": "file", "path": "extra/ggml-tiny.bin", "size": 77691713,
|
||||
"lfs": {"oid": "d" * 64, "size": 77691713}},
|
||||
]
|
||||
|
||||
MODELS = [
|
||||
{"id": "ggml-org/gemma-3-4b-it-GGUF", "downloads": 44606,
|
||||
"lastModified": "2026-07-01T00:00:00.000Z"},
|
||||
{"id": "ggml-org/gpt-oss-20b-GGUF", "downloads": 47975},
|
||||
{"noid": True},
|
||||
]
|
||||
|
||||
|
||||
class Releases(DikteTest):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.patch_attr(hub, "CACHE_DIR", self.path("cache"))
|
||||
|
||||
def test_the_tag_and_the_assets_come_back(self):
|
||||
with fake_urlopen(RELEASE) as calls:
|
||||
tag, assets = hub.release("ggml-org/whisper.cpp")
|
||||
self.assertEqual(tag, "v1.9.1")
|
||||
self.assertEqual([a.name for a in assets],
|
||||
["whisper-bin-ubuntu-x64.tar.gz", "whisper-bin-x64.zip"])
|
||||
self.assertEqual(calls[0].full_url,
|
||||
"https://api.github.com/repos/ggml-org/whisper.cpp/"
|
||||
"releases/latest")
|
||||
|
||||
def test_the_sha256_prefix_is_dropped(self):
|
||||
with fake_urlopen(RELEASE):
|
||||
_, assets = hub.release("ggml-org/whisper.cpp")
|
||||
self.assertEqual(assets[0].sha256, "a" * 64)
|
||||
|
||||
def test_a_tag_asks_for_that_tag(self):
|
||||
with fake_urlopen(RELEASE) as calls:
|
||||
hub.release("ggml-org/whisper.cpp", "v1.9.1")
|
||||
self.assertTrue(calls[0].full_url.endswith("/releases/tags/v1.9.1"))
|
||||
|
||||
def test_a_release_with_no_assets_is_an_error(self):
|
||||
with fake_urlopen({"tag_name": "v1", "assets": []}):
|
||||
with self.assertRaises(hub.HubError):
|
||||
hub.release("ggml-org/whisper.cpp")
|
||||
|
||||
def test_the_second_call_asks_nobody(self):
|
||||
with fake_urlopen(RELEASE) as calls:
|
||||
hub.release("ggml-org/whisper.cpp")
|
||||
hub.release("ggml-org/whisper.cpp")
|
||||
self.assertEqual(len(calls), 1)
|
||||
|
||||
def test_a_refresh_asks_again(self):
|
||||
with fake_urlopen(RELEASE) as calls:
|
||||
hub.release("ggml-org/whisper.cpp")
|
||||
hub.release("ggml-org/whisper.cpp", refresh=True)
|
||||
self.assertEqual(len(calls), 2)
|
||||
|
||||
def test_an_old_cache_beats_no_answer(self):
|
||||
with fake_urlopen(RELEASE):
|
||||
hub.release("ggml-org/whisper.cpp")
|
||||
# Old enough that it would normally be fetched again, and no network
|
||||
# to fetch it with.
|
||||
for path in self.path("cache").iterdir():
|
||||
os_utime(path)
|
||||
with fake_urlopen(url_error()):
|
||||
tag, assets = hub.release("ggml-org/whisper.cpp")
|
||||
self.assertEqual(tag, "v1.9.1")
|
||||
self.assertEqual(len(assets), 2)
|
||||
|
||||
def test_no_cache_and_no_network_says_so(self):
|
||||
with fake_urlopen(url_error("no route to host")):
|
||||
with self.assertRaises(hub.HubError) as caught:
|
||||
hub.release("ggml-org/whisper.cpp")
|
||||
self.assertIn("api.github.com", str(caught.exception))
|
||||
|
||||
def test_an_http_error_names_the_host_and_the_code(self):
|
||||
with fake_urlopen(http_error(404, "nope")):
|
||||
with self.assertRaises(hub.HubError) as caught:
|
||||
hub.release("ggml-org/nothing")
|
||||
self.assertIn("404", str(caught.exception))
|
||||
|
||||
|
||||
class Files(DikteTest):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.patch_attr(hub, "CACHE_DIR", self.path("cache"))
|
||||
|
||||
def test_directories_are_left_out(self):
|
||||
with fake_urlopen(TREE):
|
||||
files = hub.files("ggerganov/whisper.cpp")
|
||||
self.assertEqual([f.name for f in files],
|
||||
[".gitattributes", "ggml-base.bin", "extra/ggml-tiny.bin"])
|
||||
|
||||
def test_the_url_is_the_one_that_serves_the_bytes(self):
|
||||
with fake_urlopen(TREE):
|
||||
files = hub.files("ggerganov/whisper.cpp")
|
||||
self.assertEqual(
|
||||
files[1].url,
|
||||
"https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin")
|
||||
|
||||
def test_the_lfs_object_id_is_the_checksum(self):
|
||||
with fake_urlopen(TREE):
|
||||
files = hub.files("ggerganov/whisper.cpp")
|
||||
self.assertEqual(files[1].sha256, "c" * 64)
|
||||
self.assertEqual(files[1].size, 147951465)
|
||||
|
||||
def test_a_file_outside_lfs_has_no_checksum(self):
|
||||
with fake_urlopen(TREE):
|
||||
files = hub.files("ggerganov/whisper.cpp")
|
||||
self.assertEqual(files[0].sha256, "")
|
||||
|
||||
def test_an_answer_that_is_not_a_list_is_an_error(self):
|
||||
with fake_urlopen({"error": "Invalid username or password."}):
|
||||
with self.assertRaises(hub.HubError):
|
||||
hub.files("ggml-org/whisper.cpp")
|
||||
|
||||
|
||||
class Repos(DikteTest):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.patch_attr(hub, "CACHE_DIR", self.path("cache"))
|
||||
|
||||
def test_it_asks_for_one_author_and_for_gguf(self):
|
||||
with fake_urlopen(MODELS) as calls:
|
||||
found = hub.repos(author="ggml-org")
|
||||
self.assertIn("author=ggml-org", calls[0].full_url)
|
||||
self.assertIn("filter=gguf", calls[0].full_url)
|
||||
self.assertEqual([r.id for r in found],
|
||||
["ggml-org/gemma-3-4b-it-GGUF", "ggml-org/gpt-oss-20b-GGUF"])
|
||||
|
||||
def test_a_missing_download_count_is_zero(self):
|
||||
with fake_urlopen(MODELS):
|
||||
found = hub.repos(author="ggml-org")
|
||||
self.assertEqual(found[0].downloads, 44606)
|
||||
self.assertEqual(found[1].updated, "")
|
||||
|
||||
|
||||
def os_utime(path):
|
||||
"""Backdate a cache file past its time to live."""
|
||||
import os
|
||||
import time
|
||||
old = time.time() - hub.CACHE_TTL - 60
|
||||
os.utime(path, (old, old))
|
||||
|
||||
|
||||
class CacheOnDisk(DikteTest):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.patch_attr(hub, "CACHE_DIR", self.path("cache"))
|
||||
|
||||
def test_what_is_stored_is_what_came_back(self):
|
||||
with fake_urlopen(RELEASE):
|
||||
hub.release("ggml-org/whisper.cpp")
|
||||
stored = [json.loads(p.read_text()) for p in self.path("cache").iterdir()]
|
||||
self.assertEqual(stored[0]["tag_name"], "v1.9.1")
|
||||
|
||||
def test_a_cache_that_cannot_be_written_is_not_a_failure(self):
|
||||
self.patch_attr(hub, "CACHE_DIR", self.path("nope", "deeper"))
|
||||
self.path("nope").write_text("a file where a directory would go")
|
||||
with fake_urlopen(RELEASE):
|
||||
tag, _ = hub.release("ggml-org/whisper.cpp")
|
||||
self.assertEqual(tag, "v1.9.1")
|
||||
+93
-2
@@ -11,6 +11,7 @@ from unittest import mock
|
||||
|
||||
from PyQt6.QtWidgets import QApplication, QMessageBox
|
||||
|
||||
import cleanup
|
||||
import config as cfg
|
||||
import hotkey
|
||||
import overlay as overlay_module
|
||||
@@ -44,11 +45,20 @@ CHANGED = {
|
||||
"groq_transcribe_model": "whisper-large-v3",
|
||||
"openrouter_transcribe_model": "openai/whisper-1",
|
||||
"cleanup_enabled": False,
|
||||
"cleanup_provider": "claude",
|
||||
"cleanup_provider": "local",
|
||||
"cleanup_model": "some/other-model",
|
||||
"cleanup_claude_model": "opus",
|
||||
"cleanup_codex_model": "gpt-5",
|
||||
"cleanup_reasoning": "high",
|
||||
"local_model": "ggml-small.bin",
|
||||
"local_gpu": False,
|
||||
"local_preload": False,
|
||||
"local_threads": 6,
|
||||
"local_llm_model": "gemma-3-4b-it-Q4_K_M.gguf",
|
||||
"local_llm_repo": "ggml-org/gemma-4-E2B-it-GGUF",
|
||||
"local_llm_gpu": False,
|
||||
"local_llm_preload": True,
|
||||
"local_llm_reasoning": "low",
|
||||
"cleanup_prompt": "Only fix the punctuation.",
|
||||
"file_cleanup_prompt": "Keep the stamps where they are.",
|
||||
"transcribe_prompt": "Paraşüt, OpenFrame",
|
||||
@@ -213,7 +223,13 @@ class Settings(DikteTest):
|
||||
window = self.window(cfg.Config())
|
||||
offered = [window.transcribe_provider.itemData(i)
|
||||
for i in range(window.transcribe_provider.count())]
|
||||
self.assertEqual(offered, list(cfg.TRANSCRIBERS))
|
||||
self.assertEqual(offered, ["local"] + list(cfg.TRANSCRIBERS))
|
||||
|
||||
def test_the_cleanup_box_offers_everyone_cleanup_py_dispatches_to(self):
|
||||
window = self.window(cfg.Config())
|
||||
offered = [window.cleanup_provider.itemData(i)
|
||||
for i in range(window.cleanup_provider.count())]
|
||||
self.assertEqual(sorted(offered), sorted(cleanup.PROVIDERS))
|
||||
|
||||
def test_the_answer_to_a_test_lands_under_the_key_it_was_asked_about(self):
|
||||
"""One signal serves all three buttons, so it carries which one asked."""
|
||||
@@ -335,3 +351,78 @@ class Overlay(DikteTest):
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
|
||||
class LocalModels(DikteTest):
|
||||
"""The download boxes, without a network and without either program."""
|
||||
|
||||
def window(self, conf):
|
||||
window = settings_ui.SettingsWindow(conf)
|
||||
self.addCleanup(window.deleteLater)
|
||||
self.addCleanup(window.close)
|
||||
return window
|
||||
|
||||
def test_it_opens_where_the_missing_model_is_fixed(self):
|
||||
# Nothing can transcribe on a fresh install, which is why this window
|
||||
# was opened at all.
|
||||
window = self.window(cfg.Config())
|
||||
self.assertEqual(window.tabs.currentIndex(), window.api_tab_index)
|
||||
|
||||
def test_it_opens_where_it_was_left_when_everything_works(self):
|
||||
conf = self.config(transcribe_provider="openai", openai_api_key="sk-test")
|
||||
self.assertEqual(self.window(conf).tabs.currentIndex(), 0)
|
||||
|
||||
def test_a_model_that_is_not_here_yet_survives_a_save(self):
|
||||
# The box is filled from what is on this disk, so a model that was
|
||||
# deleted from underneath is not in the list. Dropping it on save would
|
||||
# quietly empty the setting instead of asking for the download again.
|
||||
conf = self.config(local_model="ggml-large-v3-turbo-q5_0.bin")
|
||||
with mock.patch.object(QMessageBox, "information"):
|
||||
self.window(conf)._save()
|
||||
self.assertEqual(conf["local_model"], "ggml-large-v3-turbo-q5_0.bin")
|
||||
|
||||
def test_nothing_is_fetched_for_a_window_nobody_opened(self):
|
||||
# DikteTest closes the network, so a request would fail the test. The
|
||||
# lists are asked for when the box is shown, not when it is built.
|
||||
window = self.window(cfg.Config())
|
||||
self.assertTrue(window.local_whisper._pending)
|
||||
|
||||
def test_a_model_bigger_than_two_gigabytes_counts_up_rather_than_down(self):
|
||||
# Qt's int is C++'s 32-bit one, and a 2.3 GB model is more than fits in
|
||||
# it: the count came out the far side negative, at "-1%".
|
||||
box = self.window(cfg.Config()).local_llm
|
||||
box._downloading = True
|
||||
box._report(1_048_576, 2_489_757_856)
|
||||
_app.processEvents()
|
||||
self.assertIn("2.3 GB", box.status.text())
|
||||
self.assertNotIn("-", box.status.text())
|
||||
|
||||
def test_a_long_model_name_is_not_cut_in_half(self):
|
||||
# The list under a combo box takes the box's width and elides what does
|
||||
# not fit, in the middle: "ggml-org/Qwen....7B-Base-GGUF".
|
||||
box = self.window(cfg.Config()).local_llm
|
||||
box.repo.addItem("ggml-org/a-model-with-a-name-that-runs-on-and-on-GGUF")
|
||||
box._fit_popup(box.repo)
|
||||
view = box.repo.view()
|
||||
self.assertEqual(view.textElideMode(), settings_ui.Qt.TextElideMode.ElideNone)
|
||||
widest = max(box.repo.fontMetrics().horizontalAdvance(box.repo.itemText(row))
|
||||
for row in range(box.repo.count()))
|
||||
self.assertGreaterEqual(view.minimumWidth(), widest)
|
||||
|
||||
def test_only_the_chosen_transcriber_is_on_screen(self):
|
||||
window = self.window(self.config(transcribe_provider="openai"))
|
||||
self.assertTrue(window.stt_form.isRowVisible(window.transcribe_model_row))
|
||||
self.assertFalse(window.stt_form.isRowVisible(window.local_whisper))
|
||||
window._select_data(window.transcribe_provider, "local")
|
||||
self.assertFalse(window.stt_form.isRowVisible(window.transcribe_model_row))
|
||||
self.assertTrue(window.stt_form.isRowVisible(window.local_whisper))
|
||||
|
||||
def test_only_the_chosen_cleaner_is_on_screen(self):
|
||||
window = self.window(cfg.Config())
|
||||
self.assertTrue(window.cleanup_form.isRowVisible(window.cleanup_model_row))
|
||||
self.assertFalse(window.cleanup_form.isRowVisible(window.local_llm))
|
||||
window._select_data(window.cleanup_provider, "local")
|
||||
self.assertTrue(window.cleanup_form.isRowVisible(window.local_llm))
|
||||
self.assertFalse(window.cleanup_form.isRowVisible(window.cleanup_model_row))
|
||||
# Its own thinking box, because the two default to opposite things.
|
||||
self.assertFalse(window.cleanup_form.isRowVisible(window.cleanup_reasoning))
|
||||
|
||||
@@ -234,7 +234,7 @@ class Chain(DikteTest):
|
||||
self.assertEqual(row["raw"], "uh, book it for Thursday")
|
||||
self.assertEqual(row["text"], "Book it for Thursday.")
|
||||
self.assertEqual(row["duration"], 2.0)
|
||||
self.assertEqual(row["model"], self.conf["transcribe_model"])
|
||||
self.assertEqual(row["model"], self.conf.transcribe_target().model)
|
||||
self.assertEqual(row["mode"], "")
|
||||
|
||||
def test_a_command_is_recorded_as_one(self):
|
||||
|
||||
Reference in New Issue
Block a user