Merge master: the shortcut table, local models, and a Mac still in them
Four files disagreed, and all four the same way: master had turned things the Mac branch wrote out by hand into one list to read from. Shortcuts are the whole of it. master gave every binding a row in hotkey.SHORTCUTS, so the Mac's DESKTOP_IDS is gone and CarbonHotkey reads the desktop id off that row, which also gives the new cancel key a status line on a Mac. Settings builds its four rows through master's _shortcut_row, and that one now asks _install_buttons for Install and Remove, so macOS gets a combination box and nothing to press, and everywhere else the button says the desktop's own name. dikte.py starts the listener from the same table, on macOS whatever the setting says: there is nothing installed for it to be a fallback to. The rest is two imports and a paste list that lives in paste.Desktop now.
@@ -62,7 +62,7 @@ inside the chooser and read it there every time: a constant settled at import is
|
|||||||
one no test can stand somewhere else.
|
one no test can stand somewhere else.
|
||||||
|
|
||||||
The tests are split along the same line, and almost none of them are skipped.
|
The tests are split along the same line, and almost none of them are skipped.
|
||||||
697 of the 737 run on any machine, including every line of the Wayland, X11 and
|
892 of the 935 run on any machine, including every line of the Wayland, X11 and
|
||||||
macOS backends: the programs are faked at `shutil.which`, the frameworks at the
|
macOS backends: the programs are faked at `shutil.which`, the frameworks at the
|
||||||
one function that loads them. A test class says which system it is standing on
|
one function that loads them. A test class says which system it is standing on
|
||||||
rather than avoiding the question:
|
rather than avoiding the question:
|
||||||
@@ -78,17 +78,17 @@ to a chooser cannot quietly break the platform nobody is sitting at. What the
|
|||||||
systems owe in common is written once as a contract class and subclassed by each
|
systems owe in common is written once as a contract class and subclassed by each
|
||||||
of them.
|
of them.
|
||||||
|
|
||||||
The 40 that do carry `@linux_only` are the ones that would need the real thing:
|
The 43 that do carry `@linux_only` are the ones that would need the real thing:
|
||||||
the `/dev/input` listener, KDE's shortcut file, GNOME's gsettings. Mark a test
|
the `/dev/input` listener, KDE's shortcut file, GNOME's gsettings. Mark a test
|
||||||
that way only when faking it would leave nothing to test. A test that quietly
|
that way only when faking it would leave nothing to test. A test that quietly
|
||||||
stops running on the platform you are porting to protects nothing.
|
stops running on the platform you are porting to protects nothing.
|
||||||
|
|
||||||
## What a pull request should carry
|
## What a pull request should carry
|
||||||
|
|
||||||
A change to behaviour comes with a test for it. Adding a provider means a test
|
A change to behaviour comes with a test for it. Adding a provider means a row in
|
||||||
that the request goes to the right URL with the right fields; adding a platform
|
`config.TRANSCRIBERS` and a test that the request goes to the right URL with the
|
||||||
means a test for whatever the parsing of its device list, clipboard or shortcuts
|
right fields; adding a platform means a test for whatever the parsing of its
|
||||||
looks like. Adding a setting means both halves of `settings_ui.py`: the round
|
device list, clipboard or shortcuts looks like. Adding a setting means both halves of `settings_ui.py`: the round
|
||||||
trip in `tests/test_ui.py` is what catches only one of them being written.
|
trip in `tests/test_ui.py` is what catches only one of them being written.
|
||||||
|
|
||||||
Match the surrounding code: it is plain Python with no framework, comments
|
Match the surrounding code: it is plain Python with no framework, comments
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
# Dikte
|
# Dikte
|
||||||
|
|
||||||
Press `Ctrl+Space`, talk, press again. The recording goes to OpenAI or OpenRouter
|
Press `Ctrl+Space`, talk, press again. The recording is transcribed on this
|
||||||
for transcription, a model on OpenRouter cleans it up (dropping the *uh*s, the
|
machine by default, a model cleans it up (dropping the *uh*s, the restarts, the
|
||||||
restarts, the missing punctuation), and the result lands in your clipboard and
|
missing punctuation), and the result lands in your clipboard and is pasted into
|
||||||
is pasted into whatever window you were typing in.
|
whatever window you were typing in.
|
||||||
|
|
||||||
Built for KDE Plasma 6 on Wayland. No dependencies beyond system packages:
|
Built for KDE Plasma 6 on Wayland. No dependencies beyond system packages:
|
||||||
just the Python standard library and PyQt6.
|
just the Python standard library and PyQt6.
|
||||||
@@ -17,7 +17,8 @@ just the Python standard library and PyQt6.
|
|||||||
| | |
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
| <img src="docs/settings-api.webp" width="410" alt="API and models"> | <img src="docs/settings-cleanup.webp" width="410" alt="Cleanup rules"> |
|
| <img src="docs/settings-api.webp" width="410" alt="API and models"> | <img src="docs/settings-cleanup.webp" width="410" alt="Cleanup rules"> |
|
||||||
| <img src="docs/settings-audio-file.webp" width="410" alt="Audio file"> | <img src="docs/settings-history.webp" width="410" alt="History"> |
|
| <img src="docs/settings-agent.webp" width="410" alt="Agent"> | <img src="docs/settings-meeting.webp" width="410" alt="Meeting"> |
|
||||||
|
| <img src="docs/settings-audio-file.webp" width="410" alt="Audio file"> | <img src="docs/settings-shortcuts.webp" width="410" alt="Shortcuts"> |
|
||||||
|
|
||||||
## Install
|
## Install
|
||||||
|
|
||||||
@@ -25,7 +26,7 @@ just the Python standard library and PyQt6.
|
|||||||
sudo pacman -S --needed pipewire-audio wl-clipboard ydotool ffmpeg python-pyqt6
|
sudo pacman -S --needed pipewire-audio wl-clipboard ydotool ffmpeg python-pyqt6
|
||||||
systemctl --user enable --now ydotool # needed for auto-paste
|
systemctl --user enable --now ydotool # needed for auto-paste
|
||||||
|
|
||||||
./install.sh # or: ./install.sh "Ctrl+Alt+Space"
|
./install.sh # or: ./install.sh "Meta+Space" "Meta+Shift+Space"
|
||||||
dikte # the settings window opens on first run
|
dikte # the settings window opens on first run
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -36,23 +37,27 @@ tools instead:
|
|||||||
sudo apt install pulseaudio-utils xclip xdotool ffmpeg
|
sudo apt install pulseaudio-utils xclip xdotool ffmpeg
|
||||||
```
|
```
|
||||||
|
|
||||||
`install.sh` adds the `dikte` command, a menu entry and an autostart entry. The
|
`install.sh` adds the `dikte` command, a menu entry, an autostart entry and the
|
||||||
settings window installs a GNOME or KDE global shortcut.
|
two global shortcuts, whose keys are its two arguments. `./update.sh` pulls and
|
||||||
|
puts all of that back, keeping the keys you chose; `./uninstall.sh` takes it away
|
||||||
|
again and leaves your settings and dictations alone unless you pass `--purge`.
|
||||||
|
|
||||||
Two keys go in the settings window: **OpenAI** and **OpenRouter**. Speech to text
|
Speech to text and cleanup each pick a provider in the settings window, and both
|
||||||
runs on either one (`gpt-4o-transcribe` by default), cleanup always on
|
run here by default, on models of your own. The cloud is the other option:
|
||||||
OpenRouter (`google/gemini-3.5-flash-lite`), so a single OpenRouter key can
|
speech to text on **OpenAI**, **Groq** or **OpenRouter** (`gpt-4o-transcribe`),
|
||||||
cover both. They fall back to `OPENAI_API_KEY` and `OPENROUTER_API_KEY`, and are
|
cleanup on OpenRouter (`google/gemini-3.5-flash-lite`) or, when either is
|
||||||
stored in `~/.config/dikte/config.json`, mode 600. Cleanup can be switched off,
|
installed, on Claude Code or Codex. The keys fall back to `OPENAI_API_KEY`,
|
||||||
in which case the raw transcript is pasted, and a thinking model's effort can be
|
`GROQ_API_KEY` and `OPENROUTER_API_KEY`, and are stored in
|
||||||
set next to it.
|
`~/.config/dikte/config.json`, mode 600. Cleanup can be switched off, in which
|
||||||
|
case the raw transcript is pasted, and a thinking model's effort can be set next
|
||||||
|
to it.
|
||||||
|
|
||||||
## Using it
|
## Using it
|
||||||
|
|
||||||
| What | How |
|
| What | How |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| Start / stop recording | `Ctrl+Space`, or click the tray icon |
|
| Start / stop recording | `Ctrl+Space`, or click the tray icon |
|
||||||
| Cancel a recording | Tray menu → *Cancel recording*, or `dikte cancel` |
|
| Discard the recording | `Ctrl+Alt+Space`, tray menu, or `dikte cancel` |
|
||||||
| Speak a command to an agent | Tray menu → *Ask Claude*, or `dikte ask` |
|
| Speak a command to an agent | Tray menu → *Ask Claude*, or `dikte ask` |
|
||||||
| Start / end a meeting | Tray menu → *Record a meeting*, or `dikte meeting` |
|
| Start / end a meeting | Tray menu → *Record a meeting*, or `dikte meeting` |
|
||||||
| Settings | Tray menu → *Settings*, or `dikte settings` |
|
| Settings | Tray menu → *Settings*, or `dikte settings` |
|
||||||
@@ -75,6 +80,12 @@ running.
|
|||||||
|
|
||||||
## What it does
|
## What it does
|
||||||
|
|
||||||
|
- **It all runs on this machine by default.** Speech to text on whisper.cpp and
|
||||||
|
cleanup on llama.cpp, neither installed beforehand: the settings window fetches
|
||||||
|
the program and the model, verifies the sha256 and refuses a download published
|
||||||
|
without one, then keeps a server alive while you dictate. The graphics card is
|
||||||
|
reached through CUDA, ROCm or Vulkan where the build allows. No key, no
|
||||||
|
account, nothing leaving the machine.
|
||||||
- **Silence never reaches the API.** Handed near-silence, a transcription model
|
- **Silence never reaches the API.** Handed near-silence, a transcription model
|
||||||
invents a sentence instead of returning nothing ("Thanks for watching", or in
|
invents a sentence instead of returning nothing ("Thanks for watching", or in
|
||||||
Turkish "Altyazı M.K."). A recording is dropped when nothing rose 10 dB above
|
Turkish "Altyazı M.K."). A recording is dropped when nothing rose 10 dB above
|
||||||
@@ -126,11 +137,11 @@ running.
|
|||||||
right-click to delete.
|
right-click to delete.
|
||||||
- **Turkish and English interface**, following the system locale by default.
|
- **Turkish and English interface**, following the system locale by default.
|
||||||
|
|
||||||
## The global shortcut needs one logout
|
## The global shortcuts need one logout
|
||||||
|
|
||||||
KWin only reads `kglobalshortcutsrc` at startup, so the shortcut `install.sh`
|
KWin only reads `kglobalshortcutsrc` at startup, so the shortcuts `install.sh`
|
||||||
writes will not fire until you log out and back in. Until then, Settings →
|
writes will not fire until you log out and back in. Until then, Settings →
|
||||||
Shortcut → **built-in listener** reads `/dev/input` and catches the combination
|
Shortcuts → **built-in listener** reads `/dev/input` and catches the combination
|
||||||
itself. The difference: it does not swallow the key, so `Ctrl+Space` also reaches
|
itself. The difference: it does not swallow the key, so `Ctrl+Space` also reaches
|
||||||
the focused application (some editors will pop up autocomplete). The listener
|
the focused application (some editors will pop up autocomplete). The listener
|
||||||
needs your user in the `input` group: `sudo usermod -aG input $USER`.
|
needs your user in the `input` group: `sudo usermod -aG input $USER`.
|
||||||
@@ -144,7 +155,10 @@ ipc.py one request and one reply over the local socket
|
|||||||
audio.py PCM capture: pw-record for dictation, ffmpeg for a meeting
|
audio.py PCM capture: pw-record for dictation, ffmpeg for a meeting
|
||||||
meeting.py channel split, speaker labelling, cleanup, minutes
|
meeting.py channel split, speaker labelling, cleanup, minutes
|
||||||
assistant.py running a dictation through Claude Code, Codex or OpenRouter
|
assistant.py running a dictation through Claude Code, Codex or OpenRouter
|
||||||
api.py transcription on either provider, OpenRouter cleanup (stdlib only)
|
api.py transcription and cleanup requests (stdlib only)
|
||||||
|
cleanup.py who rewrites the transcript: OpenRouter, here, Claude or Codex
|
||||||
|
ggml.py whisper.cpp and llama.cpp here: fetch, verify, keep serving
|
||||||
|
hub.py what GitHub and Hugging Face have on offer today
|
||||||
worker.py transcribe → clean up → clipboard → paste
|
worker.py transcribe → clean up → clipboard → paste
|
||||||
vad.py deciding whether a recording holds speech at all
|
vad.py deciding whether a recording holds speech at all
|
||||||
filetranscribe.py file transcription: ffmpeg, chunking, timestamps
|
filetranscribe.py file transcription: ffmpeg, chunking, timestamps
|
||||||
|
|||||||
@@ -1,9 +1,8 @@
|
|||||||
# Dikte
|
# Dikte
|
||||||
|
|
||||||
`Ctrl+Space`'e bas, konuş, tekrar bas. Ses OpenAI'ye ya da OpenRouter'a gidip
|
`Ctrl+Space`'e bas, konuş, tekrar bas. Ses varsayılan olarak bu makinede yazıya
|
||||||
yazıya çevrilir, OpenRouter'daki bir model transkripti temizler (ıı'lar,
|
çevrilir, bir model transkripti temizler (ıı'lar, tekrarlar, eksik noktalama),
|
||||||
tekrarlar, eksik noktalama), sonuç panoya kopyalanır ve o an yazdığın pencereye
|
sonuç panoya kopyalanır ve o an yazdığın pencereye yapıştırılır.
|
||||||
yapıştırılır.
|
|
||||||
|
|
||||||
KDE Plasma 6 / Wayland için yazıldı. Sistem paketleri dışında bağımlılığı yok:
|
KDE Plasma 6 / Wayland için yazıldı. Sistem paketleri dışında bağımlılığı yok:
|
||||||
sadece Python standart kütüphanesi ve PyQt6.
|
sadece Python standart kütüphanesi ve PyQt6.
|
||||||
@@ -17,7 +16,8 @@ sadece Python standart kütüphanesi ve PyQt6.
|
|||||||
| | |
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
| <img src="docs/settings-api.webp" width="410" alt="API ve modeller"> | <img src="docs/settings-cleanup.webp" width="410" alt="Temizleme kuralları"> |
|
| <img src="docs/settings-api.webp" width="410" alt="API ve modeller"> | <img src="docs/settings-cleanup.webp" width="410" alt="Temizleme kuralları"> |
|
||||||
| <img src="docs/settings-audio-file.webp" width="410" alt="Ses dosyası"> | <img src="docs/settings-history.webp" width="410" alt="Geçmiş"> |
|
| <img src="docs/settings-agent.webp" width="410" alt="Ajan"> | <img src="docs/settings-meeting.webp" width="410" alt="Toplantı"> |
|
||||||
|
| <img src="docs/settings-audio-file.webp" width="410" alt="Ses dosyası"> | <img src="docs/settings-shortcuts.webp" width="410" alt="Kısayollar"> |
|
||||||
|
|
||||||
## Kurulum
|
## Kurulum
|
||||||
|
|
||||||
@@ -25,7 +25,7 @@ sadece Python standart kütüphanesi ve PyQt6.
|
|||||||
sudo pacman -S --needed pipewire-audio wl-clipboard ydotool ffmpeg python-pyqt6
|
sudo pacman -S --needed pipewire-audio wl-clipboard ydotool ffmpeg python-pyqt6
|
||||||
systemctl --user enable --now ydotool # otomatik yapıştırma için
|
systemctl --user enable --now ydotool # otomatik yapıştırma için
|
||||||
|
|
||||||
./install.sh # ya da: ./install.sh "Ctrl+Alt+Space"
|
./install.sh # ya da: ./install.sh "Meta+Space" "Meta+Shift+Space"
|
||||||
dikte # ilk açılışta ayarlar penceresi gelir
|
dikte # ilk açılışta ayarlar penceresi gelir
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -36,13 +36,18 @@ araçlarıyla çalışır:
|
|||||||
sudo apt install pulseaudio-utils xclip xdotool ffmpeg
|
sudo apt install pulseaudio-utils xclip xdotool ffmpeg
|
||||||
```
|
```
|
||||||
|
|
||||||
`install.sh` `dikte` komutunu, menü girdisini ve oturum açılışında otomatik
|
`install.sh` `dikte` komutunu, menü girdisini, oturum açılışında otomatik
|
||||||
başlatmayı kurar. Ayarlar penceresi GNOME veya KDE global kısayolunu kurar.
|
başlatmayı ve iki global kısayolu kurar; tuşları da iki argümanı. `./update.sh`
|
||||||
|
son sürümü çeker ve bunları senin seçtiğin tuşlarla yerine koyar;
|
||||||
|
`./uninstall.sh` hepsini geri alır, `--purge` demedikçe ayarlarına ve
|
||||||
|
diktelerine dokunmaz.
|
||||||
|
|
||||||
Ayarlar penceresinde iki anahtar istenir: **OpenAI** ve **OpenRouter**. Sesi
|
Sesi yazıya çevirme ve temizleme, ayarlar penceresinde ayrı ayrı sağlayıcı
|
||||||
yazıya çevirme ikisinden birinde çalışır (varsayılan `gpt-4o-transcribe`),
|
seçer; ikisi de varsayılan olarak burada, kendi modellerinle çalışır. Bulutu
|
||||||
temizleme her zaman OpenRouter'da (`google/gemini-3.5-flash-lite`), yani tek bir
|
seçersen sesi yazıya çevirme **OpenAI**, **Groq** ya da **OpenRouter**'da
|
||||||
OpenRouter anahtarı ikisine de yeter. Boş bırakırsan `OPENAI_API_KEY` ve
|
(varsayılan `gpt-4o-transcribe`), temizleme OpenRouter'da
|
||||||
|
(`google/gemini-3.5-flash-lite`) ya da kuruluysa Claude Code veya Codex'te
|
||||||
|
çalışır. Anahtarları boş bırakırsan `OPENAI_API_KEY`, `GROQ_API_KEY` ve
|
||||||
`OPENROUTER_API_KEY` kullanılır; anahtarlar `~/.config/dikte/config.json`
|
`OPENROUTER_API_KEY` kullanılır; anahtarlar `~/.config/dikte/config.json`
|
||||||
içinde, izinler 600. Temizlemeyi tamamen kapatabilirsin, o zaman ham transkript
|
içinde, izinler 600. Temizlemeyi tamamen kapatabilirsin, o zaman ham transkript
|
||||||
yapıştırılır; modelin yanındaki kutudan düşünme seviyesini de seçebilirsin.
|
yapıştırılır; modelin yanındaki kutudan düşünme seviyesini de seçebilirsin.
|
||||||
@@ -52,7 +57,7 @@ yapıştırılır; modelin yanındaki kutudan düşünme seviyesini de seçebili
|
|||||||
| Ne | Nasıl |
|
| Ne | Nasıl |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| Kaydı başlat / bitir | `Ctrl+Space`, ya da tepsi simgesine tıkla |
|
| Kaydı başlat / bitir | `Ctrl+Space`, ya da tepsi simgesine tıkla |
|
||||||
| Kaydı iptal et | Tepsi menüsü → *Kaydı iptal et*, ya da `dikte cancel` |
|
| Kaydı iptal et | `Ctrl+Alt+Space`, tepsi menüsü, ya da `dikte cancel` |
|
||||||
| Ajana sesle komut ver | Tepsi menüsü → *Claude'a sor*, ya da `dikte ask` |
|
| Ajana sesle komut ver | Tepsi menüsü → *Claude'a sor*, ya da `dikte ask` |
|
||||||
| Toplantıyı başlat / bitir | Tepsi menüsü → *Toplantı kaydet*, ya da `dikte meeting` |
|
| Toplantıyı başlat / bitir | Tepsi menüsü → *Toplantı kaydet*, ya da `dikte meeting` |
|
||||||
| Ayarlar | Tepsi menüsü → *Ayarlar*, ya da `dikte settings` |
|
| Ayarlar | Tepsi menüsü → *Ayarlar*, ya da `dikte settings` |
|
||||||
@@ -75,6 +80,12 @@ olmasını ister.
|
|||||||
|
|
||||||
## Neler yapıyor
|
## Neler yapıyor
|
||||||
|
|
||||||
|
- **Her şey varsayılan olarak bu makinede çalışır.** Sesi yazıya çevirme
|
||||||
|
whisper.cpp, temizleme llama.cpp üzerinde; ikisini de önceden kurman gerekmez:
|
||||||
|
ayarlar penceresi programı ve modeli indirir, sha256'sını doğrular,
|
||||||
|
checksum'suz yayınlanmış bir indirmeyi reddeder, sen dikte ettikçe sunucuyu
|
||||||
|
ayakta tutar. Derleme destekliyorsa ekran kartına CUDA, ROCm ya da Vulkan
|
||||||
|
üzerinden ulaşılır. Anahtar yok, hesap yok, makineden çıkan bir şey yok.
|
||||||
- **Sessizlik API'ye gitmez.** Sessize yakın bir ses verildiğinde model boş dize
|
- **Sessizlik API'ye gitmez.** Sessize yakın bir ses verildiğinde model boş dize
|
||||||
döndürmez, bir cümle uydurur ("Altyazı M.K.", "Thanks for watching"). *O
|
döndürmez, bir cümle uydurur ("Altyazı M.K.", "Thanks for watching"). *O
|
||||||
kaydın kendi* gürültü tabanının 10 dB üstüne en az 0,3 saniye çıkan bir şey
|
kaydın kendi* gürültü tabanının 10 dB üstüne en az 0,3 saniye çıkan bir şey
|
||||||
@@ -124,11 +135,11 @@ olmasını ister.
|
|||||||
silebilirsin.
|
silebilirsin.
|
||||||
- **Türkçe ve İngilizce arayüz**, varsayılan olarak sistem dilini izler.
|
- **Türkçe ve İngilizce arayüz**, varsayılan olarak sistem dilini izler.
|
||||||
|
|
||||||
## Global kısayol için bir kez oturum kapatmak gerekir
|
## Global kısayollar için bir kez oturum kapatmak gerekir
|
||||||
|
|
||||||
KWin `kglobalshortcutsrc` dosyasını yalnızca açılışta okur, yani `install.sh`'ın
|
KWin `kglobalshortcutsrc` dosyasını yalnızca açılışta okur, yani `install.sh`'ın
|
||||||
yazdığı kısayol oturumu yeniden açana kadar tetiklenmez. O zamana kadar Ayarlar →
|
yazdığı kısayollar oturumu yeniden açana kadar tetiklenmez. O zamana kadar Ayarlar →
|
||||||
Kısayol → **yerleşik dinleyici** `/dev/input` üzerinden kombinasyonu kendisi
|
Kısayollar → **yerleşik dinleyici** `/dev/input` üzerinden kombinasyonu kendisi
|
||||||
yakalar. Tek farkı: tuşu yutmaz, yani `Ctrl+Space` odaktaki uygulamaya da iletilir
|
yakalar. Tek farkı: tuşu yutmaz, yani `Ctrl+Space` odaktaki uygulamaya da iletilir
|
||||||
(bazı editörlerde otomatik tamamlama açılabilir). Dinleyici kullanıcının `input`
|
(bazı editörlerde otomatik tamamlama açılabilir). Dinleyici kullanıcının `input`
|
||||||
grubunda olmasını gerektirir: `sudo usermod -aG input $USER`.
|
grubunda olmasını gerektirir: `sudo usermod -aG input $USER`.
|
||||||
@@ -142,7 +153,10 @@ ipc.py yerel sokette bir istek, bir cevap
|
|||||||
audio.py PCM kaydı: diktede pw-record, toplantıda ffmpeg
|
audio.py PCM kaydı: diktede pw-record, toplantıda ffmpeg
|
||||||
meeting.py kanal ayırma, konuşmacı etiketi, temizleme, tutanak
|
meeting.py kanal ayırma, konuşmacı etiketi, temizleme, tutanak
|
||||||
assistant.py dikteyi Claude Code, Codex ya da OpenRouter'dan geçirme
|
assistant.py dikteyi Claude Code, Codex ya da OpenRouter'dan geçirme
|
||||||
api.py iki sağlayıcıda transkript + OpenRouter temizleme (yalnız stdlib)
|
api.py transkript ve temizleme istekleri (yalnız stdlib)
|
||||||
|
cleanup.py transkripti kim temizler: OpenRouter, burası, Claude ya da Codex
|
||||||
|
ggml.py whisper.cpp ve llama.cpp'yi indirip burada çalıştırma
|
||||||
|
hub.py GitHub ve Hugging Face'te bugün ne olduğu
|
||||||
worker.py transkript → temizleme → pano → yapıştırma
|
worker.py transkript → temizleme → pano → yapıştırma
|
||||||
vad.py kayıtta gerçekten konuşma var mı kararı
|
vad.py kayıtta gerçekten konuşma var mı kararı
|
||||||
filetranscribe.py dosyadan transkript: ffmpeg, parçalama, zaman damgaları
|
filetranscribe.py dosyadan transkript: ffmpeg, parçalama, zaman damgaları
|
||||||
|
|||||||
@@ -1,34 +1,59 @@
|
|||||||
"""OpenAI and OpenRouter calls, stdlib only.
|
"""OpenAI, Groq, OpenRouter and this machine, stdlib only.
|
||||||
|
|
||||||
Transcription runs on either provider: OpenRouter mirrors OpenAI's
|
Transcription runs on any of the four: Groq and OpenRouter both mirror OpenAI's
|
||||||
/audio/transcriptions endpoint field for field, so one multipart request serves
|
/audio/transcriptions endpoint field for field, and ggml.py starts whisper.cpp
|
||||||
both and only the key, the base URL and the model id change. Cleanup is always
|
on that same path, so one multipart request serves all of them and only the key,
|
||||||
OpenRouter.
|
the base URL and the model id change. llama.cpp answers /chat/completions the way
|
||||||
|
OpenRouter does, so cleanup here is the same request too.
|
||||||
|
|
||||||
|
What is on this machine has no key, and its base URL is not known until a server
|
||||||
|
is up, which is the one thing this module has to fill in for it.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import collections
|
import collections
|
||||||
|
import contextlib
|
||||||
|
import http.client
|
||||||
import json
|
import json
|
||||||
import mimetypes
|
import mimetypes
|
||||||
import os
|
import os
|
||||||
import secrets
|
import secrets
|
||||||
|
import socket
|
||||||
|
import threading
|
||||||
import urllib.error
|
import urllib.error
|
||||||
import urllib.request
|
import urllib.request
|
||||||
|
|
||||||
|
import ggml
|
||||||
from i18n import t
|
from i18n import t
|
||||||
|
|
||||||
APP_URL = "https://github.com/yusufipk/dikte"
|
APP_URL = "https://github.com/yusufipk/dikte"
|
||||||
USER_AGENT = f"dikte/1.0 (+{APP_URL})"
|
USER_AGENT = f"dikte/1.0 (+{APP_URL})"
|
||||||
OPENAI_URL = "https://api.openai.com/v1"
|
OPENAI_URL = "https://api.openai.com/v1"
|
||||||
|
GROQ_URL = "https://api.groq.com/openai/v1"
|
||||||
OPENROUTER_URL = "https://openrouter.ai/api/v1"
|
OPENROUTER_URL = "https://openrouter.ai/api/v1"
|
||||||
|
|
||||||
|
# The floor for a local request. The timeouts elsewhere are sized for a hosted
|
||||||
|
# API, where a slow answer is a bill running; here the only thing being spent is
|
||||||
|
# time, and a long recording on a machine without a graphics card takes a good
|
||||||
|
# deal of it. Cutting that off would throw the work away for nothing.
|
||||||
|
LOCAL_TIMEOUT = 3600
|
||||||
|
|
||||||
# Where a transcription request goes; built by config.Config.transcribe_target().
|
# Where a transcription request goes; built by config.Config.transcribe_target().
|
||||||
# `service` is the name the user sees in an error, `provider` the one the code
|
# `service` is the name the user sees in an error, `provider` the one the code
|
||||||
# branches on.
|
# branches on.
|
||||||
Target = collections.namedtuple("Target", "provider service api_key base_url model")
|
Target = collections.namedtuple("Target", "provider service api_key base_url model")
|
||||||
|
|
||||||
|
|
||||||
def timestamp_model(provider):
|
def timestamp_model(provider, selected=""):
|
||||||
"""Only whisper-1 returns segment times, and OpenRouter namespaces the id."""
|
"""Which model answers with segment times.
|
||||||
|
|
||||||
|
OpenAI keeps them to whisper-1 and OpenRouter namespaces that id. Everything
|
||||||
|
Groq transcribes with is a whisper, so the model already chosen does it and
|
||||||
|
the fallback is only for a provider left on its default. So is everything the
|
||||||
|
local server runs, whatever the file is called, and there asking for another
|
||||||
|
model would name one it has never heard of.
|
||||||
|
"""
|
||||||
|
if provider in ("groq", "local"):
|
||||||
|
return selected or "whisper-large-v3-turbo"
|
||||||
return "openai/whisper-1" if provider == "openrouter" else "whisper-1"
|
return "openai/whisper-1" if provider == "openrouter" else "whisper-1"
|
||||||
|
|
||||||
|
|
||||||
@@ -38,6 +63,142 @@ class ApiError(Exception):
|
|||||||
self.status = status
|
self.status = status
|
||||||
|
|
||||||
|
|
||||||
|
class Aborted(Exception):
|
||||||
|
"""A request that was cut off from another thread rather than answered."""
|
||||||
|
|
||||||
|
|
||||||
|
class Aborter:
|
||||||
|
"""A Stop button that reaches the call a worker thread is blocked inside.
|
||||||
|
|
||||||
|
urlopen() hands nothing back until the server has answered, and a whisper on
|
||||||
|
this machine is minutes away from answering, so a flag read between calls is
|
||||||
|
a Stop that does nothing until the work it was meant to stop is already
|
||||||
|
done. What is registered here is cut off where it stands instead.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self._cancels = []
|
||||||
|
self.aborted = False
|
||||||
|
|
||||||
|
def abort(self):
|
||||||
|
with self._lock:
|
||||||
|
self.aborted = True
|
||||||
|
pending, self._cancels = self._cancels, []
|
||||||
|
for cancel in pending:
|
||||||
|
cancel()
|
||||||
|
|
||||||
|
def check(self):
|
||||||
|
if self.aborted:
|
||||||
|
raise Aborted
|
||||||
|
|
||||||
|
@contextlib.contextmanager
|
||||||
|
def holding(self, cancel):
|
||||||
|
"""Run `cancel` if an abort lands while this block is open."""
|
||||||
|
with self._lock:
|
||||||
|
if self.aborted:
|
||||||
|
raise Aborted
|
||||||
|
self._cancels.append(cancel)
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
with self._lock:
|
||||||
|
with contextlib.suppress(ValueError):
|
||||||
|
self._cancels.remove(cancel)
|
||||||
|
|
||||||
|
|
||||||
|
class _Sockets:
|
||||||
|
"""The connections one request is using, and whether it may still use any.
|
||||||
|
|
||||||
|
A stop can land at any point of the handful of lines urllib takes to get
|
||||||
|
from "make a connection" to "wait for the reply", so this keeps the two
|
||||||
|
halves of the answer together: what is already open is cut, and anything
|
||||||
|
opened after that is refused rather than quietly left to block.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self._conns = []
|
||||||
|
self._cut = False
|
||||||
|
|
||||||
|
def add(self, conn):
|
||||||
|
with self._lock:
|
||||||
|
if self._cut:
|
||||||
|
raise Aborted
|
||||||
|
self._conns.append(conn)
|
||||||
|
|
||||||
|
def cut(self):
|
||||||
|
with self._lock:
|
||||||
|
self._cut = True
|
||||||
|
conns = list(self._conns)
|
||||||
|
for conn in conns:
|
||||||
|
_stop_using(conn)
|
||||||
|
|
||||||
|
|
||||||
|
def _stop_using(conn):
|
||||||
|
"""Take a connection out of use, connected or not.
|
||||||
|
|
||||||
|
A connection whose socket is not open yet would open one on the next line,
|
||||||
|
so the reconnect is turned off first. One that is open is being read from,
|
||||||
|
and close() alone leaves that read waiting for bytes which are never coming
|
||||||
|
now; the shutdown is what makes it return.
|
||||||
|
"""
|
||||||
|
conn.auto_open = 0
|
||||||
|
sock = getattr(conn, "sock", None)
|
||||||
|
if sock is not None:
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
sock.shutdown(socket.SHUT_RDWR)
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
class _TrackedHTTP(urllib.request.HTTPHandler):
|
||||||
|
"""urllib's own handler, handing the connection it opens to `sockets`.
|
||||||
|
|
||||||
|
That connection is what a Stop is applied to, and urlopen() makes it out of
|
||||||
|
sight, inside the call that is about to block on it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, sockets):
|
||||||
|
super().__init__()
|
||||||
|
self._sockets = sockets
|
||||||
|
|
||||||
|
def http_open(self, req):
|
||||||
|
return self.do_open(self._connect, req)
|
||||||
|
|
||||||
|
def _connect(self, host, **kwargs):
|
||||||
|
conn = http.client.HTTPConnection(host, **kwargs)
|
||||||
|
self._sockets.add(conn)
|
||||||
|
return conn
|
||||||
|
|
||||||
|
|
||||||
|
class _TrackedHTTPS(urllib.request.HTTPSHandler):
|
||||||
|
def __init__(self, sockets):
|
||||||
|
super().__init__()
|
||||||
|
self._sockets = sockets
|
||||||
|
|
||||||
|
def https_open(self, req):
|
||||||
|
return self.do_open(self._connect, req, context=self._context)
|
||||||
|
|
||||||
|
def _connect(self, host, **kwargs):
|
||||||
|
conn = http.client.HTTPSConnection(host, **kwargs)
|
||||||
|
self._sockets.add(conn)
|
||||||
|
return conn
|
||||||
|
|
||||||
|
|
||||||
|
@contextlib.contextmanager
|
||||||
|
def _opened(req, timeout, aborter):
|
||||||
|
"""The response, left where `aborter` can cut it off."""
|
||||||
|
if aborter is None:
|
||||||
|
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||||
|
yield resp
|
||||||
|
return
|
||||||
|
sockets = _Sockets()
|
||||||
|
opener = urllib.request.build_opener(_TrackedHTTP(sockets), _TrackedHTTPS(sockets))
|
||||||
|
with aborter.holding(sockets.cut), opener.open(req, timeout=timeout) as resp:
|
||||||
|
yield resp
|
||||||
|
|
||||||
|
|
||||||
def explain(exc, service):
|
def explain(exc, service):
|
||||||
"""Turn an HTTP status into something the user can act on."""
|
"""Turn an HTTP status into something the user can act on."""
|
||||||
if exc.status in (401, 403):
|
if exc.status in (401, 403):
|
||||||
@@ -53,16 +214,21 @@ def explain(exc, service):
|
|||||||
return ApiError(f"{service}: {exc}", exc.status)
|
return ApiError(f"{service}: {exc}", exc.status)
|
||||||
|
|
||||||
|
|
||||||
def _request(url, data, headers, timeout=120):
|
def _request(url, data, headers, timeout=120, aborter=None):
|
||||||
req = urllib.request.Request(url, data=data, headers=headers, method="POST")
|
req = urllib.request.Request(url, data=data, headers=headers, method="POST")
|
||||||
try:
|
try:
|
||||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
with _opened(req, timeout, aborter) as resp:
|
||||||
return json.loads(resp.read().decode("utf-8"))
|
return json.loads(resp.read().decode("utf-8"))
|
||||||
except urllib.error.HTTPError as exc:
|
except urllib.error.HTTPError as exc:
|
||||||
body = exc.read().decode("utf-8", "replace")
|
body = exc.read().decode("utf-8", "replace")
|
||||||
raise ApiError(f"HTTP {exc.code}: {_extract_error(body)}", exc.code) from exc
|
raise ApiError(f"HTTP {exc.code}: {_extract_error(body)}", exc.code) from exc
|
||||||
except urllib.error.URLError as exc:
|
except (OSError, http.client.HTTPException) as exc:
|
||||||
raise ApiError(t("Could not connect: {reason}", reason=exc.reason)) from exc
|
# A socket that went out from under the read is this run being stopped,
|
||||||
|
# not the network failing. URLError is an OSError, so both land here.
|
||||||
|
if aborter is not None and aborter.aborted:
|
||||||
|
raise Aborted from None
|
||||||
|
raise ApiError(t("Could not connect: {reason}",
|
||||||
|
reason=getattr(exc, "reason", exc))) from exc
|
||||||
except json.JSONDecodeError as exc:
|
except json.JSONDecodeError as exc:
|
||||||
raise ApiError(t("Could not parse the response: {error}", error=exc)) from exc
|
raise ApiError(t("Could not parse the response: {error}", error=exc)) from exc
|
||||||
|
|
||||||
@@ -106,7 +272,11 @@ def _multipart(fields, file_field, file_path):
|
|||||||
|
|
||||||
|
|
||||||
def _headers(provider, api_key, content_type=None):
|
def _headers(provider, api_key, content_type=None):
|
||||||
headers = {"Authorization": f"Bearer {api_key}", "User-Agent": USER_AGENT}
|
headers = {"User-Agent": USER_AGENT}
|
||||||
|
# A server on this machine has nothing to authorise, and sending it a
|
||||||
|
# bearer token would only be a made-up one.
|
||||||
|
if api_key:
|
||||||
|
headers["Authorization"] = f"Bearer {api_key}"
|
||||||
if content_type:
|
if content_type:
|
||||||
headers["Content-Type"] = content_type
|
headers["Content-Type"] = content_type
|
||||||
if provider == "openrouter":
|
if provider == "openrouter":
|
||||||
@@ -116,17 +286,46 @@ def _headers(provider, api_key, content_type=None):
|
|||||||
return headers
|
return headers
|
||||||
|
|
||||||
|
|
||||||
|
def serving(server):
|
||||||
|
"""The base URL of a local server, started if it is not up yet.
|
||||||
|
|
||||||
|
It picks its own port, so this is the first moment its address exists.
|
||||||
|
serve() is idempotent: once it is running this costs nothing.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return server.serve()
|
||||||
|
except ggml.LocalError as exc:
|
||||||
|
raise ApiError(str(exc)) from None
|
||||||
|
|
||||||
|
|
||||||
|
def local_failure(service, server, exc):
|
||||||
|
"""A server that died mid-request, explained by its own output.
|
||||||
|
|
||||||
|
Without this the message is that the connection dropped, when the reason for
|
||||||
|
it was printed by the process at the other end.
|
||||||
|
"""
|
||||||
|
detail = server.error()
|
||||||
|
return ApiError(f"{service}: {exc}" + (f" ({detail})" if detail else ""),
|
||||||
|
exc.status)
|
||||||
|
|
||||||
|
|
||||||
def _transcribe_request(target, wav_path, language, prompt, response_format,
|
def _transcribe_request(target, wav_path, language, prompt, response_format,
|
||||||
granularity=None, timeout=300):
|
granularity=None, timeout=300, aborter=None):
|
||||||
if not target.api_key:
|
if target.provider == "local":
|
||||||
|
# The timeouts here are sized for a hosted API, where a slow answer is a
|
||||||
|
# bill running. Locally the only thing being spent is time.
|
||||||
|
target = target._replace(base_url=serving(ggml.whisper))
|
||||||
|
timeout = max(timeout, LOCAL_TIMEOUT)
|
||||||
|
elif not target.api_key:
|
||||||
raise ApiError(t("{service} API key is empty. Add it in Settings.",
|
raise ApiError(t("{service} API key is empty. Add it in Settings.",
|
||||||
service=target.service))
|
service=target.service))
|
||||||
fields = [("model", target.model), ("response_format", response_format)]
|
fields = [("model", target.model), ("response_format", response_format)]
|
||||||
if language and language != "auto":
|
if language and language != "auto":
|
||||||
fields.append(("language", language))
|
fields.append(("language", language))
|
||||||
# OpenRouter takes the hint field and throws it away, so spare it the bytes.
|
# OpenRouter takes the hint field and throws it away, so spare it the bytes.
|
||||||
# The same words still reach the cleanup model as a glossary.
|
# The same words still reach the cleanup model as a glossary. whisper.cpp
|
||||||
if prompt and target.provider == "openai":
|
# takes it as the initial prompt, the way OpenAI does.
|
||||||
|
if prompt and target.provider != "openrouter":
|
||||||
fields.append(("prompt", prompt))
|
fields.append(("prompt", prompt))
|
||||||
if granularity:
|
if granularity:
|
||||||
fields.append(("timestamp_granularities[]", granularity))
|
fields.append(("timestamp_granularities[]", granularity))
|
||||||
@@ -135,29 +334,77 @@ def _transcribe_request(target, wav_path, language, prompt, response_format,
|
|||||||
return _request(
|
return _request(
|
||||||
f"{target.base_url.rstrip('/')}/audio/transcriptions", body,
|
f"{target.base_url.rstrip('/')}/audio/transcriptions", body,
|
||||||
_headers(target.provider, target.api_key, ctype), timeout=timeout,
|
_headers(target.provider, target.api_key, ctype), timeout=timeout,
|
||||||
|
aborter=aborter,
|
||||||
)
|
)
|
||||||
except ApiError as exc:
|
except ApiError as exc:
|
||||||
|
if target.provider == "local":
|
||||||
|
raise local_failure(target.service, ggml.whisper, exc) from None
|
||||||
raise explain(exc, target.service) from None
|
raise explain(exc, target.service) from None
|
||||||
|
|
||||||
|
|
||||||
def transcribe(target, wav_path, language="", prompt="", timeout=300):
|
# Whisper marks the start of a word with a leading space, so a piece of text
|
||||||
|
# that does not begin with one continues the word before it rather than starting
|
||||||
|
# a new one. Both helpers below turn on that.
|
||||||
|
def _continues_a_word(previous, following):
|
||||||
|
return bool(previous) and not previous[-1:].isspace() and not following[:1].isspace()
|
||||||
|
|
||||||
|
|
||||||
|
def _local_text(text):
|
||||||
|
"""whisper.cpp's segments, joined back into the flowing line OpenAI returns.
|
||||||
|
|
||||||
|
Its plain text puts one segment per line, and a segment boundary falls
|
||||||
|
wherever the tokens fell, which in Turkish lands inside a word about as
|
||||||
|
often as between two. Nothing takes the line break's place: whisper's own
|
||||||
|
leading spaces are what separate the words, and a break inside "değ|iller"
|
||||||
|
has nothing on either side of it worth keeping.
|
||||||
|
"""
|
||||||
|
return "".join(text.split("\n"))
|
||||||
|
|
||||||
|
|
||||||
|
def _merge_word_splits(segments):
|
||||||
|
"""Fold a segment that begins mid-word into the one it continues.
|
||||||
|
|
||||||
|
The hosted whisper-1 hands back segments cut on sentences; whisper.cpp cuts
|
||||||
|
them on tokens, and a subtitle cue reading "değ" is not a cue. The times are
|
||||||
|
joined along with the text, so the merged segment still covers the whole
|
||||||
|
word.
|
||||||
|
"""
|
||||||
|
merged = []
|
||||||
|
for seg in segments:
|
||||||
|
text = seg.get("text") or ""
|
||||||
|
if merged and _continues_a_word(merged[-1]["text"], text):
|
||||||
|
merged[-1]["text"] += text
|
||||||
|
merged[-1]["end"] = seg.get("end") or merged[-1]["end"]
|
||||||
|
continue
|
||||||
|
merged.append({"text": text, "start": seg.get("start") or 0.0,
|
||||||
|
"end": seg.get("end") or 0.0})
|
||||||
|
return merged
|
||||||
|
|
||||||
|
|
||||||
|
def transcribe(target, wav_path, language="", prompt="", timeout=300, aborter=None):
|
||||||
data = _transcribe_request(
|
data = _transcribe_request(
|
||||||
target, wav_path, language, prompt, "json", timeout=timeout
|
target, wav_path, language, prompt, "json", timeout=timeout, aborter=aborter
|
||||||
)
|
)
|
||||||
text = (data.get("text") or "").strip()
|
text = data.get("text") or ""
|
||||||
|
if target.provider == "local":
|
||||||
|
text = _local_text(text)
|
||||||
|
text = text.strip()
|
||||||
if not text:
|
if not text:
|
||||||
raise ApiError(t("Transcript came back empty."))
|
raise ApiError(t("Transcript came back empty."))
|
||||||
return text
|
return text
|
||||||
|
|
||||||
|
|
||||||
def transcribe_segments(target, wav_path, language="", prompt="", timeout=300):
|
def transcribe_segments(target, wav_path, language="", prompt="", timeout=300,
|
||||||
|
aborter=None):
|
||||||
"""[(start_seconds, end_seconds, text)] using whisper-1's verbose response."""
|
"""[(start_seconds, end_seconds, text)] using whisper-1's verbose response."""
|
||||||
data = _transcribe_request(
|
data = _transcribe_request(
|
||||||
target._replace(model=timestamp_model(target.provider)),
|
target._replace(model=timestamp_model(target.provider, target.model)),
|
||||||
wav_path, language, prompt, "verbose_json",
|
wav_path, language, prompt, "verbose_json",
|
||||||
granularity="segment", timeout=timeout,
|
granularity="segment", timeout=timeout, aborter=aborter,
|
||||||
)
|
)
|
||||||
segments = data.get("segments") or []
|
segments = data.get("segments") or []
|
||||||
|
if target.provider == "local":
|
||||||
|
segments = _merge_word_splits(segments)
|
||||||
out = []
|
out = []
|
||||||
for seg in segments:
|
for seg in segments:
|
||||||
text = (seg.get("text") or "").strip()
|
text = (seg.get("text") or "").strip()
|
||||||
@@ -166,18 +413,55 @@ def transcribe_segments(target, wav_path, language="", prompt="", timeout=300):
|
|||||||
end = float(seg.get("end") or 0.0)
|
end = float(seg.get("end") or 0.0)
|
||||||
out.append((start, max(end, start), text))
|
out.append((start, max(end, start), text))
|
||||||
if not out:
|
if not out:
|
||||||
text = (data.get("text") or "").strip()
|
text = data.get("text") or ""
|
||||||
|
if target.provider == "local":
|
||||||
|
text = _local_text(text)
|
||||||
|
text = text.strip()
|
||||||
if not text:
|
if not text:
|
||||||
raise ApiError(t("Transcript came back empty."))
|
raise ApiError(t("Transcript came back empty."))
|
||||||
out = [(0.0, 0.0, text)]
|
out = [(0.0, 0.0, text)]
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _thinking(payload, provider, reasoning):
|
||||||
|
"""Ask for as much thinking as this provider understands, or for none.
|
||||||
|
|
||||||
|
An empty level means "whatever the model does on its own", so nothing is
|
||||||
|
sent. The two mean opposite things by that, which is why the setting is kept
|
||||||
|
per provider: OpenRouter's cleanup models answer straight away, while a local
|
||||||
|
model that was trained to think will think, and cleanup is punctuation rather
|
||||||
|
than a job worth thinking about.
|
||||||
|
"""
|
||||||
|
if not reasoning:
|
||||||
|
return
|
||||||
|
if provider == "local-llm":
|
||||||
|
# What llama.cpp passes to the chat template. The models that think read
|
||||||
|
# it; the ones that do not ignore it.
|
||||||
|
payload["chat_template_kwargs"] = {"enable_thinking": reasoning != "none"}
|
||||||
|
elif reasoning != "none":
|
||||||
|
# The thinking itself is never shown, so ask for it to be left out.
|
||||||
|
payload["reasoning"] = {"effort": reasoning, "exclude": True}
|
||||||
|
|
||||||
|
|
||||||
|
def local_ceiling(text):
|
||||||
|
"""How much of a reply is worth waiting for from a model on this machine.
|
||||||
|
|
||||||
|
Cleanup gives back what it was given, near enough, so a reply several times
|
||||||
|
the length of the transcript is a model that has lost the thread rather than
|
||||||
|
one doing the job. A small one will happily repeat the transcript until the
|
||||||
|
context is full, and every one of those tokens is a second of somebody
|
||||||
|
waiting. A hosted model is left alone: there the same runaway is rare, and a
|
||||||
|
ceiling would cut the minutes short instead.
|
||||||
|
"""
|
||||||
|
return max(512, len(text))
|
||||||
|
|
||||||
|
|
||||||
def cleanup(text, api_key, model, system_prompt, reasoning="",
|
def cleanup(text, api_key, model, system_prompt, reasoning="",
|
||||||
base_url=OPENROUTER_URL, timeout=180):
|
base_url=OPENROUTER_URL, timeout=180, provider="openrouter",
|
||||||
if not api_key:
|
service="OpenRouter", aborter=None):
|
||||||
|
if not api_key and provider != "local-llm":
|
||||||
raise ApiError(t("{service} API key is empty. Add it in Settings.",
|
raise ApiError(t("{service} API key is empty. Add it in Settings.",
|
||||||
service="OpenRouter"))
|
service=service))
|
||||||
payload = {
|
payload = {
|
||||||
"model": model,
|
"model": model,
|
||||||
"temperature": 0,
|
"temperature": 0,
|
||||||
@@ -186,25 +470,30 @@ def cleanup(text, api_key, model, system_prompt, reasoning="",
|
|||||||
{"role": "user", "content": f"<transcript>\n{text}\n</transcript>"},
|
{"role": "user", "content": f"<transcript>\n{text}\n</transcript>"},
|
||||||
],
|
],
|
||||||
}
|
}
|
||||||
# An empty level means "whatever the model does on its own"; anything else is
|
if provider == "local-llm":
|
||||||
# one of OpenRouter's efforts. The thinking itself is never shown, so ask for
|
payload["max_tokens"] = local_ceiling(text)
|
||||||
# it to be left out of the reply.
|
_thinking(payload, provider, reasoning)
|
||||||
if reasoning:
|
|
||||||
payload["reasoning"] = {"effort": reasoning, "exclude": True}
|
|
||||||
try:
|
try:
|
||||||
data = _request(
|
data = _request(
|
||||||
f"{base_url.rstrip('/')}/chat/completions",
|
f"{base_url.rstrip('/')}/chat/completions",
|
||||||
json.dumps(payload).encode("utf-8"),
|
json.dumps(payload).encode("utf-8"),
|
||||||
_headers("openrouter", api_key, "application/json"),
|
_headers(provider, api_key, "application/json"),
|
||||||
timeout=timeout,
|
timeout=timeout, aborter=aborter,
|
||||||
)
|
)
|
||||||
except ApiError as exc:
|
except ApiError as exc:
|
||||||
raise explain(exc, "OpenRouter") from None
|
raise explain(exc, service) from None
|
||||||
choices = data.get("choices") or []
|
choices = data.get("choices") or []
|
||||||
if not choices:
|
if not choices:
|
||||||
raise ApiError(_extract_error(json.dumps(data)))
|
raise ApiError(_extract_error(json.dumps(data)))
|
||||||
content = ((choices[0].get("message") or {}).get("content") or "").strip()
|
message = choices[0].get("message") or {}
|
||||||
|
content = (message.get("content") or "").strip()
|
||||||
if not content:
|
if not content:
|
||||||
|
# A thinking model can spend the whole reply on the thinking and leave
|
||||||
|
# nothing to paste. Worth naming, because the fix is a setting rather
|
||||||
|
# than a retry: cleanup is not a job that wants thinking.
|
||||||
|
if message.get("reasoning_content") or message.get("reasoning"):
|
||||||
|
raise ApiError(t("The cleanup model spent its whole reply on "
|
||||||
|
"thinking. Set Thinking to \u201cOff\u201d."))
|
||||||
raise ApiError(t("The cleanup model returned an empty reply."))
|
raise ApiError(t("The cleanup model returned an empty reply."))
|
||||||
return content
|
return content
|
||||||
|
|
||||||
@@ -297,17 +586,22 @@ def openrouter_models(api_key="", transcription=False):
|
|||||||
return sorted(m["id"] for m in models if m.get("id"))
|
return sorted(m["id"] for m in models if m.get("id"))
|
||||||
|
|
||||||
|
|
||||||
def openai_models(api_key, base_url=OPENAI_URL):
|
def openai_models(api_key, base_url=OPENAI_URL, service="OpenAI"):
|
||||||
|
"""The audio models of anything that speaks OpenAI's /models, Groq included.
|
||||||
|
|
||||||
|
`service` is only the name an error is written in, so a Groq key that is
|
||||||
|
refused says Groq rather than OpenAI.
|
||||||
|
"""
|
||||||
if not api_key:
|
if not api_key:
|
||||||
raise ApiError(t("{service} API key is empty. Add it in Settings.",
|
raise ApiError(t("{service} API key is empty. Add it in Settings.",
|
||||||
service="OpenAI"))
|
service=service))
|
||||||
try:
|
try:
|
||||||
data = _get_json(
|
data = _get_json(
|
||||||
f"{base_url.rstrip('/')}/models",
|
f"{base_url.rstrip('/')}/models",
|
||||||
{"Authorization": f"Bearer {api_key}", "User-Agent": USER_AGENT},
|
{"Authorization": f"Bearer {api_key}", "User-Agent": USER_AGENT},
|
||||||
)
|
)
|
||||||
except ApiError as exc:
|
except ApiError as exc:
|
||||||
raise explain(exc, "OpenAI") from None
|
raise explain(exc, service) from None
|
||||||
ids = [m["id"] for m in data.get("data", []) if m.get("id")]
|
ids = [m["id"] for m in data.get("data", []) if m.get("id")]
|
||||||
audio = [i for i in ids if "transcribe" in i or "whisper" in i]
|
audio = [i for i in ids if "transcribe" in i or "whisper" in i]
|
||||||
return sorted(audio or ids)
|
return sorted(audio or ids)
|
||||||
|
|||||||
@@ -75,7 +75,10 @@ CODEX_ITEMS = {
|
|||||||
CLAUDE_EFFORT = {"none": "low", "minimal": "low", "low": "low",
|
CLAUDE_EFFORT = {"none": "low", "minimal": "low", "low": "low",
|
||||||
"medium": "medium", "high": "high", "xhigh": "xhigh",
|
"medium": "medium", "high": "high", "xhigh": "xhigh",
|
||||||
"max": "max"}
|
"max": "max"}
|
||||||
CODEX_EFFORT = {"none": "minimal", "minimal": "minimal", "low": "low",
|
# "minimal" was Codex's bottom rung until the newer models replaced it with
|
||||||
|
# "none", and each of them rejects the other's word for it with a 400. "low" is
|
||||||
|
# the one every model has, so the two lowest rungs land there instead.
|
||||||
|
CODEX_EFFORT = {"none": "low", "minimal": "low", "low": "low",
|
||||||
"medium": "medium", "high": "high", "xhigh": "high",
|
"medium": "medium", "high": "high", "xhigh": "high",
|
||||||
"max": "high"}
|
"max": "high"}
|
||||||
|
|
||||||
@@ -421,7 +424,7 @@ def _conclude(found, code, stderr, session, service):
|
|||||||
if code != 0 and not found["answer"]:
|
if code != 0 and not found["answer"]:
|
||||||
if session and _session_missing(stderr):
|
if session and _session_missing(stderr):
|
||||||
raise _SessionGone()
|
raise _SessionGone()
|
||||||
raise AssistantError(_last_line(stderr) or found["failure"] or t(
|
raise AssistantError(last_line(stderr) or found["failure"] or t(
|
||||||
"{service} exited with code {code}.", service=service, code=code))
|
"{service} exited with code {code}.", service=service, code=code))
|
||||||
if found["failure"] and not found["answer"]:
|
if found["failure"] and not found["answer"]:
|
||||||
raise AssistantError(found["failure"])
|
raise AssistantError(found["failure"])
|
||||||
@@ -480,6 +483,11 @@ def _finish(proc):
|
|||||||
return stderr
|
return stderr
|
||||||
|
|
||||||
|
|
||||||
def _last_line(text):
|
def last_line(text):
|
||||||
|
"""The line worth showing out of a CLI's stderr: the last one it wrote.
|
||||||
|
|
||||||
|
Shared with cleanup, which runs the same two programs for a different job
|
||||||
|
and fails the same way when they are unhappy.
|
||||||
|
"""
|
||||||
lines = [line for line in (text or "").splitlines() if line.strip()]
|
lines = [line for line in (text or "").splitlines() if line.strip()]
|
||||||
return lines[-1].strip() if lines else ""
|
return lines[-1].strip() if lines else ""
|
||||||
|
|||||||
@@ -0,0 +1,213 @@
|
|||||||
|
"""Who rewrites the transcript once it has been heard.
|
||||||
|
|
||||||
|
Normally a small model on OpenRouter: one request, a second, a few tenths of a
|
||||||
|
cent. A machine with Claude Code or Codex on it is already paying for a model
|
||||||
|
though, and the subscription that answers "put that in my calendar on Thursday"
|
||||||
|
can just as well take the "eee"s out of a sentence. No second key, no second
|
||||||
|
bill. It costs seconds rather than one, because a CLI opens a whole session to
|
||||||
|
do it, which is the trade.
|
||||||
|
|
||||||
|
Whoever does it, the job is the same one: no tools, no files, no memory of the
|
||||||
|
last dictation. There is nothing here to look up and nothing to carry over, and
|
||||||
|
a transcript is text from a microphone rather than an instruction, so the less
|
||||||
|
the agent can reach while it reads one, the better.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
import api
|
||||||
|
import assistant
|
||||||
|
import ggml
|
||||||
|
from i18n import t
|
||||||
|
|
||||||
|
PROVIDERS = ("openrouter", "local", "claude", "codex")
|
||||||
|
|
||||||
|
|
||||||
|
class CleanupError(api.ApiError):
|
||||||
|
"""What a CLI could not do.
|
||||||
|
|
||||||
|
An ApiError because to the chain a cleanup that failed is a cleanup that
|
||||||
|
failed, whichever way it was run, and every caller already catches one and
|
||||||
|
keeps the raw transcript.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def provider(conf):
|
||||||
|
chosen = conf["cleanup_provider"]
|
||||||
|
return chosen if chosen in PROVIDERS else "openrouter"
|
||||||
|
|
||||||
|
|
||||||
|
def executable(name):
|
||||||
|
"""The CLI a provider runs, or "" when it needs none."""
|
||||||
|
return {"claude": "claude", "codex": "codex"}.get(name, "")
|
||||||
|
|
||||||
|
|
||||||
|
def model(conf):
|
||||||
|
"""Which model does the cleaning, for the history and the settings window."""
|
||||||
|
name = provider(conf)
|
||||||
|
if name == "local":
|
||||||
|
return conf["local_llm_model"]
|
||||||
|
if name == "claude":
|
||||||
|
return conf["cleanup_claude_model"].strip() or "haiku"
|
||||||
|
if name == "codex":
|
||||||
|
# Codex is left on whatever it is set to unless a model is typed in, so
|
||||||
|
# here there is only the name of the thing that did it.
|
||||||
|
return conf["cleanup_codex_model"].strip() or "codex"
|
||||||
|
return conf["cleanup_model"]
|
||||||
|
|
||||||
|
|
||||||
|
def run(text, conf, system_prompt, timeout=180, aborter=None):
|
||||||
|
"""Hand the transcript to whoever is set to clean it up.
|
||||||
|
|
||||||
|
`aborter` is only of use to the two that answer over HTTP; a CLI is stopped
|
||||||
|
between blocks instead, which is close enough when a block is seconds.
|
||||||
|
"""
|
||||||
|
name = provider(conf)
|
||||||
|
if name == "openrouter":
|
||||||
|
return api.cleanup(
|
||||||
|
text, conf.openrouter_key(), conf["cleanup_model"], system_prompt,
|
||||||
|
reasoning=conf["cleanup_reasoning"],
|
||||||
|
base_url=conf["openrouter_base_url"], timeout=timeout,
|
||||||
|
aborter=aborter,
|
||||||
|
)
|
||||||
|
if name == "local":
|
||||||
|
return _local(text, conf, system_prompt, timeout, aborter)
|
||||||
|
runner = _claude if name == "claude" else _codex
|
||||||
|
return runner(text, conf, system_prompt, timeout)
|
||||||
|
|
||||||
|
|
||||||
|
def _local(text, conf, system_prompt, timeout, aborter=None):
|
||||||
|
"""llama.cpp, on this machine, answering the request OpenRouter answers.
|
||||||
|
|
||||||
|
No key and no bill, and the address does not exist until the server is up,
|
||||||
|
which is what starting it here is for. The timeout is the hosted one raised:
|
||||||
|
the only thing being spent is time.
|
||||||
|
"""
|
||||||
|
service = t("Local model")
|
||||||
|
try:
|
||||||
|
return api.cleanup(
|
||||||
|
text, "", conf["local_llm_model"], system_prompt,
|
||||||
|
reasoning=conf["local_llm_reasoning"],
|
||||||
|
base_url=api.serving(ggml.llm),
|
||||||
|
timeout=max(timeout, api.LOCAL_TIMEOUT),
|
||||||
|
provider="local-llm", service=service, aborter=aborter,
|
||||||
|
)
|
||||||
|
except api.ApiError as exc:
|
||||||
|
# A server that died mid-request would otherwise report only that the
|
||||||
|
# connection dropped, when the reason is in its own output.
|
||||||
|
raise api.local_failure(service, ggml.llm, exc) from None
|
||||||
|
|
||||||
|
|
||||||
|
def _wrap(text):
|
||||||
|
"""The same fence the OpenRouter call puts around it: this is the material,
|
||||||
|
not the instruction, however much of it reads like one."""
|
||||||
|
return f"<transcript>\n{text}\n</transcript>"
|
||||||
|
|
||||||
|
|
||||||
|
# --- Claude Code ----------------------------------------------------------
|
||||||
|
|
||||||
|
def _claude(text, conf, system_prompt, timeout):
|
||||||
|
cmd = [
|
||||||
|
"claude", "-p", _wrap(text),
|
||||||
|
# --system-prompt rather than --append-system-prompt: the cleanup rules
|
||||||
|
# are the whole job, and Claude Code's own instructions are about
|
||||||
|
# working on a codebase.
|
||||||
|
"--system-prompt", system_prompt,
|
||||||
|
"--model", model(conf),
|
||||||
|
"--output-format", "text",
|
||||||
|
"--tools", "", # nothing to run
|
||||||
|
"--strict-mcp-config", "--mcp-config", '{"mcpServers":{}}',
|
||||||
|
"--no-session-persistence", # nothing to resume
|
||||||
|
]
|
||||||
|
effort = assistant.CLAUDE_EFFORT.get(conf["cleanup_reasoning"], "")
|
||||||
|
if effort:
|
||||||
|
cmd += ["--effort", effort]
|
||||||
|
|
||||||
|
answer = _output(cmd, timeout, "Claude")
|
||||||
|
if not answer:
|
||||||
|
raise CleanupError(t("{service} answered with nothing.", service="Claude"))
|
||||||
|
return answer
|
||||||
|
|
||||||
|
|
||||||
|
# --- Codex ----------------------------------------------------------------
|
||||||
|
|
||||||
|
def _codex(text, conf, system_prompt, timeout):
|
||||||
|
# Codex takes no system prompt of its own, so the rules ride in front of the
|
||||||
|
# transcript, kept apart from it so the two are not read as one.
|
||||||
|
body = f"{system_prompt}\n\n---\n\n{_wrap(text)}"
|
||||||
|
cmd = [
|
||||||
|
"codex", "exec",
|
||||||
|
"--sandbox", "read-only", # it has no reason to touch the disk
|
||||||
|
"--skip-git-repo-check",
|
||||||
|
"--ephemeral", # nothing to resume
|
||||||
|
"--color", "never",
|
||||||
|
"-c", 'approval_policy="never"', # there is nobody here to approve
|
||||||
|
]
|
||||||
|
if conf["cleanup_codex_model"].strip():
|
||||||
|
cmd += ["-m", conf["cleanup_codex_model"].strip()]
|
||||||
|
effort = assistant.CODEX_EFFORT.get(conf["cleanup_reasoning"], "")
|
||||||
|
if effort:
|
||||||
|
cmd += ["-c", f'model_reasoning_effort="{effort}"']
|
||||||
|
|
||||||
|
# `codex exec` prints a header, its thinking and a token count around the
|
||||||
|
# answer; the file it writes on the way out is the answer on its own.
|
||||||
|
handle, last_message = tempfile.mkstemp(prefix="dikte-cleanup-", suffix=".txt")
|
||||||
|
os.close(handle)
|
||||||
|
cmd += ["-o", last_message, body]
|
||||||
|
try:
|
||||||
|
_output(cmd, timeout, "Codex")
|
||||||
|
answer = _read(last_message)
|
||||||
|
finally:
|
||||||
|
try:
|
||||||
|
os.unlink(last_message)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
if not answer:
|
||||||
|
raise CleanupError(t("{service} answered with nothing.", service="Codex"))
|
||||||
|
return answer
|
||||||
|
|
||||||
|
|
||||||
|
def _read(path):
|
||||||
|
try:
|
||||||
|
with open(path, encoding="utf-8", errors="replace") as fh:
|
||||||
|
return fh.read().strip()
|
||||||
|
except OSError:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
# --- running a CLI --------------------------------------------------------
|
||||||
|
|
||||||
|
def _output(cmd, timeout, service):
|
||||||
|
"""Run cmd to the end and return what it printed.
|
||||||
|
|
||||||
|
It runs in the home directory rather than wherever the agent is pointed: a
|
||||||
|
project's instructions have opinions about how text should be written, and
|
||||||
|
none of them are about this transcript.
|
||||||
|
"""
|
||||||
|
binary = cmd[0]
|
||||||
|
if not shutil.which(binary):
|
||||||
|
raise CleanupError(t(
|
||||||
|
"{binary} not found. Install it, or have OpenRouter clean up "
|
||||||
|
"instead, under Settings → API and models.", binary=binary,
|
||||||
|
))
|
||||||
|
try:
|
||||||
|
done = subprocess.run(
|
||||||
|
cmd, cwd=os.path.expanduser("~"), stdin=subprocess.DEVNULL,
|
||||||
|
capture_output=True, text=True, encoding="utf-8", errors="replace",
|
||||||
|
timeout=timeout,
|
||||||
|
)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
raise CleanupError(t("{service} did not finish within {seconds} seconds.",
|
||||||
|
service=service, seconds=timeout)) from None
|
||||||
|
except OSError as exc:
|
||||||
|
raise CleanupError(t("Could not run {binary}: {error}",
|
||||||
|
binary=binary, error=exc)) from exc
|
||||||
|
if done.returncode != 0:
|
||||||
|
raise CleanupError(assistant.last_line(done.stderr) or t(
|
||||||
|
"{service} exited with code {code}.",
|
||||||
|
service=service, code=done.returncode))
|
||||||
|
return (done.stdout or "").strip()
|
||||||
@@ -25,6 +25,7 @@ from PyQt6.QtCore import QCoreApplication, QTimer
|
|||||||
import api
|
import api
|
||||||
import assistant
|
import assistant
|
||||||
import audio
|
import audio
|
||||||
|
import cleanup
|
||||||
import config as cfg
|
import config as cfg
|
||||||
import filetranscribe
|
import filetranscribe
|
||||||
import hotkey
|
import hotkey
|
||||||
@@ -44,14 +45,6 @@ GUI_VERBS = {"", "settings", "toggle", "ask", "meeting"}
|
|||||||
IDEMPOTENT_VERBS = {"cancel", "stop", "quit", "restart", "ask-cancel",
|
IDEMPOTENT_VERBS = {"cancel", "stop", "quit", "restart", "ask-cancel",
|
||||||
"ask-reset", "meeting-cancel"}
|
"ask-reset", "meeting-cancel"}
|
||||||
|
|
||||||
# Which desktop entry, name and setting belong to each of the three shortcuts.
|
|
||||||
SHORTCUTS = {
|
|
||||||
"toggle": (hotkey.DESKTOP_ID, "Dikte: start/stop recording", "shortcut"),
|
|
||||||
"ask": (hotkey.ASK_DESKTOP_ID, "Dikte: ask Claude Code", "assistant_shortcut"),
|
|
||||||
"meeting": (hotkey.MEETING_DESKTOP_ID, "Dikte: start/end a meeting recording",
|
|
||||||
"meeting_shortcut"),
|
|
||||||
}
|
|
||||||
|
|
||||||
_app = None
|
_app = None
|
||||||
|
|
||||||
|
|
||||||
@@ -662,12 +655,14 @@ def cmd_devices(opts):
|
|||||||
|
|
||||||
def cmd_models(opts):
|
def cmd_models(opts):
|
||||||
conf = cfg.Config()
|
conf = cfg.Config()
|
||||||
|
who = cfg.TRANSCRIBERS[opts.provider]
|
||||||
try:
|
try:
|
||||||
if opts.provider == "openai":
|
if opts.provider == "openrouter":
|
||||||
models = api.openai_models(conf.openai_key(), conf["openai_base_url"])
|
|
||||||
else:
|
|
||||||
models = api.openrouter_models(conf.openrouter_key(),
|
models = api.openrouter_models(conf.openrouter_key(),
|
||||||
transcription=opts.transcription)
|
transcription=opts.transcription)
|
||||||
|
else:
|
||||||
|
models = api.openai_models(conf.api_key(who.key), conf[who.url],
|
||||||
|
who.service)
|
||||||
except api.ApiError as exc:
|
except api.ApiError as exc:
|
||||||
return fail(opts, exc)
|
return fail(opts, exc)
|
||||||
return out(opts, {"ok": True, "provider": opts.provider, "models": models},
|
return out(opts, {"ok": True, "provider": opts.provider, "models": models},
|
||||||
@@ -677,19 +672,21 @@ def cmd_models(opts):
|
|||||||
def cmd_test_key(opts):
|
def cmd_test_key(opts):
|
||||||
conf = cfg.Config()
|
conf = cfg.Config()
|
||||||
results = {}
|
results = {}
|
||||||
if opts.which in ("openai", "all"):
|
for name, who in cfg.TRANSCRIBERS.items():
|
||||||
|
if opts.which not in (name, "all"):
|
||||||
|
continue
|
||||||
try:
|
try:
|
||||||
count = len(api.openai_models(conf.openai_key(), conf["openai_base_url"]))
|
if name == "openrouter":
|
||||||
results["openai"] = {"ok": True,
|
# The one key that also pays for cleanup, so it reports credit
|
||||||
"message": f"connection works, {count} models visible"}
|
# rather than a model count.
|
||||||
|
message = api.openrouter_key_status(conf.openrouter_key())
|
||||||
|
else:
|
||||||
|
count = len(api.openai_models(conf.api_key(who.key), conf[who.url],
|
||||||
|
who.service))
|
||||||
|
message = f"connection works, {count} models visible"
|
||||||
|
results[name] = {"ok": True, "message": message}
|
||||||
except api.ApiError as exc:
|
except api.ApiError as exc:
|
||||||
results["openai"] = {"ok": False, "message": str(exc)}
|
results[name] = {"ok": False, "message": str(exc)}
|
||||||
if opts.which in ("openrouter", "all"):
|
|
||||||
try:
|
|
||||||
results["openrouter"] = {"ok": True,
|
|
||||||
"message": api.openrouter_key_status(conf.openrouter_key())}
|
|
||||||
except api.ApiError as exc:
|
|
||||||
results["openrouter"] = {"ok": False, "message": str(exc)}
|
|
||||||
everything_ok = all(item["ok"] for item in results.values())
|
everything_ok = all(item["ok"] for item in results.values())
|
||||||
lines = [f"{'✓' if item['ok'] else '✗'} {name}: {item['message']}"
|
lines = [f"{'✓' if item['ok'] else '✗'} {name}: {item['message']}"
|
||||||
for name, item in results.items()]
|
for name, item in results.items()]
|
||||||
@@ -701,9 +698,9 @@ def cmd_shortcut(opts):
|
|||||||
conf = cfg.Config()
|
conf = cfg.Config()
|
||||||
if opts.shortcut == "status":
|
if opts.shortcut == "status":
|
||||||
rows = {}
|
rows = {}
|
||||||
for name, (desktop_id, _label, key) in SHORTCUTS.items():
|
for name, spec in hotkey.SHORTCUTS.items():
|
||||||
rows[name] = {"registered": hotkey.shortcut_status(desktop_id),
|
rows[name] = {"registered": hotkey.shortcut_status(spec.desktop_id),
|
||||||
"configured": conf[key]}
|
"configured": conf[spec.setting]}
|
||||||
lines = [f"{name:8} {row['registered'] or '(not installed)':16} "
|
lines = [f"{name:8} {row['registered'] or '(not installed)':16} "
|
||||||
f"setting: {row['configured'] or '(none)'}"
|
f"setting: {row['configured'] or '(none)'}"
|
||||||
for name, row in rows.items()]
|
for name, row in rows.items()]
|
||||||
@@ -711,28 +708,29 @@ def cmd_shortcut(opts):
|
|||||||
return out(opts, {"ok": True, "shortcuts": rows,
|
return out(opts, {"ok": True, "shortcuts": rows,
|
||||||
"listener": conf["evdev_hotkey"]}, "\n".join(lines))
|
"listener": conf["evdev_hotkey"]}, "\n".join(lines))
|
||||||
|
|
||||||
desktop_id, label, key = SHORTCUTS[opts.which]
|
spec = hotkey.SHORTCUTS[opts.which]
|
||||||
if opts.shortcut == "remove":
|
if opts.shortcut == "remove":
|
||||||
hotkey.remove_shortcut(desktop_id)
|
hotkey.remove_shortcut(spec.desktop_id)
|
||||||
return out(opts, {"ok": True, "removed": opts.which},
|
return out(opts, {"ok": True, "removed": opts.which},
|
||||||
f"Removed the {opts.which} shortcut.")
|
f"Removed the {opts.which} shortcut.")
|
||||||
|
|
||||||
combo = (opts.combo or conf[key] or ("Ctrl+Space" if opts.which == "toggle" else "")).strip()
|
combo = (opts.combo or conf[spec.setting] or spec.fallback).strip()
|
||||||
if not combo:
|
if not combo:
|
||||||
return fail(opts, "no combination given and none stored; pass --combo", 2)
|
return fail(opts, "no combination given and none stored; pass --combo", 2)
|
||||||
if not hotkey.valid_shortcut(combo):
|
if not hotkey.valid_shortcut(combo):
|
||||||
return fail(opts, f"cannot parse that combination: {combo}", 2)
|
return fail(opts, f"cannot parse that combination: {combo}", 2)
|
||||||
clashes = hotkey.conflicting_shortcuts(combo, desktop_id)
|
clashes = hotkey.conflicting_shortcuts(combo, spec.desktop_id)
|
||||||
if clashes and not opts.force:
|
if clashes and not opts.force:
|
||||||
return fail(opts, f"{combo} is also used by: {', '.join(clashes[:6])}. "
|
return fail(opts, f"{combo} is also used by: {', '.join(clashes[:6])}. "
|
||||||
"Pass --force to install it anyway.", 1, conflicts=clashes)
|
"Pass --force to install it anyway.", 1, conflicts=clashes)
|
||||||
|
|
||||||
ok, message = hotkey.install_shortcut(
|
ok, message = hotkey.install_shortcut(
|
||||||
combo, ipc.command_for(opts.which), name=label, desktop_id=desktop_id,
|
combo, ipc.command_for(spec.verb), name=spec.name,
|
||||||
|
desktop_id=spec.desktop_id,
|
||||||
)
|
)
|
||||||
if not ok:
|
if not ok:
|
||||||
return fail(opts, message)
|
return fail(opts, message)
|
||||||
conf[key] = combo
|
conf[spec.setting] = combo
|
||||||
try:
|
try:
|
||||||
conf.save()
|
conf.save()
|
||||||
except OSError as exc:
|
except OSError as exc:
|
||||||
@@ -767,16 +765,18 @@ def cmd_status(opts):
|
|||||||
def cmd_doctor(opts):
|
def cmd_doctor(opts):
|
||||||
"""What the settings window checks behind its buttons, in one pass."""
|
"""What the settings window checks behind its buttons, in one pass."""
|
||||||
conf = cfg.Config()
|
conf = cfg.Config()
|
||||||
programs = {name: shutil.which(name) or ""
|
wanted = ["pw-record", "wl-copy", "ydotool", "ffmpeg", "pactl", "kwriteconfig6",
|
||||||
for name in ("pw-record", "wl-copy", "ydotool", "ffmpeg",
|
assistant.executable(assistant.provider(conf)) or "claude",
|
||||||
"pactl", "kwriteconfig6",
|
cleanup.executable(cleanup.provider(conf))]
|
||||||
assistant.executable(assistant.provider(conf)) or "claude")}
|
programs = {name: shutil.which(name) or "" for name in wanted if name}
|
||||||
target = conf.transcribe_target()
|
target = conf.transcribe_target()
|
||||||
|
cleaner = cleanup.provider(conf)
|
||||||
checks = {
|
checks = {
|
||||||
"programs": programs,
|
"programs": programs,
|
||||||
"transcription": {"provider": target.provider, "model": target.model,
|
"transcription": {"provider": target.provider, "model": target.model,
|
||||||
"key": bool(target.api_key)},
|
"key": bool(target.api_key)},
|
||||||
"cleanup": {"enabled": conf["cleanup_enabled"], "model": conf["cleanup_model"],
|
"cleanup": {"enabled": conf["cleanup_enabled"], "provider": cleaner,
|
||||||
|
"model": cleanup.model(conf),
|
||||||
"key": bool(conf.openrouter_key())},
|
"key": bool(conf.openrouter_key())},
|
||||||
"agent": {"provider": assistant.provider(conf),
|
"agent": {"provider": assistant.provider(conf),
|
||||||
"directory": assistant.working_dir(conf)},
|
"directory": assistant.working_dir(conf)},
|
||||||
@@ -787,8 +787,11 @@ def cmd_doctor(opts):
|
|||||||
lines += [
|
lines += [
|
||||||
f"{'✓' if target.api_key else '✗'} {target.service} key, transcribing on "
|
f"{'✓' if target.api_key else '✗'} {target.service} key, transcribing on "
|
||||||
f"{target.model}",
|
f"{target.model}",
|
||||||
f"{'✓' if conf.openrouter_key() else '✗'} OpenRouter key, cleaning up on "
|
# Cleanup on a CLI needs no key, so what is checked is the program.
|
||||||
f"{conf['cleanup_model']}",
|
(f"{'✓' if conf.openrouter_key() else '✗'} OpenRouter key, cleaning up on "
|
||||||
|
f"{conf['cleanup_model']}") if cleaner == "openrouter" else
|
||||||
|
(f"{'✓' if programs[cleanup.executable(cleaner)] else '✗'} "
|
||||||
|
f"{cleanup.executable(cleaner)}, cleaning up on {cleanup.model(conf)}"),
|
||||||
f"{'✓' if checks['running'] else '·'} application "
|
f"{'✓' if checks['running'] else '·'} application "
|
||||||
+ ("running" if checks["running"] else "not running"),
|
+ ("running" if checks["running"] else "not running"),
|
||||||
]
|
]
|
||||||
@@ -986,30 +989,31 @@ def build_parser():
|
|||||||
# --- the machine ------------------------------------------------------
|
# --- the machine ------------------------------------------------------
|
||||||
leaf(subs, "devices", "microphones and monitors").set_defaults(func=cmd_devices)
|
leaf(subs, "devices", "microphones and monitors").set_defaults(func=cmd_devices)
|
||||||
models = leaf(subs, "models", "model ids a provider offers")
|
models = leaf(subs, "models", "model ids a provider offers")
|
||||||
models.add_argument("--provider", choices=("openrouter", "openai"),
|
models.add_argument("--provider", choices=tuple(cfg.TRANSCRIBERS),
|
||||||
default="openrouter")
|
default="openrouter")
|
||||||
models.add_argument("--transcription", action="store_true",
|
models.add_argument("--transcription", action="store_true",
|
||||||
help="only the speech-to-text ones")
|
help="only the speech-to-text ones")
|
||||||
models.set_defaults(func=cmd_models)
|
models.set_defaults(func=cmd_models)
|
||||||
test = leaf(subs, "test-key", "check the API keys")
|
test = leaf(subs, "test-key", "check the API keys")
|
||||||
test.add_argument("which", nargs="?", default="all",
|
test.add_argument("which", nargs="?", default="all",
|
||||||
choices=("all", "openai", "openrouter"))
|
choices=("all", *cfg.TRANSCRIBERS))
|
||||||
test.set_defaults(func=cmd_test_key)
|
test.set_defaults(func=cmd_test_key)
|
||||||
leaf(subs, "doctor", "keys, programs, and what is missing").set_defaults(func=cmd_doctor)
|
leaf(subs, "doctor", "keys, programs, and what is missing").set_defaults(func=cmd_doctor)
|
||||||
|
|
||||||
shortcut = leaf(subs, "shortcut", "the KDE global shortcuts")
|
shortcut = leaf(subs, "shortcut", "the desktop's global shortcuts")
|
||||||
inner = shortcut.add_subparsers(dest="shortcut", metavar="")
|
inner = shortcut.add_subparsers(dest="shortcut", metavar="")
|
||||||
shortcut.set_defaults(func=_needs_subcommand(shortcut))
|
shortcut.set_defaults(func=_needs_subcommand(shortcut))
|
||||||
leaf(inner, "status", "what is registered").set_defaults(func=cmd_shortcut)
|
leaf(inner, "status", "what is registered").set_defaults(func=cmd_shortcut)
|
||||||
install = leaf(inner, "install", "register one")
|
install = leaf(inner, "install", "register one")
|
||||||
install.add_argument("which", nargs="?", default="toggle",
|
install.add_argument("which", nargs="?", default="toggle",
|
||||||
choices=tuple(SHORTCUTS))
|
choices=tuple(hotkey.SHORTCUTS))
|
||||||
install.add_argument("--combo", help="e.g. Ctrl+Alt+Space")
|
install.add_argument("--combo", help="e.g. Ctrl+Alt+Space")
|
||||||
install.add_argument("--force", action="store_true",
|
install.add_argument("--force", action="store_true",
|
||||||
help="install it even if something else uses it")
|
help="install it even if something else uses it")
|
||||||
install.set_defaults(func=cmd_shortcut)
|
install.set_defaults(func=cmd_shortcut)
|
||||||
remove = leaf(inner, "remove", "unregister one")
|
remove = leaf(inner, "remove", "unregister one")
|
||||||
remove.add_argument("which", nargs="?", default="toggle", choices=tuple(SHORTCUTS))
|
remove.add_argument("which", nargs="?", default="toggle",
|
||||||
|
choices=tuple(hotkey.SHORTCUTS))
|
||||||
remove.set_defaults(func=cmd_shortcut)
|
remove.set_defaults(func=cmd_shortcut)
|
||||||
|
|
||||||
# --- the application --------------------------------------------------
|
# --- the application --------------------------------------------------
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
"""Settings storage, in the place this system keeps a program's settings."""
|
"""Settings storage, in the place this system keeps a program's settings."""
|
||||||
|
|
||||||
|
import collections
|
||||||
import hashlib
|
import hashlib
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
@@ -7,8 +8,10 @@ import pathlib
|
|||||||
import sys
|
import sys
|
||||||
|
|
||||||
import api
|
import api
|
||||||
|
import ggml
|
||||||
import i18n
|
import i18n
|
||||||
import paste
|
import paste
|
||||||
|
from i18n import t
|
||||||
|
|
||||||
|
|
||||||
def _xdg(var, default):
|
def _xdg(var, default):
|
||||||
@@ -377,16 +380,53 @@ DEFAULTS = {
|
|||||||
"ui_language": "auto", # auto | tr | en
|
"ui_language": "auto", # auto | tr | en
|
||||||
"openai_api_key": "",
|
"openai_api_key": "",
|
||||||
"openai_base_url": "https://api.openai.com/v1",
|
"openai_base_url": "https://api.openai.com/v1",
|
||||||
|
"groq_api_key": "",
|
||||||
|
"groq_base_url": "https://api.groq.com/openai/v1",
|
||||||
"openrouter_api_key": "",
|
"openrouter_api_key": "",
|
||||||
"openrouter_base_url": "https://openrouter.ai/api/v1",
|
"openrouter_base_url": "https://openrouter.ai/api/v1",
|
||||||
"transcribe_provider": "openai", # openai | openrouter
|
"transcribe_provider": "local", # "local", or a key of TRANSCRIBERS
|
||||||
"transcribe_model": "gpt-4o-transcribe", # used when provider is openai
|
"transcribe_model": "gpt-4o-transcribe", # used when provider is openai
|
||||||
|
"groq_transcribe_model": "whisper-large-v3-turbo",
|
||||||
"openrouter_transcribe_model": "openai/gpt-4o-transcribe",
|
"openrouter_transcribe_model": "openai/gpt-4o-transcribe",
|
||||||
"language": "tr",
|
"language": "tr",
|
||||||
"transcribe_prompt": "",
|
"transcribe_prompt": "",
|
||||||
|
|
||||||
|
# --- whisper.cpp, on this machine ---------------------------------------
|
||||||
|
# The program and the model are both fetched from Settings; empty means
|
||||||
|
# nothing has been downloaded yet, which is what opens Settings on a first
|
||||||
|
# run.
|
||||||
|
# Pointed at the suggestion rather than at nothing, so the settings window
|
||||||
|
# opens with the Download button already on the right model.
|
||||||
|
"local_model": ggml.SUGGESTED_WHISPER,
|
||||||
|
"local_threads": 0, # 0 -> whisper.cpp picks
|
||||||
|
"local_gpu": True,
|
||||||
|
"local_preload": True, # load the model while Dikte starts, rather
|
||||||
|
# than on the first dictation
|
||||||
|
"local_binary": "", # empty -> whichever copy ggml.py finds
|
||||||
|
|
||||||
"cleanup_enabled": True,
|
"cleanup_enabled": True,
|
||||||
|
"cleanup_provider": "openrouter", # a name in cleanup.PROVIDERS
|
||||||
"cleanup_model": "google/gemini-3.5-flash-lite",
|
"cleanup_model": "google/gemini-3.5-flash-lite",
|
||||||
|
"cleanup_claude_model": "haiku", # Claude Code: an alias, or a full model id
|
||||||
|
"cleanup_codex_model": "", # empty -> whatever Codex is set to
|
||||||
"cleanup_reasoning": "", # empty -> whatever the model does by default
|
"cleanup_reasoning": "", # empty -> whatever the model does by default
|
||||||
|
|
||||||
|
# --- llama.cpp, on this machine -----------------------------------------
|
||||||
|
# Kept apart from the meeting settings on purpose. Cleanup is punctuation
|
||||||
|
# and filler words, which a small model does in a moment; the minutes are a
|
||||||
|
# summary of an hour, which it does not.
|
||||||
|
"local_llm_model": "", # a file name, e.g. gemma-3-4b-it-Q4_K_M.gguf
|
||||||
|
# Where the model list is read from; the settings window offers the
|
||||||
|
# publishers ggml.py knows of and takes any other one that is typed in.
|
||||||
|
"local_llm_repo": ggml.SUGGESTED_LLM[0],
|
||||||
|
"local_llm_threads": 0,
|
||||||
|
"local_llm_gpu": True,
|
||||||
|
"local_llm_context": 8192,
|
||||||
|
"local_llm_binary": "",
|
||||||
|
"local_llm_preload": False, # heavier than whisper, so only when asked
|
||||||
|
# Off rather than empty: a model trained to think will, and 300 tokens of
|
||||||
|
# reasoning about a comma is 300 tokens of waiting.
|
||||||
|
"local_llm_reasoning": "none",
|
||||||
"cleanup_prompt": "", # empty -> language-specific default
|
"cleanup_prompt": "", # empty -> language-specific default
|
||||||
"auto_paste": True,
|
"auto_paste": True,
|
||||||
"paste_shortcut": paste.desktop().shortcuts[0], # cmd+v on a Mac
|
"paste_shortcut": paste.desktop().shortcuts[0], # cmd+v on a Mac
|
||||||
@@ -399,6 +439,10 @@ DEFAULTS = {
|
|||||||
"min_voiced_seconds": 0.3,
|
"min_voiced_seconds": 0.3,
|
||||||
"filter_hallucinations": True,
|
"filter_hallucinations": True,
|
||||||
"shortcut": "Ctrl+Space",
|
"shortcut": "Ctrl+Space",
|
||||||
|
# Ctrl+Alt+Space rather than Escape: the combination the recording started
|
||||||
|
# with, one modifier along. Escape belongs to whatever window has focus, and
|
||||||
|
# while you are dictating something else usually has it.
|
||||||
|
"cancel_shortcut": "Ctrl+Alt+Space",
|
||||||
"evdev_hotkey": False,
|
"evdev_hotkey": False,
|
||||||
"overlay_corner": "bottom-left",
|
"overlay_corner": "bottom-left",
|
||||||
"keep_audio": False,
|
"keep_audio": False,
|
||||||
@@ -453,6 +497,22 @@ LEGACY_PROMPTS = {
|
|||||||
"154fc5aca1166f00eebda705f848f0391bfbf5fe", # 1.2 English
|
"154fc5aca1166f00eebda705f848f0391bfbf5fe", # 1.2 English
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Every provider speech to text can run on, and the four settings that describe
|
||||||
|
# one. A fifth is a row here rather than another branch in transcribe_target(),
|
||||||
|
# another key row in the settings window and another line in save and load. The
|
||||||
|
# order is the order the provider box offers them in. `service` is the name the
|
||||||
|
# user sees; the environment variable that stands in for an empty key is the
|
||||||
|
# name of its setting, shouted.
|
||||||
|
Transcriber = collections.namedtuple("Transcriber", "service key url model")
|
||||||
|
TRANSCRIBERS = {
|
||||||
|
"openai": Transcriber("OpenAI", "openai_api_key", "openai_base_url",
|
||||||
|
"transcribe_model"),
|
||||||
|
"groq": Transcriber("Groq", "groq_api_key", "groq_base_url",
|
||||||
|
"groq_transcribe_model"),
|
||||||
|
"openrouter": Transcriber("OpenRouter", "openrouter_api_key",
|
||||||
|
"openrouter_base_url", "openrouter_transcribe_model"),
|
||||||
|
}
|
||||||
|
|
||||||
# Corners used to be stored with Turkish names.
|
# Corners used to be stored with Turkish names.
|
||||||
_CORNER_MIGRATION = {
|
_CORNER_MIGRATION = {
|
||||||
"sol-alt": "bottom-left", "sağ-alt": "bottom-right",
|
"sol-alt": "bottom-left", "sağ-alt": "bottom-right",
|
||||||
@@ -501,21 +561,76 @@ class Config:
|
|||||||
def get(self, key, default=None):
|
def get(self, key, default=None):
|
||||||
return self.data.get(key, DEFAULTS.get(key, default))
|
return self.data.get(key, DEFAULTS.get(key, default))
|
||||||
|
|
||||||
|
def api_key(self, setting):
|
||||||
|
"""A stored key, or the environment variable that shares its name."""
|
||||||
|
return self[setting].strip() or os.environ.get(setting.upper(), "").strip()
|
||||||
|
|
||||||
def openai_key(self):
|
def openai_key(self):
|
||||||
"""Fall back to the environment when no key is stored."""
|
return self.api_key("openai_api_key")
|
||||||
return self["openai_api_key"].strip() or os.environ.get("OPENAI_API_KEY", "").strip()
|
|
||||||
|
def groq_key(self):
|
||||||
|
return self.api_key("groq_api_key")
|
||||||
|
|
||||||
def openrouter_key(self):
|
def openrouter_key(self):
|
||||||
return self["openrouter_api_key"].strip() or os.environ.get("OPENROUTER_API_KEY", "").strip()
|
return self.api_key("openrouter_api_key")
|
||||||
|
|
||||||
def transcribe_target(self):
|
def transcribe_target(self):
|
||||||
"""Key, endpoint and model for whichever provider does speech to text."""
|
"""Key, endpoint and model for whichever provider does speech to text.
|
||||||
if self["transcribe_provider"] == "openrouter":
|
|
||||||
return api.Target("openrouter", "OpenRouter", self.openrouter_key(),
|
The local one is not in the table and leaves its base URL empty on
|
||||||
self["openrouter_base_url"],
|
purpose: the server picks a port when it starts, and reading a setting
|
||||||
self["openrouter_transcribe_model"])
|
must not be what launches a process. api.py fills the address in when it
|
||||||
return api.Target("openai", "OpenAI", self.openai_key(),
|
is about to send the request, which is the moment the server is needed
|
||||||
self["openai_base_url"], self["transcribe_model"])
|
anyway.
|
||||||
|
"""
|
||||||
|
name = self["transcribe_provider"]
|
||||||
|
if name == "local":
|
||||||
|
return api.Target("local", t("Local whisper"), "", "",
|
||||||
|
self["local_model"])
|
||||||
|
if name not in TRANSCRIBERS:
|
||||||
|
# A config written by a fork, or by a version that dropped one. The
|
||||||
|
# shipped default is not in the table, so this names the hosted one
|
||||||
|
# to land on rather than reading it from there.
|
||||||
|
name = "openai"
|
||||||
|
who = TRANSCRIBERS[name]
|
||||||
|
return api.Target(name, who.service, self.api_key(who.key),
|
||||||
|
self[who.url], self[who.model])
|
||||||
|
|
||||||
|
def transcribe_ready(self):
|
||||||
|
"""Whether speech to text could run right now, without opening Settings."""
|
||||||
|
if self["transcribe_provider"] == "local":
|
||||||
|
return self.local_whisper_ready()
|
||||||
|
return bool(self.transcribe_target().api_key)
|
||||||
|
|
||||||
|
def local_whisper_ready(self):
|
||||||
|
return bool(ggml.program_path(ggml.WHISPER, self["local_binary"])
|
||||||
|
and self["local_model"]
|
||||||
|
and ggml.have_model(ggml.whisper_model_path(self["local_model"])))
|
||||||
|
|
||||||
|
def local_llm_ready(self):
|
||||||
|
return bool(ggml.program_path(ggml.LLAMA, self["local_llm_binary"])
|
||||||
|
and self["local_llm_model"]
|
||||||
|
and ggml.have_model(ggml.llm_model_path(self["local_llm_model"])))
|
||||||
|
|
||||||
|
def apply_local(self):
|
||||||
|
"""Hand the local settings to the servers, restarting what they change."""
|
||||||
|
ggml.whisper.configure(
|
||||||
|
model=self["local_model"],
|
||||||
|
threads=int(self["local_threads"]),
|
||||||
|
gpu=bool(self["local_gpu"]),
|
||||||
|
binary=self["local_binary"],
|
||||||
|
)
|
||||||
|
ggml.llm.configure(
|
||||||
|
model=self["local_llm_model"],
|
||||||
|
threads=int(self["local_llm_threads"]),
|
||||||
|
gpu=bool(self["local_llm_gpu"]),
|
||||||
|
binary=self["local_llm_binary"],
|
||||||
|
context=int(self["local_llm_context"]),
|
||||||
|
)
|
||||||
|
|
||||||
|
def uses_local_llm(self):
|
||||||
|
"""Whether anything is set to run the local cleanup model."""
|
||||||
|
return self["cleanup_provider"] == "local"
|
||||||
|
|
||||||
def cleanup_prompt(self, with_timestamps=False, with_speakers=False,
|
def cleanup_prompt(self, with_timestamps=False, with_speakers=False,
|
||||||
subtitles=False):
|
subtitles=False):
|
||||||
|
|||||||
@@ -7,9 +7,13 @@ terminal talks to. Every verb it answers is in cli.py, which is also what runs
|
|||||||
command line says "there is no instance to talk to, so be one".
|
command line says "there is no instance to talk to, so be one".
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import contextlib
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
|
import signal
|
||||||
|
import socket
|
||||||
import sys
|
import sys
|
||||||
|
import threading
|
||||||
|
|
||||||
# A Wayland client cannot place a window in a screen corner, so the indicator
|
# A Wayland client cannot place a window in a screen corner, so the indicator
|
||||||
# is drawn through XWayland.
|
# is drawn through XWayland.
|
||||||
@@ -25,7 +29,7 @@ if sys.platform == "darwin":
|
|||||||
os.environ.get("PATH", "")) if part
|
os.environ.get("PATH", "")) if part
|
||||||
)
|
)
|
||||||
|
|
||||||
from PyQt6.QtCore import QTimer, QElapsedTimer # noqa: E402
|
from PyQt6.QtCore import QTimer, QElapsedTimer, QSocketNotifier # noqa: E402
|
||||||
from PyQt6.QtGui import QAction, QIcon # noqa: E402
|
from PyQt6.QtGui import QAction, QIcon # noqa: E402
|
||||||
from PyQt6.QtNetwork import QLocalServer, QLocalSocket # noqa: E402
|
from PyQt6.QtNetwork import QLocalServer, QLocalSocket # noqa: E402
|
||||||
from PyQt6.QtWidgets import QApplication, QMenu, QSystemTrayIcon # noqa: E402
|
from PyQt6.QtWidgets import QApplication, QMenu, QSystemTrayIcon # noqa: E402
|
||||||
@@ -34,6 +38,7 @@ import assistant # noqa: E402
|
|||||||
import audio # noqa: E402
|
import audio # noqa: E402
|
||||||
import cli # noqa: E402
|
import cli # noqa: E402
|
||||||
import config as cfg # noqa: E402
|
import config as cfg # noqa: E402
|
||||||
|
import ggml # noqa: E402
|
||||||
import hotkey # noqa: E402
|
import hotkey # noqa: E402
|
||||||
import i18n # noqa: E402
|
import i18n # noqa: E402
|
||||||
import ipc # noqa: E402
|
import ipc # noqa: E402
|
||||||
@@ -101,6 +106,9 @@ class Dikte:
|
|||||||
self.meeting_recorder = audio.MeetingRecorder()
|
self.meeting_recorder = audio.MeetingRecorder()
|
||||||
self.meetings = MeetingPipeline(self.conf)
|
self.meetings = MeetingPipeline(self.conf)
|
||||||
self.evdev = hotkey.listener()
|
self.evdev = hotkey.listener()
|
||||||
|
# Before anything of ours is started: a server from a Dikte that was
|
||||||
|
# killed outright is still holding a model in memory.
|
||||||
|
ggml.sweep()
|
||||||
|
|
||||||
self.recorder.level.connect(self._on_level)
|
self.recorder.level.connect(self._on_level)
|
||||||
self.recorder.stopped.connect(self._on_recorded)
|
self.recorder.stopped.connect(self._on_recorded)
|
||||||
@@ -161,8 +169,10 @@ class Dikte:
|
|||||||
self.ask_cancel_action.setEnabled(False)
|
self.ask_cancel_action.setEnabled(False)
|
||||||
self.menu.addAction(self.ask_cancel_action)
|
self.menu.addAction(self.ask_cancel_action)
|
||||||
|
|
||||||
self.cancel_action = QAction(t("Cancel recording"), self.menu)
|
self.cancel_action = QAction(t("Discard the recording"), self.menu)
|
||||||
self.cancel_action.triggered.connect(self.cancel)
|
# The inner method, so that a menu click is never mistaken for the KDE
|
||||||
|
# shortcut echoing the built-in listener's press.
|
||||||
|
self.cancel_action.triggered.connect(self._cancel)
|
||||||
self.cancel_action.setEnabled(False)
|
self.cancel_action.setEnabled(False)
|
||||||
self.menu.addAction(self.cancel_action)
|
self.menu.addAction(self.cancel_action)
|
||||||
self.menu.addSeparator()
|
self.menu.addSeparator()
|
||||||
@@ -315,6 +325,9 @@ class Dikte:
|
|||||||
def toggle_meeting(self):
|
def toggle_meeting(self):
|
||||||
self._external("meeting", self._toggle_meeting)
|
self._external("meeting", self._toggle_meeting)
|
||||||
|
|
||||||
|
def cancel(self):
|
||||||
|
self._external("cancel", self._cancel)
|
||||||
|
|
||||||
def _external(self, name, handler):
|
def _external(self, name, handler):
|
||||||
# The built-in listener sees the key press the instant it happens, so a
|
# The built-in listener sees the key press the instant it happens, so a
|
||||||
# toggle arriving right behind one is the KDE shortcut catching up on
|
# toggle arriving right behind one is the KDE shortcut catching up on
|
||||||
@@ -335,7 +348,8 @@ class Dikte:
|
|||||||
if timer is None:
|
if timer is None:
|
||||||
timer = self.last_evdev[name] = QElapsedTimer()
|
timer = self.last_evdev[name] = QElapsedTimer()
|
||||||
timer.restart()
|
timer.restart()
|
||||||
handlers = {"meeting": self._toggle_meeting, "ask": self._toggle_ask}
|
handlers = {"meeting": self._toggle_meeting, "ask": self._toggle_ask,
|
||||||
|
"cancel": self._cancel}
|
||||||
handlers.get(name, self._toggle)()
|
handlers.get(name, self._toggle)()
|
||||||
|
|
||||||
def _retire_listener(self):
|
def _retire_listener(self):
|
||||||
@@ -529,7 +543,7 @@ class Dikte:
|
|||||||
self.ask_overlay.show_busy(t("Transcribing…"))
|
self.ask_overlay.show_busy(t("Transcribing…"))
|
||||||
self.recorder.stop()
|
self.recorder.stop()
|
||||||
|
|
||||||
def cancel(self):
|
def _cancel(self):
|
||||||
"""Throw away whichever recording is running."""
|
"""Throw away whichever recording is running."""
|
||||||
if not self.recording:
|
if not self.recording:
|
||||||
return
|
return
|
||||||
@@ -553,7 +567,7 @@ class Dikte:
|
|||||||
def cancel_ask(self):
|
def cancel_ask(self):
|
||||||
"""Call off the agent, whether it is still recording or already working."""
|
"""Call off the agent, whether it is still recording or already working."""
|
||||||
if self.ask_state == RECORDING:
|
if self.ask_state == RECORDING:
|
||||||
self.cancel()
|
self._cancel()
|
||||||
elif self.ask_state == BUSY:
|
elif self.ask_state == BUSY:
|
||||||
self.ask_overlay.show_busy(t("Stopping…"))
|
self.ask_overlay.show_busy(t("Stopping…"))
|
||||||
self.ask_pipeline.cancel()
|
self.ask_pipeline.cancel()
|
||||||
@@ -801,10 +815,7 @@ class Dikte:
|
|||||||
|
|
||||||
def open_settings(self):
|
def open_settings(self):
|
||||||
if self.settings_window is None:
|
if self.settings_window is None:
|
||||||
self.settings_window = SettingsWindow(
|
self.settings_window = SettingsWindow(self.conf, self.meetings)
|
||||||
self.conf, launch_command(), meeting_command(), self.meetings,
|
|
||||||
ask_command(),
|
|
||||||
)
|
|
||||||
self.settings_window.applied.connect(self._apply_settings)
|
self.settings_window.applied.connect(self._apply_settings)
|
||||||
self.settings_window.finished.connect(self._settings_closed)
|
self.settings_window.finished.connect(self._settings_closed)
|
||||||
self.settings_window.show()
|
self.settings_window.show()
|
||||||
@@ -815,18 +826,52 @@ class Dikte:
|
|||||||
# Don't drop the object while its own signal is still being delivered.
|
# Don't drop the object while its own signal is still being delivered.
|
||||||
QTimer.singleShot(0, lambda: setattr(self, "settings_window", None))
|
QTimer.singleShot(0, lambda: setattr(self, "settings_window", None))
|
||||||
|
|
||||||
|
def _apply_local(self):
|
||||||
|
"""Pass the local settings on, and hold the models ready if asked to.
|
||||||
|
|
||||||
|
Loading a model takes a second or two for whisper and longer for an LLM.
|
||||||
|
Doing it while Dikte starts rather than on the first dictation is the
|
||||||
|
whole reason a server is kept alive instead of running the program once
|
||||||
|
per recording; the checkboxes are there for the machine whose memory is
|
||||||
|
wanted elsewhere.
|
||||||
|
"""
|
||||||
|
self.conf.apply_local()
|
||||||
|
wanted = []
|
||||||
|
if self.conf["transcribe_provider"] == "local":
|
||||||
|
if self.conf["local_preload"] and self.conf.local_whisper_ready():
|
||||||
|
wanted.append((ggml.whisper, "whisper"))
|
||||||
|
else:
|
||||||
|
ggml.whisper.stop() # give the memory back when it is not in use
|
||||||
|
if self.conf.uses_local_llm():
|
||||||
|
if self.conf["local_llm_preload"] and self.conf.local_llm_ready():
|
||||||
|
wanted.append((ggml.llm, "llama"))
|
||||||
|
else:
|
||||||
|
ggml.llm.stop()
|
||||||
|
|
||||||
|
def warm():
|
||||||
|
for server, name in wanted:
|
||||||
|
try:
|
||||||
|
server.serve()
|
||||||
|
except ggml.LocalError as exc:
|
||||||
|
# Not worth an indicator: the first dictation raises the
|
||||||
|
# same thing where the user can act on it.
|
||||||
|
print(f"dikte: {name}: {exc}", file=sys.stderr)
|
||||||
|
|
||||||
|
if wanted:
|
||||||
|
threading.Thread(target=warm, daemon=True).start()
|
||||||
|
|
||||||
def _apply_settings(self):
|
def _apply_settings(self):
|
||||||
self.overlay.corner = self.conf["overlay_corner"]
|
self.overlay.corner = self.conf["overlay_corner"]
|
||||||
self.ask_overlay.corner = self.conf["overlay_corner"]
|
self.ask_overlay.corner = self.conf["overlay_corner"]
|
||||||
|
self._apply_local()
|
||||||
self._build_tray()
|
self._build_tray()
|
||||||
self._refresh_tray()
|
self._refresh_tray()
|
||||||
# Where the desktop has no shortcut registry of its own, the listener is
|
# Where the desktop has no shortcut registry of its own, the listener is
|
||||||
# not the fallback the setting offers to turn on: it is the only way the
|
# not the fallback the setting offers to turn on: it is the only way the
|
||||||
# keys arrive at all, so it runs whatever the setting says.
|
# keys arrive at all, so it runs whatever the setting says.
|
||||||
if self.conf["evdev_hotkey"] or not hotkey.installs_shortcuts():
|
if self.conf["evdev_hotkey"] or not hotkey.installs_shortcuts():
|
||||||
self.evdev.start({"toggle": self.conf["shortcut"],
|
self.evdev.start({name: self.conf[spec.setting]
|
||||||
"ask": self.conf["assistant_shortcut"],
|
for name, spec in hotkey.SHORTCUTS.items()})
|
||||||
"meeting": self.conf["meeting_shortcut"]})
|
|
||||||
else:
|
else:
|
||||||
self.evdev.stop()
|
self.evdev.stop()
|
||||||
|
|
||||||
@@ -850,6 +895,9 @@ class Dikte:
|
|||||||
self.meeting_recorder.stop()
|
self.meeting_recorder.stop()
|
||||||
self.overlay.dismiss()
|
self.overlay.dismiss()
|
||||||
self.ask_overlay.dismiss()
|
self.ask_overlay.dismiss()
|
||||||
|
# Also on the restart path, which replaces the process without ever
|
||||||
|
# reaching atexit and would otherwise leave the models in memory.
|
||||||
|
ggml.stop_all()
|
||||||
self.tray.hide()
|
self.tray.hide()
|
||||||
|
|
||||||
|
|
||||||
@@ -865,19 +913,6 @@ def _clock(seconds):
|
|||||||
else f"{minutes}:{secs:02d}")
|
else f"{minutes}:{secs:02d}")
|
||||||
|
|
||||||
|
|
||||||
def launch_command():
|
|
||||||
"""The command the KDE shortcut will run."""
|
|
||||||
return ipc.command_for("toggle")
|
|
||||||
|
|
||||||
|
|
||||||
def meeting_command():
|
|
||||||
return ipc.command_for("meeting")
|
|
||||||
|
|
||||||
|
|
||||||
def ask_command():
|
|
||||||
return ipc.command_for("ask")
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
argv = sys.argv[1:]
|
argv = sys.argv[1:]
|
||||||
# Anything typed at a terminal is the command line's business, including
|
# Anything typed at a terminal is the command line's business, including
|
||||||
@@ -888,6 +923,41 @@ def main():
|
|||||||
return run_app([arg for arg in argv if arg != "--gui"])
|
return run_app([arg for arg in argv if arg != "--gui"])
|
||||||
|
|
||||||
|
|
||||||
|
def install_signal_handlers(app):
|
||||||
|
"""Quit properly on the signals a session sends, rather than dying where we stand.
|
||||||
|
|
||||||
|
Qt spends its time blocked inside C, and a Python signal handler only runs
|
||||||
|
between bytecodes, so on its own it would not run until the next event
|
||||||
|
arrived, which for an idle tray icon may be never. set_wakeup_fd writes the
|
||||||
|
signal number to a socket instead, and a notifier turns that into an event
|
||||||
|
Qt does deliver.
|
||||||
|
|
||||||
|
Worth the trouble because of what shutdown() does: a logout sends SIGTERM,
|
||||||
|
and without this a whisper.cpp or llama.cpp server outlives the session
|
||||||
|
holding its model in memory. SIGKILL cannot be caught at all, which is what
|
||||||
|
ggml.sweep() is for.
|
||||||
|
|
||||||
|
Returns the objects it made; they have to stay alive to keep working.
|
||||||
|
"""
|
||||||
|
reader, writer = socket.socketpair()
|
||||||
|
reader.setblocking(False)
|
||||||
|
writer.setblocking(False)
|
||||||
|
signal.set_wakeup_fd(writer.fileno())
|
||||||
|
notifier = QSocketNotifier(reader.fileno(), QSocketNotifier.Type.Read)
|
||||||
|
|
||||||
|
def woken():
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
reader.recv(64)
|
||||||
|
app.quit() # aboutToQuit runs shutdown()
|
||||||
|
|
||||||
|
notifier.activated.connect(woken)
|
||||||
|
for sig in (signal.SIGINT, signal.SIGTERM, signal.SIGHUP):
|
||||||
|
# A handler that does nothing, so that the default action, stopping the
|
||||||
|
# process where it stands, is replaced by the wakeup above.
|
||||||
|
signal.signal(sig, lambda *_: None)
|
||||||
|
return reader, writer, notifier
|
||||||
|
|
||||||
|
|
||||||
def run_app(args):
|
def run_app(args):
|
||||||
command = args[0] if args else ""
|
command = args[0] if args else ""
|
||||||
|
|
||||||
@@ -895,6 +965,12 @@ def run_app(args):
|
|||||||
app.setApplicationName("Dikte")
|
app.setApplicationName("Dikte")
|
||||||
app.setDesktopFileName("dikte")
|
app.setDesktopFileName("dikte")
|
||||||
app.setQuitOnLastWindowClosed(False)
|
app.setQuitOnLastWindowClosed(False)
|
||||||
|
# Before Dikte is built, because building it is what may start a server, and
|
||||||
|
# a signal arriving in the middle of that would otherwise take the default
|
||||||
|
# action and leave the server behind. A signal this early lands in the
|
||||||
|
# socket and is delivered as soon as the event loop starts. Held in a name
|
||||||
|
# so that the notifier and its socket outlive this function.
|
||||||
|
signal_plumbing = install_signal_handlers(app) # noqa: F841
|
||||||
|
|
||||||
if not QSystemTrayIcon.isSystemTrayAvailable():
|
if not QSystemTrayIcon.isSystemTrayAvailable():
|
||||||
print("dikte: no system tray found, running anyway")
|
print("dikte: no system tray found, running anyway")
|
||||||
@@ -943,7 +1019,10 @@ def run_app(args):
|
|||||||
|
|
||||||
# No key for the chosen transcription provider means nothing can work yet,
|
# No key for the chosen transcription provider means nothing can work yet,
|
||||||
# so the settings window is the only useful thing to open.
|
# so the settings window is the only useful thing to open.
|
||||||
if command == "settings" or not dikte.conf.transcribe_target().api_key:
|
# A transcription provider that cannot run yet, whether that is a missing
|
||||||
|
# API key or a model nobody has downloaded, means nothing can work, so the
|
||||||
|
# settings window is the only useful thing to open.
|
||||||
|
if command == "settings" or not dikte.conf.transcribe_ready():
|
||||||
dikte.open_settings()
|
dikte.open_settings()
|
||||||
elif command == "toggle":
|
elif command == "toggle":
|
||||||
QTimer.singleShot(0, dikte.toggle)
|
QTimer.singleShot(0, dikte.toggle)
|
||||||
|
|||||||
|
After Width: | Height: | Size: 92 KiB |
|
Before Width: | Height: | Size: 61 KiB After Width: | Height: | Size: 71 KiB |
|
Before Width: | Height: | Size: 68 KiB After Width: | Height: | Size: 48 KiB |
|
Before Width: | Height: | Size: 99 KiB After Width: | Height: | Size: 116 KiB |
|
Before Width: | Height: | Size: 71 KiB After Width: | Height: | Size: 70 KiB |
|
Before Width: | Height: | Size: 66 KiB |
|
After Width: | Height: | Size: 83 KiB |
|
After Width: | Height: | Size: 58 KiB |
@@ -17,6 +17,8 @@ import wave
|
|||||||
from PyQt6.QtCore import QObject, pyqtSignal
|
from PyQt6.QtCore import QObject, pyqtSignal
|
||||||
|
|
||||||
import api
|
import api
|
||||||
|
import cleanup
|
||||||
|
import ggml
|
||||||
from i18n import t
|
from i18n import t
|
||||||
|
|
||||||
CHUNK_SECONDS = 600 # 10 min ≈ 19 MB at 16 kHz mono s16
|
CHUNK_SECONDS = 600 # 10 min ≈ 19 MB at 16 kHz mono s16
|
||||||
@@ -28,8 +30,9 @@ MIN_SUBTITLE_SECONDS = 1.5 # how long a cue with no end time of its own stays
|
|||||||
STAMP_RE = re.compile(r"^\[(?:(\d+):)?(\d{1,2}):(\d{2})\]\s*")
|
STAMP_RE = re.compile(r"^\[(?:(\d+):)?(\d{1,2}):(\d{2})\]\s*")
|
||||||
|
|
||||||
|
|
||||||
class Cancelled(Exception):
|
# What a stopped run comes back with, wherever it was stopped: the request that
|
||||||
pass
|
# was cut off raises it from api, and the steps in between raise it themselves.
|
||||||
|
Cancelled = api.Aborted
|
||||||
|
|
||||||
|
|
||||||
class FileTranscriber(QObject):
|
class FileTranscriber(QObject):
|
||||||
@@ -41,7 +44,9 @@ class FileTranscriber(QObject):
|
|||||||
super().__init__(parent)
|
super().__init__(parent)
|
||||||
self.conf = conf
|
self.conf = conf
|
||||||
self._thread = None
|
self._thread = None
|
||||||
self._stop = threading.Event()
|
self._abort = api.Aborter()
|
||||||
|
# The server on this machine the work is with, when it is with one.
|
||||||
|
self._local = None
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def busy(self):
|
def busy(self):
|
||||||
@@ -50,18 +55,26 @@ class FileTranscriber(QObject):
|
|||||||
def start(self, path, timestamps, do_cleanup):
|
def start(self, path, timestamps, do_cleanup):
|
||||||
if self.busy:
|
if self.busy:
|
||||||
return
|
return
|
||||||
self._stop.clear()
|
self._abort = api.Aborter() # the last one is spent
|
||||||
self._thread = threading.Thread(
|
self._thread = threading.Thread(
|
||||||
target=self._work, args=(path, timestamps, do_cleanup), daemon=True
|
target=self._work, args=(path, timestamps, do_cleanup), daemon=True
|
||||||
)
|
)
|
||||||
self._thread.start()
|
self._thread.start()
|
||||||
|
|
||||||
def stop(self):
|
def stop(self):
|
||||||
self._stop.set()
|
"""Cut the run off where it stands, rather than at the next step."""
|
||||||
|
self._abort.abort()
|
||||||
|
# Closing the socket is nothing to a server on this machine: it is a
|
||||||
|
# process of ours, and it would grind on to the end of the chunk with
|
||||||
|
# nobody left to hand the answer to. Stopping it is what stops the
|
||||||
|
# work; the next run starts it again. Killing waits on the process, so
|
||||||
|
# not on the thread the window is drawn from.
|
||||||
|
local = self._local
|
||||||
|
if local is not None:
|
||||||
|
threading.Thread(target=local.stop, daemon=True).start()
|
||||||
|
|
||||||
def _check(self):
|
def _check(self):
|
||||||
if self._stop.is_set():
|
self._abort.check()
|
||||||
raise Cancelled
|
|
||||||
|
|
||||||
def _work(self, path, timestamps, do_cleanup):
|
def _work(self, path, timestamps, do_cleanup):
|
||||||
conf = self.conf
|
conf = self.conf
|
||||||
@@ -72,7 +85,7 @@ class FileTranscriber(QObject):
|
|||||||
|
|
||||||
workdir = tempfile.mkdtemp(prefix="dikte-file-")
|
workdir = tempfile.mkdtemp(prefix="dikte-file-")
|
||||||
self.progress.emit(t("Converting audio…"))
|
self.progress.emit(t("Converting audio…"))
|
||||||
wav_path = _to_wav(path, workdir)
|
wav_path = _to_wav(path, workdir, self._abort)
|
||||||
self._check()
|
self._check()
|
||||||
|
|
||||||
chunks = split_wav(wav_path, workdir)
|
chunks = split_wav(wav_path, workdir)
|
||||||
@@ -80,6 +93,7 @@ class FileTranscriber(QObject):
|
|||||||
self.progress.emit(t("Splitting into {count} chunks…", count=len(chunks)))
|
self.progress.emit(t("Splitting into {count} chunks…", count=len(chunks)))
|
||||||
|
|
||||||
target = conf.transcribe_target()
|
target = conf.transcribe_target()
|
||||||
|
self._local = ggml.whisper if target.provider == "local" else None
|
||||||
pieces = []
|
pieces = []
|
||||||
segments = []
|
segments = []
|
||||||
for index, (chunk_path, offset) in enumerate(chunks, start=1):
|
for index, (chunk_path, offset) in enumerate(chunks, start=1):
|
||||||
@@ -95,6 +109,7 @@ class FileTranscriber(QObject):
|
|||||||
chunk_path,
|
chunk_path,
|
||||||
language=conf["language"],
|
language=conf["language"],
|
||||||
prompt=conf["transcribe_prompt"],
|
prompt=conf["transcribe_prompt"],
|
||||||
|
aborter=self._abort,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
pieces = [f"[{format_timestamp(start)}] {line}"
|
pieces = [f"[{format_timestamp(start)}] {line}"
|
||||||
@@ -105,6 +120,7 @@ class FileTranscriber(QObject):
|
|||||||
chunk_path,
|
chunk_path,
|
||||||
language=conf["language"],
|
language=conf["language"],
|
||||||
prompt=conf["transcribe_prompt"],
|
prompt=conf["transcribe_prompt"],
|
||||||
|
aborter=self._abort,
|
||||||
))
|
))
|
||||||
|
|
||||||
text = "\n".join(pieces) if timestamps else " ".join(pieces)
|
text = "\n".join(pieces) if timestamps else " ".join(pieces)
|
||||||
@@ -121,23 +137,18 @@ class FileTranscriber(QObject):
|
|||||||
except (api.ApiError, OSError, subprocess.SubprocessError, wave.Error) as exc:
|
except (api.ApiError, OSError, subprocess.SubprocessError, wave.Error) as exc:
|
||||||
self.failed.emit(str(exc))
|
self.failed.emit(str(exc))
|
||||||
finally:
|
finally:
|
||||||
|
self._local = None
|
||||||
if workdir:
|
if workdir:
|
||||||
shutil.rmtree(workdir, ignore_errors=True)
|
shutil.rmtree(workdir, ignore_errors=True)
|
||||||
|
|
||||||
def _cleanup(self, text, timestamps):
|
def _cleanup(self, text, timestamps):
|
||||||
conf = self.conf
|
conf = self.conf
|
||||||
|
self._local = ggml.llm if cleanup.provider(conf) == "local" else None
|
||||||
prompt = conf.cleanup_prompt(with_timestamps=timestamps, subtitles=True)
|
prompt = conf.cleanup_prompt(with_timestamps=timestamps, subtitles=True)
|
||||||
out = []
|
out = []
|
||||||
for block in split_text(text, timestamps):
|
for block in split_text(text, timestamps):
|
||||||
self._check()
|
self._check()
|
||||||
out.append(api.cleanup(
|
out.append(cleanup.run(block, conf, prompt, aborter=self._abort))
|
||||||
block,
|
|
||||||
conf.openrouter_key(),
|
|
||||||
conf["cleanup_model"],
|
|
||||||
prompt,
|
|
||||||
reasoning=conf["cleanup_reasoning"],
|
|
||||||
base_url=conf["openrouter_base_url"],
|
|
||||||
))
|
|
||||||
return ("\n" if timestamps else "\n\n").join(out)
|
return ("\n" if timestamps else "\n\n").join(out)
|
||||||
|
|
||||||
|
|
||||||
@@ -200,17 +211,34 @@ def to_srt(text, segments):
|
|||||||
return "\n\n".join(blocks) + "\n" if blocks else ""
|
return "\n\n".join(blocks) + "\n" if blocks else ""
|
||||||
|
|
||||||
|
|
||||||
def _to_wav(path, workdir):
|
def _reap(proc):
|
||||||
|
"""Leave nothing running behind a conversion that did not finish."""
|
||||||
|
if proc.poll() is None:
|
||||||
|
proc.kill()
|
||||||
|
proc.wait()
|
||||||
|
|
||||||
|
|
||||||
|
def _to_wav(path, workdir, aborter=None):
|
||||||
out = os.path.join(workdir, "audio.wav")
|
out = os.path.join(workdir, "audio.wav")
|
||||||
res = subprocess.run(
|
proc = subprocess.Popen(
|
||||||
["ffmpeg", "-nostdin", "-y", "-i", path, "-vn",
|
["ffmpeg", "-nostdin", "-y", "-i", path, "-vn",
|
||||||
"-ac", "1", "-ar", str(RATE), "-c:a", "pcm_s16le", out],
|
"-ac", "1", "-ar", str(RATE), "-c:a", "pcm_s16le", out],
|
||||||
capture_output=True, text=True,
|
stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
|
||||||
|
text=True,
|
||||||
)
|
)
|
||||||
if res.returncode != 0 or not os.path.exists(out):
|
# A two hour film is a minute of ffmpeg, which is a minute of a Stop button
|
||||||
tail = (res.stderr or "").strip().splitlines()
|
# doing nothing unless the abort reaches the process itself.
|
||||||
|
with contextlib.ExitStack() as stack:
|
||||||
|
stack.callback(_reap, proc)
|
||||||
|
if aborter is not None:
|
||||||
|
stack.enter_context(aborter.holding(proc.kill))
|
||||||
|
_stdout, stderr = proc.communicate()
|
||||||
|
if aborter is not None:
|
||||||
|
aborter.check()
|
||||||
|
if proc.returncode != 0 or not os.path.exists(out):
|
||||||
|
tail = (stderr or "").strip().splitlines()
|
||||||
raise api.ApiError(t("Could not read the file: {error}",
|
raise api.ApiError(t("Could not read the file: {error}",
|
||||||
error=tail[-1] if tail else res.returncode))
|
error=tail[-1] if tail else proc.returncode))
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,792 @@
|
|||||||
|
"""Speech to text and cleanup on this machine: whisper.cpp and llama.cpp.
|
||||||
|
|
||||||
|
Two programs, one treatment. Fetch a release from GitHub, unpack it under the
|
||||||
|
data directory, fetch a model from Hugging Face, then keep one server alive on a
|
||||||
|
port of its own. Both of them speak the shape api.py already sends to the hosted
|
||||||
|
providers, so what the rest of Dikte sees is a base URL and nothing else:
|
||||||
|
whisper-server is started on `--inference-path /v1/audio/transcriptions`, the
|
||||||
|
exact path api.py builds, and llama-server answers /v1/chat/completions the way
|
||||||
|
OpenRouter does.
|
||||||
|
|
||||||
|
A server rather than a one-shot run, because the model is the slow part. Loading
|
||||||
|
a large whisper model takes a second or two while transcribing a few seconds of
|
||||||
|
speech takes a fraction of one, and an LLM is worse: a server pays that once and
|
||||||
|
a run per dictation pays it every time.
|
||||||
|
|
||||||
|
Nothing downloaded is trusted for having arrived. Every file is checked against
|
||||||
|
the sha256 its index published, and the bytes go to a `.part` that is only
|
||||||
|
renamed once the whole thing is there, so an interrupted download can never be
|
||||||
|
mistaken for a working one.
|
||||||
|
|
||||||
|
This module imports hub and the string table, and nothing else of Dikte's: it
|
||||||
|
knows how to fetch a file and how to run a process, and nothing about dictation.
|
||||||
|
Its errors leave as LocalError and api.py turns them into the ApiError the
|
||||||
|
interface already knows how to show.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import atexit
|
||||||
|
import collections
|
||||||
|
import ctypes.util
|
||||||
|
import hashlib
|
||||||
|
import http.client
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import pathlib
|
||||||
|
import platform
|
||||||
|
import shutil
|
||||||
|
import signal
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import tarfile
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
import hub
|
||||||
|
from i18n import t
|
||||||
|
|
||||||
|
HOST = "127.0.0.1"
|
||||||
|
# The path api.py asks for, so its URL and the server's line up.
|
||||||
|
INFERENCE_PATH = "/v1/audio/transcriptions"
|
||||||
|
|
||||||
|
DATA_DIR = (pathlib.Path(os.environ.get("XDG_DATA_HOME")
|
||||||
|
or os.path.expanduser("~/.local/share")) / "dikte")
|
||||||
|
BIN_DIR = DATA_DIR / "bin"
|
||||||
|
MODELS_DIR = DATA_DIR / "models"
|
||||||
|
|
||||||
|
# Loading a large model onto a GPU is the slow part of a start, and on a cold
|
||||||
|
# page cache a large LLM read from a spinning disk is slower still.
|
||||||
|
STARTUP_TIMEOUT = 180.0
|
||||||
|
DOWNLOAD_CHUNK = 1 << 20
|
||||||
|
|
||||||
|
# `health` is the path that answers only once the model is in memory. whisper
|
||||||
|
# does not have one and does not need one: it binds its port after the model is
|
||||||
|
# loaded, so the port opening is the signal.
|
||||||
|
Program = collections.namedtuple("Program", "name repo binary health")
|
||||||
|
|
||||||
|
WHISPER = Program("whisper", "ggml-org/whisper.cpp", "whisper-server", "")
|
||||||
|
LLAMA = Program("llama", "ggml-org/llama.cpp", "llama-server", "/health")
|
||||||
|
|
||||||
|
# Where the models are listed. Neither list is written into Dikte: a catalogue
|
||||||
|
# in the source means a release of Dikte for every model somebody else
|
||||||
|
# publishes.
|
||||||
|
WHISPER_MODELS_REPO = "ggerganov/whisper.cpp"
|
||||||
|
LLM_AUTHOR = "ggml-org"
|
||||||
|
|
||||||
|
# What the whisper repository holds besides models: Core ML encoders for Apple
|
||||||
|
# hardware and the odd loose file.
|
||||||
|
WHISPER_PREFIX = "ggml-"
|
||||||
|
WHISPER_SUFFIX = ".bin"
|
||||||
|
|
||||||
|
# What a GGUF repository holds besides the model: mmproj is the vision half of a
|
||||||
|
# multimodal model, mtp a draft head for speculative decoding. Neither is a model
|
||||||
|
# a server can be started on, and offering them is offering a failure.
|
||||||
|
GGUF_SKIP = ("mmproj", "mtp-")
|
||||||
|
# Big enough for a 12B at Q4 and far past anything cleanup wants; the point is
|
||||||
|
# to keep a 400 GB frontier model out of a list somebody might click.
|
||||||
|
GGUF_MAX_BYTES = 16 << 30
|
||||||
|
|
||||||
|
# Suggestions, not a catalogue: the list itself is fetched, and these are only
|
||||||
|
# the rows that float to the top of it. Small instruction-following models,
|
||||||
|
# because cleanup is punctuation and filler words rather than anything that
|
||||||
|
# wants thinking about.
|
||||||
|
SUGGESTED_LLM = (
|
||||||
|
"ggml-org/gemma-3-4b-it-GGUF",
|
||||||
|
"ggml-org/gemma-4-E2B-it-GGUF",
|
||||||
|
"ggml-org/gemma-4-E4B-it-GGUF",
|
||||||
|
"ggml-org/SmolLM3-3B-GGUF",
|
||||||
|
)
|
||||||
|
# Turbo at q5_0 is smaller than `small` and better than it, which makes the
|
||||||
|
# usual "start small" advice point at the same file as "start good".
|
||||||
|
SUGGESTED_WHISPER = "ggml-large-v3-turbo-q5_0.bin"
|
||||||
|
|
||||||
|
|
||||||
|
class LocalError(Exception):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def human_size(count):
|
||||||
|
for unit in ("B", "KB", "MB", "GB"):
|
||||||
|
if count < 1024 or unit == "GB":
|
||||||
|
return f"{count:.0f} {unit}" if unit == "B" else f"{count:.1f} {unit}"
|
||||||
|
count /= 1024.0
|
||||||
|
return f"{count:.1f} GB"
|
||||||
|
|
||||||
|
|
||||||
|
# --- fetching -------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def download(item, target, on_progress=None, should_stop=None, require_hash=True):
|
||||||
|
"""Fetch one hub.Item to `target`. True when it landed, False when stopped.
|
||||||
|
|
||||||
|
The bytes go to a `.part` that is renamed only after both the length and the
|
||||||
|
hash agree with what the index said. A truncated file would otherwise sit
|
||||||
|
there looking installed and fail much later, inside a server, as a corrupt
|
||||||
|
model; a file that is the right length but the wrong content is worse, and
|
||||||
|
this is a program as often as it is a model.
|
||||||
|
|
||||||
|
A file whose index published no hash is refused rather than taken on trust.
|
||||||
|
Everything fetched here is either run or parsed by something written in C++,
|
||||||
|
and GitHub did not always publish a digest: a release old enough to predate
|
||||||
|
that would otherwise install unchecked, which is the one case where this
|
||||||
|
would matter most and say least.
|
||||||
|
"""
|
||||||
|
target = pathlib.Path(target)
|
||||||
|
if require_hash and not item.sha256:
|
||||||
|
raise LocalError(t("{name} is published without a checksum, so there is "
|
||||||
|
"no way to tell what arrived. Nothing was installed.",
|
||||||
|
name=item.name))
|
||||||
|
part = target.with_name(target.name + ".part")
|
||||||
|
try:
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
except OSError as exc:
|
||||||
|
raise LocalError(t("Could not create {path}: {error}",
|
||||||
|
path=target.parent, error=exc)) from exc
|
||||||
|
|
||||||
|
request = urllib.request.Request(item.url, headers={"User-Agent": hub.USER_AGENT})
|
||||||
|
digest = hashlib.sha256()
|
||||||
|
done = 0
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=60) as response:
|
||||||
|
total = int(response.headers.get("Content-Length") or item.size or 0)
|
||||||
|
with open(part, "wb") as out:
|
||||||
|
while True:
|
||||||
|
if should_stop is not None and should_stop():
|
||||||
|
part.unlink(missing_ok=True)
|
||||||
|
return False
|
||||||
|
block = response.read(DOWNLOAD_CHUNK)
|
||||||
|
if not block:
|
||||||
|
break
|
||||||
|
out.write(block)
|
||||||
|
digest.update(block)
|
||||||
|
done += len(block)
|
||||||
|
# More than was announced: a body that does not end is the
|
||||||
|
# one way this loop could run until the disk is full.
|
||||||
|
if total and done > total:
|
||||||
|
part.unlink(missing_ok=True)
|
||||||
|
raise LocalError(t("{name} is longer than it said it "
|
||||||
|
"would be.", name=item.name))
|
||||||
|
if on_progress is not None:
|
||||||
|
on_progress(done, total)
|
||||||
|
# A proxy notice or an error page that came back as 200 would otherwise
|
||||||
|
# be renamed into place and only fail when something tries to read it.
|
||||||
|
if total and done != total:
|
||||||
|
part.unlink(missing_ok=True)
|
||||||
|
raise LocalError(t("The download stopped early ({done} of {total}).",
|
||||||
|
done=human_size(done), total=human_size(total)))
|
||||||
|
if item.sha256 and digest.hexdigest() != item.sha256:
|
||||||
|
part.unlink(missing_ok=True)
|
||||||
|
raise LocalError(t("{name} does not match its published checksum. "
|
||||||
|
"Nothing was installed.", name=item.name))
|
||||||
|
part.replace(target)
|
||||||
|
return True
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
part.unlink(missing_ok=True)
|
||||||
|
exc.close() # it holds the response body open until it is collected
|
||||||
|
raise LocalError(t("Could not download {name}: HTTP {code}",
|
||||||
|
name=item.name, code=exc.code)) from exc
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
part.unlink(missing_ok=True)
|
||||||
|
raise LocalError(t("Could not download {name}: {error}",
|
||||||
|
name=item.name, error=exc.reason)) from exc
|
||||||
|
except OSError as exc:
|
||||||
|
# A connection cut mid-body arrives here too, and gigabytes in is
|
||||||
|
# exactly where that happens.
|
||||||
|
part.unlink(missing_ok=True)
|
||||||
|
raise LocalError(t("Could not write {name}: {error}",
|
||||||
|
name=item.name, error=exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
# --- the programs ---------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _arch():
|
||||||
|
machine = platform.machine().lower()
|
||||||
|
if machine in ("aarch64", "arm64"):
|
||||||
|
return "arm64"
|
||||||
|
return "x64"
|
||||||
|
|
||||||
|
|
||||||
|
def _has_vulkan():
|
||||||
|
"""Whether a Vulkan loader is installed, which decides which build to fetch.
|
||||||
|
|
||||||
|
llama.cpp publishes no CUDA build for Linux, so Vulkan is what a graphics
|
||||||
|
card gets here. The build without it is smaller and runs on the CPU, and
|
||||||
|
fetching the Vulkan one for a machine that cannot load it would only make
|
||||||
|
the download bigger.
|
||||||
|
"""
|
||||||
|
return bool(ctypes.util.find_library("vulkan"))
|
||||||
|
|
||||||
|
|
||||||
|
def _wanted_assets(program):
|
||||||
|
"""Asset name endings to accept, best first."""
|
||||||
|
arch = _arch()
|
||||||
|
if program is LLAMA and _has_vulkan():
|
||||||
|
return (f"bin-ubuntu-vulkan-{arch}.tar.gz", f"bin-ubuntu-{arch}.tar.gz")
|
||||||
|
return (f"bin-ubuntu-{arch}.tar.gz",)
|
||||||
|
|
||||||
|
|
||||||
|
def _install_record(program):
|
||||||
|
return BIN_DIR / program.name / "installed.json"
|
||||||
|
|
||||||
|
|
||||||
|
def installed_program(program):
|
||||||
|
"""The binary Dikte downloaded, or "" when there is none that still runs."""
|
||||||
|
try:
|
||||||
|
record = json.loads(_install_record(program).read_text(encoding="utf-8"))
|
||||||
|
path = record.get("binary") or ""
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return ""
|
||||||
|
return path if os.path.isfile(path) and os.access(path, os.X_OK) else ""
|
||||||
|
|
||||||
|
|
||||||
|
def installed_version(program):
|
||||||
|
try:
|
||||||
|
record = json.loads(_install_record(program).read_text(encoding="utf-8"))
|
||||||
|
return record.get("tag") or ""
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def program_path(program, custom=""):
|
||||||
|
"""Which copy of the program to run, or "" when there is none.
|
||||||
|
|
||||||
|
A system one wins over a downloaded one. The distribution package is built
|
||||||
|
against whatever the machine has, which on this platform means it may reach
|
||||||
|
the graphics card, while the release binaries carry CPU backends only.
|
||||||
|
"""
|
||||||
|
custom = (custom or "").strip()
|
||||||
|
if custom:
|
||||||
|
return custom if os.path.isfile(custom) and os.access(custom, os.X_OK) else ""
|
||||||
|
return shutil.which(program.binary) or installed_program(program)
|
||||||
|
|
||||||
|
|
||||||
|
def system_program(program):
|
||||||
|
"""Whether the program came from the system rather than from Dikte."""
|
||||||
|
return bool(shutil.which(program.binary))
|
||||||
|
|
||||||
|
|
||||||
|
def _find_binary(root, name):
|
||||||
|
for path in sorted(pathlib.Path(root).rglob(name)):
|
||||||
|
if path.is_file():
|
||||||
|
return path
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _extract(archive, into):
|
||||||
|
"""Unpack a release tarball, refusing anything that reaches outside `into`.
|
||||||
|
|
||||||
|
The archives lay their libraries next to their binaries and are linked with
|
||||||
|
an $ORIGIN runpath, so a whole directory is what has to survive the trip and
|
||||||
|
the binary cannot be lifted out of it.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with tarfile.open(archive, "r:gz") as tar:
|
||||||
|
try:
|
||||||
|
tar.extractall(into, filter="data")
|
||||||
|
except TypeError: # Python without the extraction filters
|
||||||
|
tar.extractall(into)
|
||||||
|
except (tarfile.TarError, OSError) as exc:
|
||||||
|
raise LocalError(t("Could not unpack {name}: {error}",
|
||||||
|
name=os.path.basename(str(archive)), error=exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def install_program(program, tag="", on_progress=None, should_stop=None,
|
||||||
|
refresh=False):
|
||||||
|
"""Fetch and unpack a release. The path to the binary, or "" when stopped.
|
||||||
|
|
||||||
|
`tag` is empty for whatever the project released last, which is the point:
|
||||||
|
a version pinned in Dikte's source would mean a release of Dikte every time
|
||||||
|
whisper.cpp has one.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
tag, assets = hub.release(program.repo, tag or "latest", refresh=refresh)
|
||||||
|
except hub.HubError as exc:
|
||||||
|
raise LocalError(str(exc)) from exc
|
||||||
|
|
||||||
|
item = None
|
||||||
|
for ending in _wanted_assets(program):
|
||||||
|
item = next((a for a in assets if a.name.endswith(ending)), None)
|
||||||
|
if item:
|
||||||
|
break
|
||||||
|
if item is None:
|
||||||
|
raise LocalError(t("{repo} {tag} has no build for this machine.",
|
||||||
|
repo=program.repo, tag=tag))
|
||||||
|
|
||||||
|
into = BIN_DIR / program.name / tag
|
||||||
|
shutil.rmtree(into, ignore_errors=True)
|
||||||
|
archive = BIN_DIR / program.name / item.name
|
||||||
|
try:
|
||||||
|
if not download(item, archive, on_progress, should_stop):
|
||||||
|
return ""
|
||||||
|
_extract(archive, into)
|
||||||
|
binary = _find_binary(into, program.binary)
|
||||||
|
if binary is None:
|
||||||
|
raise LocalError(t("{name} was not in the download.",
|
||||||
|
name=program.binary))
|
||||||
|
binary.chmod(binary.stat().st_mode | 0o111)
|
||||||
|
_install_record(program).write_text(
|
||||||
|
json.dumps({"tag": tag, "binary": str(binary)}), encoding="utf-8")
|
||||||
|
except OSError as exc:
|
||||||
|
raise LocalError(t("Could not install {name}: {error}",
|
||||||
|
name=program.name, error=exc)) from exc
|
||||||
|
finally:
|
||||||
|
try:
|
||||||
|
archive.unlink(missing_ok=True)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
_drop_old_versions(program, keep=tag)
|
||||||
|
return str(binary)
|
||||||
|
|
||||||
|
|
||||||
|
def _drop_old_versions(program, keep):
|
||||||
|
"""Leave one unpacked release behind, not one per update."""
|
||||||
|
root = BIN_DIR / program.name
|
||||||
|
try:
|
||||||
|
for path in root.iterdir():
|
||||||
|
if path.is_dir() and path.name != keep:
|
||||||
|
shutil.rmtree(path, ignore_errors=True)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
# --- the models -----------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def whisper_models(refresh=False):
|
||||||
|
"""[hub.Item] for every whisper model on offer, smallest first."""
|
||||||
|
try:
|
||||||
|
files = hub.files(WHISPER_MODELS_REPO, refresh=refresh)
|
||||||
|
except hub.HubError as exc:
|
||||||
|
raise LocalError(str(exc)) from exc
|
||||||
|
models = [f for f in files
|
||||||
|
if f.name.startswith(WHISPER_PREFIX) and f.name.endswith(WHISPER_SUFFIX)
|
||||||
|
and f.size > 0]
|
||||||
|
return sorted(models, key=lambda f: f.size)
|
||||||
|
|
||||||
|
|
||||||
|
def llm_repos(refresh=False):
|
||||||
|
"""Repository ids for the GGUF models on offer, suggestions first."""
|
||||||
|
try:
|
||||||
|
found = [r.id for r in hub.repos(author=LLM_AUTHOR, refresh=refresh)]
|
||||||
|
except hub.HubError:
|
||||||
|
# A menu rather than a catalogue: with nothing to show, the suggestions
|
||||||
|
# are still worth showing, and whatever is wrong with the network will
|
||||||
|
# say so where it matters, when a download is asked for.
|
||||||
|
found = []
|
||||||
|
if not found:
|
||||||
|
return list(SUGGESTED_LLM)
|
||||||
|
first = [r for r in SUGGESTED_LLM if r in found]
|
||||||
|
return first + [r for r in found if r not in first]
|
||||||
|
|
||||||
|
|
||||||
|
def llm_quants(repo, refresh=False):
|
||||||
|
"""[hub.Item] for the model files in one GGUF repository, smallest first."""
|
||||||
|
try:
|
||||||
|
files = hub.files(repo, refresh=refresh)
|
||||||
|
except hub.HubError as exc:
|
||||||
|
raise LocalError(str(exc)) from exc
|
||||||
|
out = []
|
||||||
|
for item in files:
|
||||||
|
name = item.name.rsplit("/", 1)[-1]
|
||||||
|
if not name.endswith(".gguf") or name.startswith(GGUF_SKIP):
|
||||||
|
continue
|
||||||
|
# A model split across files needs all of them and a different command
|
||||||
|
# line; anything cleanup wants fits in one.
|
||||||
|
if "-of-000" in name or not 0 < item.size <= GGUF_MAX_BYTES:
|
||||||
|
continue
|
||||||
|
out.append(item)
|
||||||
|
return sorted(out, key=lambda f: f.size)
|
||||||
|
|
||||||
|
|
||||||
|
def whisper_model_path(name):
|
||||||
|
return MODELS_DIR / "whisper" / name
|
||||||
|
|
||||||
|
|
||||||
|
def llm_model_path(name):
|
||||||
|
return MODELS_DIR / "llm" / name.rsplit("/", 1)[-1]
|
||||||
|
|
||||||
|
|
||||||
|
def have_model(path):
|
||||||
|
path = pathlib.Path(path)
|
||||||
|
return path.is_file() and path.stat().st_size > 0
|
||||||
|
|
||||||
|
|
||||||
|
def installed_whisper_models():
|
||||||
|
return sorted(p.name for p in (MODELS_DIR / "whisper").glob("*.bin"))
|
||||||
|
|
||||||
|
|
||||||
|
def installed_llm_models():
|
||||||
|
return sorted(p.name for p in (MODELS_DIR / "llm").glob("*.gguf"))
|
||||||
|
|
||||||
|
|
||||||
|
def delete_model(path):
|
||||||
|
try:
|
||||||
|
pathlib.Path(path).unlink()
|
||||||
|
except FileNotFoundError:
|
||||||
|
pass
|
||||||
|
except OSError as exc:
|
||||||
|
raise LocalError(t("Could not delete the model: {error}", error=exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
# --- one server -----------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _free_port():
|
||||||
|
"""A port nothing is listening on, handed straight to the server.
|
||||||
|
|
||||||
|
Between closing this socket and the server binding it, something else could
|
||||||
|
take it; that is why a start retries rather than trusting the number.
|
||||||
|
"""
|
||||||
|
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||||
|
sock.bind((HOST, 0))
|
||||||
|
return sock.getsockname()[1]
|
||||||
|
|
||||||
|
|
||||||
|
def _listening(port):
|
||||||
|
try:
|
||||||
|
with socket.create_connection((HOST, port), timeout=0.5):
|
||||||
|
return True
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _healthy(port, path):
|
||||||
|
"""Whether the model is in memory, for a server that says so.
|
||||||
|
|
||||||
|
Spoken over http.client rather than urllib because this never leaves the
|
||||||
|
machine: it is the same question as _listening, one layer up.
|
||||||
|
"""
|
||||||
|
connection = http.client.HTTPConnection(HOST, port, timeout=2)
|
||||||
|
try:
|
||||||
|
connection.request("GET", path)
|
||||||
|
# 503 for as long as the model is still being read in.
|
||||||
|
return connection.getresponse().status == 200
|
||||||
|
except (http.client.HTTPException, OSError):
|
||||||
|
return False
|
||||||
|
finally:
|
||||||
|
connection.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _tail(path, lines=3):
|
||||||
|
try:
|
||||||
|
with open(path, encoding="utf-8", errors="replace") as fh:
|
||||||
|
found = [line.strip() for line in fh if line.strip()]
|
||||||
|
except OSError:
|
||||||
|
return ""
|
||||||
|
return " | ".join(found[-lines:])
|
||||||
|
|
||||||
|
|
||||||
|
class Server:
|
||||||
|
"""One process, started when something needs it and stopped when nothing does.
|
||||||
|
|
||||||
|
`build` turns the settings into a command line; everything else about
|
||||||
|
running a server is the same for both programs.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, program, build, defaults):
|
||||||
|
self.program = program
|
||||||
|
self._build = build
|
||||||
|
self._settings = dict(defaults)
|
||||||
|
# Two locks on purpose. `_lock` is held for the length of a dictionary
|
||||||
|
# lookup, so the interface can ask what is running while a model is
|
||||||
|
# being loaded; `_starting` is held across the start itself, which can
|
||||||
|
# take a minute and which two threads must not both do.
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self._starting = threading.Lock()
|
||||||
|
self._proc = None
|
||||||
|
self._port = 0
|
||||||
|
self._log = ""
|
||||||
|
self._key = None
|
||||||
|
|
||||||
|
# ---- settings --------------------------------------------------------
|
||||||
|
|
||||||
|
def configure(self, **changes):
|
||||||
|
"""Apply settings. A server started on the old ones is stopped."""
|
||||||
|
with self._lock:
|
||||||
|
for key, value in changes.items():
|
||||||
|
if value is not None and key in self._settings:
|
||||||
|
self._settings[key] = value
|
||||||
|
stale = self._proc is not None and self._key != self._settings_key()
|
||||||
|
if stale:
|
||||||
|
self.stop()
|
||||||
|
|
||||||
|
def settings(self):
|
||||||
|
with self._lock:
|
||||||
|
return dict(self._settings)
|
||||||
|
|
||||||
|
def _settings_key(self):
|
||||||
|
"""What a running server would have to be restarted for."""
|
||||||
|
return json.dumps(self._settings, sort_keys=True, default=str)
|
||||||
|
|
||||||
|
# ---- process ---------------------------------------------------------
|
||||||
|
|
||||||
|
@property
|
||||||
|
def running(self):
|
||||||
|
with self._lock:
|
||||||
|
return self._proc is not None and self._proc.poll() is None
|
||||||
|
|
||||||
|
def base_url(self):
|
||||||
|
with self._lock:
|
||||||
|
return f"http://{HOST}:{self._port}/v1" if self._port else ""
|
||||||
|
|
||||||
|
def error(self):
|
||||||
|
"""The last thing the server printed, for a failure after it started."""
|
||||||
|
with self._lock:
|
||||||
|
log = self._log
|
||||||
|
return _tail(log) if log else ""
|
||||||
|
|
||||||
|
def serve(self):
|
||||||
|
"""The base URL of a server that is up and running the current settings."""
|
||||||
|
ready = self._current_url()
|
||||||
|
if ready:
|
||||||
|
return ready
|
||||||
|
with self._starting:
|
||||||
|
# Somebody may have started it while this thread waited its turn.
|
||||||
|
ready = self._current_url()
|
||||||
|
if ready:
|
||||||
|
return ready
|
||||||
|
self.stop()
|
||||||
|
with self._lock:
|
||||||
|
settings, key = dict(self._settings), self._settings_key()
|
||||||
|
proc, port, log = self._launch(settings)
|
||||||
|
with self._lock:
|
||||||
|
self._proc, self._port, self._log, self._key = proc, port, log, key
|
||||||
|
return self.base_url()
|
||||||
|
|
||||||
|
def _current_url(self):
|
||||||
|
with self._lock:
|
||||||
|
up = self._proc is not None and self._proc.poll() is None
|
||||||
|
return (f"http://{HOST}:{self._port}/v1"
|
||||||
|
if up and self._key == self._settings_key() else "")
|
||||||
|
|
||||||
|
def _launch(self, settings):
|
||||||
|
args = self._build(settings) # raises LocalError when unusable
|
||||||
|
last = ""
|
||||||
|
for _ in range(3):
|
||||||
|
port = _free_port()
|
||||||
|
log = DATA_DIR / f"{self.program.name}-server.log"
|
||||||
|
try:
|
||||||
|
log.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
sink = open(log, "wb")
|
||||||
|
except OSError as exc:
|
||||||
|
raise LocalError(t("Could not start {name}: {error}",
|
||||||
|
name=self.program.name, error=exc)) from exc
|
||||||
|
try:
|
||||||
|
with sink:
|
||||||
|
proc = subprocess.Popen(
|
||||||
|
args + ["--host", HOST, "--port", str(port)],
|
||||||
|
stdout=sink, stderr=subprocess.STDOUT,
|
||||||
|
stdin=subprocess.DEVNULL,
|
||||||
|
)
|
||||||
|
except OSError as exc:
|
||||||
|
raise LocalError(t("Could not start {name}: {error}",
|
||||||
|
name=self.program.name, error=exc)) from exc
|
||||||
|
|
||||||
|
# Written before it is ready rather than after, so that a kill
|
||||||
|
# during the model load leaves something for the sweep to find.
|
||||||
|
self._remember(proc.pid)
|
||||||
|
try:
|
||||||
|
ready = self._wait_ready(proc, port)
|
||||||
|
except BaseException:
|
||||||
|
# Whatever went wrong while waiting, the process is ours and
|
||||||
|
# nothing else is left holding a reference to it. Leaving it
|
||||||
|
# running would leak a loaded model with nobody to ask it
|
||||||
|
# anything, which is the whole failure this class is careful
|
||||||
|
# about elsewhere.
|
||||||
|
self._kill(proc)
|
||||||
|
self._forget()
|
||||||
|
raise
|
||||||
|
if ready:
|
||||||
|
return proc, port, str(log)
|
||||||
|
last = _tail(log)
|
||||||
|
self._forget()
|
||||||
|
# A port taken between the probe and the bind is the one failure
|
||||||
|
# worth another go; anything else will fail the same way again.
|
||||||
|
if "address" not in last.lower() and "bind" not in last.lower():
|
||||||
|
break
|
||||||
|
raise LocalError(t("{name} did not start: {error}",
|
||||||
|
name=self.program.binary, error=last or t("no output")))
|
||||||
|
|
||||||
|
def _wait_ready(self, proc, port):
|
||||||
|
deadline = time.monotonic() + STARTUP_TIMEOUT
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
if proc.poll() is not None:
|
||||||
|
return False
|
||||||
|
if _listening(port):
|
||||||
|
# whisper binds after the model is loaded, so the open port is
|
||||||
|
# the answer. llama binds first and answers /health with 503
|
||||||
|
# until it is ready.
|
||||||
|
if not self.program.health or _healthy(port, self.program.health):
|
||||||
|
return True
|
||||||
|
time.sleep(0.1)
|
||||||
|
self._kill(proc)
|
||||||
|
return False
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _kill(proc, gently=False):
|
||||||
|
"""Stop a process of ours, and wait for it rather than assume."""
|
||||||
|
if proc is None or proc.poll() is not None:
|
||||||
|
return
|
||||||
|
if gently:
|
||||||
|
proc.terminate()
|
||||||
|
try:
|
||||||
|
proc.wait(timeout=5)
|
||||||
|
return
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
pass
|
||||||
|
proc.kill()
|
||||||
|
try:
|
||||||
|
proc.wait(timeout=5)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
with self._lock:
|
||||||
|
proc, self._proc = self._proc, None
|
||||||
|
self._port, self._log, self._key = 0, "", None
|
||||||
|
self._kill(proc, gently=True)
|
||||||
|
if proc is not None:
|
||||||
|
self._forget()
|
||||||
|
|
||||||
|
# ---- servers a killed Dikte left behind -------------------------------
|
||||||
|
|
||||||
|
def _pid_file(self):
|
||||||
|
return DATA_DIR / f"{self.program.name}-server.pid"
|
||||||
|
|
||||||
|
def _remember(self, pid):
|
||||||
|
try:
|
||||||
|
path = self._pid_file()
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(str(pid))
|
||||||
|
except OSError:
|
||||||
|
pass # the sweep is a safety net, not something to fail a run over
|
||||||
|
|
||||||
|
def _forget(self):
|
||||||
|
try:
|
||||||
|
self._pid_file().unlink()
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def _is_ours(self, pid):
|
||||||
|
"""Whether that pid is still the server this Dikte started.
|
||||||
|
|
||||||
|
Asked because pids are handed out again: by the time anyone looks, the
|
||||||
|
number could belong to something else entirely, and killing it would be
|
||||||
|
a good deal worse than the leak being cleaned up. The program name alone
|
||||||
|
could be somebody else's copy; the name together with Dikte's own data
|
||||||
|
directory on the command line could not.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
blob = pathlib.Path(f"/proc/{pid}/cmdline").read_bytes()
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
|
return (self.program.binary.encode() in blob
|
||||||
|
and str(DATA_DIR).encode() in blob)
|
||||||
|
|
||||||
|
def sweep(self):
|
||||||
|
"""Kill a server a previous Dikte left behind. True when one was found.
|
||||||
|
|
||||||
|
stop() and atexit cover every exit that gets to run code. A SIGKILL does
|
||||||
|
not, and neither does a session torn down from under it, and the server
|
||||||
|
would then sit there holding the model with nothing left alive to ask it
|
||||||
|
anything.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
pid = int(self._pid_file().read_text().strip())
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return False
|
||||||
|
self._forget()
|
||||||
|
if not self._is_ours(pid):
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
os.kill(pid, signal.SIGTERM)
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
# --- the two of them ------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _whisper_args(settings):
|
||||||
|
binary = program_path(WHISPER, settings["binary"])
|
||||||
|
if not binary:
|
||||||
|
raise LocalError(t("whisper.cpp is not installed. Settings → API and "
|
||||||
|
"models → Download."))
|
||||||
|
model = whisper_model_path(settings["model"])
|
||||||
|
if not settings["model"] or not have_model(model):
|
||||||
|
raise LocalError(t("No whisper model has been downloaded yet. "
|
||||||
|
"Settings → API and models → Download."))
|
||||||
|
args = [
|
||||||
|
binary, "-m", str(model),
|
||||||
|
"--inference-path", INFERENCE_PATH,
|
||||||
|
# Whatever language the request does not name. api.py leaves the field
|
||||||
|
# out when the language is "auto", and the server's own default is
|
||||||
|
# English rather than detection.
|
||||||
|
"-l", "auto",
|
||||||
|
# Stock phrases invented for near-silence come from non-speech tokens,
|
||||||
|
# and verbose_json otherwise pays for a language probability sweep
|
||||||
|
# nothing here reads.
|
||||||
|
"-sns", "-nlp",
|
||||||
|
]
|
||||||
|
if int(settings["threads"]) > 0:
|
||||||
|
args += ["-t", str(int(settings["threads"]))]
|
||||||
|
if not settings["gpu"]:
|
||||||
|
args.append("-ng")
|
||||||
|
return args
|
||||||
|
|
||||||
|
|
||||||
|
def _llm_args(settings):
|
||||||
|
binary = program_path(LLAMA, settings["binary"])
|
||||||
|
if not binary:
|
||||||
|
raise LocalError(t("llama.cpp is not installed. Settings → API and "
|
||||||
|
"models → Download."))
|
||||||
|
model = llm_model_path(settings["model"])
|
||||||
|
if not settings["model"] or not have_model(model):
|
||||||
|
raise LocalError(t("No local cleanup model has been downloaded yet. "
|
||||||
|
"Settings → API and models → Download."))
|
||||||
|
args = [binary, "-m", str(model), "-c", str(int(settings["context"]))]
|
||||||
|
# All of them, or as many as fit: llama.cpp stops offloading when the card
|
||||||
|
# is full rather than failing, and a build with no GPU backend ignores it.
|
||||||
|
args += ["-ngl", "99" if settings["gpu"] else "0"]
|
||||||
|
if int(settings["threads"]) > 0:
|
||||||
|
args += ["-t", str(int(settings["threads"]))]
|
||||||
|
return args
|
||||||
|
|
||||||
|
|
||||||
|
whisper = Server(WHISPER, _whisper_args, {
|
||||||
|
"model": "",
|
||||||
|
"threads": 0,
|
||||||
|
"gpu": True,
|
||||||
|
"binary": "",
|
||||||
|
})
|
||||||
|
|
||||||
|
llm = Server(LLAMA, _llm_args, {
|
||||||
|
"model": "",
|
||||||
|
"threads": 0,
|
||||||
|
"gpu": True,
|
||||||
|
"binary": "",
|
||||||
|
# A dictation and its prompt are short. This is sized for the longest
|
||||||
|
# cleanup block rather than for a conversation, and it is what the model
|
||||||
|
# costs in memory beyond its own weights.
|
||||||
|
"context": 8192,
|
||||||
|
})
|
||||||
|
|
||||||
|
SERVERS = (whisper, llm)
|
||||||
|
|
||||||
|
|
||||||
|
def sweep():
|
||||||
|
"""Clean up after a Dikte that was killed outright. True when one was found."""
|
||||||
|
return any([server.sweep() for server in SERVERS])
|
||||||
|
|
||||||
|
|
||||||
|
def stop_all():
|
||||||
|
for server in SERVERS:
|
||||||
|
server.stop()
|
||||||
|
|
||||||
|
|
||||||
|
# Dikte stops the servers itself on quit and on restart; this catches the paths
|
||||||
|
# that skip that, such as an unhandled exception on the way out.
|
||||||
|
atexit.register(stop_all)
|
||||||
@@ -9,6 +9,7 @@ there the listener is not a fallback but the whole mechanism.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import ast
|
import ast
|
||||||
|
import collections
|
||||||
import ctypes
|
import ctypes
|
||||||
import ctypes.util
|
import ctypes.util
|
||||||
import glob
|
import glob
|
||||||
@@ -27,17 +28,34 @@ from PyQt6.QtCore import QObject, pyqtSignal
|
|||||||
from i18n import t
|
from i18n import t
|
||||||
|
|
||||||
DESKTOP_ID = "dikte-toggle.desktop"
|
DESKTOP_ID = "dikte-toggle.desktop"
|
||||||
|
CANCEL_DESKTOP_ID = "dikte-cancel.desktop"
|
||||||
MEETING_DESKTOP_ID = "dikte-meeting.desktop"
|
MEETING_DESKTOP_ID = "dikte-meeting.desktop"
|
||||||
ASK_DESKTOP_ID = "dikte-ask.desktop"
|
ASK_DESKTOP_ID = "dikte-ask.desktop"
|
||||||
# The name each binding is started under, and the entry it is installed as.
|
|
||||||
DESKTOP_IDS = {"toggle": DESKTOP_ID, "meeting": MEETING_DESKTOP_ID,
|
|
||||||
"ask": ASK_DESKTOP_ID}
|
|
||||||
APPLICATIONS_DIR = pathlib.Path.home() / ".local/share/applications"
|
APPLICATIONS_DIR = pathlib.Path.home() / ".local/share/applications"
|
||||||
DESKTOP_FILE = APPLICATIONS_DIR / DESKTOP_ID
|
DESKTOP_FILE = APPLICATIONS_DIR / DESKTOP_ID
|
||||||
SHORTCUTS_FILE = pathlib.Path.home() / ".config/kglobalshortcutsrc"
|
SHORTCUTS_FILE = pathlib.Path.home() / ".config/kglobalshortcutsrc"
|
||||||
GNOME_MEDIA_SCHEMA = "org.gnome.settings-daemon.plugins.media-keys"
|
GNOME_MEDIA_SCHEMA = "org.gnome.settings-daemon.plugins.media-keys"
|
||||||
GNOME_BINDING_SCHEMA = "org.gnome.settings-daemon.plugins.media-keys.custom-keybinding"
|
GNOME_BINDING_SCHEMA = "org.gnome.settings-daemon.plugins.media-keys.custom-keybinding"
|
||||||
|
|
||||||
|
Shortcut = collections.namedtuple("Shortcut", "verb desktop_id name setting fallback")
|
||||||
|
|
||||||
|
# Every global shortcut in one place, because there are four of them and the
|
||||||
|
# command line, the settings window and the installer each used to carry their
|
||||||
|
# own copy of the list. `fallback` is what to register when the setting is
|
||||||
|
# empty: only the toggle has one, since it is the key the application is
|
||||||
|
# unusable without.
|
||||||
|
SHORTCUTS = {
|
||||||
|
"toggle": Shortcut("toggle", DESKTOP_ID, "Dikte: start/stop recording",
|
||||||
|
"shortcut", "Ctrl+Space"),
|
||||||
|
"cancel": Shortcut("cancel", CANCEL_DESKTOP_ID, "Dikte: discard the recording",
|
||||||
|
"cancel_shortcut", ""),
|
||||||
|
"ask": Shortcut("ask", ASK_DESKTOP_ID, "Dikte: ask Claude Code",
|
||||||
|
"assistant_shortcut", ""),
|
||||||
|
"meeting": Shortcut("meeting", MEETING_DESKTOP_ID,
|
||||||
|
"Dikte: start/end a meeting recording",
|
||||||
|
"meeting_shortcut", ""),
|
||||||
|
}
|
||||||
|
|
||||||
# --- evdev key codes (linux/input-event-codes.h) --------------------------
|
# --- evdev key codes (linux/input-event-codes.h) --------------------------
|
||||||
|
|
||||||
EV_KEY = 0x01
|
EV_KEY = 0x01
|
||||||
@@ -318,9 +336,9 @@ class CarbonHotkey(QObject):
|
|||||||
continue
|
continue
|
||||||
self._registrations.append(reference)
|
self._registrations.append(reference)
|
||||||
self._names[identifier] = name
|
self._names[identifier] = name
|
||||||
desktop_id = DESKTOP_IDS.get(name)
|
spec = SHORTCUTS.get(name)
|
||||||
if desktop_id:
|
if spec:
|
||||||
_REGISTERED[desktop_id] = shortcut
|
_REGISTERED[spec.desktop_id] = shortcut
|
||||||
return bool(self._registrations)
|
return bool(self._registrations)
|
||||||
|
|
||||||
def stop(self):
|
def stop(self):
|
||||||
@@ -642,10 +660,14 @@ def remove_kde_shortcut(desktop_id=DESKTOP_ID):
|
|||||||
(APPLICATIONS_DIR / desktop_id).unlink(missing_ok=True)
|
(APPLICATIONS_DIR / desktop_id).unlink(missing_ok=True)
|
||||||
except OSError:
|
except OSError:
|
||||||
pass
|
pass
|
||||||
|
# kwriteconfig6 deletes keys rather than groups, so both of the ones KDE
|
||||||
|
# keeps in there go and the empty group is left behind harmlessly.
|
||||||
|
for key in ("_launch", "_k_friendly_name"):
|
||||||
try:
|
try:
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
["kwriteconfig6", "--notify", "--file", "kglobalshortcutsrc",
|
["kwriteconfig6", "--notify", "--file", "kglobalshortcutsrc",
|
||||||
"--group", "services", "--group", desktop_id, "--key", "_launch", "--delete"],
|
"--group", "services", "--group", desktop_id,
|
||||||
|
"--key", key, "--delete"],
|
||||||
capture_output=True, timeout=10,
|
capture_output=True, timeout=10,
|
||||||
)
|
)
|
||||||
except (subprocess.SubprocessError, OSError):
|
except (subprocess.SubprocessError, OSError):
|
||||||
|
|||||||
@@ -0,0 +1,186 @@
|
|||||||
|
"""Where the programs and the models come from: GitHub releases and Hugging Face.
|
||||||
|
|
||||||
|
Both answer plain JSON over HTTPS without a key, and both publish a sha256 for
|
||||||
|
every file they hand out: GitHub as the asset digest, Hugging Face as the LFS
|
||||||
|
object id. Nothing that lands on disk is trusted for having arrived, which
|
||||||
|
matters more here than it usually would, because half of what is fetched is a
|
||||||
|
program Dikte then runs.
|
||||||
|
|
||||||
|
The lists are read rather than kept. A model catalogue written into the source
|
||||||
|
means a release of Dikte for every new model, and a pinned whisper.cpp version
|
||||||
|
means one for every whisper.cpp release; both of those are somebody else's news,
|
||||||
|
not Dikte's. Answers are cached for a few hours, and a cache that has gone stale
|
||||||
|
is still a better answer than none when the network is down.
|
||||||
|
|
||||||
|
Nothing here imports the rest of Dikte apart from the string table: this module
|
||||||
|
knows two websites and nothing about dictation.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import collections
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import pathlib
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
from i18n import t
|
||||||
|
|
||||||
|
GITHUB_API = "https://api.github.com"
|
||||||
|
HF_API = "https://huggingface.co/api"
|
||||||
|
HF_FILES = "https://huggingface.co"
|
||||||
|
USER_AGENT = "dikte/1.0 (+https://github.com/yusufipk/dikte)"
|
||||||
|
|
||||||
|
CACHE_DIR = (pathlib.Path(os.environ.get("XDG_CACHE_HOME")
|
||||||
|
or os.path.expanduser("~/.cache")) / "dikte")
|
||||||
|
# Long enough that opening the settings window twice in an evening asks nobody
|
||||||
|
# anything, short enough that a model published this morning is offered today.
|
||||||
|
CACHE_TTL = 6 * 3600
|
||||||
|
|
||||||
|
# `sha256` is empty for the few files neither side stores in LFS; those are the
|
||||||
|
# small ones, and a checksum is only worth having where there is something to
|
||||||
|
# check.
|
||||||
|
Item = collections.namedtuple("Item", "name url size sha256")
|
||||||
|
Repo = collections.namedtuple("Repo", "id downloads updated")
|
||||||
|
|
||||||
|
|
||||||
|
class HubError(Exception):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _get(url, timeout=20):
|
||||||
|
request = urllib.request.Request(url, headers={
|
||||||
|
"User-Agent": USER_AGENT,
|
||||||
|
"Accept": "application/json",
|
||||||
|
})
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||||
|
return json.loads(response.read().decode("utf-8"))
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
exc.close() # it holds the response body open until it is collected
|
||||||
|
raise HubError(t("{url} answered HTTP {code}.",
|
||||||
|
url=urllib.parse.urlsplit(url).netloc, code=exc.code)) from exc
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
raise HubError(t("Could not reach {url}: {error}",
|
||||||
|
url=urllib.parse.urlsplit(url).netloc,
|
||||||
|
error=exc.reason)) from exc
|
||||||
|
except (ValueError, OSError) as exc:
|
||||||
|
raise HubError(t("Could not read the answer from {url}: {error}",
|
||||||
|
url=urllib.parse.urlsplit(url).netloc, error=exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _cache_file(key):
|
||||||
|
safe = "".join(c if c.isalnum() or c in "-._" else "-" for c in key)
|
||||||
|
return CACHE_DIR / f"{safe}.json"
|
||||||
|
|
||||||
|
|
||||||
|
def _read_cache(key, ttl):
|
||||||
|
"""What was stored under this key, or None. `ttl` of 0 ignores the age."""
|
||||||
|
path = _cache_file(key)
|
||||||
|
try:
|
||||||
|
age = time.time() - path.stat().st_mtime
|
||||||
|
if ttl and age > ttl:
|
||||||
|
return None
|
||||||
|
return json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _write_cache(key, payload):
|
||||||
|
try:
|
||||||
|
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
_cache_file(key).write_text(json.dumps(payload), encoding="utf-8")
|
||||||
|
except OSError:
|
||||||
|
pass # a cache that cannot be written is not a failed lookup
|
||||||
|
|
||||||
|
|
||||||
|
def _fetch(key, url, ttl=CACHE_TTL, refresh=False):
|
||||||
|
"""The JSON at `url`, from the cache when it is fresh enough.
|
||||||
|
|
||||||
|
A lookup that fails falls back to the cache however old it is: an offline
|
||||||
|
settings window that shows yesterday's list is worth a great deal more than
|
||||||
|
one that shows an error.
|
||||||
|
"""
|
||||||
|
if not refresh:
|
||||||
|
cached = _read_cache(key, ttl)
|
||||||
|
if cached is not None:
|
||||||
|
return cached
|
||||||
|
try:
|
||||||
|
payload = _get(url)
|
||||||
|
except HubError:
|
||||||
|
stale = _read_cache(key, 0)
|
||||||
|
if stale is not None:
|
||||||
|
return stale
|
||||||
|
raise
|
||||||
|
_write_cache(key, payload)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def _digest(value):
|
||||||
|
"""GitHub writes its digests as "sha256:…"; Hugging Face writes the hash."""
|
||||||
|
value = (value or "").strip()
|
||||||
|
return value.split(":", 1)[1] if value.startswith("sha256:") else value
|
||||||
|
|
||||||
|
|
||||||
|
def release(repo, tag="latest", refresh=False):
|
||||||
|
"""(tag, [Item]) for one GitHub release, newest when no tag is given."""
|
||||||
|
where = "latest" if tag in ("", "latest") else f"tags/{tag}"
|
||||||
|
data = _fetch(f"gh-{repo}-{tag or 'latest'}",
|
||||||
|
f"{GITHUB_API}/repos/{repo}/releases/{where}", refresh=refresh)
|
||||||
|
if not isinstance(data, dict) or not data.get("assets"):
|
||||||
|
raise HubError(t("{repo} has no downloadable release.", repo=repo))
|
||||||
|
assets = [Item(a.get("name") or "", a.get("browser_download_url") or "",
|
||||||
|
int(a.get("size") or 0), _digest(a.get("digest")))
|
||||||
|
for a in data["assets"] if a.get("browser_download_url")]
|
||||||
|
return data.get("tag_name") or tag, assets
|
||||||
|
|
||||||
|
|
||||||
|
def files(repo, revision="main", refresh=False):
|
||||||
|
"""[Item] for every file in a Hugging Face repository.
|
||||||
|
|
||||||
|
The size is there whether or not the file is in LFS; the hash is only there
|
||||||
|
when it is, which for anything worth downloading it always is.
|
||||||
|
"""
|
||||||
|
data = _fetch(f"hf-tree-{repo}-{revision}",
|
||||||
|
f"{HF_API}/models/{repo}/tree/{revision}?recursive=true",
|
||||||
|
refresh=refresh)
|
||||||
|
if not isinstance(data, list):
|
||||||
|
raise HubError(t("{repo} did not return a file list.", repo=repo))
|
||||||
|
out = []
|
||||||
|
for entry in data:
|
||||||
|
if entry.get("type") != "file":
|
||||||
|
continue
|
||||||
|
path = entry.get("path") or ""
|
||||||
|
lfs = entry.get("lfs") or {}
|
||||||
|
out.append(Item(
|
||||||
|
path,
|
||||||
|
f"{HF_FILES}/{repo}/resolve/{revision}/{urllib.parse.quote(path)}",
|
||||||
|
int(lfs.get("size") or entry.get("size") or 0),
|
||||||
|
_digest(lfs.get("oid") or lfs.get("sha256")),
|
||||||
|
))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def repos(author="", search="", limit=40, refresh=False):
|
||||||
|
"""[Repo] of GGUF repositories, newest first.
|
||||||
|
|
||||||
|
Filtered by author on purpose. Hugging Face's own trending list is open to
|
||||||
|
everyone and reads like it: asking it for the popular GGUF today answers
|
||||||
|
with a wall of roleplay merges, which is not what a dictation transcript
|
||||||
|
wants cleaning up. An author is a small enough thing to trust and a large
|
||||||
|
enough one to keep the list current without Dikte being updated.
|
||||||
|
"""
|
||||||
|
query = {"filter": "gguf", "sort": "lastModified", "direction": "-1",
|
||||||
|
"limit": str(limit)}
|
||||||
|
if author:
|
||||||
|
query["author"] = author
|
||||||
|
if search:
|
||||||
|
query["search"] = search
|
||||||
|
url = f"{HF_API}/models?{urllib.parse.urlencode(query)}"
|
||||||
|
data = _fetch(f"hf-models-{author}-{search}-{limit}", url, refresh=refresh)
|
||||||
|
if not isinstance(data, list):
|
||||||
|
raise HubError(t("Hugging Face did not return a model list."))
|
||||||
|
return [Repo(m.get("id") or "", int(m.get("downloads") or 0),
|
||||||
|
m.get("lastModified") or "")
|
||||||
|
for m in data if m.get("id")]
|
||||||
@@ -56,7 +56,7 @@ TR = {
|
|||||||
"Start recording": "Kaydı başlat",
|
"Start recording": "Kaydı başlat",
|
||||||
"Stop and transcribe": "Kaydı bitir ve yaz",
|
"Stop and transcribe": "Kaydı bitir ve yaz",
|
||||||
"Working…": "İşleniyor…",
|
"Working…": "İşleniyor…",
|
||||||
"Cancel recording": "Kaydı iptal et",
|
"Discard the recording": "Kaydı iptal et",
|
||||||
"Settings…": "Ayarlar…",
|
"Settings…": "Ayarlar…",
|
||||||
"Restart": "Yeniden başlat",
|
"Restart": "Yeniden başlat",
|
||||||
"Quit": "Çık",
|
"Quit": "Çık",
|
||||||
@@ -140,6 +140,7 @@ TR = {
|
|||||||
"Cleanup rules": "Temizleme kuralları",
|
"Cleanup rules": "Temizleme kuralları",
|
||||||
"Audio file": "Ses dosyası",
|
"Audio file": "Ses dosyası",
|
||||||
"Shortcut": "Kısayol",
|
"Shortcut": "Kısayol",
|
||||||
|
"Shortcuts": "Kısayollar",
|
||||||
"History": "Geçmiş",
|
"History": "Geçmiş",
|
||||||
"Save": "Kaydet",
|
"Save": "Kaydet",
|
||||||
"Saved successfully.": "Başarıyla kaydedildi.",
|
"Saved successfully.": "Başarıyla kaydedildi.",
|
||||||
@@ -185,6 +186,7 @@ TR = {
|
|||||||
"Model": "Model",
|
"Model": "Model",
|
||||||
"Provider": "Sağlayıcı",
|
"Provider": "Sağlayıcı",
|
||||||
"sk-… (falls back to OPENAI_API_KEY)": "sk-… (boşsa OPENAI_API_KEY kullanılır)",
|
"sk-… (falls back to OPENAI_API_KEY)": "sk-… (boşsa OPENAI_API_KEY kullanılır)",
|
||||||
|
"gsk_… (falls back to GROQ_API_KEY)": "gsk_… (boşsa GROQ_API_KEY kullanılır)",
|
||||||
"sk-or-… (falls back to OPENROUTER_API_KEY)": "sk-or-… (boşsa OPENROUTER_API_KEY kullanılır)",
|
"sk-or-… (falls back to OPENROUTER_API_KEY)": "sk-or-… (boşsa OPENROUTER_API_KEY kullanılır)",
|
||||||
"Test": "Test et",
|
"Test": "Test et",
|
||||||
"Trying…": "Deneniyor…",
|
"Trying…": "Deneniyor…",
|
||||||
@@ -192,6 +194,18 @@ TR = {
|
|||||||
"Connection works. {count} audio models visible.":
|
"Connection works. {count} audio models visible.":
|
||||||
"Bağlantı tamam. {count} ses modeli görünüyor.",
|
"Bağlantı tamam. {count} ses modeli görünüyor.",
|
||||||
"Clean the transcript with a model": "Transkripti bir modelle temizle",
|
"Clean the transcript with a model": "Transkripti bir modelle temizle",
|
||||||
|
"OpenRouter is the quickest and the only one that needs nothing installed. "
|
||||||
|
"Claude Code and Codex clean up on the subscription you already have, "
|
||||||
|
"without a second key, and take a few seconds longer because each one opens "
|
||||||
|
"a session to do it.":
|
||||||
|
"En hızlısı OpenRouter'dır ve kurulu bir program istemeyen tek seçenektir. "
|
||||||
|
"Claude Code ile Codex, temizliği hâlihazırda ödediğin abonelik üzerinden "
|
||||||
|
"yapar, ikinci bir anahtar istemez; her biri bunun için bir oturum açtığından "
|
||||||
|
"birkaç saniye daha uzun sürer.",
|
||||||
|
"{binary} is not on your PATH, so cleanup would fail and the raw transcript "
|
||||||
|
"would be pasted. Install it, or pick another one above.":
|
||||||
|
"{binary} PATH'te değil; temizleme başarısız olur ve ham transkript "
|
||||||
|
"yapıştırılır. Kur ya da yukarıdan başka birini seç.",
|
||||||
"Thinking": "Düşünme",
|
"Thinking": "Düşünme",
|
||||||
"Model's own default": "Modelin kendi varsayılanı",
|
"Model's own default": "Modelin kendi varsayılanı",
|
||||||
"Off": "Kapalı",
|
"Off": "Kapalı",
|
||||||
@@ -289,6 +303,13 @@ TR = {
|
|||||||
"Global kısayol kurulu değil. Toplantı tepsi menüsünden de başlatılabilir.",
|
"Global kısayol kurulu değil. Toplantı tepsi menüsünden de başlatılabilir.",
|
||||||
"No global shortcut installed. The tray menu asks it too.":
|
"No global shortcut installed. The tray menu asks it too.":
|
||||||
"Global kısayol kurulu değil. Tepsi menüsünden de soru sorulabilir.",
|
"Global kısayol kurulu değil. Tepsi menüsünden de soru sorulabilir.",
|
||||||
|
"No global shortcut installed. The tray menu discards it too.":
|
||||||
|
"Global kısayol kurulu değil. Kayıt tepsi menüsünden de iptal edilebilir.",
|
||||||
|
"Start and stop": "Başlat ve bitir",
|
||||||
|
"Throws the recording away without transcribing it. Works on a dictation "
|
||||||
|
"and on a command for the agent alike, whichever is running.":
|
||||||
|
"Kaydı yazıya dökmeden atar. Hangisi çalışıyorsa ona işler: dikteye de, "
|
||||||
|
"ajana verilen komuta da.",
|
||||||
"Shortcut saved: {shortcut}": "Kısayol kaydedildi: {shortcut}",
|
"Shortcut saved: {shortcut}": "Kısayol kaydedildi: {shortcut}",
|
||||||
"Could not register the GNOME shortcut: {error}":
|
"Could not register the GNOME shortcut: {error}":
|
||||||
"GNOME kısayolu kaydedilemedi: {error}",
|
"GNOME kısayolu kaydedilemedi: {error}",
|
||||||
@@ -397,6 +418,14 @@ TR = {
|
|||||||
"It was not allowed to use: {tools}": "Şunları kullanmasına izin yoktu: {tools}",
|
"It was not allowed to use: {tools}": "Şunları kullanmasına izin yoktu: {tools}",
|
||||||
"The model returned an empty reply.": "Model boş cevap döndürdü.",
|
"The model returned an empty reply.": "Model boş cevap döndürdü.",
|
||||||
|
|
||||||
|
# --- cleanup, when a CLI does it ----------------------------------------
|
||||||
|
"{binary} not found. Install it, or have OpenRouter clean up instead, "
|
||||||
|
"under Settings → API and models.":
|
||||||
|
"{binary} bulunamadı. Kur ya da Ayarlar → API ve modeller sekmesinden "
|
||||||
|
"temizliği OpenRouter'a bırak.",
|
||||||
|
"{service} did not finish within {seconds} seconds.":
|
||||||
|
"{service} {seconds} saniye içinde bitmedi.",
|
||||||
|
|
||||||
# --- settings: the agent ------------------------------------------------
|
# --- settings: the agent ------------------------------------------------
|
||||||
"Agent": "Ajan",
|
"Agent": "Ajan",
|
||||||
"This shortcut records the same way dictation does, but the transcript is "
|
"This shortcut records the same way dictation does, but the transcript is "
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Dikte installer: dependency check, launchers, KDE shortcut.
|
# Dikte installer: dependency check, launchers, global shortcuts.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
@@ -8,6 +8,9 @@ BIN_DIR="$HOME/.local/bin"
|
|||||||
APP_DIR="$HOME/.local/share/applications"
|
APP_DIR="$HOME/.local/share/applications"
|
||||||
AUTOSTART_DIR="$HOME/.config/autostart"
|
AUTOSTART_DIR="$HOME/.config/autostart"
|
||||||
SHORTCUT="${1:-Ctrl+Space}"
|
SHORTCUT="${1:-Ctrl+Space}"
|
||||||
|
# Without the colon, so that a second argument given as "" stays empty. That is
|
||||||
|
# how update.sh says "this one was turned off", as against not saying anything.
|
||||||
|
CANCEL_SHORTCUT="${2-Ctrl+Alt+Space}"
|
||||||
|
|
||||||
say() { printf ' %s\n' "$1"; }
|
say() { printf ' %s\n' "$1"; }
|
||||||
ok() { printf ' \033[32m✓\033[0m %s\n' "$1"; }
|
ok() { printf ' \033[32m✓\033[0m %s\n' "$1"; }
|
||||||
@@ -88,33 +91,47 @@ StartupNotify=false
|
|||||||
EOF
|
EOF
|
||||||
ok "Will start automatically on login"
|
ok "Will start automatically on login"
|
||||||
|
|
||||||
# 4. KDE global shortcut ---------------------------------------------------
|
# 4. Global shortcuts ------------------------------------------------------
|
||||||
cat > "$APP_DIR/dikte-toggle.desktop" <<EOF
|
# Two of them: one to start and stop, one to throw the recording away. The
|
||||||
[Desktop Entry]
|
# second is worth a key of its own because stopping is the step there is no
|
||||||
Exec=$PY $DIR/dikte.py toggle
|
# taking back, being what sends the audio off to be transcribed.
|
||||||
Name=Dikte: start/stop recording
|
#
|
||||||
NoDisplay=true
|
# Dikte registers them rather than this script writing the files itself: it
|
||||||
StartupNotify=false
|
# knows which desktop it is on, and it stores the combination in the settings
|
||||||
Type=Application
|
# as well, which is where the built-in listener reads it from. A key written
|
||||||
X-KDE-GlobalAccel-CommandShortcut=true
|
# to only one of the two places is a key that half works.
|
||||||
EOF
|
if [[ "$SHORTCUT" == "$CANCEL_SHORTCUT" ]]; then
|
||||||
|
warn "Both arguments are $SHORTCUT, so the discard key was left out."
|
||||||
|
say "Pass two different combinations, or set it in Settings → Shortcuts."
|
||||||
|
CANCEL_SHORTCUT=""
|
||||||
|
fi
|
||||||
|
|
||||||
if [[ "${XDG_CURRENT_DESKTOP:-}" == *GNOME* || "${XDG_CURRENT_DESKTOP:-}" == *gnome* ]]; then
|
register() { # which combination label
|
||||||
ok "GNOME detected"
|
if out="$("$PY" "$DIR/dikte.py" shortcut install "$1" --combo "$2" 2>&1)"; then
|
||||||
say "Open Dikte Settings > Shortcut to install the global shortcut."
|
ok "$3: $2"
|
||||||
elif command -v kwriteconfig6 >/dev/null; then
|
|
||||||
kwriteconfig6 --notify --file kglobalshortcutsrc \
|
|
||||||
--group services --group dikte-toggle.desktop \
|
|
||||||
--key _launch "$SHORTCUT"
|
|
||||||
ok "KDE shortcut registered: $SHORTCUT"
|
|
||||||
warn "KWin only reads this at startup, so the shortcut goes live after your"
|
|
||||||
say "next login. Until then open Settings → Shortcut and turn on the"
|
|
||||||
say "built-in listener to use it right away."
|
|
||||||
else
|
else
|
||||||
warn "No supported shortcut manager found. Add the shortcut in desktop settings."
|
# One line: the rest of what it has to say about KWin is printed below.
|
||||||
|
warn "${out%%$'\n'*}"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
if python3 -c 'import PyQt6.QtWidgets' 2>/dev/null; then
|
||||||
|
register toggle "$SHORTCUT" "Start and stop"
|
||||||
|
if [[ -n "$CANCEL_SHORTCUT" ]]; then
|
||||||
|
register cancel "$CANCEL_SHORTCUT" "Discard the recording"
|
||||||
|
fi
|
||||||
|
if [[ "${XDG_CURRENT_DESKTOP:-}" != *[Gg][Nn][Oo][Mm][Ee]* ]]; then
|
||||||
|
warn "KWin only reads these at startup, so they go live after your next"
|
||||||
|
say "login. Until then open Settings → Shortcuts and turn on the"
|
||||||
|
say "built-in listener to use them right away."
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
warn "PyQt6 is missing, so no shortcut was registered. Install it, then run:"
|
||||||
|
say "dikte shortcut install toggle --combo '$SHORTCUT'"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo
|
echo
|
||||||
ok "Done. Start it with: dikte"
|
ok "Done. Start it with: dikte"
|
||||||
say "The settings window opens on first run; add an OpenAI, Groq or OpenRouter key."
|
say "The settings window opens on first run: download a speech model, or add"
|
||||||
|
say "an OpenAI or OpenRouter key instead."
|
||||||
echo
|
echo
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ import wave
|
|||||||
from PyQt6.QtCore import QObject, pyqtSignal
|
from PyQt6.QtCore import QObject, pyqtSignal
|
||||||
|
|
||||||
import api
|
import api
|
||||||
|
import cleanup
|
||||||
import config as cfg
|
import config as cfg
|
||||||
import filetranscribe
|
import filetranscribe
|
||||||
import vad
|
import vad
|
||||||
@@ -208,14 +209,7 @@ class MeetingPipeline(QObject):
|
|||||||
if len(blocks) > 1:
|
if len(blocks) > 1:
|
||||||
self._say(t("Cleaning up {index}/{count}…",
|
self._say(t("Cleaning up {index}/{count}…",
|
||||||
index=index, count=len(blocks)))
|
index=index, count=len(blocks)))
|
||||||
out.append(api.cleanup(
|
out.append(cleanup.run(block, conf, prompt, timeout=600))
|
||||||
block,
|
|
||||||
conf.openrouter_key(),
|
|
||||||
conf["cleanup_model"],
|
|
||||||
prompt,
|
|
||||||
reasoning=conf["cleanup_reasoning"],
|
|
||||||
base_url=conf["openrouter_base_url"],
|
|
||||||
))
|
|
||||||
return "\n".join(out)
|
return "\n".join(out)
|
||||||
|
|
||||||
def _write(self, doc_path, minutes, transcript, entry):
|
def _write(self, doc_path, minutes, transcript, entry):
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ atexit.register(shutil.rmtree, _SANDBOX, True)
|
|||||||
# A key sitting in the environment would otherwise reach the code that falls
|
# A key sitting in the environment would otherwise reach the code that falls
|
||||||
# back to it, and the tests for "there is no key" would pass only on a machine
|
# back to it, and the tests for "there is no key" would pass only on a machine
|
||||||
# without one.
|
# without one.
|
||||||
for _var in ("OPENAI_API_KEY", "OPENROUTER_API_KEY"):
|
for _var in ("OPENAI_API_KEY", "GROQ_API_KEY", "OPENROUTER_API_KEY"):
|
||||||
os.environ.pop(_var, None)
|
os.environ.pop(_var, None)
|
||||||
|
|
||||||
# The interface language leaks through module-level state, so the tests fix it
|
# The interface language leaks through module-level state, so the tests fix it
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ import sys
|
|||||||
import tempfile
|
import tempfile
|
||||||
import unittest
|
import unittest
|
||||||
import urllib.error
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
import wave
|
import wave
|
||||||
from unittest import mock
|
from unittest import mock
|
||||||
|
|
||||||
@@ -40,6 +41,12 @@ linux_only = unittest.skipUnless(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _no_network(*args, **kwargs):
|
||||||
|
raise AssertionError(
|
||||||
|
"a test reached the network; wrap the call in support.fake_urlopen"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _no_exec(*args, **kwargs):
|
def _no_exec(*args, **kwargs):
|
||||||
raise AssertionError(
|
raise AssertionError(
|
||||||
"a test reached os.execv, which would replace the test process with the "
|
"a test reached os.execv, which would replace the test process with the "
|
||||||
@@ -78,6 +85,11 @@ class DikteTest(unittest.TestCase):
|
|||||||
# with it and hang, so it fails loudly here instead.
|
# with it and hang, so it fails loudly here instead.
|
||||||
self.patch_attr(os, "execv", _no_exec)
|
self.patch_attr(os, "execv", _no_exec)
|
||||||
|
|
||||||
|
# Every way out of here goes through urllib, so closing it is enough to
|
||||||
|
# keep the suite offline. A test that means to answer a request patches
|
||||||
|
# this again through fake_urlopen.
|
||||||
|
self.patch_attr(urllib.request, "urlopen", _no_network)
|
||||||
|
|
||||||
# ---- helpers ---------------------------------------------------------
|
# ---- helpers ---------------------------------------------------------
|
||||||
|
|
||||||
def path(self, *parts):
|
def path(self, *parts):
|
||||||
|
|||||||
@@ -3,13 +3,21 @@
|
|||||||
Nothing here reaches the network. What is checked is the request that would have
|
Nothing here reaches the network. What is checked is the request that would have
|
||||||
gone out, because that is what a new provider changes and what an old one
|
gone out, because that is what a new provider changes and what an old one
|
||||||
notices: the URL, the headers, the fields of the multipart body, the JSON.
|
notices: the URL, the headers, the fields of the multipart body, the JSON.
|
||||||
|
|
||||||
|
Stopping one is the exception. Cutting a request off is done to the socket it
|
||||||
|
is blocked on, and a faked urlopen has no socket to cut, so those tests talk to
|
||||||
|
a server of their own on the loopback interface.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import http.server
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
import unittest
|
import unittest
|
||||||
|
|
||||||
import api
|
import api
|
||||||
|
import ggml
|
||||||
from tests.support import (
|
from tests.support import (
|
||||||
DikteTest,
|
DikteTest,
|
||||||
fake_urlopen,
|
fake_urlopen,
|
||||||
@@ -21,6 +29,7 @@ from tests.support import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
OPENAI = api.Target("openai", "OpenAI", "sk-test", api.OPENAI_URL, "gpt-4o-transcribe")
|
OPENAI = api.Target("openai", "OpenAI", "sk-test", api.OPENAI_URL, "gpt-4o-transcribe")
|
||||||
|
GROQ = api.Target("groq", "Groq", "gsk-test", api.GROQ_URL, "whisper-large-v3-turbo")
|
||||||
OPENROUTER = api.Target("openrouter", "OpenRouter", "sk-or-test",
|
OPENROUTER = api.Target("openrouter", "OpenRouter", "sk-or-test",
|
||||||
api.OPENROUTER_URL, "openai/gpt-4o-transcribe")
|
api.OPENROUTER_URL, "openai/gpt-4o-transcribe")
|
||||||
|
|
||||||
@@ -32,6 +41,18 @@ class TimestampModel(unittest.TestCase):
|
|||||||
def test_openrouter_namespaces_the_id(self):
|
def test_openrouter_namespaces_the_id(self):
|
||||||
self.assertEqual(api.timestamp_model("openrouter"), "openai/whisper-1")
|
self.assertEqual(api.timestamp_model("openrouter"), "openai/whisper-1")
|
||||||
|
|
||||||
|
def test_groq_keeps_the_model_that_was_chosen(self):
|
||||||
|
"""Every model it transcribes with is a whisper, so all of them do times."""
|
||||||
|
self.assertEqual(api.timestamp_model("groq", "whisper-large-v3"),
|
||||||
|
"whisper-large-v3")
|
||||||
|
|
||||||
|
def test_groq_with_nothing_chosen_falls_back(self):
|
||||||
|
self.assertEqual(api.timestamp_model("groq"), "whisper-large-v3-turbo")
|
||||||
|
|
||||||
|
def test_the_others_ignore_what_was_chosen(self):
|
||||||
|
self.assertEqual(api.timestamp_model("openai", "gpt-4o-transcribe"),
|
||||||
|
"whisper-1")
|
||||||
|
|
||||||
|
|
||||||
class Explain(DikteTest):
|
class Explain(DikteTest):
|
||||||
def error(self, status):
|
def error(self, status):
|
||||||
@@ -176,13 +197,28 @@ class Transcribe(DikteTest):
|
|||||||
self.assertEqual(multipart_fields(calls[0])["language"], "tr")
|
self.assertEqual(multipart_fields(calls[0])["language"], "tr")
|
||||||
self.assertNotIn("language", multipart_fields(calls[1]))
|
self.assertNotIn("language", multipart_fields(calls[1]))
|
||||||
|
|
||||||
def test_the_glossary_goes_to_openai_only(self):
|
def test_the_glossary_goes_everywhere_but_openrouter(self):
|
||||||
"""OpenRouter takes the field and throws it away, so spare it the bytes."""
|
"""OpenRouter takes the field and throws it away, so spare it the bytes."""
|
||||||
with fake_urlopen({"text": "hi"}) as calls:
|
with fake_urlopen({"text": "hi"}) as calls:
|
||||||
api.transcribe(OPENAI, self.wav, prompt="Paraşüt, OpenFrame")
|
api.transcribe(OPENAI, self.wav, prompt="Paraşüt, OpenFrame")
|
||||||
|
api.transcribe(GROQ, self.wav, prompt="Paraşüt, OpenFrame")
|
||||||
api.transcribe(OPENROUTER, self.wav, prompt="Paraşüt, OpenFrame")
|
api.transcribe(OPENROUTER, self.wav, prompt="Paraşüt, OpenFrame")
|
||||||
self.assertIn("prompt", multipart_fields(calls[0]))
|
self.assertIn("prompt", multipart_fields(calls[0]))
|
||||||
self.assertNotIn("prompt", multipart_fields(calls[1]))
|
self.assertIn("prompt", multipart_fields(calls[1]))
|
||||||
|
self.assertNotIn("prompt", multipart_fields(calls[2]))
|
||||||
|
|
||||||
|
def test_groq_goes_to_groq(self):
|
||||||
|
with fake_urlopen({"text": "hi"}) as calls:
|
||||||
|
api.transcribe(GROQ, self.wav)
|
||||||
|
self.assertEqual(calls[0].full_url,
|
||||||
|
"https://api.groq.com/openai/v1/audio/transcriptions")
|
||||||
|
self.assertEqual(multipart_fields(calls[0])["model"], "whisper-large-v3-turbo")
|
||||||
|
|
||||||
|
def test_a_refused_groq_key_is_explained_in_groq_s_name(self):
|
||||||
|
with fake_urlopen(http_error(401, '{"error": {"message": "bad key"}}')), \
|
||||||
|
self.assertRaises(api.ApiError) as caught:
|
||||||
|
api.transcribe(GROQ, self.wav)
|
||||||
|
self.assertIn("Groq", str(caught.exception))
|
||||||
|
|
||||||
def test_openrouter_is_attributed(self):
|
def test_openrouter_is_attributed(self):
|
||||||
with fake_urlopen({"text": "hi"}) as calls:
|
with fake_urlopen({"text": "hi"}) as calls:
|
||||||
@@ -242,6 +278,12 @@ class TranscribeSegments(DikteTest):
|
|||||||
api.transcribe_segments(OPENROUTER, self.wav)
|
api.transcribe_segments(OPENROUTER, self.wav)
|
||||||
self.assertEqual(multipart_fields(calls[0])["model"], "openai/whisper-1")
|
self.assertEqual(multipart_fields(calls[0])["model"], "openai/whisper-1")
|
||||||
|
|
||||||
|
def test_groq_stays_on_the_model_it_was_given(self):
|
||||||
|
target = GROQ._replace(model="whisper-large-v3")
|
||||||
|
with fake_urlopen(self.reply([{"start": 0, "end": 1, "text": "hi"}])) as calls:
|
||||||
|
api.transcribe_segments(target, self.wav)
|
||||||
|
self.assertEqual(multipart_fields(calls[0])["model"], "whisper-large-v3")
|
||||||
|
|
||||||
def test_the_segments_come_back_as_numbers(self):
|
def test_the_segments_come_back_as_numbers(self):
|
||||||
with fake_urlopen(self.reply([
|
with fake_urlopen(self.reply([
|
||||||
{"start": "0.5", "end": "2.25", "text": " hello "},
|
{"start": "0.5", "end": "2.25", "text": " hello "},
|
||||||
@@ -432,6 +474,229 @@ class ModelLists(DikteTest):
|
|||||||
with self.assertRaises(api.ApiError):
|
with self.assertRaises(api.ApiError):
|
||||||
api.openai_models("")
|
api.openai_models("")
|
||||||
|
|
||||||
|
def test_the_same_list_read_from_groq(self):
|
||||||
|
with fake_urlopen({"data": [{"id": "llama-3.3-70b"},
|
||||||
|
{"id": "whisper-large-v3"}]}) as calls:
|
||||||
|
models = api.openai_models("gsk-test", api.GROQ_URL, "Groq")
|
||||||
|
self.assertEqual(calls[0].full_url, "https://api.groq.com/openai/v1/models")
|
||||||
|
self.assertEqual(models, ["whisper-large-v3"])
|
||||||
|
|
||||||
|
def test_a_missing_groq_key_says_groq(self):
|
||||||
|
with self.assertRaises(api.ApiError) as caught:
|
||||||
|
api.openai_models("", api.GROQ_URL, "Groq")
|
||||||
|
self.assertIn("Groq", str(caught.exception))
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
||||||
|
|
||||||
|
class FakeServer:
|
||||||
|
"""A ggml.Server as far as api.py is concerned."""
|
||||||
|
|
||||||
|
def __init__(self, url="http://127.0.0.1:9999/v1", fails="", log=""):
|
||||||
|
self.url = url
|
||||||
|
self.fails = fails
|
||||||
|
self.log = log
|
||||||
|
self.starts = 0
|
||||||
|
|
||||||
|
def serve(self):
|
||||||
|
self.starts += 1
|
||||||
|
if self.fails:
|
||||||
|
raise ggml.LocalError(self.fails)
|
||||||
|
return self.url
|
||||||
|
|
||||||
|
def error(self):
|
||||||
|
return self.log
|
||||||
|
|
||||||
|
|
||||||
|
LOCAL = api.Target("local", "Local whisper", "", "", "ggml-base.bin")
|
||||||
|
|
||||||
|
|
||||||
|
class TranscribeHere(DikteTest):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.wav = str(self.path("clip.wav"))
|
||||||
|
os.makedirs(self.root, exist_ok=True)
|
||||||
|
with open(self.wav, "wb") as fh:
|
||||||
|
fh.write(b"RIFFfake")
|
||||||
|
self.server = FakeServer()
|
||||||
|
self.patch_attr(ggml, "whisper", self.server)
|
||||||
|
|
||||||
|
def test_the_address_comes_from_the_server_it_starts(self):
|
||||||
|
with fake_urlopen({"text": "hello"}) as calls:
|
||||||
|
api.transcribe(LOCAL, self.wav)
|
||||||
|
self.assertEqual(self.server.starts, 1)
|
||||||
|
self.assertEqual(calls[0].full_url,
|
||||||
|
"http://127.0.0.1:9999/v1/audio/transcriptions")
|
||||||
|
|
||||||
|
def test_nothing_local_is_authorised(self):
|
||||||
|
with fake_urlopen({"text": "hello"}) as calls:
|
||||||
|
api.transcribe(LOCAL, self.wav)
|
||||||
|
self.assertNotIn("Authorization", calls[0].headers)
|
||||||
|
|
||||||
|
def test_a_server_that_will_not_start_is_the_error_shown(self):
|
||||||
|
self.patch_attr(ggml, "whisper", FakeServer(fails="no model downloaded"))
|
||||||
|
with self.assertRaises(api.ApiError) as caught:
|
||||||
|
api.transcribe(LOCAL, self.wav)
|
||||||
|
self.assertIn("no model downloaded", str(caught.exception))
|
||||||
|
|
||||||
|
def test_a_server_that_dies_mid_request_says_what_it_printed(self):
|
||||||
|
self.patch_attr(ggml, "whisper", FakeServer(log="out of memory"))
|
||||||
|
with fake_urlopen(url_error("connection reset")):
|
||||||
|
with self.assertRaises(api.ApiError) as caught:
|
||||||
|
api.transcribe(LOCAL, self.wav)
|
||||||
|
self.assertIn("out of memory", str(caught.exception))
|
||||||
|
|
||||||
|
def test_the_hint_reaches_whisper_as_its_initial_prompt(self):
|
||||||
|
with fake_urlopen({"text": "hi"}) as calls:
|
||||||
|
api.transcribe(LOCAL, self.wav, prompt="Dikte, Paraşüt")
|
||||||
|
self.assertEqual(multipart_fields(calls[0])["prompt"], "Dikte, Paraşüt")
|
||||||
|
|
||||||
|
def test_a_word_broken_over_two_lines_is_put_back_together(self):
|
||||||
|
# whisper.cpp cuts on tokens and writes one segment per line, which in
|
||||||
|
# Turkish lands inside a word about as often as between two.
|
||||||
|
with fake_urlopen({"text": "Onlar akraba değ\niller. Ve\n devamı."}):
|
||||||
|
# The line break inside a word leaves nothing in its place; the
|
||||||
|
# one between two words is where whisper's own leading space is.
|
||||||
|
self.assertEqual(api.transcribe(LOCAL, self.wav),
|
||||||
|
"Onlar akraba değiller. Ve devamı.")
|
||||||
|
|
||||||
|
def test_a_local_timeout_is_not_a_hosted_one(self):
|
||||||
|
# Nothing is being spent but time, and a long file on a machine without
|
||||||
|
# a graphics card takes a good deal of it.
|
||||||
|
with fake_urlopen({"text": "hi"}):
|
||||||
|
api.transcribe(LOCAL, self.wav, timeout=300)
|
||||||
|
self.assertGreaterEqual(api.LOCAL_TIMEOUT, 600)
|
||||||
|
|
||||||
|
def test_segments_that_continue_a_word_are_merged(self):
|
||||||
|
reply = {"segments": [
|
||||||
|
{"start": 0.0, "end": 1.0, "text": " Onlar akraba değ"},
|
||||||
|
{"start": 1.0, "end": 1.4, "text": "iller."},
|
||||||
|
{"start": 2.0, "end": 3.0, "text": " Başka bir cümle."},
|
||||||
|
]}
|
||||||
|
with fake_urlopen(reply):
|
||||||
|
out = api.transcribe_segments(LOCAL, self.wav)
|
||||||
|
self.assertEqual([text for _, _, text in out],
|
||||||
|
["Onlar akraba değiller.", "Başka bir cümle."])
|
||||||
|
self.assertEqual(out[0][1], 1.4) # the merged cue covers the whole word
|
||||||
|
|
||||||
|
def test_the_loaded_model_is_the_one_asked_for_again(self):
|
||||||
|
with fake_urlopen({"segments": [{"start": 0, "end": 1, "text": " hi"}]}) as calls:
|
||||||
|
api.transcribe_segments(LOCAL, self.wav)
|
||||||
|
self.assertEqual(multipart_fields(calls[0])["model"], "ggml-base.bin")
|
||||||
|
|
||||||
|
|
||||||
|
class Stopping(unittest.TestCase):
|
||||||
|
"""The Stop button, from the far end: a request already blocked on a reply.
|
||||||
|
|
||||||
|
The one that matters is a whisper on this machine, which answers minutes
|
||||||
|
after it was asked, so it is a real socket here rather than a fake urlopen.
|
||||||
|
Nothing leaves the loopback interface.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
answering = threading.Event()
|
||||||
|
|
||||||
|
class Slow(http.server.BaseHTTPRequestHandler):
|
||||||
|
def do_POST(self):
|
||||||
|
self.rfile.read(int(self.headers.get("Content-Length") or 0))
|
||||||
|
answering.set()
|
||||||
|
time.sleep(30) # the model, thinking
|
||||||
|
|
||||||
|
def log_message(self, *args):
|
||||||
|
pass
|
||||||
|
|
||||||
|
self.answering = answering
|
||||||
|
self.server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Slow)
|
||||||
|
threading.Thread(target=self.server.serve_forever, daemon=True).start()
|
||||||
|
self.addCleanup(self.server.server_close)
|
||||||
|
self.addCleanup(self.server.shutdown)
|
||||||
|
self.url = f"http://127.0.0.1:{self.server.server_address[1]}/v1/x"
|
||||||
|
|
||||||
|
def post(self, aborter, out):
|
||||||
|
try:
|
||||||
|
api._request(self.url, b"{}", {}, timeout=30, aborter=aborter)
|
||||||
|
out.append("answered")
|
||||||
|
except BaseException as exc: # noqa: BLE001 - the type is the result
|
||||||
|
out.append(type(exc).__name__)
|
||||||
|
|
||||||
|
def test_a_request_waiting_on_a_reply_is_cut_off(self):
|
||||||
|
aborter, out = api.Aborter(), []
|
||||||
|
thread = threading.Thread(target=self.post, args=(aborter, out))
|
||||||
|
thread.start()
|
||||||
|
self.assertTrue(self.answering.wait(10))
|
||||||
|
aborter.abort()
|
||||||
|
thread.join(timeout=10)
|
||||||
|
self.assertFalse(thread.is_alive())
|
||||||
|
self.assertEqual(out, ["Aborted"])
|
||||||
|
|
||||||
|
def test_a_request_that_starts_after_the_stop_never_goes_out(self):
|
||||||
|
aborter, out = api.Aborter(), []
|
||||||
|
aborter.abort()
|
||||||
|
self.post(aborter, out)
|
||||||
|
self.assertEqual(out, ["Aborted"])
|
||||||
|
self.assertFalse(self.answering.is_set())
|
||||||
|
|
||||||
|
def test_without_one_the_request_is_the_plain_urllib_one(self):
|
||||||
|
"""Everything that is not stoppable keeps the opener it always had."""
|
||||||
|
with fake_urlopen({"text": "hi"}) as calls:
|
||||||
|
api._request(self.url, b"{}", {})
|
||||||
|
self.assertEqual(len(calls), 1)
|
||||||
|
|
||||||
|
|
||||||
|
class Sockets(unittest.TestCase):
|
||||||
|
"""The few lines urllib takes between making a connection and blocking on
|
||||||
|
it. A stop that lands in there must not leave the request waiting out its
|
||||||
|
hour-long local timeout."""
|
||||||
|
|
||||||
|
class FakeConn:
|
||||||
|
auto_open = 1
|
||||||
|
sock = None
|
||||||
|
closed = False
|
||||||
|
|
||||||
|
def close(self):
|
||||||
|
self.closed = True
|
||||||
|
|
||||||
|
def test_a_connection_opened_after_the_stop_is_refused(self):
|
||||||
|
sockets = api._Sockets()
|
||||||
|
sockets.cut()
|
||||||
|
with self.assertRaises(api.Aborted):
|
||||||
|
sockets.add(self.FakeConn())
|
||||||
|
|
||||||
|
def test_one_that_is_already_open_is_closed_where_it_stands(self):
|
||||||
|
sockets, conn = api._Sockets(), self.FakeConn()
|
||||||
|
sockets.add(conn)
|
||||||
|
sockets.cut()
|
||||||
|
self.assertTrue(conn.closed)
|
||||||
|
|
||||||
|
def test_one_with_no_socket_yet_is_stopped_from_making_another(self):
|
||||||
|
"""close() leaves auto_open on, and the next line would reconnect."""
|
||||||
|
sockets, conn = api._Sockets(), self.FakeConn()
|
||||||
|
sockets.add(conn)
|
||||||
|
sockets.cut()
|
||||||
|
self.assertEqual(conn.auto_open, 0)
|
||||||
|
|
||||||
|
|
||||||
|
class Aborter(unittest.TestCase):
|
||||||
|
def test_what_was_registered_is_run_once_the_stop_lands(self):
|
||||||
|
aborter, cut = api.Aborter(), []
|
||||||
|
with aborter.holding(lambda: cut.append(True)):
|
||||||
|
aborter.abort()
|
||||||
|
self.assertEqual(cut, [True])
|
||||||
|
|
||||||
|
def test_a_block_that_ended_is_not_cut_afterwards(self):
|
||||||
|
aborter, cut = api.Aborter(), []
|
||||||
|
with aborter.holding(lambda: cut.append(True)):
|
||||||
|
pass
|
||||||
|
aborter.abort()
|
||||||
|
self.assertEqual(cut, [])
|
||||||
|
|
||||||
|
def test_a_stop_that_already_landed_stops_the_next_step_too(self):
|
||||||
|
aborter = api.Aborter()
|
||||||
|
aborter.abort()
|
||||||
|
with self.assertRaises(api.Aborted):
|
||||||
|
aborter.check()
|
||||||
|
with self.assertRaises(api.Aborted):
|
||||||
|
with aborter.holding(lambda: None):
|
||||||
|
pass
|
||||||
|
|||||||
@@ -79,9 +79,12 @@ class Effort(unittest.TestCase):
|
|||||||
self.assertEqual(assistant.CODEX_EFFORT["xhigh"], "high")
|
self.assertEqual(assistant.CODEX_EFFORT["xhigh"], "high")
|
||||||
self.assertEqual(assistant.CODEX_EFFORT["max"], "high")
|
self.assertEqual(assistant.CODEX_EFFORT["max"], "high")
|
||||||
|
|
||||||
def test_claude_has_no_rung_below_low(self):
|
def test_neither_one_asks_for_a_rung_below_low(self):
|
||||||
self.assertEqual(assistant.CLAUDE_EFFORT["none"], "low")
|
# Claude has none; Codex has one, but calls it "minimal" on the older
|
||||||
self.assertEqual(assistant.CLAUDE_EFFORT["minimal"], "low")
|
# models and "none" on the newer ones, and refuses the wrong word.
|
||||||
|
for scale in (assistant.CLAUDE_EFFORT, assistant.CODEX_EFFORT):
|
||||||
|
self.assertEqual(scale["none"], "low")
|
||||||
|
self.assertEqual(scale["minimal"], "low")
|
||||||
|
|
||||||
def test_an_empty_setting_asks_for_nothing(self):
|
def test_an_empty_setting_asks_for_nothing(self):
|
||||||
self.assertEqual(assistant.CLAUDE_EFFORT.get("", ""), "")
|
self.assertEqual(assistant.CLAUDE_EFFORT.get("", ""), "")
|
||||||
@@ -232,10 +235,10 @@ class SessionMissing(unittest.TestCase):
|
|||||||
self.assertFalse(assistant._session_missing(text))
|
self.assertFalse(assistant._session_missing(text))
|
||||||
|
|
||||||
def test_the_last_line_is_the_one_worth_showing(self):
|
def test_the_last_line_is_the_one_worth_showing(self):
|
||||||
self.assertEqual(assistant._last_line("warning\n\nreal error\n"),
|
self.assertEqual(assistant.last_line("warning\n\nreal error\n"),
|
||||||
"real error")
|
"real error")
|
||||||
self.assertEqual(assistant._last_line(""), "")
|
self.assertEqual(assistant.last_line(""), "")
|
||||||
self.assertEqual(assistant._last_line(None), "")
|
self.assertEqual(assistant.last_line(None), "")
|
||||||
|
|
||||||
|
|
||||||
class Conclude(DikteTest):
|
class Conclude(DikteTest):
|
||||||
|
|||||||
@@ -0,0 +1,286 @@
|
|||||||
|
"""Who cleans the transcript up, and what they are asked.
|
||||||
|
|
||||||
|
The CLIs are faked at subprocess.run: what the tests read is the argument list
|
||||||
|
each one is given, where the answer is picked up from, and what happens to the
|
||||||
|
chain when the program is missing, slow or unhappy. The OpenRouter path is the
|
||||||
|
one that was always there and is checked here only for still being taken.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import unittest
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
import api
|
||||||
|
import cleanup
|
||||||
|
import ggml
|
||||||
|
from tests.support import DikteTest, fake_urlopen, sent_json, url_error
|
||||||
|
from tests.test_api import FakeServer, chat_reply
|
||||||
|
|
||||||
|
|
||||||
|
def fake_run(stdout="", code=0, stderr="", last_message=""):
|
||||||
|
"""Stand in for subprocess.run, writing the file Codex would have written."""
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
def run(cmd, **kwargs):
|
||||||
|
calls.append(cmd)
|
||||||
|
if last_message and "-o" in cmd:
|
||||||
|
with open(cmd[cmd.index("-o") + 1], "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(last_message)
|
||||||
|
return subprocess.CompletedProcess(cmd, code, stdout, stderr)
|
||||||
|
|
||||||
|
return mock.patch.object(subprocess, "run", side_effect=run), calls
|
||||||
|
|
||||||
|
|
||||||
|
class Provider(DikteTest):
|
||||||
|
def test_the_default_is_still_openrouter(self):
|
||||||
|
self.assertEqual(cleanup.provider(self.config()), "openrouter")
|
||||||
|
|
||||||
|
def test_a_provider_this_version_does_not_have(self):
|
||||||
|
self.assertEqual(
|
||||||
|
cleanup.provider(self.config(cleanup_provider="ollama")), "openrouter")
|
||||||
|
|
||||||
|
def test_each_one_is_recognised(self):
|
||||||
|
for name in cleanup.PROVIDERS:
|
||||||
|
with self.subTest(name=name):
|
||||||
|
self.assertEqual(
|
||||||
|
cleanup.provider(self.config(cleanup_provider=name)), name)
|
||||||
|
|
||||||
|
def test_what_each_one_runs(self):
|
||||||
|
self.assertEqual(cleanup.executable("claude"), "claude")
|
||||||
|
self.assertEqual(cleanup.executable("codex"), "codex")
|
||||||
|
self.assertEqual(cleanup.executable("openrouter"), "")
|
||||||
|
|
||||||
|
def test_the_model_named_in_the_history_is_the_one_that_did_it(self):
|
||||||
|
self.assertEqual(cleanup.model(self.config(cleanup_model="some/model")),
|
||||||
|
"some/model")
|
||||||
|
self.assertEqual(
|
||||||
|
cleanup.model(self.config(cleanup_provider="claude")), "haiku")
|
||||||
|
self.assertEqual(
|
||||||
|
cleanup.model(self.config(cleanup_provider="claude",
|
||||||
|
cleanup_claude_model="opus")), "opus")
|
||||||
|
# Codex on its own default has no model id to report, only a name.
|
||||||
|
self.assertEqual(
|
||||||
|
cleanup.model(self.config(cleanup_provider="codex")), "codex")
|
||||||
|
self.assertEqual(
|
||||||
|
cleanup.model(self.config(cleanup_provider="codex",
|
||||||
|
cleanup_codex_model="gpt-5.4")), "gpt-5.4")
|
||||||
|
|
||||||
|
|
||||||
|
class OpenRouter(DikteTest):
|
||||||
|
def test_it_is_still_one_request_with_the_settings_as_they_were(self):
|
||||||
|
conf = self.config(openrouter_api_key="sk-or-test",
|
||||||
|
cleanup_model="some/model", cleanup_reasoning="low")
|
||||||
|
with mock.patch.object(api, "cleanup", return_value="Done.") as call:
|
||||||
|
self.assertEqual(cleanup.run("uh, done", conf, "the rules"), "Done.")
|
||||||
|
text, key, model, prompt = call.call_args.args
|
||||||
|
self.assertEqual((text, key, model, prompt),
|
||||||
|
("uh, done", "sk-or-test", "some/model", "the rules"))
|
||||||
|
self.assertEqual(call.call_args.kwargs["reasoning"], "low")
|
||||||
|
|
||||||
|
def test_no_cli_is_started_for_it(self):
|
||||||
|
conf = self.config(openrouter_api_key="sk-or-test")
|
||||||
|
patcher, calls = fake_run(stdout="never")
|
||||||
|
with patcher, mock.patch.object(api, "cleanup", return_value="Done."):
|
||||||
|
cleanup.run("uh, done", conf, "the rules")
|
||||||
|
self.assertEqual(calls, [])
|
||||||
|
|
||||||
|
|
||||||
|
class ClaudeCode(DikteTest):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.conf = self.config(cleanup_provider="claude")
|
||||||
|
self.patch_attr(cleanup.shutil, "which", lambda name: f"/usr/bin/{name}")
|
||||||
|
|
||||||
|
def run_cleanup(self, text="uh, book it", **kwargs):
|
||||||
|
patcher, calls = fake_run(**kwargs)
|
||||||
|
with patcher:
|
||||||
|
answer = cleanup.run(text, self.conf, "the rules")
|
||||||
|
return answer, calls[0]
|
||||||
|
|
||||||
|
def test_the_transcript_goes_in_fenced_and_the_rules_go_in_as_the_prompt(self):
|
||||||
|
answer, cmd = self.run_cleanup(stdout="Book it.\n")
|
||||||
|
self.assertEqual(answer, "Book it.")
|
||||||
|
self.assertEqual(cmd[0], "claude")
|
||||||
|
self.assertIn("<transcript>\nuh, book it\n</transcript>", cmd)
|
||||||
|
self.assertEqual(cmd[cmd.index("--system-prompt") + 1], "the rules")
|
||||||
|
self.assertEqual(cmd[cmd.index("--model") + 1], "haiku")
|
||||||
|
|
||||||
|
def test_it_is_given_nothing_to_run_and_nothing_to_remember(self):
|
||||||
|
_, cmd = self.run_cleanup(stdout="Book it.")
|
||||||
|
self.assertEqual(cmd[cmd.index("--tools") + 1], "")
|
||||||
|
self.assertIn("--strict-mcp-config", cmd)
|
||||||
|
self.assertIn("--no-session-persistence", cmd)
|
||||||
|
|
||||||
|
def test_the_thinking_setting_is_carried_over_in_its_own_words(self):
|
||||||
|
self.conf["cleanup_reasoning"] = "none"
|
||||||
|
_, cmd = self.run_cleanup(stdout="Book it.")
|
||||||
|
self.assertEqual(cmd[cmd.index("--effort") + 1], "low")
|
||||||
|
|
||||||
|
def test_no_thinking_setting_means_no_flag(self):
|
||||||
|
_, cmd = self.run_cleanup(stdout="Book it.")
|
||||||
|
self.assertNotIn("--effort", cmd)
|
||||||
|
|
||||||
|
def test_a_model_of_your_own(self):
|
||||||
|
self.conf["cleanup_claude_model"] = "claude-sonnet-5"
|
||||||
|
_, cmd = self.run_cleanup(stdout="Book it.")
|
||||||
|
self.assertEqual(cmd[cmd.index("--model") + 1], "claude-sonnet-5")
|
||||||
|
|
||||||
|
def test_an_answer_of_nothing_is_a_failure_rather_than_an_empty_paste(self):
|
||||||
|
with self.assertRaises(cleanup.CleanupError):
|
||||||
|
self.run_cleanup(stdout=" \n")
|
||||||
|
|
||||||
|
def test_the_last_line_of_the_complaint_is_what_gets_shown(self):
|
||||||
|
with self.assertRaises(cleanup.CleanupError) as caught:
|
||||||
|
self.run_cleanup(code=1, stderr="a warning\nout of credit\n")
|
||||||
|
self.assertEqual(str(caught.exception), "out of credit")
|
||||||
|
|
||||||
|
def test_a_failure_is_the_same_kind_the_chain_already_catches(self):
|
||||||
|
# worker, the file transcriber and the meeting all keep the raw
|
||||||
|
# transcript when an ApiError comes out of here.
|
||||||
|
self.assertTrue(issubclass(cleanup.CleanupError, api.ApiError))
|
||||||
|
|
||||||
|
def test_a_program_that_is_not_installed_says_so_before_running_anything(self):
|
||||||
|
self.patch_attr(cleanup.shutil, "which", lambda name: "")
|
||||||
|
with self.assertRaises(cleanup.CleanupError) as caught:
|
||||||
|
self.run_cleanup(stdout="Book it.")
|
||||||
|
self.assertIn("claude", str(caught.exception))
|
||||||
|
|
||||||
|
def test_a_run_that_never_ends(self):
|
||||||
|
def run(cmd, **kwargs):
|
||||||
|
raise subprocess.TimeoutExpired(cmd, 180)
|
||||||
|
|
||||||
|
with mock.patch.object(subprocess, "run", side_effect=run):
|
||||||
|
with self.assertRaises(cleanup.CleanupError) as caught:
|
||||||
|
cleanup.run("uh, book it", self.conf, "the rules")
|
||||||
|
self.assertIn("180", str(caught.exception))
|
||||||
|
|
||||||
|
|
||||||
|
class Codex(DikteTest):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.conf = self.config(cleanup_provider="codex")
|
||||||
|
self.patch_attr(cleanup.shutil, "which", lambda name: f"/usr/bin/{name}")
|
||||||
|
|
||||||
|
def run_cleanup(self, text="uh, book it", **kwargs):
|
||||||
|
patcher, calls = fake_run(**kwargs)
|
||||||
|
with patcher:
|
||||||
|
answer = cleanup.run(text, self.conf, "the rules")
|
||||||
|
return answer, calls[0]
|
||||||
|
|
||||||
|
def test_the_rules_ride_in_front_of_the_transcript(self):
|
||||||
|
answer, cmd = self.run_cleanup(last_message="Book it.\n")
|
||||||
|
self.assertEqual(answer, "Book it.")
|
||||||
|
self.assertEqual(cmd[:2], ["codex", "exec"])
|
||||||
|
self.assertEqual(cmd[-1],
|
||||||
|
"the rules\n\n---\n\n<transcript>\nuh, book it\n</transcript>")
|
||||||
|
|
||||||
|
def test_the_answer_is_read_from_the_file_rather_than_the_noise_on_stdout(self):
|
||||||
|
answer, _ = self.run_cleanup(
|
||||||
|
stdout="workdir: /home\nmodel: gpt-5.4\ntokens used 400\n",
|
||||||
|
last_message="Book it.",
|
||||||
|
)
|
||||||
|
self.assertEqual(answer, "Book it.")
|
||||||
|
|
||||||
|
def test_that_file_does_not_stay_behind(self):
|
||||||
|
_, cmd = self.run_cleanup(last_message="Book it.")
|
||||||
|
self.assertFalse(os.path.exists(cmd[cmd.index("-o") + 1]))
|
||||||
|
|
||||||
|
def test_it_may_read_but_not_write_and_has_nobody_to_ask(self):
|
||||||
|
_, cmd = self.run_cleanup(last_message="Book it.")
|
||||||
|
self.assertEqual(cmd[cmd.index("--sandbox") + 1], "read-only")
|
||||||
|
self.assertIn('approval_policy="never"', cmd)
|
||||||
|
self.assertIn("--ephemeral", cmd)
|
||||||
|
|
||||||
|
def test_the_model_is_left_alone_until_one_is_typed_in(self):
|
||||||
|
_, cmd = self.run_cleanup(last_message="Book it.")
|
||||||
|
self.assertNotIn("-m", cmd)
|
||||||
|
self.conf["cleanup_codex_model"] = "gpt-5.4"
|
||||||
|
_, cmd = self.run_cleanup(last_message="Book it.")
|
||||||
|
self.assertEqual(cmd[cmd.index("-m") + 1], "gpt-5.4")
|
||||||
|
|
||||||
|
def test_the_thinking_setting_lands_on_the_nearest_rung_codex_has(self):
|
||||||
|
self.conf["cleanup_reasoning"] = "xhigh"
|
||||||
|
_, cmd = self.run_cleanup(last_message="Book it.")
|
||||||
|
self.assertIn('model_reasoning_effort="high"', cmd)
|
||||||
|
|
||||||
|
def test_an_answer_of_nothing(self):
|
||||||
|
with self.assertRaises(cleanup.CleanupError):
|
||||||
|
self.run_cleanup(stdout="tokens used 400", last_message="")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
||||||
|
|
||||||
|
class Here(DikteTest):
|
||||||
|
"""llama.cpp, answering the request OpenRouter answers."""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.conf = self.config(cleanup_provider="local",
|
||||||
|
local_llm_model="gemma.gguf")
|
||||||
|
self.server = FakeServer()
|
||||||
|
self.patch_attr(ggml, "llm", self.server)
|
||||||
|
|
||||||
|
def test_the_address_comes_from_the_server_it_starts(self):
|
||||||
|
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||||
|
self.assertEqual(cleanup.run("uh, done", self.conf, "the rules"),
|
||||||
|
"Done.")
|
||||||
|
self.assertEqual(self.server.starts, 1)
|
||||||
|
self.assertEqual(calls[0].full_url,
|
||||||
|
"http://127.0.0.1:9999/v1/chat/completions")
|
||||||
|
|
||||||
|
def test_no_key_is_wanted_and_none_is_sent(self):
|
||||||
|
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||||
|
cleanup.run("uh, done", self.conf, "the rules")
|
||||||
|
self.assertNotIn("Authorization", calls[0].headers)
|
||||||
|
|
||||||
|
def test_thinking_is_turned_off_in_the_words_llama_cpp_uses(self):
|
||||||
|
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||||
|
cleanup.run("uh, done", self.conf, "the rules")
|
||||||
|
self.assertEqual(sent_json(calls[0])["chat_template_kwargs"],
|
||||||
|
{"enable_thinking": False})
|
||||||
|
|
||||||
|
def test_the_models_own_default_asks_for_nothing(self):
|
||||||
|
self.conf["local_llm_reasoning"] = ""
|
||||||
|
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||||
|
cleanup.run("uh, done", self.conf, "the rules")
|
||||||
|
self.assertNotIn("chat_template_kwargs", sent_json(calls[0]))
|
||||||
|
|
||||||
|
def test_a_reply_longer_than_the_transcript_is_cut_off(self):
|
||||||
|
# A small model will repeat the transcript until the context is full,
|
||||||
|
# and every one of those tokens is a second of somebody waiting.
|
||||||
|
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||||
|
cleanup.run("x" * 4000, self.conf, "the rules")
|
||||||
|
self.assertEqual(sent_json(calls[0])["max_tokens"], 4000)
|
||||||
|
|
||||||
|
def test_a_short_dictation_still_gets_room_to_answer(self):
|
||||||
|
with fake_urlopen(chat_reply("Done.")) as calls:
|
||||||
|
cleanup.run("uh, done", self.conf, "the rules")
|
||||||
|
self.assertEqual(sent_json(calls[0])["max_tokens"], 512)
|
||||||
|
|
||||||
|
def test_a_reply_that_was_all_thinking_names_the_setting_that_fixes_it(self):
|
||||||
|
reply = {"choices": [{"message": {"content": "", "reasoning": "hmm"}}]}
|
||||||
|
with fake_urlopen(reply), self.assertRaises(api.ApiError) as caught:
|
||||||
|
cleanup.run("uh, done", self.conf, "the rules")
|
||||||
|
self.assertIn("Thinking", str(caught.exception))
|
||||||
|
|
||||||
|
def test_a_server_that_will_not_start_is_the_error_shown(self):
|
||||||
|
self.patch_attr(ggml, "llm", FakeServer(fails="llama.cpp is not installed"))
|
||||||
|
with self.assertRaises(api.ApiError) as caught:
|
||||||
|
cleanup.run("uh, done", self.conf, "the rules")
|
||||||
|
self.assertIn("llama.cpp", str(caught.exception))
|
||||||
|
|
||||||
|
def test_a_server_that_dies_mid_request_says_what_it_printed(self):
|
||||||
|
self.patch_attr(ggml, "llm", FakeServer(log="out of memory"))
|
||||||
|
with fake_urlopen(url_error("connection reset")):
|
||||||
|
with self.assertRaises(api.ApiError) as caught:
|
||||||
|
cleanup.run("uh, done", self.conf, "the rules")
|
||||||
|
self.assertIn("out of memory", str(caught.exception))
|
||||||
|
|
||||||
|
def test_no_cli_is_started_for_it(self):
|
||||||
|
patcher, calls = fake_run(stdout="never")
|
||||||
|
with patcher, fake_urlopen(chat_reply("Done.")):
|
||||||
|
cleanup.run("uh, done", self.conf, "the rules")
|
||||||
|
self.assertEqual(calls, [])
|
||||||
@@ -14,8 +14,9 @@ from unittest import mock
|
|||||||
|
|
||||||
import cli
|
import cli
|
||||||
import config as cfg
|
import config as cfg
|
||||||
|
import hotkey
|
||||||
import ipc
|
import ipc
|
||||||
from tests.support import DikteTest
|
from tests.support import DikteTest, fake_urlopen
|
||||||
|
|
||||||
|
|
||||||
class Options:
|
class Options:
|
||||||
@@ -144,6 +145,22 @@ class Parser(unittest.TestCase):
|
|||||||
self.assertIsNone(opts.verb)
|
self.assertIsNone(opts.verb)
|
||||||
self.assertEqual(opts.func, cli.cmd_plain)
|
self.assertEqual(opts.func, cli.cmd_plain)
|
||||||
|
|
||||||
|
def test_every_global_shortcut_runs_a_verb_that_exists(self):
|
||||||
|
"""A shortcut registers a command line; a verb the parser never heard of
|
||||||
|
is a key that does nothing at all when it is pressed."""
|
||||||
|
for name, spec in hotkey.SHORTCUTS.items():
|
||||||
|
with self.subTest(name=name):
|
||||||
|
opts = self.parse(spec.verb)
|
||||||
|
self.assertTrue(callable(opts.func))
|
||||||
|
|
||||||
|
def test_every_shortcut_can_be_installed_and_removed_by_name(self):
|
||||||
|
for name in hotkey.SHORTCUTS:
|
||||||
|
with self.subTest(name=name):
|
||||||
|
self.assertEqual(self.parse("shortcut", "install", name).which,
|
||||||
|
name)
|
||||||
|
self.assertEqual(self.parse("shortcut", "remove", name).which,
|
||||||
|
name)
|
||||||
|
|
||||||
def test_every_verb_is_wired_to_something(self):
|
def test_every_verb_is_wired_to_something(self):
|
||||||
for verb in ("record", "toggle", "start", "stop", "cancel", "ask",
|
for verb in ("record", "toggle", "start", "stop", "cancel", "ask",
|
||||||
"session", "transcribe", "meeting", "meetings", "history",
|
"session", "transcribe", "meeting", "meetings", "history",
|
||||||
@@ -316,6 +333,67 @@ class ConfigCommands(DikteTest):
|
|||||||
{"cleanup", "subtitles", "meeting", "agent"})
|
{"cleanup", "subtitles", "meeting", "agent"})
|
||||||
|
|
||||||
|
|
||||||
|
class Providers(DikteTest):
|
||||||
|
"""The terminal reaches every provider the settings window does."""
|
||||||
|
|
||||||
|
def run_cmd(self, func, **values):
|
||||||
|
with captured() as (out, err):
|
||||||
|
code = func(Options(**values))
|
||||||
|
return code, out.getvalue(), err.getvalue()
|
||||||
|
|
||||||
|
def test_a_provider_the_settings_window_offers_is_a_choice_here_too(self):
|
||||||
|
parser = cli.build_parser()
|
||||||
|
for provider in cfg.TRANSCRIBERS:
|
||||||
|
with self.subTest(provider=provider):
|
||||||
|
opts = parser.parse_args(["models", "--provider", provider])
|
||||||
|
self.assertEqual(opts.provider, provider)
|
||||||
|
self.assertEqual(parser.parse_args(["test-key", provider]).which,
|
||||||
|
provider)
|
||||||
|
|
||||||
|
def test_the_model_list_is_read_from_the_chosen_provider(self):
|
||||||
|
self.write_config({"groq_api_key": "gsk-test"})
|
||||||
|
with fake_urlopen({"data": [{"id": "whisper-large-v3"}]}) as calls:
|
||||||
|
code, out, _ = self.run_cmd(cli.cmd_models, provider="groq",
|
||||||
|
transcription=False)
|
||||||
|
self.assertEqual(code, 0)
|
||||||
|
self.assertEqual(calls[0].full_url, "https://api.groq.com/openai/v1/models")
|
||||||
|
self.assertEqual(out.strip(), "whisper-large-v3")
|
||||||
|
|
||||||
|
def test_a_key_that_is_not_there_is_reported_under_its_own_name(self):
|
||||||
|
code, out, _ = self.run_cmd(cli.cmd_test_key, which="groq")
|
||||||
|
self.assertEqual(code, 1)
|
||||||
|
self.assertIn("groq", out)
|
||||||
|
self.assertIn("Groq", out)
|
||||||
|
|
||||||
|
|
||||||
|
class Doctor(DikteTest):
|
||||||
|
"""One pass over everything the settings window checks behind its buttons."""
|
||||||
|
|
||||||
|
def run_doctor(self, as_json=True, **settings):
|
||||||
|
self.write_config(settings)
|
||||||
|
with mock.patch.object(ipc, "send", return_value=None), \
|
||||||
|
captured() as (out, _err):
|
||||||
|
cli.cmd_doctor(Options(json=as_json))
|
||||||
|
return json.loads(out.getvalue()) if as_json else out.getvalue()
|
||||||
|
|
||||||
|
def test_cleanup_on_openrouter_is_a_question_about_the_key(self):
|
||||||
|
reply = self.run_doctor(cleanup_model="some/model")
|
||||||
|
self.assertEqual(reply["cleanup"]["provider"], "openrouter")
|
||||||
|
self.assertEqual(reply["cleanup"]["model"], "some/model")
|
||||||
|
self.assertIn("OpenRouter key, cleaning up on some/model",
|
||||||
|
self.run_doctor(as_json=False, cleanup_model="some/model"))
|
||||||
|
|
||||||
|
def test_cleanup_on_a_cli_is_a_question_about_the_program(self):
|
||||||
|
reply = self.run_doctor(cleanup_provider="codex",
|
||||||
|
cleanup_codex_model="gpt-5.4")
|
||||||
|
self.assertEqual(reply["cleanup"]["provider"], "codex")
|
||||||
|
self.assertEqual(reply["cleanup"]["model"], "gpt-5.4")
|
||||||
|
self.assertIn("codex", reply["programs"])
|
||||||
|
self.assertIn("codex, cleaning up on gpt-5.4",
|
||||||
|
self.run_doctor(as_json=False, cleanup_provider="codex",
|
||||||
|
cleanup_codex_model="gpt-5.4"))
|
||||||
|
|
||||||
|
|
||||||
class Finding(DikteTest):
|
class Finding(DikteTest):
|
||||||
def test_no_history_at_all(self):
|
def test_no_history_at_all(self):
|
||||||
self.assertIsNone(cli._find_history("last"))
|
self.assertIsNone(cli._find_history("last"))
|
||||||
|
|||||||
@@ -12,7 +12,9 @@ import unittest
|
|||||||
from unittest import mock
|
from unittest import mock
|
||||||
|
|
||||||
import api
|
import api
|
||||||
|
import cleanup
|
||||||
import config as cfg
|
import config as cfg
|
||||||
|
import ggml
|
||||||
import i18n
|
import i18n
|
||||||
import paste
|
import paste
|
||||||
from tests.support import DikteTest
|
from tests.support import DikteTest
|
||||||
@@ -126,10 +128,23 @@ class Keys(DikteTest):
|
|||||||
def test_no_key_anywhere(self):
|
def test_no_key_anywhere(self):
|
||||||
self.assertEqual(cfg.Config().openai_key(), "")
|
self.assertEqual(cfg.Config().openai_key(), "")
|
||||||
|
|
||||||
|
def test_every_provider_falls_back_to_the_variable_of_its_own_name(self):
|
||||||
|
with mock.patch.dict(os.environ, {"GROQ_API_KEY": "gsk-env"}):
|
||||||
|
self.assertEqual(cfg.Config().groq_key(), "gsk-env")
|
||||||
|
|
||||||
|
|
||||||
class TranscribeTarget(DikteTest):
|
class TranscribeTarget(DikteTest):
|
||||||
def test_openai_by_default(self):
|
def test_this_machine_by_default(self):
|
||||||
target = self.config(openai_api_key="sk-test").transcribe_target()
|
target = cfg.Config().transcribe_target()
|
||||||
|
self.assertEqual(target.provider, "local")
|
||||||
|
self.assertEqual(target.api_key, "")
|
||||||
|
# Empty on purpose: the server picks a port when it starts, and reading
|
||||||
|
# a setting must not be what starts it.
|
||||||
|
self.assertEqual(target.base_url, "")
|
||||||
|
|
||||||
|
def test_openai_when_it_is_picked(self):
|
||||||
|
target = self.config(transcribe_provider="openai",
|
||||||
|
openai_api_key="sk-test").transcribe_target()
|
||||||
self.assertEqual(target.provider, "openai")
|
self.assertEqual(target.provider, "openai")
|
||||||
self.assertEqual(target.service, "OpenAI")
|
self.assertEqual(target.service, "OpenAI")
|
||||||
self.assertEqual(target.api_key, "sk-test")
|
self.assertEqual(target.api_key, "sk-test")
|
||||||
@@ -146,8 +161,24 @@ class TranscribeTarget(DikteTest):
|
|||||||
self.assertEqual(target.api_key, "sk-or-test")
|
self.assertEqual(target.api_key, "sk-or-test")
|
||||||
self.assertEqual(target.model, "openai/whisper-1")
|
self.assertEqual(target.model, "openai/whisper-1")
|
||||||
|
|
||||||
|
def test_groq_when_it_is_picked(self):
|
||||||
|
conf = self.config(transcribe_provider="groq", groq_api_key="gsk-test",
|
||||||
|
groq_transcribe_model="whisper-large-v3")
|
||||||
|
target = conf.transcribe_target()
|
||||||
|
self.assertEqual(target.provider, "groq")
|
||||||
|
self.assertEqual(target.service, "Groq")
|
||||||
|
self.assertEqual(target.api_key, "gsk-test")
|
||||||
|
self.assertEqual(target.base_url, api.GROQ_URL)
|
||||||
|
self.assertEqual(target.model, "whisper-large-v3")
|
||||||
|
|
||||||
|
def test_a_provider_this_version_has_never_heard_of(self):
|
||||||
|
"""A config written by a fork, or by a version that dropped one."""
|
||||||
|
target = self.config(transcribe_provider="deepgram").transcribe_target()
|
||||||
|
self.assertEqual(target.provider, "openai")
|
||||||
|
|
||||||
def test_a_self_hosted_endpoint(self):
|
def test_a_self_hosted_endpoint(self):
|
||||||
conf = self.config(openai_base_url="http://localhost:8080/v1")
|
conf = self.config(transcribe_provider="openai",
|
||||||
|
openai_base_url="http://localhost:8080/v1")
|
||||||
self.assertEqual(conf.transcribe_target().base_url, "http://localhost:8080/v1")
|
self.assertEqual(conf.transcribe_target().base_url, "http://localhost:8080/v1")
|
||||||
|
|
||||||
|
|
||||||
@@ -459,3 +490,69 @@ class Directories(unittest.TestCase):
|
|||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
||||||
|
|
||||||
|
class LocalCleanup(DikteTest):
|
||||||
|
def test_the_local_model_is_what_the_history_records(self):
|
||||||
|
conf = self.config(cleanup_provider="local",
|
||||||
|
local_llm_model="gemma-3-4b-it-Q4_K_M.gguf")
|
||||||
|
self.assertEqual(cleanup.provider(conf), "local")
|
||||||
|
self.assertEqual(cleanup.model(conf), "gemma-3-4b-it-Q4_K_M.gguf")
|
||||||
|
|
||||||
|
def test_it_needs_no_program_on_the_path(self):
|
||||||
|
# whisper.cpp and llama.cpp are fetched rather than installed, so unlike
|
||||||
|
# Claude Code and Codex there is no executable to look for.
|
||||||
|
self.assertEqual(cleanup.executable("local"), "")
|
||||||
|
|
||||||
|
def test_the_minutes_do_not_follow_the_cleanup_provider(self):
|
||||||
|
# A 4B model here will strip the filler words out of a dictation and
|
||||||
|
# will not write up an hour long meeting.
|
||||||
|
conf = self.config(cleanup_provider="local")
|
||||||
|
self.assertEqual(conf["meeting_model"], cfg.DEFAULTS["meeting_model"])
|
||||||
|
|
||||||
|
def test_only_the_cleanup_setting_asks_for_the_local_model(self):
|
||||||
|
self.assertFalse(cfg.Config().uses_local_llm())
|
||||||
|
self.assertTrue(self.config(cleanup_provider="local").uses_local_llm())
|
||||||
|
|
||||||
|
|
||||||
|
class ReadyToRun(DikteTest):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.patch_attr(ggml, "MODELS_DIR", self.path("models"))
|
||||||
|
|
||||||
|
def install(self, name):
|
||||||
|
path = ggml.whisper_model_path(name)
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_bytes(b"model")
|
||||||
|
|
||||||
|
def test_a_missing_program_is_not_ready(self):
|
||||||
|
with mock.patch("shutil.which", return_value=None):
|
||||||
|
self.install("ggml-base.bin")
|
||||||
|
conf = self.config(local_model="ggml-base.bin")
|
||||||
|
self.assertFalse(conf.transcribe_ready())
|
||||||
|
|
||||||
|
def test_a_missing_model_is_not_ready_either(self):
|
||||||
|
with mock.patch("shutil.which", return_value="/usr/bin/whisper-server"):
|
||||||
|
conf = self.config(local_model="ggml-base.bin")
|
||||||
|
self.assertFalse(conf.transcribe_ready())
|
||||||
|
|
||||||
|
def test_both_halves_in_place(self):
|
||||||
|
with mock.patch("shutil.which", return_value="/usr/bin/whisper-server"):
|
||||||
|
self.install("ggml-base.bin")
|
||||||
|
conf = self.config(local_model="ggml-base.bin")
|
||||||
|
self.assertTrue(conf.transcribe_ready())
|
||||||
|
|
||||||
|
def test_a_hosted_provider_is_ready_when_it_has_a_key(self):
|
||||||
|
conf = self.config(transcribe_provider="openai", openai_api_key="sk-test")
|
||||||
|
self.assertTrue(conf.transcribe_ready())
|
||||||
|
|
||||||
|
def test_the_settings_reach_the_servers(self):
|
||||||
|
conf = self.config(local_model="ggml-base.bin", local_threads=4,
|
||||||
|
local_gpu=False, local_llm_model="gemma.gguf",
|
||||||
|
local_llm_context=4096)
|
||||||
|
conf.apply_local()
|
||||||
|
self.addCleanup(ggml.whisper.configure, model="", threads=0, gpu=True)
|
||||||
|
self.assertEqual(ggml.whisper.settings()["model"], "ggml-base.bin")
|
||||||
|
self.assertEqual(ggml.whisper.settings()["threads"], 4)
|
||||||
|
self.assertFalse(ggml.whisper.settings()["gpu"])
|
||||||
|
self.assertEqual(ggml.llm.settings()["context"], 4096)
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ made up a stamp nobody recorded.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import contextlib
|
import contextlib
|
||||||
|
import time
|
||||||
import unittest
|
import unittest
|
||||||
import wave
|
import wave
|
||||||
from unittest import mock
|
from unittest import mock
|
||||||
@@ -170,7 +171,7 @@ class Transcriber(DikteTest):
|
|||||||
worker.failed.connect(failures.append)
|
worker.failed.connect(failures.append)
|
||||||
worker.progress.connect(progress.append)
|
worker.progress.connect(progress.append)
|
||||||
|
|
||||||
def to_wav(path, workdir):
|
def to_wav(path, workdir, aborter=None):
|
||||||
return make_wav(self.path("converted.wav"), tone(1.0))
|
return make_wav(self.path("converted.wav"), tone(1.0))
|
||||||
|
|
||||||
with mock.patch.object(ft, "_to_wav", side_effect=to_wav), \
|
with mock.patch.object(ft, "_to_wav", side_effect=to_wav), \
|
||||||
@@ -225,6 +226,40 @@ class Transcriber(DikteTest):
|
|||||||
_, _, _, cleanup_call = self.run_chain(cleanup=True, transcript="")
|
_, _, _, cleanup_call = self.run_chain(cleanup=True, transcript="")
|
||||||
cleanup_call.assert_not_called()
|
cleanup_call.assert_not_called()
|
||||||
|
|
||||||
|
def test_a_stopped_run_is_not_a_failure(self):
|
||||||
|
def stopped(*args, **kwargs):
|
||||||
|
raise api.Aborted
|
||||||
|
done, failures, progress, _ = self.run_chain(fail=stopped)
|
||||||
|
self.assertEqual(failures, [])
|
||||||
|
self.assertEqual(done, [])
|
||||||
|
self.assertEqual(progress[-1], "Stopped.")
|
||||||
|
|
||||||
|
def test_the_request_is_handed_the_stop_to_watch(self):
|
||||||
|
worker = ft.FileTranscriber(self.conf)
|
||||||
|
with mock.patch.object(ft, "_to_wav", side_effect=lambda *a: self.source), \
|
||||||
|
mock.patch.object(ft.shutil, "which", return_value="/usr/bin/ffmpeg"), \
|
||||||
|
mock.patch.object(api, "transcribe", return_value="text") as call:
|
||||||
|
worker._work(self.source, False, False)
|
||||||
|
self.assertIs(call.call_args.kwargs["aborter"], worker._abort)
|
||||||
|
|
||||||
|
def test_stopping_a_local_run_stops_the_model_with_it(self):
|
||||||
|
"""Closing the socket is nothing to a process of ours: it would grind on
|
||||||
|
to the end of the chunk with nobody left to hand the answer to."""
|
||||||
|
worker = ft.FileTranscriber(self.conf)
|
||||||
|
worker._local = mock.Mock()
|
||||||
|
worker.stop()
|
||||||
|
self.assertTrue(worker._abort.aborted)
|
||||||
|
for _ in range(100):
|
||||||
|
if worker._local.stop.called:
|
||||||
|
break
|
||||||
|
time.sleep(0.01)
|
||||||
|
worker._local.stop.assert_called_once_with()
|
||||||
|
|
||||||
|
def test_a_run_that_is_over_leaves_the_model_alone(self):
|
||||||
|
worker = ft.FileTranscriber(self.conf)
|
||||||
|
worker.stop()
|
||||||
|
self.assertTrue(worker._abort.aborted)
|
||||||
|
|
||||||
def test_a_second_start_while_one_is_running_is_ignored(self):
|
def test_a_second_start_while_one_is_running_is_ignored(self):
|
||||||
worker = ft.FileTranscriber(self.conf)
|
worker = ft.FileTranscriber(self.conf)
|
||||||
worker._thread = mock.Mock(is_alive=lambda: True)
|
worker._thread = mock.Mock(is_alive=lambda: True)
|
||||||
|
|||||||
@@ -0,0 +1,664 @@
|
|||||||
|
"""Fetching a program and a model, and keeping a server alive on them.
|
||||||
|
|
||||||
|
No network and no whisper.cpp: the downloads are answered from memory, and the
|
||||||
|
servers are stand-in scripts that take the same arguments and open their port
|
||||||
|
when they are told to, which is the only thing the code waits on.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import contextlib
|
||||||
|
import hashlib
|
||||||
|
import io
|
||||||
|
import os
|
||||||
|
import signal
|
||||||
|
import sys
|
||||||
|
import tarfile
|
||||||
|
import textwrap
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
import ggml
|
||||||
|
import hub
|
||||||
|
from tests.support import (DikteTest, fake_urlopen, http_error, json_body,
|
||||||
|
linux_only, url_error)
|
||||||
|
|
||||||
|
|
||||||
|
def body(data, length=None):
|
||||||
|
"""What urlopen hands back for a download: a reader with a length header."""
|
||||||
|
class Body:
|
||||||
|
def __init__(self):
|
||||||
|
self._buf = io.BytesIO(data)
|
||||||
|
self.headers = {"Content-Length":
|
||||||
|
str(len(data) if length is None else length)}
|
||||||
|
|
||||||
|
def read(self, count=-1):
|
||||||
|
return self._buf.read(count)
|
||||||
|
|
||||||
|
def __enter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *_):
|
||||||
|
return False
|
||||||
|
return Body()
|
||||||
|
|
||||||
|
|
||||||
|
def item(name, data, url="https://example.invalid/f", sha=True):
|
||||||
|
return hub.Item(name, url, len(data),
|
||||||
|
hashlib.sha256(data).hexdigest() if sha else "")
|
||||||
|
|
||||||
|
|
||||||
|
@contextlib.contextmanager
|
||||||
|
def serving(release, archive):
|
||||||
|
"""Answer by what is being asked for rather than by what came before.
|
||||||
|
|
||||||
|
An install asks GitHub what the release is and then asks for one file out of
|
||||||
|
it, and the first of those two comes from the cache the second time around.
|
||||||
|
Answering in order would then hand the archive request the release listing.
|
||||||
|
"""
|
||||||
|
def opener(request, timeout=None):
|
||||||
|
url = request.full_url
|
||||||
|
if "api.github.com" in url:
|
||||||
|
return json_body(release)
|
||||||
|
return body(archive)
|
||||||
|
|
||||||
|
with mock.patch("urllib.request.urlopen", side_effect=opener) as calls:
|
||||||
|
yield calls
|
||||||
|
|
||||||
|
|
||||||
|
def tarball(entries):
|
||||||
|
"""A .tar.gz laid out the way the releases are: one directory of files."""
|
||||||
|
buf = io.BytesIO()
|
||||||
|
with tarfile.open(fileobj=buf, mode="w:gz") as tar:
|
||||||
|
for name, content in entries.items():
|
||||||
|
info = tarfile.TarInfo(name)
|
||||||
|
info.size = len(content)
|
||||||
|
info.mode = 0o755
|
||||||
|
tar.addfile(info, io.BytesIO(content))
|
||||||
|
return buf.getvalue()
|
||||||
|
|
||||||
|
|
||||||
|
class Local(DikteTest):
|
||||||
|
"""A test with its own bin, models and cache directories."""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.patch_attr(ggml, "DATA_DIR", self.path("data"))
|
||||||
|
self.patch_attr(ggml, "BIN_DIR", self.path("data", "bin"))
|
||||||
|
self.patch_attr(ggml, "MODELS_DIR", self.path("data", "models"))
|
||||||
|
self.patch_attr(hub, "CACHE_DIR", self.path("cache"))
|
||||||
|
|
||||||
|
|
||||||
|
# --- downloading ----------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class Download(Local):
|
||||||
|
def test_it_lands_and_the_part_file_is_gone(self):
|
||||||
|
data = b"a model, more or less" * 100
|
||||||
|
target = self.path("data", "models", "m.bin")
|
||||||
|
with fake_urlopen(body(data)):
|
||||||
|
self.assertTrue(ggml.download(item("m.bin", data), target))
|
||||||
|
self.assertEqual(target.read_bytes(), data)
|
||||||
|
self.assertFalse(target.with_name("m.bin.part").exists())
|
||||||
|
|
||||||
|
def test_a_wrong_checksum_installs_nothing(self):
|
||||||
|
data = b"the bytes that arrived"
|
||||||
|
wrong = hub.Item("m.bin", "https://example.invalid/f", len(data), "f" * 64)
|
||||||
|
target = self.path("data", "models", "m.bin")
|
||||||
|
with fake_urlopen(body(data)):
|
||||||
|
with self.assertRaises(ggml.LocalError) as caught:
|
||||||
|
ggml.download(wrong, target)
|
||||||
|
self.assertIn("checksum", str(caught.exception))
|
||||||
|
self.assertFalse(target.exists())
|
||||||
|
self.assertFalse(target.with_name("m.bin.part").exists())
|
||||||
|
|
||||||
|
def test_a_body_shorter_than_its_header_installs_nothing(self):
|
||||||
|
data = b"half of it"
|
||||||
|
target = self.path("data", "models", "m.bin")
|
||||||
|
with fake_urlopen(body(data, length=len(data) * 2)):
|
||||||
|
with self.assertRaises(ggml.LocalError):
|
||||||
|
ggml.download(item("m.bin", data), target)
|
||||||
|
self.assertFalse(target.exists())
|
||||||
|
|
||||||
|
def test_a_file_with_no_published_checksum_is_refused(self):
|
||||||
|
# Everything fetched here is run or parsed by something written in C++,
|
||||||
|
# and GitHub did not always publish a digest.
|
||||||
|
data = b"a program, say"
|
||||||
|
target = self.path("data", "models", "m.bin")
|
||||||
|
with fake_urlopen(body(data)):
|
||||||
|
with self.assertRaises(ggml.LocalError) as caught:
|
||||||
|
ggml.download(item("m.bin", data, sha=False), target)
|
||||||
|
self.assertIn("checksum", str(caught.exception))
|
||||||
|
self.assertFalse(target.exists())
|
||||||
|
|
||||||
|
def test_nothing_is_asked_for_before_it_is_refused(self):
|
||||||
|
# The refusal is not worth a gigabyte of somebody's bandwidth first.
|
||||||
|
with fake_urlopen(body(b"never read")) as calls:
|
||||||
|
with self.assertRaises(ggml.LocalError):
|
||||||
|
ggml.download(item("m.bin", b"x", sha=False), self.path("m.bin"))
|
||||||
|
self.assertEqual(calls, [])
|
||||||
|
|
||||||
|
def test_stopping_leaves_nothing_behind(self):
|
||||||
|
data = b"x" * (ggml.DOWNLOAD_CHUNK * 3)
|
||||||
|
target = self.path("data", "models", "m.bin")
|
||||||
|
with fake_urlopen(body(data)):
|
||||||
|
landed = ggml.download(item("m.bin", data), target,
|
||||||
|
should_stop=lambda: True)
|
||||||
|
self.assertFalse(landed)
|
||||||
|
self.assertFalse(target.exists())
|
||||||
|
self.assertFalse(target.with_name("m.bin.part").exists())
|
||||||
|
|
||||||
|
def test_progress_is_reported_against_the_total(self):
|
||||||
|
data = b"y" * (ggml.DOWNLOAD_CHUNK + 5)
|
||||||
|
seen = []
|
||||||
|
with fake_urlopen(body(data)):
|
||||||
|
ggml.download(item("m.bin", data), self.path("data", "m.bin"),
|
||||||
|
on_progress=lambda done, total: seen.append((done, total)))
|
||||||
|
self.assertEqual(seen[-1], (len(data), len(data)))
|
||||||
|
self.assertGreater(len(seen), 1)
|
||||||
|
|
||||||
|
def test_a_refused_connection_says_which_file(self):
|
||||||
|
with fake_urlopen(url_error("no route to host")):
|
||||||
|
with self.assertRaises(ggml.LocalError) as caught:
|
||||||
|
ggml.download(item("m.bin", b"x"), self.path("data", "m.bin"))
|
||||||
|
self.assertIn("m.bin", str(caught.exception))
|
||||||
|
|
||||||
|
def test_an_http_error_is_not_written_to_disk(self):
|
||||||
|
target = self.path("data", "m.bin")
|
||||||
|
with fake_urlopen(http_error(404)):
|
||||||
|
with self.assertRaises(ggml.LocalError):
|
||||||
|
ggml.download(item("m.bin", b"x"), target)
|
||||||
|
self.assertFalse(target.exists())
|
||||||
|
|
||||||
|
|
||||||
|
# --- installing a program -------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class InstallProgram(Local):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
# Built once, because the release listing has to publish its checksum
|
||||||
|
# and a tarball is not the same bytes twice.
|
||||||
|
self.archive = tarball({
|
||||||
|
"whisper-bin-ubuntu-x64/whisper-server": b"#!/bin/sh\nexit 0\n",
|
||||||
|
"whisper-bin-ubuntu-x64/libwhisper.so": b"not really a library",
|
||||||
|
})
|
||||||
|
|
||||||
|
def release(self, *names, archive=None):
|
||||||
|
digest = hashlib.sha256(self.archive if archive is None else archive)
|
||||||
|
return {"tag_name": "v1.9.1", "assets": [
|
||||||
|
{"name": name, "browser_download_url": f"https://example.invalid/{name}",
|
||||||
|
"size": 10, "digest": "sha256:" + digest.hexdigest()}
|
||||||
|
for name in names]}
|
||||||
|
|
||||||
|
def install(self, *names, archive=None):
|
||||||
|
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||||
|
blob = self.archive if archive is None else archive
|
||||||
|
with serving(self.release(*names, archive=blob), blob) as calls:
|
||||||
|
path = ggml.install_program(ggml.WHISPER)
|
||||||
|
return path, [call.args[0].full_url for call in calls.call_args_list]
|
||||||
|
|
||||||
|
def test_the_binary_and_its_libraries_land_together(self):
|
||||||
|
path, _ = self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||||
|
self.assertTrue(os.path.isfile(path))
|
||||||
|
self.assertTrue(os.access(path, os.X_OK))
|
||||||
|
self.assertTrue(os.path.isfile(os.path.join(os.path.dirname(path),
|
||||||
|
"libwhisper.so")))
|
||||||
|
|
||||||
|
def test_the_build_for_this_machine_is_the_one_fetched(self):
|
||||||
|
_, urls = self.install("whisper-bin-x64.zip", "whisper-bin-ubuntu-arm64.tar.gz",
|
||||||
|
"whisper-bin-ubuntu-x64.tar.gz")
|
||||||
|
self.assertTrue(urls[1].endswith("whisper-bin-ubuntu-x64.tar.gz"))
|
||||||
|
|
||||||
|
def test_a_release_with_nothing_for_this_machine_says_so(self):
|
||||||
|
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||||
|
with fake_urlopen(self.release("whisper-bin-Win32.zip")):
|
||||||
|
with self.assertRaises(ggml.LocalError) as caught:
|
||||||
|
ggml.install_program(ggml.WHISPER)
|
||||||
|
self.assertIn("this machine", str(caught.exception))
|
||||||
|
|
||||||
|
def test_what_was_installed_is_remembered(self):
|
||||||
|
path, _ = self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||||
|
self.assertEqual(ggml.installed_program(ggml.WHISPER), path)
|
||||||
|
self.assertEqual(ggml.installed_version(ggml.WHISPER), "v1.9.1")
|
||||||
|
|
||||||
|
def test_a_record_pointing_at_a_deleted_binary_counts_for_nothing(self):
|
||||||
|
path, _ = self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||||
|
os.unlink(path)
|
||||||
|
self.assertEqual(ggml.installed_program(ggml.WHISPER), "")
|
||||||
|
|
||||||
|
def test_the_archive_is_not_kept(self):
|
||||||
|
self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||||
|
left = list((self.path("data", "bin", "whisper")).glob("*.tar.gz"))
|
||||||
|
self.assertEqual(left, [])
|
||||||
|
|
||||||
|
def test_the_previous_version_is_swept_up(self):
|
||||||
|
self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||||
|
old = self.path("data", "bin", "whisper", "v1.9.0")
|
||||||
|
old.mkdir(parents=True)
|
||||||
|
(old / "whisper-server").write_bytes(b"older")
|
||||||
|
self.install("whisper-bin-ubuntu-x64.tar.gz")
|
||||||
|
self.assertFalse(old.exists())
|
||||||
|
|
||||||
|
def test_an_archive_without_the_binary_is_refused(self):
|
||||||
|
empty = tarball({"whisper-bin-ubuntu-x64/README": b"nothing here"})
|
||||||
|
with self.assertRaises(ggml.LocalError) as caught:
|
||||||
|
self.install("whisper-bin-ubuntu-x64.tar.gz", archive=empty)
|
||||||
|
self.assertIn("whisper-server", str(caught.exception))
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_release_without_a_published_checksum_is_refused(self):
|
||||||
|
# GitHub did not always publish one, and whisper.cpp v1.8.0 and older
|
||||||
|
# still have none.
|
||||||
|
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||||
|
listing = {"tag_name": "v1.8.0", "assets": [
|
||||||
|
{"name": "whisper-bin-ubuntu-x64.tar.gz",
|
||||||
|
"browser_download_url": "https://example.invalid/w.tar.gz",
|
||||||
|
"size": 10}]}
|
||||||
|
with serving(listing, self.archive):
|
||||||
|
with self.assertRaises(ggml.LocalError) as caught:
|
||||||
|
ggml.install_program(ggml.WHISPER)
|
||||||
|
self.assertIn("checksum", str(caught.exception))
|
||||||
|
self.assertEqual(ggml.installed_program(ggml.WHISPER), "")
|
||||||
|
|
||||||
|
def test_an_archive_that_is_not_what_was_promised_installs_nothing(self):
|
||||||
|
listing = self.release("whisper-bin-ubuntu-x64.tar.gz")
|
||||||
|
other = tarball({"whisper-bin-ubuntu-x64/whisper-server": b"#!/bin/sh\nrm -rf\n"})
|
||||||
|
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||||
|
with serving(listing, other):
|
||||||
|
with self.assertRaises(ggml.LocalError) as caught:
|
||||||
|
ggml.install_program(ggml.WHISPER)
|
||||||
|
self.assertIn("checksum", str(caught.exception))
|
||||||
|
self.assertEqual(ggml.installed_program(ggml.WHISPER), "")
|
||||||
|
|
||||||
|
def test_an_archive_cannot_write_outside_the_directory_it_is_opened_in(self):
|
||||||
|
# An archive is not a trusted thing to unpack: a member named ../../ is
|
||||||
|
# how one writes over a file it was never given.
|
||||||
|
escape = tarball({"../../../escaped": b"should not land"})
|
||||||
|
path = self.path("data", "bin", "whisper", "v1.9.1")
|
||||||
|
with self.assertRaises(ggml.LocalError):
|
||||||
|
self.install("whisper-bin-ubuntu-x64.tar.gz", archive=escape)
|
||||||
|
self.assertFalse(self.path("escaped").exists())
|
||||||
|
self.assertFalse((path.parent.parent / "escaped").exists())
|
||||||
|
|
||||||
|
def test_a_symlink_out_of_the_directory_does_not_survive_either(self):
|
||||||
|
buf = io.BytesIO()
|
||||||
|
with tarfile.open(fileobj=buf, mode="w:gz") as tar:
|
||||||
|
info = tarfile.TarInfo("whisper-bin-ubuntu-x64/whisper-server")
|
||||||
|
info.type, info.linkname = tarfile.SYMTYPE, "/etc/passwd"
|
||||||
|
tar.addfile(info)
|
||||||
|
with self.assertRaises(ggml.LocalError):
|
||||||
|
self.install("whisper-bin-ubuntu-x64.tar.gz", archive=buf.getvalue())
|
||||||
|
|
||||||
|
def test_everything_is_asked_for_over_tls(self):
|
||||||
|
for url in (hub.GITHUB_API, hub.HF_API, hub.HF_FILES):
|
||||||
|
with self.subTest(url=url):
|
||||||
|
self.assertTrue(url.startswith("https://"))
|
||||||
|
|
||||||
|
def test_llama_takes_the_vulkan_build_when_there_is_a_loader(self):
|
||||||
|
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||||
|
self.patch_attr(ggml, "_has_vulkan", lambda: True)
|
||||||
|
self.assertEqual(ggml._wanted_assets(ggml.LLAMA)[0],
|
||||||
|
"bin-ubuntu-vulkan-x64.tar.gz")
|
||||||
|
|
||||||
|
def test_llama_falls_back_to_the_plain_build_without_one(self):
|
||||||
|
self.patch_attr(ggml, "_arch", lambda: "x64")
|
||||||
|
self.patch_attr(ggml, "_has_vulkan", lambda: False)
|
||||||
|
self.assertEqual(ggml._wanted_assets(ggml.LLAMA), ("bin-ubuntu-x64.tar.gz",))
|
||||||
|
|
||||||
|
|
||||||
|
class WhichCopyRuns(Local):
|
||||||
|
def test_a_system_build_wins_over_a_downloaded_one(self):
|
||||||
|
self.patch_attr(ggml, "installed_program", lambda program: "/data/whisper-server")
|
||||||
|
with mock.patch("shutil.which", return_value="/usr/bin/whisper-server"):
|
||||||
|
self.assertEqual(ggml.program_path(ggml.WHISPER), "/usr/bin/whisper-server")
|
||||||
|
|
||||||
|
def test_the_downloaded_one_is_used_when_there_is_no_system_build(self):
|
||||||
|
self.patch_attr(ggml, "installed_program", lambda program: "/data/whisper-server")
|
||||||
|
with mock.patch("shutil.which", return_value=None):
|
||||||
|
self.assertEqual(ggml.program_path(ggml.WHISPER), "/data/whisper-server")
|
||||||
|
|
||||||
|
def test_a_setting_pointing_at_nothing_is_no_program(self):
|
||||||
|
self.assertEqual(ggml.program_path(ggml.WHISPER, "/nowhere/whisper-server"), "")
|
||||||
|
|
||||||
|
def test_a_setting_pointing_at_a_program_wins(self):
|
||||||
|
mine = self.path("mine")
|
||||||
|
mine.write_text("#!/bin/sh\n")
|
||||||
|
mine.chmod(0o755)
|
||||||
|
with mock.patch("shutil.which", return_value="/usr/bin/whisper-server"):
|
||||||
|
self.assertEqual(ggml.program_path(ggml.WHISPER, str(mine)), str(mine))
|
||||||
|
|
||||||
|
|
||||||
|
# --- the lists ------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
WHISPER_TREE = [
|
||||||
|
{"type": "file", "path": "ggml-base.bin", "size": 147951465,
|
||||||
|
"lfs": {"oid": "a" * 64}},
|
||||||
|
{"type": "file", "path": "ggml-large-v3-turbo-q5_0.bin", "size": 574041195,
|
||||||
|
"lfs": {"oid": "b" * 64}},
|
||||||
|
{"type": "file", "path": "ggml-base-encoder.mlmodelc.zip", "size": 37922638,
|
||||||
|
"lfs": {"oid": "c" * 64}},
|
||||||
|
{"type": "file", "path": "README.md", "size": 3196},
|
||||||
|
]
|
||||||
|
|
||||||
|
GGUF_TREE = [
|
||||||
|
{"type": "file", "path": "gemma-3-4b-it-Q4_K_M.gguf", "size": 2489000000,
|
||||||
|
"lfs": {"oid": "a" * 64}},
|
||||||
|
{"type": "file", "path": "gemma-3-4b-it-Q8_0.gguf", "size": 4130000000,
|
||||||
|
"lfs": {"oid": "b" * 64}},
|
||||||
|
{"type": "file", "path": "mmproj-model-f16.gguf", "size": 851000000,
|
||||||
|
"lfs": {"oid": "c" * 64}},
|
||||||
|
{"type": "file", "path": "mtp-gemma-4-E4B-it-Q4_0.gguf", "size": 59000000,
|
||||||
|
"lfs": {"oid": "d" * 64}},
|
||||||
|
{"type": "file", "path": "huge-00001-of-00009.gguf", "size": 40000000000,
|
||||||
|
"lfs": {"oid": "e" * 64}},
|
||||||
|
{"type": "file", "path": "README.md", "size": 100},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class Catalogue(Local):
|
||||||
|
def test_only_models_are_offered_and_the_small_ones_first(self):
|
||||||
|
with fake_urlopen(WHISPER_TREE):
|
||||||
|
models = ggml.whisper_models()
|
||||||
|
self.assertEqual([m.name for m in models],
|
||||||
|
["ggml-base.bin", "ggml-large-v3-turbo-q5_0.bin"])
|
||||||
|
|
||||||
|
def test_the_core_ml_encoders_are_not_models(self):
|
||||||
|
with fake_urlopen(WHISPER_TREE):
|
||||||
|
names = [m.name for m in ggml.whisper_models()]
|
||||||
|
self.assertNotIn("ggml-base-encoder.mlmodelc.zip", names)
|
||||||
|
|
||||||
|
def test_the_projector_and_the_draft_head_are_not_models(self):
|
||||||
|
with fake_urlopen(GGUF_TREE):
|
||||||
|
names = [q.name for q in ggml.llm_quants("ggml-org/gemma-3-4b-it-GGUF")]
|
||||||
|
self.assertEqual(names,
|
||||||
|
["gemma-3-4b-it-Q4_K_M.gguf", "gemma-3-4b-it-Q8_0.gguf"])
|
||||||
|
|
||||||
|
def test_a_model_split_across_files_is_left_out(self):
|
||||||
|
with fake_urlopen(GGUF_TREE):
|
||||||
|
names = [q.name for q in ggml.llm_quants("ggml-org/gemma-3-4b-it-GGUF")]
|
||||||
|
self.assertNotIn("huge-00001-of-00009.gguf", names)
|
||||||
|
|
||||||
|
def test_the_suggestions_come_first_and_the_rest_follow(self):
|
||||||
|
listing = [{"id": "ggml-org/something-new-GGUF"},
|
||||||
|
{"id": ggml.SUGGESTED_LLM[0]}]
|
||||||
|
with fake_urlopen(listing):
|
||||||
|
found = ggml.llm_repos()
|
||||||
|
self.assertEqual(found[0], ggml.SUGGESTED_LLM[0])
|
||||||
|
self.assertIn("ggml-org/something-new-GGUF", found)
|
||||||
|
|
||||||
|
def test_an_unreachable_list_still_offers_the_suggestions(self):
|
||||||
|
with fake_urlopen(url_error()):
|
||||||
|
self.assertEqual(ggml.llm_repos(), list(ggml.SUGGESTED_LLM))
|
||||||
|
|
||||||
|
def test_an_unreachable_whisper_list_is_an_error_worth_showing(self):
|
||||||
|
with fake_urlopen(url_error()):
|
||||||
|
with self.assertRaises(ggml.LocalError):
|
||||||
|
ggml.whisper_models()
|
||||||
|
|
||||||
|
def test_what_is_on_disk_is_read_from_disk(self):
|
||||||
|
self.assertEqual(ggml.installed_whisper_models(), [])
|
||||||
|
path = ggml.whisper_model_path("ggml-base.bin")
|
||||||
|
path.parent.mkdir(parents=True)
|
||||||
|
path.write_bytes(b"model")
|
||||||
|
self.assertEqual(ggml.installed_whisper_models(), ["ggml-base.bin"])
|
||||||
|
self.assertTrue(ggml.have_model(path))
|
||||||
|
|
||||||
|
def test_an_empty_file_is_not_a_model(self):
|
||||||
|
path = ggml.llm_model_path("ggml-org/x-GGUF/model.gguf")
|
||||||
|
path.parent.mkdir(parents=True)
|
||||||
|
path.write_bytes(b"")
|
||||||
|
self.assertFalse(ggml.have_model(path))
|
||||||
|
|
||||||
|
def test_a_model_is_named_by_its_file_not_its_repository(self):
|
||||||
|
self.assertEqual(ggml.llm_model_path("ggml-org/x-GGUF/model.gguf").name,
|
||||||
|
"model.gguf")
|
||||||
|
|
||||||
|
|
||||||
|
# --- keeping a server alive -----------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
STAND_IN = textwrap.dedent("""
|
||||||
|
import http.server, sys, threading, time
|
||||||
|
|
||||||
|
args = sys.argv[1:]
|
||||||
|
|
||||||
|
def opt(name, default=""):
|
||||||
|
return args[args.index(name) + 1] if name in args else default
|
||||||
|
|
||||||
|
if "--die" in args:
|
||||||
|
print("could not load model: no such file")
|
||||||
|
sys.exit(2)
|
||||||
|
|
||||||
|
time.sleep(float(opt("--wait", "0")))
|
||||||
|
|
||||||
|
started = time.monotonic()
|
||||||
|
healthy_after = float(opt("--healthy-after", "0"))
|
||||||
|
|
||||||
|
class Handler(http.server.BaseHTTPRequestHandler):
|
||||||
|
def do_GET(self):
|
||||||
|
ok = time.monotonic() - started >= healthy_after
|
||||||
|
self.send_response(200 if ok else 503)
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(b"{}")
|
||||||
|
|
||||||
|
def log_message(self, *a):
|
||||||
|
pass
|
||||||
|
|
||||||
|
server = http.server.HTTPServer((opt("--host"), int(opt("--port"))), Handler)
|
||||||
|
print("listening on " + opt("--port"), flush=True)
|
||||||
|
server.serve_forever()
|
||||||
|
""")
|
||||||
|
|
||||||
|
|
||||||
|
class Servers(Local):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.path("data").mkdir(parents=True, exist_ok=True)
|
||||||
|
# Named for the program and kept inside the data directory, because that
|
||||||
|
# is what the sweep looks for on a command line.
|
||||||
|
self.script = self.path("data", "whisper-server.py")
|
||||||
|
self.script.write_text(STAND_IN)
|
||||||
|
self.addCleanup(ggml.stop_all)
|
||||||
|
self.servers = []
|
||||||
|
|
||||||
|
def server(self, program=ggml.WHISPER, **settings):
|
||||||
|
defaults = {"extra": []}
|
||||||
|
defaults.update(settings)
|
||||||
|
made = ggml.Server(
|
||||||
|
program,
|
||||||
|
lambda values: [sys.executable, str(self.script)] + list(values["extra"]),
|
||||||
|
defaults,
|
||||||
|
)
|
||||||
|
self.servers.append(made)
|
||||||
|
self.addCleanup(made.stop)
|
||||||
|
return made
|
||||||
|
|
||||||
|
def test_a_started_server_hands_back_its_address(self):
|
||||||
|
server = self.server()
|
||||||
|
url = server.serve()
|
||||||
|
self.assertRegex(url, r"^http://127\.0\.0\.1:\d+/v1$")
|
||||||
|
self.assertTrue(server.running)
|
||||||
|
|
||||||
|
def test_the_second_call_does_not_start_a_second_one(self):
|
||||||
|
server = self.server()
|
||||||
|
first = server.serve()
|
||||||
|
self.assertEqual(server.serve(), first)
|
||||||
|
|
||||||
|
def test_a_settings_change_stops_what_was_running(self):
|
||||||
|
server = self.server()
|
||||||
|
server.serve()
|
||||||
|
server.configure(extra=["--wait", "0"])
|
||||||
|
self.assertFalse(server.running)
|
||||||
|
|
||||||
|
def test_the_new_settings_are_what_the_next_start_uses(self):
|
||||||
|
server = self.server()
|
||||||
|
server.serve()
|
||||||
|
server.configure(extra=["--healthy-after", "0"])
|
||||||
|
second = server.serve()
|
||||||
|
self.assertTrue(server.running)
|
||||||
|
self.assertTrue(second)
|
||||||
|
|
||||||
|
def test_a_program_that_dies_reports_what_it_printed(self):
|
||||||
|
server = self.server(extra=["--die"])
|
||||||
|
with self.assertRaises(ggml.LocalError) as caught:
|
||||||
|
server.serve()
|
||||||
|
self.assertIn("no such file", str(caught.exception))
|
||||||
|
self.assertFalse(server.running)
|
||||||
|
|
||||||
|
def test_a_model_that_is_still_loading_is_not_ready_yet(self):
|
||||||
|
# llama binds its port first and answers /health with 503 until the
|
||||||
|
# model is in memory, so the open port on its own is not the signal.
|
||||||
|
server = self.server(program=ggml.LLAMA, extra=["--healthy-after", "0.4"])
|
||||||
|
started = time.monotonic()
|
||||||
|
server.serve()
|
||||||
|
self.assertGreaterEqual(time.monotonic() - started, 0.4)
|
||||||
|
|
||||||
|
def test_a_start_that_never_becomes_ready_gives_up(self):
|
||||||
|
self.patch_attr(ggml, "STARTUP_TIMEOUT", 0.5)
|
||||||
|
server = self.server(program=ggml.LLAMA, extra=["--healthy-after", "30"])
|
||||||
|
with self.assertRaises(ggml.LocalError):
|
||||||
|
server.serve()
|
||||||
|
|
||||||
|
def test_stopping_leaves_nothing_running(self):
|
||||||
|
server = self.server()
|
||||||
|
server.serve()
|
||||||
|
server.stop()
|
||||||
|
self.assertFalse(server.running)
|
||||||
|
self.assertEqual(server.base_url(), "")
|
||||||
|
|
||||||
|
def test_the_last_thing_it_printed_is_available(self):
|
||||||
|
server = self.server()
|
||||||
|
server.serve()
|
||||||
|
self.assertIn("listening", server.error())
|
||||||
|
|
||||||
|
def test_asking_what_is_running_does_not_wait_for_a_start(self):
|
||||||
|
"""A model being loaded must not freeze the settings window.
|
||||||
|
|
||||||
|
The interface asks a running server what it is doing while a start is in
|
||||||
|
flight, and a lock held across the whole start would stop it dead.
|
||||||
|
"""
|
||||||
|
server = self.server(extra=["--wait", "0.6"])
|
||||||
|
answers = []
|
||||||
|
|
||||||
|
def start():
|
||||||
|
server.serve()
|
||||||
|
|
||||||
|
thread = __import__("threading").Thread(target=start)
|
||||||
|
thread.start()
|
||||||
|
try:
|
||||||
|
time.sleep(0.15)
|
||||||
|
began = time.monotonic()
|
||||||
|
answers.append(server.settings())
|
||||||
|
answers.append(server.running)
|
||||||
|
self.assertLess(time.monotonic() - began, 0.2)
|
||||||
|
finally:
|
||||||
|
thread.join(timeout=10)
|
||||||
|
|
||||||
|
@linux_only
|
||||||
|
def test_a_server_a_killed_dikte_left_behind_is_swept_up(self):
|
||||||
|
server = self.server()
|
||||||
|
server.serve()
|
||||||
|
# What a SIGKILL of Dikte leaves: the child still running, the pid file
|
||||||
|
# still on disk, and nothing left that knows about either.
|
||||||
|
proc, server._proc = server._proc, None
|
||||||
|
self.assertTrue(server.sweep())
|
||||||
|
self.assertEqual(proc.wait(timeout=5), -signal.SIGTERM)
|
||||||
|
|
||||||
|
@linux_only
|
||||||
|
def test_a_pid_that_belongs_to_something_else_is_left_alone(self):
|
||||||
|
server = self.server()
|
||||||
|
server._remember(os.getpid()) # this test runner, not a server
|
||||||
|
self.assertFalse(server.sweep())
|
||||||
|
|
||||||
|
def test_no_pid_file_is_nothing_to_sweep(self):
|
||||||
|
self.assertFalse(self.server().sweep())
|
||||||
|
|
||||||
|
def test_a_start_that_goes_wrong_takes_its_process_with_it(self):
|
||||||
|
started = []
|
||||||
|
|
||||||
|
def explode(inner, proc, port):
|
||||||
|
started.append(proc)
|
||||||
|
raise RuntimeError("something in the wait went wrong")
|
||||||
|
|
||||||
|
self.patch_attr(ggml.Server, "_wait_ready", explode)
|
||||||
|
server = self.server()
|
||||||
|
with self.assertRaises(RuntimeError):
|
||||||
|
server.serve()
|
||||||
|
# Nothing else holds a reference to it, so leaving it running would leak
|
||||||
|
# a loaded model with nobody left to ask it anything.
|
||||||
|
self.assertIsNotNone(started[0].poll())
|
||||||
|
self.assertFalse(server.sweep()) # and the pid file went with it
|
||||||
|
|
||||||
|
|
||||||
|
class Arguments(Local):
|
||||||
|
"""What the two command lines say, since neither program is here to say it."""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.binary = self.path("whisper-server")
|
||||||
|
self.binary.write_text("#!/bin/sh\n")
|
||||||
|
self.binary.chmod(0o755)
|
||||||
|
|
||||||
|
def whisper_model(self, name="ggml-base.bin"):
|
||||||
|
path = ggml.whisper_model_path(name)
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_bytes(b"model")
|
||||||
|
return name
|
||||||
|
|
||||||
|
def test_the_inference_path_is_the_one_api_py_builds(self):
|
||||||
|
args = ggml._whisper_args({"binary": str(self.binary), "gpu": True,
|
||||||
|
"threads": 0, "model": self.whisper_model()})
|
||||||
|
self.assertIn("--inference-path", args)
|
||||||
|
self.assertEqual(args[args.index("--inference-path") + 1],
|
||||||
|
"/v1/audio/transcriptions")
|
||||||
|
|
||||||
|
def test_detection_rather_than_english_when_nothing_is_asked_for(self):
|
||||||
|
args = ggml._whisper_args({"binary": str(self.binary), "gpu": True,
|
||||||
|
"threads": 0, "model": self.whisper_model()})
|
||||||
|
self.assertEqual(args[args.index("-l") + 1], "auto")
|
||||||
|
|
||||||
|
def test_the_graphics_card_is_turned_off_rather_than_asked_for(self):
|
||||||
|
settings = {"binary": str(self.binary), "gpu": False, "threads": 2,
|
||||||
|
"model": self.whisper_model()}
|
||||||
|
args = ggml._whisper_args(settings)
|
||||||
|
self.assertIn("-ng", args)
|
||||||
|
self.assertEqual(args[args.index("-t") + 1], "2")
|
||||||
|
|
||||||
|
def test_a_missing_model_is_a_message_about_settings(self):
|
||||||
|
with self.assertRaises(ggml.LocalError) as caught:
|
||||||
|
ggml._whisper_args({"binary": str(self.binary), "gpu": True,
|
||||||
|
"threads": 0, "model": "ggml-nothing.bin"})
|
||||||
|
self.assertIn("Settings", str(caught.exception))
|
||||||
|
|
||||||
|
def test_a_missing_program_says_so_before_a_missing_model(self):
|
||||||
|
with mock.patch("shutil.which", return_value=None):
|
||||||
|
with self.assertRaises(ggml.LocalError) as caught:
|
||||||
|
ggml._whisper_args({"binary": "", "gpu": True, "threads": 0,
|
||||||
|
"model": self.whisper_model()})
|
||||||
|
self.assertIn("whisper.cpp", str(caught.exception))
|
||||||
|
|
||||||
|
def test_the_layers_go_to_the_card_when_there_is_one(self):
|
||||||
|
model = ggml.llm_model_path("m.gguf")
|
||||||
|
model.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
model.write_bytes(b"gguf")
|
||||||
|
args = ggml._llm_args({"binary": str(self.binary), "gpu": True,
|
||||||
|
"threads": 0, "model": "m.gguf", "context": 4096})
|
||||||
|
self.assertEqual(args[args.index("-ngl") + 1], "99")
|
||||||
|
self.assertEqual(args[args.index("-c") + 1], "4096")
|
||||||
|
|
||||||
|
def test_no_card_means_no_layers_offloaded(self):
|
||||||
|
model = ggml.llm_model_path("m.gguf")
|
||||||
|
model.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
model.write_bytes(b"gguf")
|
||||||
|
args = ggml._llm_args({"binary": str(self.binary), "gpu": False,
|
||||||
|
"threads": 0, "model": "m.gguf", "context": 4096})
|
||||||
|
self.assertEqual(args[args.index("-ngl") + 1], "0")
|
||||||
|
|
||||||
|
|
||||||
|
class Sizes(DikteTest):
|
||||||
|
def test_bytes_are_written_the_way_a_download_is_talked_about(self):
|
||||||
|
self.assertEqual(ggml.human_size(512), "512 B")
|
||||||
|
self.assertEqual(ggml.human_size(574041195), "547.4 MB")
|
||||||
|
self.assertEqual(ggml.human_size(3_095_033_483), "2.9 GB")
|
||||||
|
|
||||||
@@ -6,6 +6,7 @@ import subprocess
|
|||||||
import unittest
|
import unittest
|
||||||
from unittest import mock
|
from unittest import mock
|
||||||
|
|
||||||
|
import config as cfg
|
||||||
import hotkey
|
import hotkey
|
||||||
from tests.support import DikteTest, FakeCompleted, linux_only
|
from tests.support import DikteTest, FakeCompleted, linux_only
|
||||||
|
|
||||||
@@ -56,6 +57,28 @@ class ParseShortcut(unittest.TestCase):
|
|||||||
self.assertEqual(hotkey.parse_shortcut(None), (None, None))
|
self.assertEqual(hotkey.parse_shortcut(None), (None, None))
|
||||||
|
|
||||||
|
|
||||||
|
class Table(unittest.TestCase):
|
||||||
|
"""The one list of global shortcuts. The command line, the settings window
|
||||||
|
and install.sh read it instead of keeping a copy each, so what it has to
|
||||||
|
hold together is checked here rather than in three places."""
|
||||||
|
|
||||||
|
def test_every_shortcut_remembers_itself_in_a_real_setting(self):
|
||||||
|
for name, spec in hotkey.SHORTCUTS.items():
|
||||||
|
with self.subTest(name=name):
|
||||||
|
self.assertIn(spec.setting, cfg.DEFAULTS)
|
||||||
|
|
||||||
|
def test_no_two_share_a_desktop_entry(self):
|
||||||
|
ids = [spec.desktop_id for spec in hotkey.SHORTCUTS.values()]
|
||||||
|
self.assertEqual(len(ids), len(set(ids)))
|
||||||
|
|
||||||
|
def test_only_the_toggle_falls_back_to_a_key_of_its_own(self):
|
||||||
|
"""The rest are off until you pick one, and emptying the box is how you
|
||||||
|
turn them off again."""
|
||||||
|
self.assertEqual(hotkey.SHORTCUTS["toggle"].fallback, "Ctrl+Space")
|
||||||
|
self.assertEqual([name for name, spec in hotkey.SHORTCUTS.items()
|
||||||
|
if spec.fallback], ["toggle"])
|
||||||
|
|
||||||
|
|
||||||
class ModsMatch(unittest.TestCase):
|
class ModsMatch(unittest.TestCase):
|
||||||
"""The combination has to be exact, or Ctrl+Space fires on Ctrl+Shift+Space."""
|
"""The combination has to be exact, or Ctrl+Space fires on Ctrl+Shift+Space."""
|
||||||
|
|
||||||
@@ -122,6 +145,23 @@ class Bindings(DikteTest):
|
|||||||
thread.assert_called_once()
|
thread.assert_called_once()
|
||||||
self.assertEqual(len(listener._bindings[57]), 2)
|
self.assertEqual(len(listener._bindings[57]), 2)
|
||||||
|
|
||||||
|
def test_starting_and_discarding_do_not_fire_on_each_other(self):
|
||||||
|
"""The two defaults are one modifier apart on the same key code, so the
|
||||||
|
modifier set is the only thing keeping them apart."""
|
||||||
|
listener = hotkey.EvdevHotkey()
|
||||||
|
self.addCleanup(listener.stop)
|
||||||
|
with mock.patch.object(listener, "_open_devices", return_value=[99]), \
|
||||||
|
mock.patch.object(hotkey.threading, "Thread"):
|
||||||
|
listener.start({"toggle": "Ctrl+Space", "cancel": "Ctrl+Alt+Space"})
|
||||||
|
|
||||||
|
def fired(held):
|
||||||
|
return [name for mods, name in listener._bindings[57]
|
||||||
|
if hotkey.EvdevHotkey._mods_match(held, mods)]
|
||||||
|
|
||||||
|
self.assertEqual(fired({29}), ["toggle"]) # ctrl
|
||||||
|
self.assertEqual(fired({29, 56}), ["cancel"]) # ctrl + alt
|
||||||
|
self.assertEqual(fired({29, 42}), []) # ctrl + shift
|
||||||
|
|
||||||
|
|
||||||
class Chooser(DikteTest):
|
class Chooser(DikteTest):
|
||||||
"""Which desktop is asked to register the shortcut."""
|
"""Which desktop is asked to register the shortcut."""
|
||||||
|
|||||||
@@ -0,0 +1,184 @@
|
|||||||
|
"""What GitHub and Hugging Face are asked, and what is believed of the answer."""
|
||||||
|
|
||||||
|
import json
|
||||||
|
|
||||||
|
import hub
|
||||||
|
from tests.support import DikteTest, fake_urlopen, http_error, url_error
|
||||||
|
|
||||||
|
RELEASE = {
|
||||||
|
"tag_name": "v1.9.1",
|
||||||
|
"assets": [
|
||||||
|
{"name": "whisper-bin-ubuntu-x64.tar.gz",
|
||||||
|
"browser_download_url": "https://example.invalid/ubuntu-x64.tar.gz",
|
||||||
|
"size": 9379235, "digest": "sha256:" + "a" * 64},
|
||||||
|
{"name": "whisper-bin-x64.zip",
|
||||||
|
"browser_download_url": "https://example.invalid/win.zip",
|
||||||
|
"size": 100, "digest": "sha256:" + "b" * 64},
|
||||||
|
{"name": "no-url-here.zip", "size": 1},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
TREE = [
|
||||||
|
{"type": "file", "path": ".gitattributes", "size": 1477},
|
||||||
|
{"type": "file", "path": "ggml-base.bin", "size": 147951465,
|
||||||
|
"lfs": {"oid": "c" * 64, "size": 147951465}},
|
||||||
|
{"type": "directory", "path": "extra"},
|
||||||
|
{"type": "file", "path": "extra/ggml-tiny.bin", "size": 77691713,
|
||||||
|
"lfs": {"oid": "d" * 64, "size": 77691713}},
|
||||||
|
]
|
||||||
|
|
||||||
|
MODELS = [
|
||||||
|
{"id": "ggml-org/gemma-3-4b-it-GGUF", "downloads": 44606,
|
||||||
|
"lastModified": "2026-07-01T00:00:00.000Z"},
|
||||||
|
{"id": "ggml-org/gpt-oss-20b-GGUF", "downloads": 47975},
|
||||||
|
{"noid": True},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class Releases(DikteTest):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.patch_attr(hub, "CACHE_DIR", self.path("cache"))
|
||||||
|
|
||||||
|
def test_the_tag_and_the_assets_come_back(self):
|
||||||
|
with fake_urlopen(RELEASE) as calls:
|
||||||
|
tag, assets = hub.release("ggml-org/whisper.cpp")
|
||||||
|
self.assertEqual(tag, "v1.9.1")
|
||||||
|
self.assertEqual([a.name for a in assets],
|
||||||
|
["whisper-bin-ubuntu-x64.tar.gz", "whisper-bin-x64.zip"])
|
||||||
|
self.assertEqual(calls[0].full_url,
|
||||||
|
"https://api.github.com/repos/ggml-org/whisper.cpp/"
|
||||||
|
"releases/latest")
|
||||||
|
|
||||||
|
def test_the_sha256_prefix_is_dropped(self):
|
||||||
|
with fake_urlopen(RELEASE):
|
||||||
|
_, assets = hub.release("ggml-org/whisper.cpp")
|
||||||
|
self.assertEqual(assets[0].sha256, "a" * 64)
|
||||||
|
|
||||||
|
def test_a_tag_asks_for_that_tag(self):
|
||||||
|
with fake_urlopen(RELEASE) as calls:
|
||||||
|
hub.release("ggml-org/whisper.cpp", "v1.9.1")
|
||||||
|
self.assertTrue(calls[0].full_url.endswith("/releases/tags/v1.9.1"))
|
||||||
|
|
||||||
|
def test_a_release_with_no_assets_is_an_error(self):
|
||||||
|
with fake_urlopen({"tag_name": "v1", "assets": []}):
|
||||||
|
with self.assertRaises(hub.HubError):
|
||||||
|
hub.release("ggml-org/whisper.cpp")
|
||||||
|
|
||||||
|
def test_the_second_call_asks_nobody(self):
|
||||||
|
with fake_urlopen(RELEASE) as calls:
|
||||||
|
hub.release("ggml-org/whisper.cpp")
|
||||||
|
hub.release("ggml-org/whisper.cpp")
|
||||||
|
self.assertEqual(len(calls), 1)
|
||||||
|
|
||||||
|
def test_a_refresh_asks_again(self):
|
||||||
|
with fake_urlopen(RELEASE) as calls:
|
||||||
|
hub.release("ggml-org/whisper.cpp")
|
||||||
|
hub.release("ggml-org/whisper.cpp", refresh=True)
|
||||||
|
self.assertEqual(len(calls), 2)
|
||||||
|
|
||||||
|
def test_an_old_cache_beats_no_answer(self):
|
||||||
|
with fake_urlopen(RELEASE):
|
||||||
|
hub.release("ggml-org/whisper.cpp")
|
||||||
|
# Old enough that it would normally be fetched again, and no network
|
||||||
|
# to fetch it with.
|
||||||
|
for path in self.path("cache").iterdir():
|
||||||
|
os_utime(path)
|
||||||
|
with fake_urlopen(url_error()):
|
||||||
|
tag, assets = hub.release("ggml-org/whisper.cpp")
|
||||||
|
self.assertEqual(tag, "v1.9.1")
|
||||||
|
self.assertEqual(len(assets), 2)
|
||||||
|
|
||||||
|
def test_no_cache_and_no_network_says_so(self):
|
||||||
|
with fake_urlopen(url_error("no route to host")):
|
||||||
|
with self.assertRaises(hub.HubError) as caught:
|
||||||
|
hub.release("ggml-org/whisper.cpp")
|
||||||
|
self.assertIn("api.github.com", str(caught.exception))
|
||||||
|
|
||||||
|
def test_an_http_error_names_the_host_and_the_code(self):
|
||||||
|
with fake_urlopen(http_error(404, "nope")):
|
||||||
|
with self.assertRaises(hub.HubError) as caught:
|
||||||
|
hub.release("ggml-org/nothing")
|
||||||
|
self.assertIn("404", str(caught.exception))
|
||||||
|
|
||||||
|
|
||||||
|
class Files(DikteTest):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.patch_attr(hub, "CACHE_DIR", self.path("cache"))
|
||||||
|
|
||||||
|
def test_directories_are_left_out(self):
|
||||||
|
with fake_urlopen(TREE):
|
||||||
|
files = hub.files("ggerganov/whisper.cpp")
|
||||||
|
self.assertEqual([f.name for f in files],
|
||||||
|
[".gitattributes", "ggml-base.bin", "extra/ggml-tiny.bin"])
|
||||||
|
|
||||||
|
def test_the_url_is_the_one_that_serves_the_bytes(self):
|
||||||
|
with fake_urlopen(TREE):
|
||||||
|
files = hub.files("ggerganov/whisper.cpp")
|
||||||
|
self.assertEqual(
|
||||||
|
files[1].url,
|
||||||
|
"https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin")
|
||||||
|
|
||||||
|
def test_the_lfs_object_id_is_the_checksum(self):
|
||||||
|
with fake_urlopen(TREE):
|
||||||
|
files = hub.files("ggerganov/whisper.cpp")
|
||||||
|
self.assertEqual(files[1].sha256, "c" * 64)
|
||||||
|
self.assertEqual(files[1].size, 147951465)
|
||||||
|
|
||||||
|
def test_a_file_outside_lfs_has_no_checksum(self):
|
||||||
|
with fake_urlopen(TREE):
|
||||||
|
files = hub.files("ggerganov/whisper.cpp")
|
||||||
|
self.assertEqual(files[0].sha256, "")
|
||||||
|
|
||||||
|
def test_an_answer_that_is_not_a_list_is_an_error(self):
|
||||||
|
with fake_urlopen({"error": "Invalid username or password."}):
|
||||||
|
with self.assertRaises(hub.HubError):
|
||||||
|
hub.files("ggml-org/whisper.cpp")
|
||||||
|
|
||||||
|
|
||||||
|
class Repos(DikteTest):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.patch_attr(hub, "CACHE_DIR", self.path("cache"))
|
||||||
|
|
||||||
|
def test_it_asks_for_one_author_and_for_gguf(self):
|
||||||
|
with fake_urlopen(MODELS) as calls:
|
||||||
|
found = hub.repos(author="ggml-org")
|
||||||
|
self.assertIn("author=ggml-org", calls[0].full_url)
|
||||||
|
self.assertIn("filter=gguf", calls[0].full_url)
|
||||||
|
self.assertEqual([r.id for r in found],
|
||||||
|
["ggml-org/gemma-3-4b-it-GGUF", "ggml-org/gpt-oss-20b-GGUF"])
|
||||||
|
|
||||||
|
def test_a_missing_download_count_is_zero(self):
|
||||||
|
with fake_urlopen(MODELS):
|
||||||
|
found = hub.repos(author="ggml-org")
|
||||||
|
self.assertEqual(found[0].downloads, 44606)
|
||||||
|
self.assertEqual(found[1].updated, "")
|
||||||
|
|
||||||
|
|
||||||
|
def os_utime(path):
|
||||||
|
"""Backdate a cache file past its time to live."""
|
||||||
|
import os
|
||||||
|
import time
|
||||||
|
old = time.time() - hub.CACHE_TTL - 60
|
||||||
|
os.utime(path, (old, old))
|
||||||
|
|
||||||
|
|
||||||
|
class CacheOnDisk(DikteTest):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.patch_attr(hub, "CACHE_DIR", self.path("cache"))
|
||||||
|
|
||||||
|
def test_what_is_stored_is_what_came_back(self):
|
||||||
|
with fake_urlopen(RELEASE):
|
||||||
|
hub.release("ggml-org/whisper.cpp")
|
||||||
|
stored = [json.loads(p.read_text()) for p in self.path("cache").iterdir()]
|
||||||
|
self.assertEqual(stored[0]["tag_name"], "v1.9.1")
|
||||||
|
|
||||||
|
def test_a_cache_that_cannot_be_written_is_not_a_failure(self):
|
||||||
|
self.patch_attr(hub, "CACHE_DIR", self.path("nope", "deeper"))
|
||||||
|
self.path("nope").write_text("a file where a directory would go")
|
||||||
|
with fake_urlopen(RELEASE):
|
||||||
|
tag, _ = hub.release("ggml-org/whisper.cpp")
|
||||||
|
self.assertEqual(tag, "v1.9.1")
|
||||||
@@ -13,7 +13,9 @@ from unittest import mock
|
|||||||
|
|
||||||
from PyQt6.QtWidgets import QApplication, QMessageBox
|
from PyQt6.QtWidgets import QApplication, QMessageBox
|
||||||
|
|
||||||
|
import cleanup
|
||||||
import config as cfg
|
import config as cfg
|
||||||
|
import hotkey
|
||||||
import overlay as overlay_module
|
import overlay as overlay_module
|
||||||
import paste
|
import paste
|
||||||
import settings_ui
|
import settings_ui
|
||||||
@@ -39,13 +41,27 @@ CHANGED = {
|
|||||||
"filter_hallucinations": False,
|
"filter_hallucinations": False,
|
||||||
"keep_audio": True,
|
"keep_audio": True,
|
||||||
"openai_api_key": "sk-test-key",
|
"openai_api_key": "sk-test-key",
|
||||||
|
"groq_api_key": "gsk-test-key",
|
||||||
"openrouter_api_key": "sk-or-test-key",
|
"openrouter_api_key": "sk-or-test-key",
|
||||||
"transcribe_provider": "openrouter",
|
"transcribe_provider": "openrouter",
|
||||||
"transcribe_model": "whisper-1",
|
"transcribe_model": "whisper-1",
|
||||||
|
"groq_transcribe_model": "whisper-large-v3",
|
||||||
"openrouter_transcribe_model": "openai/whisper-1",
|
"openrouter_transcribe_model": "openai/whisper-1",
|
||||||
"cleanup_enabled": False,
|
"cleanup_enabled": False,
|
||||||
|
"cleanup_provider": "local",
|
||||||
"cleanup_model": "some/other-model",
|
"cleanup_model": "some/other-model",
|
||||||
|
"cleanup_claude_model": "opus",
|
||||||
|
"cleanup_codex_model": "gpt-5",
|
||||||
"cleanup_reasoning": "high",
|
"cleanup_reasoning": "high",
|
||||||
|
"local_model": "ggml-small.bin",
|
||||||
|
"local_gpu": False,
|
||||||
|
"local_preload": False,
|
||||||
|
"local_threads": 6,
|
||||||
|
"local_llm_model": "gemma-3-4b-it-Q4_K_M.gguf",
|
||||||
|
"local_llm_repo": "ggml-org/gemma-4-E2B-it-GGUF",
|
||||||
|
"local_llm_gpu": False,
|
||||||
|
"local_llm_preload": True,
|
||||||
|
"local_llm_reasoning": "low",
|
||||||
"cleanup_prompt": "Only fix the punctuation.",
|
"cleanup_prompt": "Only fix the punctuation.",
|
||||||
"file_cleanup_prompt": "Keep the stamps where they are.",
|
"file_cleanup_prompt": "Keep the stamps where they are.",
|
||||||
"transcribe_prompt": "Paraşüt, OpenFrame",
|
"transcribe_prompt": "Paraşüt, OpenFrame",
|
||||||
@@ -77,6 +93,7 @@ CHANGED = {
|
|||||||
"file_timestamps": True,
|
"file_timestamps": True,
|
||||||
"file_cleanup": False,
|
"file_cleanup": False,
|
||||||
"shortcut": "Ctrl+Alt+Space",
|
"shortcut": "Ctrl+Alt+Space",
|
||||||
|
"cancel_shortcut": "Meta+Shift+Space",
|
||||||
"evdev_hotkey": True,
|
"evdev_hotkey": True,
|
||||||
"history_limit": 50,
|
"history_limit": 50,
|
||||||
}
|
}
|
||||||
@@ -105,7 +122,7 @@ class Settings(DikteTest):
|
|||||||
self.path("kglobalshortcutsrc")))
|
self.path("kglobalshortcutsrc")))
|
||||||
|
|
||||||
def window(self, conf):
|
def window(self, conf):
|
||||||
window = settings_ui.SettingsWindow(conf, "dikte toggle")
|
window = settings_ui.SettingsWindow(conf)
|
||||||
self.addCleanup(window.deleteLater)
|
self.addCleanup(window.deleteLater)
|
||||||
self.addCleanup(window.close)
|
self.addCleanup(window.close)
|
||||||
return window
|
return window
|
||||||
@@ -133,6 +150,20 @@ class Settings(DikteTest):
|
|||||||
with self.subTest(key=key):
|
with self.subTest(key=key):
|
||||||
self.assertEqual(stored[key], value)
|
self.assertEqual(stored[key], value)
|
||||||
|
|
||||||
|
def test_the_model_box_on_screen_belongs_to_whoever_cleans_up(self):
|
||||||
|
"""An OpenRouter id and a Claude alias are not the same field."""
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
boxes = {"openrouter": window.cleanup_model_row,
|
||||||
|
"claude": window.cleanup_claude_model,
|
||||||
|
"codex": window.cleanup_codex_model}
|
||||||
|
for provider, box in boxes.items():
|
||||||
|
with self.subTest(provider=provider):
|
||||||
|
window._select_data(window.cleanup_provider, provider)
|
||||||
|
shown = [name for name, other in boxes.items()
|
||||||
|
if not other.isHidden()]
|
||||||
|
self.assertEqual(shown, [provider])
|
||||||
|
self.assertFalse(box.isHidden())
|
||||||
|
|
||||||
def test_the_settings_the_window_does_not_show_are_left_alone(self):
|
def test_the_settings_the_window_does_not_show_are_left_alone(self):
|
||||||
"""A tab nobody wrote must not reset what the command line set."""
|
"""A tab nobody wrote must not reset what the command line set."""
|
||||||
self.write_config({"silence_db": -42.0, "speech_margin_db": 15.0,
|
self.write_config({"silence_db": -42.0, "speech_margin_db": 15.0,
|
||||||
@@ -143,6 +174,37 @@ class Settings(DikteTest):
|
|||||||
self.assertEqual(stored["speech_margin_db"], 15.0)
|
self.assertEqual(stored["speech_margin_db"], 15.0)
|
||||||
self.assertEqual(stored["openrouter_base_url"], "http://localhost:1234/v1")
|
self.assertEqual(stored["openrouter_base_url"], "http://localhost:1234/v1")
|
||||||
|
|
||||||
|
def test_every_global_shortcut_has_a_row_of_its_own(self):
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
self.assertEqual(set(window._shortcut_rows), set(hotkey.SHORTCUTS))
|
||||||
|
|
||||||
|
def test_emptying_a_shortcut_turns_it_off_but_not_the_toggle(self):
|
||||||
|
"""The application is unusable without the toggle, so that one box
|
||||||
|
falls back. The rest stay empty, which is how they are switched off."""
|
||||||
|
conf = cfg.Config()
|
||||||
|
window = self.window(conf)
|
||||||
|
for box, _status, _missing in window._shortcut_rows.values():
|
||||||
|
box.setCurrentText("")
|
||||||
|
window._save()
|
||||||
|
self.assertEqual(conf["shortcut"], "Ctrl+Space")
|
||||||
|
self.assertEqual(conf["cancel_shortcut"], "")
|
||||||
|
self.assertEqual(conf["assistant_shortcut"], "")
|
||||||
|
self.assertEqual(conf["meeting_shortcut"], "")
|
||||||
|
|
||||||
|
def test_installing_the_discard_key_writes_its_own_entry(self):
|
||||||
|
conf = cfg.Config()
|
||||||
|
window = self.window(conf)
|
||||||
|
window._shortcut_rows["cancel"][0].setCurrentText("Meta+Shift+Space")
|
||||||
|
with mock.patch.object(settings_ui.hotkey, "install_shortcut",
|
||||||
|
return_value=(True, "saved")) as install:
|
||||||
|
window._install_shortcut("cancel")
|
||||||
|
combo, command = install.call_args.args
|
||||||
|
self.assertEqual(combo, "Meta+Shift+Space")
|
||||||
|
self.assertTrue(command.endswith(" cancel"))
|
||||||
|
self.assertEqual(install.call_args.kwargs["desktop_id"],
|
||||||
|
hotkey.CANCEL_DESKTOP_ID)
|
||||||
|
self.assertEqual(conf["cancel_shortcut"], "Meta+Shift+Space")
|
||||||
|
|
||||||
def test_a_prompt_left_at_its_default_is_stored_as_empty(self):
|
def test_a_prompt_left_at_its_default_is_stored_as_empty(self):
|
||||||
"""So that switching the interface language switches the prompt too."""
|
"""So that switching the interface language switches the prompt too."""
|
||||||
conf = cfg.Config()
|
conf = cfg.Config()
|
||||||
@@ -154,14 +216,44 @@ class Settings(DikteTest):
|
|||||||
def test_each_provider_keeps_its_own_transcription_model(self):
|
def test_each_provider_keeps_its_own_transcription_model(self):
|
||||||
self.write_config({"transcribe_provider": "openai",
|
self.write_config({"transcribe_provider": "openai",
|
||||||
"transcribe_model": "gpt-4o-transcribe",
|
"transcribe_model": "gpt-4o-transcribe",
|
||||||
|
"groq_transcribe_model": "whisper-large-v3",
|
||||||
"openrouter_transcribe_model": "openai/whisper-1"})
|
"openrouter_transcribe_model": "openai/whisper-1"})
|
||||||
conf = cfg.Config()
|
conf = cfg.Config()
|
||||||
window = self.window(conf)
|
window = self.window(conf)
|
||||||
|
for provider in ("groq", "openrouter"):
|
||||||
window.transcribe_provider.setCurrentIndex(
|
window.transcribe_provider.setCurrentIndex(
|
||||||
window.transcribe_provider.findData("openrouter"))
|
window.transcribe_provider.findData(provider))
|
||||||
window._save()
|
window._save()
|
||||||
self.assertEqual(conf["transcribe_provider"], "openrouter")
|
self.assertEqual(conf["transcribe_provider"], "openrouter")
|
||||||
self.assertEqual(conf["transcribe_model"], "gpt-4o-transcribe")
|
self.assertEqual(conf["transcribe_model"], "gpt-4o-transcribe")
|
||||||
|
self.assertEqual(conf["groq_transcribe_model"], "whisper-large-v3")
|
||||||
|
|
||||||
|
def test_the_provider_box_offers_every_provider_config_knows(self):
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
offered = [window.transcribe_provider.itemData(i)
|
||||||
|
for i in range(window.transcribe_provider.count())]
|
||||||
|
self.assertEqual(offered, ["local"] + list(cfg.TRANSCRIBERS))
|
||||||
|
|
||||||
|
def test_the_cleanup_box_offers_everyone_cleanup_py_dispatches_to(self):
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
offered = [window.cleanup_provider.itemData(i)
|
||||||
|
for i in range(window.cleanup_provider.count())]
|
||||||
|
self.assertEqual(sorted(offered), sorted(cleanup.PROVIDERS))
|
||||||
|
|
||||||
|
def test_the_answer_to_a_test_lands_under_the_key_it_was_asked_about(self):
|
||||||
|
"""One signal serves all three buttons, so it carries which one asked."""
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
window._on_test_done("groq", True, "it works")
|
||||||
|
button, answer = window._testers["groq"]
|
||||||
|
self.assertEqual(answer.text(), "✓ it works")
|
||||||
|
self.assertTrue(button.isEnabled())
|
||||||
|
self.assertEqual(window._testers["openai"][1].text(), "")
|
||||||
|
|
||||||
|
def test_a_key_lands_in_the_field_of_its_own_provider(self):
|
||||||
|
self.write_config({"groq_api_key": "gsk-mine"})
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
self.assertEqual(window.groq_key.text(), "gsk-mine")
|
||||||
|
self.assertEqual(window.openai_key.text(), "")
|
||||||
|
|
||||||
def test_saving_applies_the_lowered_history_limit_at_once(self):
|
def test_saving_applies_the_lowered_history_limit_at_once(self):
|
||||||
for index in range(10):
|
for index in range(10):
|
||||||
@@ -183,6 +275,43 @@ class Settings(DikteTest):
|
|||||||
window = self.window(cfg.Config())
|
window = self.window(cfg.Config())
|
||||||
self.assertEqual(window.windowTitle(), "Dikte Ayarları")
|
self.assertEqual(window.windowTitle(), "Dikte Ayarları")
|
||||||
|
|
||||||
|
def test_the_audio_file_switches_are_kept_without_the_save_button(self):
|
||||||
|
"""They are ticked to transcribe one file, not to fill in a form."""
|
||||||
|
self.write_config({"file_timestamps": False, "file_cleanup": True})
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
window.file_timestamps.setChecked(True)
|
||||||
|
window.file_cleanup.setChecked(False)
|
||||||
|
stored = self.read_config_file()
|
||||||
|
self.assertTrue(stored["file_timestamps"])
|
||||||
|
self.assertFalse(stored["file_cleanup"])
|
||||||
|
|
||||||
|
def test_loading_the_audio_file_tab_is_not_taken_for_a_change(self):
|
||||||
|
self.write_config({"file_timestamps": True, "file_cleanup": False})
|
||||||
|
conf = cfg.Config()
|
||||||
|
with mock.patch.object(conf, "save") as save:
|
||||||
|
window = self.window(conf)
|
||||||
|
save.assert_not_called()
|
||||||
|
self.assertTrue(window.file_timestamps.isChecked())
|
||||||
|
self.assertFalse(window.file_cleanup.isChecked())
|
||||||
|
|
||||||
|
def test_the_run_button_comes_back_when_the_stop_lands(self):
|
||||||
|
"""In whichever language, since the worker says so through t() too."""
|
||||||
|
for language in ("auto", "tr"):
|
||||||
|
with self.subTest(language=language):
|
||||||
|
self.write_config({"ui_language": language})
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
window.file_run.setEnabled(False)
|
||||||
|
window._on_file_progress(settings_ui.t("Stopped."))
|
||||||
|
self.assertTrue(window.file_run.isEnabled())
|
||||||
|
|
||||||
|
def test_stop_leaves_nothing_to_press_twice(self):
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
with mock.patch.object(window.transcriber, "stop") as stop:
|
||||||
|
window.file_stop.setEnabled(True)
|
||||||
|
window._stop_file()
|
||||||
|
stop.assert_called_once_with()
|
||||||
|
self.assertFalse(window.file_stop.isEnabled())
|
||||||
|
|
||||||
|
|
||||||
class MacSettings(Settings):
|
class MacSettings(Settings):
|
||||||
"""The same window and the same round trip, standing on a Mac.
|
"""The same window and the same round trip, standing on a Mac.
|
||||||
@@ -296,3 +425,78 @@ class Overlay(DikteTest):
|
|||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
||||||
|
|
||||||
|
class LocalModels(DikteTest):
|
||||||
|
"""The download boxes, without a network and without either program."""
|
||||||
|
|
||||||
|
def window(self, conf):
|
||||||
|
window = settings_ui.SettingsWindow(conf)
|
||||||
|
self.addCleanup(window.deleteLater)
|
||||||
|
self.addCleanup(window.close)
|
||||||
|
return window
|
||||||
|
|
||||||
|
def test_it_opens_where_the_missing_model_is_fixed(self):
|
||||||
|
# Nothing can transcribe on a fresh install, which is why this window
|
||||||
|
# was opened at all.
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
self.assertEqual(window.tabs.currentIndex(), window.api_tab_index)
|
||||||
|
|
||||||
|
def test_it_opens_where_it_was_left_when_everything_works(self):
|
||||||
|
conf = self.config(transcribe_provider="openai", openai_api_key="sk-test")
|
||||||
|
self.assertEqual(self.window(conf).tabs.currentIndex(), 0)
|
||||||
|
|
||||||
|
def test_a_model_that_is_not_here_yet_survives_a_save(self):
|
||||||
|
# The box is filled from what is on this disk, so a model that was
|
||||||
|
# deleted from underneath is not in the list. Dropping it on save would
|
||||||
|
# quietly empty the setting instead of asking for the download again.
|
||||||
|
conf = self.config(local_model="ggml-large-v3-turbo-q5_0.bin")
|
||||||
|
with mock.patch.object(QMessageBox, "information"):
|
||||||
|
self.window(conf)._save()
|
||||||
|
self.assertEqual(conf["local_model"], "ggml-large-v3-turbo-q5_0.bin")
|
||||||
|
|
||||||
|
def test_nothing_is_fetched_for_a_window_nobody_opened(self):
|
||||||
|
# DikteTest closes the network, so a request would fail the test. The
|
||||||
|
# lists are asked for when the box is shown, not when it is built.
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
self.assertTrue(window.local_whisper._pending)
|
||||||
|
|
||||||
|
def test_a_model_bigger_than_two_gigabytes_counts_up_rather_than_down(self):
|
||||||
|
# Qt's int is C++'s 32-bit one, and a 2.3 GB model is more than fits in
|
||||||
|
# it: the count came out the far side negative, at "-1%".
|
||||||
|
box = self.window(cfg.Config()).local_llm
|
||||||
|
box._downloading = True
|
||||||
|
box._report(1_048_576, 2_489_757_856)
|
||||||
|
_app.processEvents()
|
||||||
|
self.assertIn("2.3 GB", box.status.text())
|
||||||
|
self.assertNotIn("-", box.status.text())
|
||||||
|
|
||||||
|
def test_a_long_model_name_is_not_cut_in_half(self):
|
||||||
|
# The list under a combo box takes the box's width and elides what does
|
||||||
|
# not fit, in the middle: "ggml-org/Qwen....7B-Base-GGUF".
|
||||||
|
box = self.window(cfg.Config()).local_llm
|
||||||
|
box.repo.addItem("ggml-org/a-model-with-a-name-that-runs-on-and-on-GGUF")
|
||||||
|
box._fit_popup(box.repo)
|
||||||
|
view = box.repo.view()
|
||||||
|
self.assertEqual(view.textElideMode(), settings_ui.Qt.TextElideMode.ElideNone)
|
||||||
|
widest = max(box.repo.fontMetrics().horizontalAdvance(box.repo.itemText(row))
|
||||||
|
for row in range(box.repo.count()))
|
||||||
|
self.assertGreaterEqual(view.minimumWidth(), widest)
|
||||||
|
|
||||||
|
def test_only_the_chosen_transcriber_is_on_screen(self):
|
||||||
|
window = self.window(self.config(transcribe_provider="openai"))
|
||||||
|
self.assertTrue(window.stt_form.isRowVisible(window.transcribe_model_row))
|
||||||
|
self.assertFalse(window.stt_form.isRowVisible(window.local_whisper))
|
||||||
|
window._select_data(window.transcribe_provider, "local")
|
||||||
|
self.assertFalse(window.stt_form.isRowVisible(window.transcribe_model_row))
|
||||||
|
self.assertTrue(window.stt_form.isRowVisible(window.local_whisper))
|
||||||
|
|
||||||
|
def test_only_the_chosen_cleaner_is_on_screen(self):
|
||||||
|
window = self.window(cfg.Config())
|
||||||
|
self.assertTrue(window.cleanup_form.isRowVisible(window.cleanup_model_row))
|
||||||
|
self.assertFalse(window.cleanup_form.isRowVisible(window.local_llm))
|
||||||
|
window._select_data(window.cleanup_provider, "local")
|
||||||
|
self.assertTrue(window.cleanup_form.isRowVisible(window.local_llm))
|
||||||
|
self.assertFalse(window.cleanup_form.isRowVisible(window.cleanup_model_row))
|
||||||
|
# Its own thinking box, because the two default to opposite things.
|
||||||
|
self.assertFalse(window.cleanup_form.isRowVisible(window.cleanup_reasoning))
|
||||||
|
|||||||
@@ -234,7 +234,7 @@ class Chain(DikteTest):
|
|||||||
self.assertEqual(row["raw"], "uh, book it for Thursday")
|
self.assertEqual(row["raw"], "uh, book it for Thursday")
|
||||||
self.assertEqual(row["text"], "Book it for Thursday.")
|
self.assertEqual(row["text"], "Book it for Thursday.")
|
||||||
self.assertEqual(row["duration"], 2.0)
|
self.assertEqual(row["duration"], 2.0)
|
||||||
self.assertEqual(row["model"], self.conf["transcribe_model"])
|
self.assertEqual(row["model"], self.conf.transcribe_target().model)
|
||||||
self.assertEqual(row["mode"], "")
|
self.assertEqual(row["mode"], "")
|
||||||
|
|
||||||
def test_a_command_is_recorded_as_one(self):
|
def test_a_command_is_recorded_as_one(self):
|
||||||
|
|||||||
@@ -0,0 +1,154 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Dikte uninstaller: takes back what install.sh put down, and nothing else
|
||||||
|
# unless asked. Your settings and your dictations survive a plain run; --purge
|
||||||
|
# is the word that deletes them.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
PY="$(command -v python3 || true)"
|
||||||
|
USER_NAME="$(id -un)"
|
||||||
|
BIN_DIR="$HOME/.local/bin"
|
||||||
|
APP_DIR="$HOME/.local/share/applications"
|
||||||
|
AUTOSTART_DIR="$HOME/.config/autostart"
|
||||||
|
CONFIG_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/dikte"
|
||||||
|
DATA_DIR="${XDG_DATA_HOME:-$HOME/.local/share}/dikte"
|
||||||
|
|
||||||
|
PURGE=0
|
||||||
|
ASSUME_YES=0
|
||||||
|
|
||||||
|
say() { printf ' %s\n' "$1"; }
|
||||||
|
ok() { printf ' \033[32m✓\033[0m %s\n' "$1"; }
|
||||||
|
warn() { printf ' \033[33m!\033[0m %s\n' "$1"; }
|
||||||
|
gone() { printf ' \033[90m·\033[0m %s\n' "$1"; }
|
||||||
|
# "1 dictation", "3 dictations": how many is the point of printing it at all.
|
||||||
|
count() {
|
||||||
|
if (( $1 == 1 )); then printf '%s %s' "$1" "$2"; else printf '%s %s' "$1" "$3"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<EOF
|
||||||
|
Usage: ./uninstall.sh [--purge] [--yes]
|
||||||
|
|
||||||
|
--purge also delete the settings ($CONFIG_DIR)
|
||||||
|
and the dictations, meetings and recordings ($DATA_DIR)
|
||||||
|
--yes do not ask before deleting those
|
||||||
|
|
||||||
|
Without --purge nothing you have written is touched, and the source directory
|
||||||
|
is left alone either way.
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
for arg in "$@"; do
|
||||||
|
case "$arg" in
|
||||||
|
--purge) PURGE=1 ;;
|
||||||
|
--yes|-y) ASSUME_YES=1 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) printf 'uninstall.sh: unknown option: %s\n' "$arg" >&2; usage >&2; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# A symlink whose target is gone is still a file to remove, hence -L.
|
||||||
|
remove() {
|
||||||
|
if [[ -e "$1" || -L "$1" ]]; then
|
||||||
|
rm -f "$1"
|
||||||
|
ok "Removed $1"
|
||||||
|
else
|
||||||
|
gone "Was not there: $1"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "Uninstalling Dikte"
|
||||||
|
echo "──────────────────"
|
||||||
|
|
||||||
|
# 1. Global shortcuts ------------------------------------------------------
|
||||||
|
# Handed to Dikte while it can still run, because it is the half that knows
|
||||||
|
# whether they went into KDE's kglobalshortcutsrc or GNOME's gsettings.
|
||||||
|
if [[ -n "$PY" ]] && python3 -c 'import PyQt6.QtWidgets' 2>/dev/null; then
|
||||||
|
for which in toggle cancel ask meeting; do
|
||||||
|
"$PY" "$DIR/dikte.py" shortcut remove "$which" >/dev/null 2>&1 || true
|
||||||
|
done
|
||||||
|
ok "Global shortcuts unregistered"
|
||||||
|
say "KWin reads that file at startup, so the keys are free after your next login."
|
||||||
|
else
|
||||||
|
warn "PyQt6 is missing, so the shortcuts were left registered."
|
||||||
|
say "Remove them in your desktop's shortcut settings."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 2. The running instance --------------------------------------------------
|
||||||
|
# It holds a tray icon and a socket; asking it to quit is tidier than pulling
|
||||||
|
# its launchers out from under it.
|
||||||
|
if pgrep -u "$USER_NAME" -f 'dikte\.py' >/dev/null 2>&1; then
|
||||||
|
[[ -n "$PY" ]] && "$PY" "$DIR/dikte.py" quit >/dev/null 2>&1 || true
|
||||||
|
sleep 0.5
|
||||||
|
if pgrep -u "$USER_NAME" -f 'dikte\.py' >/dev/null 2>&1; then
|
||||||
|
warn "Dikte is still running; close it from the tray icon"
|
||||||
|
else
|
||||||
|
ok "Stopped the running instance"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 3. Launchers -------------------------------------------------------------
|
||||||
|
# Only our own symlink goes: a file of the same name that somebody else put
|
||||||
|
# there is not ours to delete.
|
||||||
|
if [[ -L "$BIN_DIR/dikte" ]]; then
|
||||||
|
remove "$BIN_DIR/dikte"
|
||||||
|
elif [[ -e "$BIN_DIR/dikte" ]]; then
|
||||||
|
warn "$BIN_DIR/dikte is not our symlink, leaving it alone"
|
||||||
|
else
|
||||||
|
gone "Was not there: $BIN_DIR/dikte"
|
||||||
|
fi
|
||||||
|
remove "$APP_DIR/dikte.desktop"
|
||||||
|
remove "$AUTOSTART_DIR/dikte.desktop"
|
||||||
|
# Removing the shortcut takes its desktop file with it, but an install from
|
||||||
|
# before this script existed may have left one behind on a desktop that never
|
||||||
|
# used them.
|
||||||
|
for id in dikte-toggle dikte-cancel dikte-ask dikte-meeting; do
|
||||||
|
if [[ -e "$APP_DIR/$id.desktop" ]]; then
|
||||||
|
remove "$APP_DIR/$id.desktop"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# 4. Settings and dictations -----------------------------------------------
|
||||||
|
echo
|
||||||
|
if ((PURGE)); then
|
||||||
|
warn "--purge also deletes:"
|
||||||
|
if [[ -f "$CONFIG_DIR/config.json" ]]; then
|
||||||
|
say "$CONFIG_DIR/config.json (your API keys and every setting)"
|
||||||
|
fi
|
||||||
|
if [[ -f "$DATA_DIR/history.jsonl" ]]; then
|
||||||
|
# grep -c rather than wc -l: a last line with no newline is still a dictation.
|
||||||
|
say "$DATA_DIR/history.jsonl ($(count "$(grep -c '' "$DATA_DIR/history.jsonl" 2>/dev/null || echo 0)" dictation dictations))"
|
||||||
|
fi
|
||||||
|
if [[ -d "$DATA_DIR/meetings" ]]; then
|
||||||
|
say "$DATA_DIR/meetings ($(count "$(find "$DATA_DIR/meetings" -name '*.md' | wc -l)" meeting meetings))"
|
||||||
|
fi
|
||||||
|
if [[ -d "$DATA_DIR/recordings" ]]; then
|
||||||
|
say "$DATA_DIR/recordings ($(du -sh "$DATA_DIR/recordings" | cut -f1) of audio)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ((!ASSUME_YES)); then
|
||||||
|
if [[ -t 0 ]]; then
|
||||||
|
printf ' Type yes to delete them: '
|
||||||
|
read -r reply
|
||||||
|
[[ "$reply" == "yes" ]] || { PURGE=0; say "Kept."; }
|
||||||
|
else
|
||||||
|
PURGE=0
|
||||||
|
warn "Not a terminal, so nothing was deleted. Pass --yes if you meant it."
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ((PURGE)); then
|
||||||
|
rm -rf "$CONFIG_DIR" "$DATA_DIR"
|
||||||
|
ok "Settings and dictations deleted"
|
||||||
|
else
|
||||||
|
say "Settings kept: $CONFIG_DIR"
|
||||||
|
say "Dictations kept: $DATA_DIR"
|
||||||
|
say "Delete them too with: ./uninstall.sh --purge"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo
|
||||||
|
ok "Done."
|
||||||
|
say "The source directory is untouched: $DIR"
|
||||||
|
echo
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Dikte updater: pull, put the launchers back, restart what was running.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
PY="$(command -v python3 || true)"
|
||||||
|
USER_NAME="$(id -un)"
|
||||||
|
|
||||||
|
say() { printf ' %s\n' "$1"; }
|
||||||
|
ok() { printf ' \033[32m✓\033[0m %s\n' "$1"; }
|
||||||
|
warn() { printf ' \033[33m!\033[0m %s\n' "$1"; }
|
||||||
|
die() { printf ' \033[31m✗\033[0m %s\n' "$1"; echo; exit 1; }
|
||||||
|
|
||||||
|
# The combination stored in the settings, which is where Dikte itself reads it
|
||||||
|
# from and the one place that is the same on KDE and on GNOME.
|
||||||
|
setting() {
|
||||||
|
[[ -n "$PY" ]] || return 0
|
||||||
|
"$PY" "$DIR/dikte.py" config get "$1" 2>/dev/null || true
|
||||||
|
}
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "Updating Dikte"
|
||||||
|
echo "──────────────"
|
||||||
|
|
||||||
|
cd "$DIR"
|
||||||
|
|
||||||
|
# 1. Somewhere there is something to pull ----------------------------------
|
||||||
|
command -v git >/dev/null || die "git not found; update by downloading the source again"
|
||||||
|
git rev-parse --git-dir >/dev/null 2>&1 \
|
||||||
|
|| die "$DIR is not a git checkout; update by downloading the source again"
|
||||||
|
|
||||||
|
before="$(git rev-parse HEAD)"
|
||||||
|
|
||||||
|
# 2. Is there anything to come? ---------------------------------------------
|
||||||
|
# Asked before anything else is complained about: an unfinished afternoon in
|
||||||
|
# the working tree is nobody's problem on a day when nothing has been
|
||||||
|
# published. Fetching leaves the working tree alone.
|
||||||
|
git fetch --quiet || die "Could not reach the remote."
|
||||||
|
upstream="$(git rev-parse '@{u}' 2>/dev/null)" \
|
||||||
|
|| die "This branch is not tracking a remote one; pull by hand."
|
||||||
|
|
||||||
|
if [[ "$before" == "$upstream" ]]; then
|
||||||
|
echo
|
||||||
|
ok "Already up to date ($(git log -1 --format=%s))"
|
||||||
|
echo
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 3. Only now, your own edits -----------------------------------------------
|
||||||
|
# They would be overwritten by a fast-forward or would block it, and either way
|
||||||
|
# that is your call to make, not this script's. Untracked files are counted
|
||||||
|
# too: a fast-forward that adds a file of that name stops on them.
|
||||||
|
if [[ -n "$(git status --porcelain)" ]]; then
|
||||||
|
warn "There is an update waiting, but you have changes of your own here:"
|
||||||
|
git --no-pager status --short | sed 's/^/ /'
|
||||||
|
say "Commit them, or put them aside with: git stash --include-untracked"
|
||||||
|
die "Nothing was updated."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --ff-only: an update should be somebody else's commits arriving, never a
|
||||||
|
# merge this script decided to make on your behalf. The fetch above already
|
||||||
|
# brought them, so this touches no network.
|
||||||
|
# advice off: git's suggestion is a merge or a rebase, and which of those you
|
||||||
|
# want is the sentence below, not a wall of hints.
|
||||||
|
if ! merge_log="$(git -c advice.diverging=false merge --ff-only '@{u}' 2>&1)"; then
|
||||||
|
printf '%s\n' "$merge_log" | sed 's/^/ /'
|
||||||
|
say "Your branch has commits the remote does not. To put them on top of the"
|
||||||
|
say "update instead: git pull --rebase"
|
||||||
|
die "Could not fast-forward."
|
||||||
|
fi
|
||||||
|
after="$(git rev-parse HEAD)"
|
||||||
|
|
||||||
|
echo
|
||||||
|
say "What arrived:"
|
||||||
|
git --no-pager log --oneline "$before..$after" | sed 's/^/ /'
|
||||||
|
echo
|
||||||
|
|
||||||
|
# 4. Launchers --------------------------------------------------------------
|
||||||
|
# An update can add a dependency or move a file, so the installer runs again.
|
||||||
|
# It would otherwise register its own defaults over the keys you chose, so it
|
||||||
|
# is told what those are. Read before the installer runs, since it is the one
|
||||||
|
# writing them.
|
||||||
|
shortcut="$(setting shortcut)"
|
||||||
|
cancel_shortcut="$(setting cancel_shortcut)"
|
||||||
|
# Positional, so a chosen discard key cannot be passed without the other one.
|
||||||
|
"$DIR/install.sh" "${shortcut:-Ctrl+Space}" "${cancel_shortcut:-}"
|
||||||
|
|
||||||
|
# 5. The running instance ---------------------------------------------------
|
||||||
|
# It is still running the code from before the pull.
|
||||||
|
if pgrep -u "$USER_NAME" -f 'dikte\.py' >/dev/null 2>&1; then
|
||||||
|
if [[ -n "$PY" ]] && "$PY" "$DIR/dikte.py" restart >/dev/null 2>&1; then
|
||||||
|
ok "Restarted, so the new version is the one running"
|
||||||
|
else
|
||||||
|
warn "Could not restart it; use the tray menu → Restart"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
say "Dikte was not running. Start it with: dikte"
|
||||||
|
fi
|
||||||
|
echo
|
||||||
@@ -18,6 +18,7 @@ from PyQt6.QtCore import QObject, pyqtSignal
|
|||||||
import api
|
import api
|
||||||
import assistant
|
import assistant
|
||||||
import audio
|
import audio
|
||||||
|
import cleanup
|
||||||
import config as cfg
|
import config as cfg
|
||||||
import i18n
|
import i18n
|
||||||
import paste
|
import paste
|
||||||
@@ -112,14 +113,7 @@ class Pipeline(QObject):
|
|||||||
if (conf["assistant_cleanup"] if ask else conf["cleanup_enabled"]):
|
if (conf["assistant_cleanup"] if ask else conf["cleanup_enabled"]):
|
||||||
self.stage.emit(t("Cleaning up…"))
|
self.stage.emit(t("Cleaning up…"))
|
||||||
try:
|
try:
|
||||||
text = api.cleanup(
|
text = cleanup.run(raw, conf, conf.cleanup_prompt())
|
||||||
raw,
|
|
||||||
conf.openrouter_key(),
|
|
||||||
conf["cleanup_model"],
|
|
||||||
conf.cleanup_prompt(),
|
|
||||||
reasoning=conf["cleanup_reasoning"],
|
|
||||||
base_url=conf["openrouter_base_url"],
|
|
||||||
)
|
|
||||||
except api.ApiError as exc:
|
except api.ApiError as exc:
|
||||||
# Keep the transcript, but never let the failure pass unseen:
|
# Keep the transcript, but never let the failure pass unseen:
|
||||||
# a rejected key would otherwise look like working dictation.
|
# a rejected key would otherwise look like working dictation.
|
||||||
@@ -159,7 +153,7 @@ class Pipeline(QObject):
|
|||||||
"duration": round(duration, 1),
|
"duration": round(duration, 1),
|
||||||
"elapsed": round(time.monotonic() - started, 1),
|
"elapsed": round(time.monotonic() - started, 1),
|
||||||
"model": target.model,
|
"model": target.model,
|
||||||
"cleanup_model": conf["cleanup_model"] if conf["cleanup_enabled"] else "",
|
"cleanup_model": cleanup.model(conf) if conf["cleanup_enabled"] else "",
|
||||||
"cleanup_error": warning,
|
"cleanup_error": warning,
|
||||||
"mode": "ask" if ask else "",
|
"mode": "ask" if ask else "",
|
||||||
"question": question,
|
"question": question,
|
||||||
|
|||||||