mirror of
https://github.com/yusufipk/dikte.git
synced 2026-09-11 19:06:11 +00:00
Do not let a shrugging sysconf turn a workstation into a tiny machine
Three from the review of the change before this one. sysconf answers -1 for a limit it holds to be indeterminate, and CPython hands that back rather than raising, so the page count times the page size came out negative. A negative is truthy, so it went past the check for a machine nothing could be read from and floored at half a gigabyte: a 64 GB workstation was told every model past 512 MB was too big for it, the suggestion dropped to small-q5_1, and the machine line read "Memory: -4096 B". Anything not positive is now the unknown machine it always was. The memory is read once and kept. It does not change while Dikte runs, and a thirty row list asked seventy times per draw, which on the Mac path is seventy processes started on the interface thread every time a download finished, a model was deleted or a publisher changed. And "test-" is matched as a plain substring, so it was also inside "Latest-" and dropped a publisher nothing is wrong with. Anchored the way every other mark in that list already is.
This commit is contained in:
+28
-1
@@ -130,8 +130,11 @@ GGUF_MAX_BYTES = 16 << 30
|
||||
# on: a vision or audio tower with no text half worth running, a speech model,
|
||||
# and the base models, which continue text rather than following an instruction
|
||||
# and answer a cleanup prompt by carrying on writing the transcript.
|
||||
# Matched as plain substrings, so every one of these carries its own
|
||||
# delimiters: an unanchored "test-" is also inside "Latest-" and would drop a
|
||||
# publisher that is perfectly usable.
|
||||
LLM_REPO_SKIP = ("-Base-GGUF", "-VL-", "-Vision-", "-Omni-", "-Video-",
|
||||
"-TTS-", "parakeet", "test-")
|
||||
"-TTS-", "parakeet", "/test-")
|
||||
|
||||
GB = 1 << 30
|
||||
|
||||
@@ -219,6 +222,8 @@ MEMORY_OVERHEAD = GB
|
||||
# anything. Enough for the smallest whisper models and for a sub-billion
|
||||
# cleanup model, which is what such a machine can run.
|
||||
MEMORY_FLOOR = GB // 2
|
||||
# What total_memory() read the one time it asked. None until it has.
|
||||
_MEMORY = None
|
||||
|
||||
|
||||
class LocalError(Exception):
|
||||
@@ -713,6 +718,28 @@ def total_memory():
|
||||
Zero is a real answer and not a failure: every caller treats an unknown
|
||||
machine as one big enough for whatever it is looking at, because a wrong
|
||||
"too big" is worse advice than none.
|
||||
|
||||
Read once and kept. The memory in a machine does not change while Dikte
|
||||
runs, and a list of thirty rows asks this question seventy times: on the
|
||||
Mac path below, where the answer comes from a program rather than a
|
||||
library call, that was seventy processes started on the interface thread
|
||||
every time a list was drawn.
|
||||
"""
|
||||
global _MEMORY
|
||||
if _MEMORY is None:
|
||||
_MEMORY = max(_read_memory(), 0)
|
||||
return _MEMORY
|
||||
|
||||
|
||||
def _read_memory():
|
||||
"""What the system says, which on a bad day is a negative number.
|
||||
|
||||
sysconf answers -1 for a limit it holds to be indeterminate, and CPython
|
||||
hands that straight back rather than raising, so the product below can
|
||||
come out negative. The caller floors it at zero, which is the answer for
|
||||
a machine nothing could be read from: a 64 GB workstation whose sysconf
|
||||
shrugged was otherwise being told every model past 512 MB was too big
|
||||
for it.
|
||||
"""
|
||||
try:
|
||||
return os.sysconf("SC_PAGE_SIZE") * os.sysconf("SC_PHYS_PAGES")
|
||||
|
||||
Reference in New Issue
Block a user