lhl

@lhl.bsky.social

488 Followers 327 Following

Easily distracted, currently building open source AI. Living online since FidoNet

A checklist of everything shisad now does to ensure that its supply-chain and deployment is fully hardened.
New table of Shaberi scores (GPT-4.1 judge)
Shisa V2 405B 日本語上手！
JA MT-Bench Radar Charts: Our post-training has improved the Japanese performance of the Llama 3.1 405B Instruct base model across all evaluation categories. Shisa V2 405B's JA MT-Bench scores are competitive with the flagship models published by both leading US and Chinese Frontier Labs.
Shisa V2 405B scores above GPT-4o latest in JA MT-Bench
PyTorch FA perf on Strix Halo (gfx1151) is quite awful.
DPO mini-sweep. The calculated-scaled LR did end up being the best overall performer.
Chart showing JA + EN performance of Shisa V2 and a new 405B FFT vs others
Each iteration does slightly better on accuracy and reward margins
SimPO vs DPO training curves
Funny meme (sorry too much to type but if you can send to a VLM it can explain it)
SGLang W8A8 (Marlin) vs MLC q4fp16_1 throughput and latency
LLM Inference Efficiency Chart
Table of 128GB RAM inference AIO options (+ Nvidia Jetson for reference)
DeepSeek v3 Performance on vLLM and SGLang: sglang beats the pants off of vllm on DeepSeek-v3 atm.
W7900 vs RTX 3090 Performance - speculative decoding improves throughput for both by >50%. The RTX 3090 is about 60% higher and it's non-SD perf actually beats the W7900 w/ SD perf. Mean TTFT remains about the same w/ or w/o SD. The RTX 3090 is about 2.5X faster.
pp512/tg128 t/s vs Power Limit for RTX 3090, very little tg128 dropoff from 420-320W
Chart of Llama 3 8B performance showing llama.cpp to still run bs=1 inferencing faster than vLLM. MLC's q4f16_1 quants run even faster, although llama.cpp has a lower TTFT