Show HN: Janus – Go binary that runs GGUF models via Vulkan on AMD/Intel/Nvidia

 (github.com)

38 points | by Maverick617 2 hours ago

3 comments

  • peddling-brink 36 minutes ago
    > llama.cpp via Vulkan (AMD / Intel / NVIDIA) or CPU fallback

    I got excited about someone paying attention to intel. Oh well.

  • dlcarrier 35 minutes ago
    From what I've seen, Vulkan adds a lot of overhead on Intel hardware.
  • PcChip 1 hour ago
    I didn't see any benchmarks against vllm, sglang, exllama, etc
    [-]
    • rancor 1 hour ago
      Since this is basically a wrapper around libllama.so, I would assume that the performance is roughly the same as llama.cpp upstream.