Track llama.cpp performance with Bencher

[epompeii]

There are several places in the llama.cpp project where benchmarks are tracked in markdown tables.

For example:

• Performance of llama.cpp on Apple Silicon M-series #4167
• ggml : x2 speed for WASM by optimizing SIMD #11453

Would there be any interest in tracking performance over time using Bencher: https://github.com/bencherdev/bencher

Bencher supports the concept of Testbeds, so the benchmark results for each backend would stay separate from one another.
I'm the creator and maintainer of Bencher, so I'm more than happy to help answer any questions if this is of interest.

[slaren]

llama-bench supports outputting the benchmark results in multiple formats. It could be extended so that it supports exporting the results in a format that Bencher can read.