compare llama-bench: add option to plot

[am17an]

Creates graph with varying one variable (default: n_depth) and keeping others constant, added --plot (for plot location) and --plot_x (for x-axis) in the script

example graph

[JohannesGaessler]

Thank you, this would be useful for my work. For more general use consider also adding support for plotting the performance of only a single commit. Non-developer users are frequently more interested in determining the performance of their setup rather than the performance of the software so they would be I think more interested in running llama-bench for only a single commit and plotting the performance of that. In practical terms the interface change that I envision would be to make --compare optional. The opposite case where you may want to compare more than two commits could also be interesting. (These things should be in separate PRs.)

[JohannesGaessler]

Regarding capitalization: I'm going by the standards that are common in my line of work unrelated to llama.cpp (high energy physics) where the guideline is not to capitalize every word in plot titles and axis names. Because the PRETTY_NAMES are capitalized this is going to be inconsistent with a join on ", " but this is I think a minor issue and would not block a merge.

[CISC]

Thank you, this would be useful for my work. For more general use consider also adding support for plotting the performance of only a single commit. Non-developer users are frequently more interested in determining the performance of their setup rather than the performance of the software so they would be I think more interested in running llama-bench for only a single commit and plotting the performance of that. In practical terms the interface change that I envision would be to make --compare optional. The opposite case where you may want to compare more than two commits could also be interesting. (These things should be in separate PRs.)

Just FYI, I haven't forgotten about adding Mermaid support, but I was working on comparing across columns first (ie, comparing two different models or options) and got stuck in SQL hell so it got stashed and put on the back burner. :(

[JohannesGaessler]

Sorry, but I'm not going to turn down this PR when it does what I need with the tools that I'm used to.

[CISC]

Sorry, but I'm not going to turn down this PR when it does what I need with the tools that I'm used to.

Oh, no, I wasn't suggesting that, this PR looks great!

[JohannesGaessler]

Test plot I made for a currently open PR:

I manually edited the code to get a log scale.

[JohannesGaessler]

If there's nothing else from your side I'll merge.