llama : add DeepSeek-v2-Chat support

[DirtyKnightForVi]

please support deepseek-ai/DeepSeek-V2-Chat

https://huggingface.co/deepseek-ai/DeepSeek-V2-Chat

[SinanAkkoyun]

That would be awesome.

[jeff31415]

Impressive model, and potentially a CPU friendly model(if you have >96GB memory)

[SinanAkkoyun]

@ggerganov I'd be very interested in helping, I want to get into porting models to inference engines

Would you be so kind to provide a rough outline of what needs to be done here? I'd then submit a draft PR and ask for little details that don't work

[ggerganov]

Interesting - can we get a rundown of the multi-head latent KV cache technique:

@SinanAkkoyun Look at PRs that have already been merged and add support for new model arches

[DirtyKnightForVi]

Sure thing. Here's their tech report: https://github.com/deepseek-ai/DeepSeek-V2/blob/main/deepseek-v2-tech-report.pdf

[ggerganov]

Thanks, very cool work! Adding this to the roadmap to give it more visibility

[taozhiyuai]

+1

[fairydreaming]

I'm working on it right now: https://youtu.be/1AG-GUtDvaw
The code needs some cleanup, so it's not published yet.

[SinanAkkoyun]

@fairydreaming Oh wow how awesome!! How does the ppl look?