will ./server support parallel decoding as well? #3693
Description
Expected Behavior
The parallel examples look promising. I'm wondering if ./server will also support an -np argument to process requests in parallel. This way, the user can send np prompts at a time.
Current Behavior
Currently, ./server processes requests sequentially.
Environment and Context
macOS Sonoma, M1 Pro chip