InferencevLLM: Keeping AI answers on time can matter more than generating more tokens
Separating long-prompt processing from answer generation can prevent one request from slowing everyone else. The trade is extra transfer time and a more complex service.