4.1 Working with clients on requirements

Key points

1. A stakeholder is anyone affected by the project, such as the client, users and operators. Agreeing on goals first sets the requirements for everything else. Then you define who owns what, and the roles. MLOps means machine learning operations.

   What NVIDIA says (1)

   “Align stakeholders on the goals, create an organizational structure that defines who owns what, then define responsibilities and roles”

   — What is MLOps?

2. A model card summarizes what a model does and its limits. Writing for clients and users helps agree on fitness for use.

   What NVIDIA says (1)

   “They are built to communicate to nontechnical stakeholders, customers using the software, and even students.”

   — Enhancing AI Transparency and Ethical Considerations with Model Card++

3. ISL and OSL mean input and output sequence length. Longer inputs raise time to first token (TTFT). Longer outputs raise the time between tokens. Knowing the mix lets you size the deployment.

   What NVIDIA says (2)

   “A longer ISL will increase the memory requirement for the prefill stage and thus increase the TTFT.”

   — LLM Inference Benchmarking: Fundamental Concepts

   “It is important to understand the distribution of inputs and outputs in your LLM deployment to best optimize your hardwar”

   — LLM Inference Benchmarking: Fundamental Concepts

4. The primary modality is chosen from the focus of the application. Here the focus is text Q&A, so images get text descriptions. RAG means retrieval-augmented generation.

   What NVIDIA says (1)

   “Another option is to pick a primary modality based on the focus of the application and ground all other modalities in the primary modality.”

   — An Easy Introduction to Multimodal Retrieval-Augmented Generation

Key terms

• Model card: A short document that describes a model, its intended use and its limits for developers and users.
• MLOps: Practices for building, deploying and tracking AI models reliably.
• Stakeholder: Anyone affected by an AI project, such as the client, users and operators.

Sample question

Gartner, as quoted by NVIDIA, gives three first steps for starting MLOps. What is the first?