AI-300 Exam Dumps - Operationalizing Machine Learning and Generative AI Solutions

Detect when the system is slow at returning output: Latency

Identify when the system cannot sustain expected request volume: Throughput

Measure the duration of handling a request: Response time

Latency measures delay experienced while waiting for model output and is therefore the appropriate metric for identifying degradation in perceived responsiveness. Microsoft describes latency as the time required to obtain a response from the model and exposes metrics such as Time to Response/Time to First Token and Time Between Tokens for generative AI workloads. An increase indicates that users are waiting longer for output to begin or continue.

Throughput represents the amount of workload the system can process over a unit of time. Microsoft describes system-level throughput in terms such as requests per minute and tokens per minute. Consequently, throughput is the correct metric when determining whether a deployment can sustain the required request volume as demand increases.

Response time represents the overall duration required to handle an individual request. It is appropriate for measuring end-to-end request processing and validating request-duration SLOs. Microsoft specifically includes latency, throughput, and response times among the performance metrics candidates are expected to monitor for generative AI applications.

Concurrency measures simultaneous active requests and can influence capacity and latency, but none of these requirements directly asks for simultaneous-request count.

Study Guide Reference: Implement generative AI quality assurance and observability — continuous monitoring, latency, throughput, response times, production troubleshooting, and SLO monitoring.

Azure OpenAI Service fine-tuning requires training data in JSON Lines (JSONL) format — a text file where each line is a self-contained, valid JSON object representing one training example. For chat fine-tuning, each line contains a messages array with system, user, and assistant turns. JSONL is ideal for machine-learning datasets because it is streamable, easy to generate programmatically, and directly supported by the Azure OpenAI fine-tuning API. CSV (option A) and TSV (option B) are tabular formats that cannot natively represent the nested, multi-turn conversation structure required by chat models. Plain JSON (option D) would require loading the entire file as a single object, which is impractical for large datasets and is not accepted by the Azure OpenAI fine-tuning endpoint. Always use JSONL with UTF-8 encoding for Azure OpenAI training files.

Microsoft Learn Reference Topic: Prepare training data for Azure OpenAI fine-tuning – JSONL file format

Model version: A version of a registered model. When a new model is added to the Model Registry, it is added as Version 1. Each model registered to the same model name increments the version number.