DP-750 Exam Dumps - Implementing Data Engineering Solutions Using Azure Databricks

The correct answer is D. Breaking the pipeline into separate tasks for ingestion, cleansing, and curation is the foundation of well-designed Lakeflow Jobs pipelines. Each task should own one responsibility — when a task does too much, debugging a failure becomes a hunt through unrelated code, and retry logic becomes expensive because you re-execute work that already succeeded.

Contoso ' s planned changes explicitly call for ' a clear execution order and dependencies ' and ' orchestrate multi-step ingestion and transformation workflows. ' Separate tasks map directly to those goals: Lakeflow Jobs tracks each task ' s status independently, so if cleansing fails, ingestion doesn ' t re-run.

Option A bundles everything into one notebook, which means a curation bug forces a full re-ingestion. Option B copies logic three times — any future change must be applied in triplicate, which is a maintenance hazard. Option C forces everything through SQL MERGE, which is the wrong tool for raw-event ingestion and doesn ' t address cleansing or schema drift.

The symptoms indicate data skew. Spark partitions the stage’s work into tasks, but a few partitions contain substantially more records than the others. Tasks processing the smaller partitions finish quickly, leaving the cluster underutilized while the stage waits for the oversized partitions. Caching would concern whether reused data is stored in memory and does not inherently explain an uneven distribution of input records. Shuffling transfers data between executors and can be expensive, but the decisive evidence is that only a few tasks process far more records. Spilling occurs when an executor cannot keep intermediate data in memory and writes it to disk; the scenario does not identify high spill volumes. Examining per-task input sizes in the Spark UI would confirm the skewed partition distribution.

To record historical changes: Implement a Type 2 slowly changing dimension (SCD).

To support temporal analysis: Use Delta Lake time travel.

A Type 2 slowly changing dimension preserves customer-profile history by inserting a new record whenever a tracked attribute changes instead of overwriting the existing record. Effective dates, expiration dates, version values, or current-record indicators can identify which version applied during a particular period. This provides persistent business history for long-term analysis. Delta Lake time travel supports temporal analysis of the physical table by allowing users to query an earlier version with VERSION AS OF or TIMESTAMP AS OF. Time travel is useful for auditing and reproducing previous results, but its availability depends on retained Delta log entries and data files. Therefore, it should not replace a Type 2 SCD for permanent customer history. Together, the two features satisfy the historical-record and earlier-version requirements.

Enable audit logging for: The Databricks account

Send audit logs to: Log Analytics

Enabling audit logging at the Databricks account level provides centralized coverage of account and workspace activities, including Unity Catalog data-access events. Enabling logging only for Workspace1 would provide a narrower configuration and would not offer the same centralized account-level auditing model. Individual Unity Catalog tables and the Azure subscription are not the appropriate Databricks audit-log scope selections. Log Analytics is the suitable destination because it supports rapid ingestion and immediate querying of log records through Azure Monitor Logs. Auditors can search, filter, correlate, and analyze the captured events without first developing a separate data-processing solution. An Azure Storage account is more suitable for inexpensive archival, while Event Hubs is primarily a streaming transport that normally requires an additional consumer before investigators can analyze the records.

The correct answer is A — Yes.

df.filter(df.order_amount.isNotNull()) is the correct PySpark pattern for excluding null rows. The isNotNull() method is a Column method that returns True for every row where order_amount has a value and False for rows where it is null. Spark ' s filter keeps only the rows where the condition evaluates to True, producing a DataFrame with all null order_amount rows removed.

This works correctly because isNotNull() is explicitly null-aware — unlike the != None comparison in Q52, it doesn ' t rely on Python equality semantics. Under the hood it maps to the SQL expression order_amount IS NOT NULL, which is unambiguous in both SQL and Spark.

Both df.filter(df.order_amount.isNotNull()) and df.dropna(subset=[ ' order_amount ' ]) produce identical results. The choice between them is stylistic — isNotNull() reads more explicitly as a filter condition, while dropna is more compact when handling multiple columns.

The correct answer is B — No.

A Databricks Access Connector is an Azure resource that provides a managed identity for Databricks to authenticate to Azure Storage services. It is an authentication bridge — nothing more. Creating one doesn ' t register any external database, doesn ' t create any Unity Catalog objects representing DB1, and doesn ' t enable querying of SQL Server data from Databricks.

An Access Connector is a prerequisite for setting up storage credentials and external locations in Unity Catalog (to access ADLS Gen2, for example), but it has no role in Lakehouse Federation or foreign catalog creation. It cannot expose an external database ' s schemas and tables in Unity Catalog alongside native objects.

The correct solution for this scenario is a foreign catalog (Q47), which requires a connection object pointing to DB1 — not an Access Connector.

The expect_or_fail decorator validates every incoming record against the expression amount > 0 and stops the table update when a violation occurs. This exactly meets the requirement that invalid data cause the pipeline update to fail. Option A applies multiple expectations and drops violating records instead of failing the update. Option B also discards invalid records while allowing valid records to continue processing. Option D records expectation metrics but retains invalid records and allows the update to continue. These alternative behaviors are useful when invalid data can be tolerated, quarantined, or monitored, but they do not satisfy strict enforcement. The fail action provides transactional protection because the affected update does not successfully publish data that violates the stated quality rule.

Diagnosing OOM errors requires analysing actual runtime execution behaviour. The Spark UI is the primary tool — it ' s accessible from the cluster detail page and captures rich per-stage and per-task metrics without any extra setup.

In the Executors tab, look at storage memory used, execution memory used, memory spill to disk, and GC time per executor. An executor showing high memory spill is a strong indicator that it ' s processing more data than it can hold in memory — often caused by data skew, where one partition is far larger than the others.

The Stages tab shows task distribution — if one task in a stage is processing 10x more data than its peers, that ' s data skew causing memory pressure on that specific executor. Ganglia (available on older runtimes) provides node-level OS metrics like heap usage over time, which can confirm whether memory pressure is sustained or spiky.

These built-in tools give a complete picture of the root cause before making any configuration changes.