A Spark job is taking longer than expected. Using the Spark UI, a data engineer notes that the Min, Median, and Max Durations for tasks in a particular stage show the minimum and median time to complete a task as roughly the same, but the max duration for a task to be roughly 100 times as long as the minimum.
Which situation is causing increased duration of the overall job?

• A. Spill resulting from attached volume storage being too small.
• B. Credential validation errors while pulling data from an external system.
• C. Network latency due to some cluster nodes being in different regions from the source data
• D. Skew caused by more data being assigned to a subset of spark-partitions.
• E. Task queueing resulting from improper thread pool assignment.

A data engineering team uses Databricks Lakehouse Monitoring to track the percent_null metric for a critical column in their Delta table.
The profile metrics table (prod_catalog.prod_schema.customer_data_profile_metrics) stores hourly percent_null values.
The team wants to:
Trigger an alert when the daily average of percent_null exceeds 5% for
three consecutive days.
Ensure that notifications are not spammed during sustained issues.

• A. SELECT SUM(CASE WHEN percent_null > 5 THEN 1 ELSE 0 END) AS violation_days FROM prod_catalog.prod_schema.customer_data_profile_metrics WHERE window.end >= CURRENT_TIMESTAMP - INTERVAL '3' DAY Alert Condition: violation_days >= 3 Notification Frequency: Just once
• B. WITH daily_avg AS (
  SELECT DATE_TRUNC('DAY', window.end) AS day,
  AVG(percent_null) AS avg_null
  FROM prod_catalog.prod_schema.customer_data_profile_metrics
  GROUP BY DATE_TRUNC('DAY', window.end)
  )
  SELECT day, avg_null
  FROM daily_avg
  ORDER BY day DESC
  LIMIT 3
  Alert Condition: ALL avg_null > 5 for the latest 3 rows
  Notification Frequency: Just once
• C. SELECT AVG(percent_null) AS daily_avg
  FROM prod_catalog.prod_schema.customer_data_profile_metrics
  WHERE window.end >= CURRENT_TIMESTAMP - INTERVAL '3' DAY
  Alert Condition: daily_avg > 5
  Notification Frequency: Each time alert is evaluated
• D. SELECT percent_null
  FROM prod_catalog.prod_schema.customer_data_profile_metrics
  WHERE window.end >= CURRENT_TIMESTAMP - INTERVAL '1' DAY
  Alert Condition: percent_null > 5
  Notification Frequency: At most every 24 hours

The data governance team has instituted a requirement that the "user" table containing Personal Identifiable Information (PII) must have the appropriate masking on the SSN column. This means that anyone outside of the HRAdminGroup should see masked social security numbers as ***-**-
****.
The team created a masking function:

What does the data governance team need to do next to achieve this goal?

• A. CREATE TABLE users
  (name STRING, int STRING);
  ALTER TABLE users ALTER COLUMN ssn CREATE MASK if is_member('HRAdminGroup');
• B. CREATE TABLE users
  (name STRING, ssn STRING);
  ALTER TABLE users ALTER COLUMN ssn SET MASK ssn_mask;
• C. CREATE TABLE users
  (name STRING);
  ALTER TABLE users CREATE COLUMN ssn CREATE MASK ssn_mask;
• D. CREATE TABLE users
  (name STRING, ssn INT MASKED ssn_mask);

A company processes semi-structured JSON files from an external source using Auto Loader in a classic Databricks job. Occasionally, records arrive with null critical fields, invalid types, or unexpected nested schema variations. The engineer must ensure that malformed or non- conforming records are not dropped silently and are captured in a separate quarantine table. The pipeline should continue processing good records into the Bronze layer without failing the job, and the approach must support both batch and streaming ingestion.
The data engineer needs to build a robust ingestion pattern that automatically routes bad records to a quarantine Delta table, while still ingesting good records into the Bronze layer for further processing.
Which approach fulfills the quarantine mechanism in this ingestion architecture?

• A. Use Auto Loader with failFast mode to set to false, and enable schema evolution; invalid records will be silently ignored during ingestion.
• B. Use Auto Loader with LDP and implement an EXPECT () constraint with a record audit logic to route bad records.
• C. Use Lakeflow Spark Declarative Pipelines with a SQL pipeline; configure it to drop rows with nulls using where critical_fields is not null, and rely on audit logs for malformed data.
• D. Create a notebook job with inferSchema=True, write a streaming query with .foreachBatch() and catch exceptions using try/except to redirect failed batches to quarantine.