Databricks-Certified-Professional-Data-Engineer試験無料問題集「Databricks Certified Professional Data Engineer 認定」

A. Network I/O never spikes

B. CPU Utilization is around 75%

C. Bytes Received never exceeds 80 million bytes per second

D. The five Minute Load Average remains consistent/flat

E. Total Disk Space remains constant

A. All Delta Lake transactions are ACID compliance against a single table, and Databricks does not enforce foreign key constraints.

B. Foreign keys must reference a primary key field; multi-table inserts must leverage Delta Lake's upsert functionality.

C. Databricks only allows foreign key constraints on hashed identifiers, which avoid collisions in highly- parallel writes.

D. Committing to multiple tables simultaneously requires taking out multiple table locks and can lead to a state of deadlock.

A. Maintain data quality rules in a Delta table outside of this pipeline's target schema, providing the schema name as a pipeline parameter.

B. Maintain data quality rules in a separate Databricks notebook that each DLT notebook of file.

C. Use global Python variables to make expectations visible across DLT notebooks included in the same pipeline.

D. Add data quality constraints to tables in this pipeline using an external job with access to pipeline configuration files.

A. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 1 GB.

B. Optimized writes use logical partitions instead of directory partitions; because partition boundaries are only represented in metadata, fewer small files are written.

C. Data is queued in a messaging bus instead of committing data directly to memory; all data is committed from the messaging bus in one batch once the job is complete.

D. Before a Jobs cluster terminates, optimize is executed on all tables modified during the most recent job.

E. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 128 MB.

A. New files are identified by listing the input directory; the target table is materialized by directory querying all valid files in the source directory.

B. Cloud vendor-specific queue storage and notification services are configured to track newly arriving files; new files are incrementally and impotently into the target Delta Lake table.

C. Webhook trigger Databricks job to run anytime new data arrives in a source directory; new data automatically merged into target tables using rules inferred from the data.

D. New files are identified by listing the input directory; new files are incrementally and idempotently loaded into the target Delta Lake table.

A. Set spark.sql.adaptive.advisoryPartitionSizeInBytes to 512 MB bytes, ingest the data, execute the narrow transformations, coalesce to 2,048 partitions (1TB*1024*1024/512), and then write to parquet.

B. Set spark.sql.shuffle.partitions to 512, ingest the data, execute the narrow transformations, and then write to parquet.

C. Set spark.sql.shuffle.partitions to 2,048 partitions (1TB*1024*1024/512), ingest the data, execute the narrow transformations, optimize the data by sorting it (which automatically repartitions the data), and then write to parquet.

D. Ingest the data, execute the narrow transformations, repartition to 2,048 partitions (1TB* 1024*1024
/512), and then write to parquet.

E. Set spark.sql.files.maxPartitionBytes to 512 MB, ingest the data, execute the narrow transformations, and then write to parquet.

A. The trigger interval cannot be modified without modifying the checkpoint directory; to maintain the current stream state, increase the number of shuffle partitions to maximize parallelism.

B. Increase the trigger interval to 30 seconds; setting the trigger interval near the maximum execution time observed for each batch is always best practice to ensure no records are dropped.

C. Decrease the trigger interval to 5 seconds; triggering batches more frequently allows idle executors to begin processing the next batch while longer running tasks from previous batches finish.

D. Decrease the trigger interval to 5 seconds; triggering batches more frequently may prevent records from backing up and large batches from causing spill.

E. Use the trigger once option and configure a Databricks job to execute the query every 10 seconds; this ensures all backlogged records are processed with each batch.

A. "event_time"

B. window("event_time", "10 minutes").alias("time")

C. to_interval("event_time", "5 minutes").alias("time")

D. lag("event_time", "10 minutes").alias("time")

E. window("event_time", "5 minutes").alias("time")

A. Set the skipChangeCommits flag to true on bpm_stats

B. Set the SkipChangeCommits flag to true raw_lot

C. Set the pipelines, reset, allowed property to false on bpm_stats

D. Set the pipelines, reset, allowed property to false on raw_iot