Databricks-Certified-Professional-Data-Engineer問題集合格保証付きの合格できるDatabricks-Certified-Professional-Data-Engineer試験2026年更新 [Q30-Q54]

Databricks-Certified-Professional-Data-Engineer問題集合格保証付きの合格できるDatabricks-Certified-Professional-Data-Engineer試験2026年更新

Databricks-Certified-Professional-Data-Engineer試験問題集を試そう！ベストDatabricks-Certified-Professional-Data-Engineer試験問題トレーニングを提供していますGoShiken

試験は、Databricksを使用して大規模なデータ処理パイプラインを設計、構築、および維持することに熟練したデータエンジニアが自分の専門知識を証明したい場合に最適です。認定試験はデータエンジニアの特定の役割と責任に合わせて調整されており、データインジェスション、データ変換、データストレージ、およびデータ分析などのトピックをカバーしています。試験に合格することで、候補者はDatabricksを使用して複雑なビッグデータ課題を解決する能力を証明できます。

Databricks認定プロフェッショナルデータエンジニア試験は、特にDatabricksプラットフォームでビッグデータテクノロジーを扱う専門家のスキルと知識を検証する認定プログラムです。この試験は、データパイプラインを設計、構築、および維持し、機械学習ワークフローを実装し、DataBricksプラットフォームでパフォーマンスを最適化する候補者の能力をテストするように設計されています。この認定は、データエンジニア、データアーキテクト、およびこの分野での専門知識を実証したいビッグデータの専門家に最適です。

 

質問 # 30
The Databricks workspace administrator has configured interactive clusters for each of the data engineering groups. To control costs, clusters are set to terminate after 30 minutes of inactivity. Each user should be able to execute workloads against their assigned clusters at any time of the day.
Assuming users have been added to a workspace but not granted any permissions, which of the following describes the minimal permissions a user would need to start and attach to an already configured cluster.

• A. "Can Restart" privileges on the required cluster
• B. "Can Manage" privileges on the required cluster
• C. Cluster creation allowed. "Can Restart" privileges on the required cluster
• D. Cluster creation allowed. "Can Attach To" privileges on the required cluster
• E. Workspace Admin privileges, cluster creation allowed. "Can Attach To" privileges on the required cluster

正解：A

解説：
Explanation
https://learn.microsoft.com/en-us/azure/databricks/security/auth-authz/access-control/cluster-acl
https://docs.databricks.com/en/security/auth-authz/access-control/cluster-acl.html

質問 # 31
The data architect has decided that once data has been ingested from external sources into the Databricks Lakehouse, table access controls will be leveraged to manage permissions for all production tables and views.
The following logic was executed to grant privileges for interactive queries on a production database to the core engineering group.
GRANT USAGE ON DATABASE prod TO eng;
GRANT SELECT ON DATABASE prod TO eng;
Assuming these are the only privileges that have been granted to the eng group and that these users are not workspace administrators, which statement describes their privileges?

• A. Group members are able to query all tables and views in the prod database, but cannot create or edit anything in the database.
• B. Group members are able to create, query, and modify all tables and views in the prod database, but cannot define custom functions.
• C. Group members have full permissions on the prod database and can also assign permissions to other users or groups.
• D. Group members are able to list all tables in the prod database but are not able to see the results of any queries on those tables.
• E. Group members are able to query and modify all tables and views in the prod database, but cannot create new tables or views.

正解：A

解説：
The GRANT USAGE ON DATABASE prod TO eng command grants the eng group the permission to use the prod database, which means they can list and access the tables and views in the database. The GRANT SELECT ON DATABASE prod TO eng command grants the eng group the permission to select data from the tables and views in the prod database, which means they can query the data using SQL or DataFrame API.
However, these commands do not grant the eng group any other permissions, such as creating, modifying, or deleting tables and views, or defining custom functions. Therefore, the eng group members are able to query all tables and views in the prod database, but cannot create or edit anything in the database. References:
* Grant privileges on a database: https://docs.databricks.com/en/security/auth-authz/table-acls/grant- privileges-database.html
* Privileges you can grant on Hive metastore objects: https://docs.databricks.com/en/security/auth-authz
/table-acls/privileges.html

質問 # 32
Which configuration parameter directly affects the size of a spark-partition upon ingestion of data into Spark?

• A. spark.sql.adaptive.coalescePartitions.minPartitionNum
• B. spark.sql.autoBroadcastJoinThreshold
• C. spark.sql.adaptive.advisoryPartitionSizeInBytes
• D. spark.sql.files.openCostInBytes
• E. spark.sql.files.maxPartitionBytes

正解：E

解説：
This is the correct answer because spark.sql.files.maxPartitionBytes is a configuration parameter that directly affects the size of a spark-partition upon ingestion of data into Spark. This parameter configures the maximum number of bytes to pack into a single partition when reading files from file-based sources such as Parquet, JSON and ORC. The default value is 128 MB, which means each partition will be roughly 128 MB in size, unless there are too many small files or only one large file. Verified Reference: [Databricks Certified Data Engineer Professional], under "Spark Configuration" section; Databricks Documentation, under "Available Properties - spark.sql.files.maxPartitionBytes" section.

質問 # 33
Which of the following describes a scenario in which a data engineer will want to use a Job cluster instead of
an all-purpose cluster?

• A. A data engineer needs to manually investigate a production error
• B. An automated workflow needs to be run every 30 minutes
• C. A data team needs to collaborate on the development of a machine learning model
• D. An ad-hoc analytics report needs to be developed while minimizing compute costs
• E. A Databricks SQL query needs to be scheduled for upward reporting

正解：B

質問 # 34
A member of the data engineering team has submitted a short notebook that they wish to schedule as part of a larger data pipeline. Assume that the commands provided below produce the logically correct results when run as presented.

Which command should be removed from the notebook before scheduling it as a job?

• A. Cmd 3
• B. Cmd 2
• C. Cmd 6
• D. Cmd 5
• E. Cmd 4

正解：C

解説：
Cmd 6 is the command that should be removed from the notebook before scheduling it as a job. This command is selecting all the columns from the finalDF dataframe and displaying them in the notebook. This is not necessary for the job, as the finalDF dataframe is already written to a table in Cmd 7. Displaying the dataframe in the notebook will only consume resources and time, and it will not affect the output of the job.
Therefore, Cmd 6 is redundant and should be removed.
The other commands are essential for the job, as they perform the following tasks:
* Cmd 1: Reads the raw_data table into a Spark dataframe called rawDF.
* Cmd 2: Prints the schema of the rawDF dataframe, which is useful for debugging and understanding the data structure.
* Cmd 3: Selects all the columns from the rawDF dataframe, as well as the nested columns from the values struct column, and creates a new dataframe called flattenedDF.
* Cmd 4: Drops the values column from the flattenedDF dataframe, as it is no longer needed after flattening, and creates a new dataframe called finalDF.
* Cmd 5: Explains the physical plan of the finalDF dataframe, which is useful for optimizing and tuning the performance of the job.
* Cmd 7: Writes the finalDF dataframe to a table called flat_data, using the append mode to add new data to the existing table.

質問 # 35
A member of the data engineering team has submitted a short notebook that they wish to schedule as part of a larger data pipeline. Assume that the commands provided below produce the logically correct results when run as presented.

Which command should be removed from the notebook before scheduling it as a job?

• A. Cmd 3
• B. Cmd 2
• C. Cmd 6
• D. Cmd 5
• E. Cmd 4

正解：C

解説：
Cmd 6 is the command that should be removed from the notebook before scheduling it as a job. This command is selecting all the columns from the finalDF dataframe and displaying them in the notebook. This is not necessary for the job, as the finalDF dataframe is already written to a table in Cmd 7. Displaying the dataframe in the notebook will only consume resources and time, and it will not affect the output of the job. Therefore, Cmd 6 is redundant and should be removed.
The other commands are essential for the job, as they perform the following tasks:
Cmd 1: Reads the raw_data table into a Spark dataframe called rawDF.
Cmd 2: Prints the schema of the rawDF dataframe, which is useful for debugging and understanding the data structure.
Cmd 3: Selects all the columns from the rawDF dataframe, as well as the nested columns from the values struct column, and creates a new dataframe called flattenedDF.
Cmd 4: Drops the values column from the flattenedDF dataframe, as it is no longer needed after flattening, and creates a new dataframe called finalDF.
Cmd 5: Explains the physical plan of the finalDF dataframe, which is useful for optimizing and tuning the performance of the job.
Cmd 7: Writes the finalDF dataframe to a table called flat_data, using the append mode to add new data to the existing table.

質問 # 36
How do you upgrade an existing workspace managed table to a unity catalog table?

• A. Create table table_name format = UNITY as select * from old_table_name
• B. Create or replace table_name format = UNITY using deep clone old_table_name
• C. Create table catalog_name.schema_name.table_name
  as select * from hive_metastore.old_schema.old_table
• D. Create table table_name as select * from hive_metastore.old_schema.old_table
• E. ALTER TABLE table_name SET UNITY_CATALOG = TRUE

正解：C

解説：
Explanation
The answer is Create table catalog_name.schema_name.table_name as select * from hive_metastore.old_schema.old_table Basically, we are moving the data from an internal hive metastore to a metastore and catalog that is registered in the Unity catalog.
note: if it is a managed table the data is copied to a different storage account, for a large tables this can take a lot of time. For an external table the process is different.
Managed table: Upgrade a managed to Unity Catalog
External table: Upgrade an external table to Unity Catalog

質問 # 37
Assuming that the Databricks CLI has been installed and configured correctly, which Databricks CLI command can be used to upload a custom Python Wheel to object storage mounted with the DBFS for use with a production job?

• A. configure
• B. fs
• C. workspace
• D. libraries
• E. jobs

正解：D

解説：
The libraries command group allows you to install, uninstall, and list libraries on Databricks clusters. You can use the libraries install command to install a custom Python Wheel on a cluster by specifying the --whl option and the path to the wheel file. For example, you can use the following command to install a custom Python Wheel named mylib-0.1-py3-none-any.whl on a cluster with the id 1234-567890-abcde123:
databricks libraries install --cluster-id 1234-567890-abcde123 --whl dbfs:/mnt/mylib/mylib-0.1-py3-none-any.
whl
This will upload the custom Python Wheel to the cluster and make it available for use with a production job.
You can also use the libraries uninstall command to uninstall a library from a cluster, and the libraries list command to list the libraries installed on a cluster.
References:
* Libraries CLI (legacy): https://docs.databricks.com/en/archive/dev-tools/cli/libraries-cli.html
* Library operations: https://docs.databricks.com/en/dev-tools/cli/commands.html#library-operations
* Install or update the Databricks CLI: https://docs.databricks.com/en/dev-tools/cli/install.html

質問 # 38
The Delta Live Tables Pipeline is configured to run in Development mode using the Triggered Pipeline Mode.
what is the expected outcome after clicking Start to update the pipeline?

• A. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped
• B. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist after the pipeline is stopped to allow for additional development and testing
• C. All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated
• D. All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional development and testing
• E. All datasets will be updated continuously and the pipeline will not shut down. The compute resources will persist with the pipeline

正解：E

解説：
Explanation
The answer is All datasets will be updated once and the pipeline will shut down. The compute re-sources will persist to allow for additional testing.
DLT pipeline supports two modes Development and Production, you can switch between the two based on the stage of your development and deployment lifecycle.
Development and production modes
When you run your pipeline in development mode, the Delta Live Tables system:
*Reuses a cluster to avoid the overhead of restarts.
*Disables pipeline retries so you can immediately detect and fix errors.
In production mode, the Delta Live Tables system:
*Restarts the cluster for specific recoverable errors, including memory leaks and stale credentials.
*Retries execution in the event of specific errors, for example, a failure to start a cluster.
Use the buttons in the Pipelines UI to switch between develop-ment and production modes. By default, pipelines run in development mode.
Switching between development and production modes only controls cluster and pipeline execution behavior.
Storage locations must be configured as part of pipeline settings and are not affected when switching between modes.
Please review additional DLT concepts using below link
https://docs.databricks.com/data-engineering/delta-live-tables/delta-live-tables-concepts.html#delta-live-tables-c

質問 # 39
Each configuration below is identical to the extent that each cluster has 400 GB total of RAM, 160 total cores and only one Executor per VM.
Given a job with at least one wide transformation, which of the following cluster configurations will result in maximum performance?

• A. * Total VMs: 4
  * 100 GB per Executor
  * 40 Cores/Executor
• B. * Total VMs: 8
  * 50 GB per Executor
  * 20 Cores / Executor
• C. * Total VMs:2
  * 200 GB per Executor
  * 80 Cores / Executor
• D. * Total VMs; 1
  * 400 GB per Executor
  * 160 Cores / Executor

正解：B

解説：
This is the correct answer because it is the cluster configuration that will result in maximum performance for a job with at least one wide transformation. A wide transformation is a type of transformation that requires shuffling data across partitions, such as join, groupBy, or orderBy. Shuffling can be expensive and time-consuming, especially if there are too many or too few partitions. Therefore, it is important to choose a cluster configuration that can balance the trade-off between parallelism and network overhead. In this case, having 8 VMs with 50 GB per executor and 20 cores per executor will create 8 partitions, each with enough memory and CPU resources to handle the shuffling efficiently. Having fewer VMs with more memory and cores per executor will create fewer partitions, which will reduce parallelism and increase the size of each shuffle block. Having more VMs with less memory and cores per executor will create more partitions, which will increase parallelism but also increase the network overhead and the number of shuffle files. Verified References: [Databricks Certified Data Engineer Professional], under "Performance Tuning" section; Databricks Documentation, under "Cluster configurations" section.

質問 # 40
A data engineer is using a Databricks SQL query to monitor the performance of an ELT job. The ELT job is triggered by a specific number of input records being ready to process. The Databricks SQL query returns the number of minutes since the job's most recent runtime. Which of the following approaches can enable the data engineering team to be notified if the ELT job has not been run in an hour?

• A. They can set up an Alert for the query to notify when the ELT job fails.
• B. They can set up an Alert for the accompanying dashboard to notify when it has not re-freshed in 60 minutes.
• C. They can set up an Alert for the query to notify them if the returned value is greater than 60.
• D. This type of alert is not possible in Databricks
• E. They can set up an Alert for the accompanying dashboard to notify them if the returned value is greater than 60.

正解：C

解説：
Explanation
The answer is, They can set up an Alert for the query to notify them if the returned value is greater than 60.
The important thing to note here is that alert can only be setup on query not on the dashboard, query can return a value, which is used if alert can be triggered.

質問 # 41
If you create a database sample_db with the statement CREATE DATABASE sample_db what will be the default location of the database in DBFS?

• A. Default Location, dbfs:/user/hive/warehouse
• B. Statement fails "Unable to create database without location"
• C. Default location, DBFS:/user/
• D. Default Storage account
• E. Default location, /user/db/

正解：A

解説：
Explanation
The Answer is dbfs:/user/hive/warehouse this is the default location where spark stores user data-bases, the default can be changed using spark.sql.warehouse.dir a parameter. You can also provide a custom location using the LOCATION keyword.
Here is how this works,
Graphical user interface, text, application, email Description automatically generated

Default location

FYI, This can be changed used using cluster spark config or session config.
Modify spark.sql.warehouse.dir location to change the default location
Graphical user interface, text, application Description automatically generated

質問 # 42
Which of the following scenarios is the best fit for AUTO LOADER?

• A. Efficiently process new data incrementally from cloud object storage
• B. Incrementally process new data from relational databases like MySQL
• C. Efficiently move data incrementally from one delta table to another delta table
• D. Incrementally process new data from streaming data sources like Kafka into delta lake
• E. Efficiently copy data from one data lake location to another data lake location

正解：A

解説：
Explanation
The answer is, Efficiently process new data incrementally from cloud object storage, AU-TO LOADER only supports ingesting files stored in a cloud object storage. Auto Loader cannot process streaming data sources like Kafka or Delta streams, use Structured streaming for these data sources.
Diagram Description automatically generated

Auto Loader and Cloud Storage Integration
Auto Loader supports a couple of ways to ingest data incrementally
1.Directory listing - List Directory and maintain the state in RocksDB, supports incremental file listing
2.File notification - Uses a trigger+queue to store the file notification which can be later used to retrieve the file, unlike Directory listing File notification can scale up to millions of files per day.
[OPTIONAL]
Auto Loader vs COPY INTO?
Auto Loader
Auto Loader incrementally and efficiently processes new data files as they arrive in cloud storage without any additional setup. Auto Loader provides a new Structured Streaming source called cloudFiles. Given an input directory path on the cloud file storage, the cloudFiles source automatically processes new files as they arrive, with the option of also processing existing files in that directory.
When to use Auto Loader instead of the COPY INTO?
*You want to load data from a file location that contains files in the order of millions or higher. Auto Loader can discover files more efficiently than the COPY INTO SQL command and can split file processing into multiple batches.
*You do not plan to load subsets of previously uploaded files. With Auto Loader, it can be more difficult to reprocess subsets of files. However, you can use the COPY INTO SQL command to reload subsets of files while an Auto Loader stream is simultaneously running.

質問 # 43
The data governance team is reviewing code used for deleting records for compliance with GDPR. They note the following logic is used to delete records from the Delta Lake table namedusers.

Assuming thatuser_idis a unique identifying key and thatdelete_requestscontains all users that have requested deletion, which statement describes whether successfully executing the above logic guarantees that the records to be deleted are no longer accessible and why?

• A. Yes; the Delta cache immediately updates to reflect the latest data files recorded to disk.
• B. No; the Delta Lake delete command only provides ACID guarantees when combined with the merge into command.
• C. No; files containing deleted records may still be accessible with time travel until a vacuum command is used to remove invalidated data files.
• D. No; the Delta cache may return records from previous versions of the table until the cluster is restarted.
• E. Yes; Delta Lake ACID guarantees provide assurance that the delete command succeeded fully and permanently purged these records.

正解：C

解説：
The code uses the DELETE FROM command to delete records from the users table that match a condition based on a join with another table called delete_requests, which contains all users that have requested deletion.
The DELETE FROM command deletes records from a Delta Lake table by creating a new version of the table that does not contain the deleted records. However, this does not guarantee that the records to be deleted are no longer accessible, because Delta Lake supports time travel, which allows querying previous versions of the table using a timestamp or version number. Therefore, files containing deleted records may still be accessible with time travel until a vacuum command is used to remove invalidated data files from physical storage.
Verified References: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "Delete from a table" section; Databricks Documentation, under "Remove files no longer referenced by a Delta table" section.

質問 # 44
Which of the following is true of Delta Lake and the Lakehouse?

• A. Z-order can only be applied to numeric values stored in Delta Lake tables
• B. Views in the Lakehouse maintain a valid cache of the most recent versions of source tables at all times.
• C. Delta Lake automatically collects statistics on the first 32 columns of each table which are leveraged in data skipping based on query filters.
• D. Because Parquet compresses data row by row. strings will only be compressed when a character is repeated multiple times.
• E. Primary and foreign key constraints can be leveraged to ensure duplicate values are never entered into a dimension table.

正解：C

解説：
https://docs.delta.io/2.0.0/table-properties.html
Delta Lake automatically collects statistics on the first 32 columns of each table, which are leveraged in data skipping based on query filters1. Data skipping is a performance optimization technique that aims to avoid reading irrelevant data from the storage layer1. By collecting statistics such as min/max values, null counts, and bloom filters, Delta Lake can efficiently prune unnecessary files or partitions from the query plan1. This can significantly improve the query performance and reduce the I/O cost.
The other options are false because:
Parquet compresses data column by column, not row by row2. This allows for better compression ratios, especially for repeated or similar values within a column2.
Views in the Lakehouse do not maintain a valid cache of the most recent versions of source tables at all times3. Views are logical constructs that are defined by a SQL query on one or more base tables3. Views are not materialized by default, which means they do not store any data, but only the query definition3. Therefore, views always reflect the latest state of the source tables when queried3. However, views can be cached manually using the CACHE TABLE or CREATE TABLE AS SELECT commands.
Primary and foreign key constraints can not be leveraged to ensure duplicate values are never entered into a dimension table. Delta Lake does not support enforcing primary and foreign key constraints on tables. Constraints are logical rules that define the integrity and validity of the data in a table. Delta Lake relies on the application logic or the user to ensure the data quality and consistency.
Z-order can be applied to any values stored in Delta Lake tables, not only numeric values. Z-order is a technique to optimize the layout of the data files by sorting them on one or more columns. Z-order can improve the query performance by clustering related values together and enabling more efficient data skipping. Z-order can be applied to any column that has a defined ordering, such as numeric, string, date, or boolean values.

質問 # 45
Incorporating unit tests into a PySpark application requires upfront attention to the design of your jobs, or a potentially significant refactoring of existing code.
Which statement describes a main benefit that offset this additional effort?

• A. Troubleshooting is easier since all steps are isolated and tested individually
• B. Validates a complete use case of your application
• C. Ensures that all steps interact correctly to achieve the desired end result
• D. Improves the quality of your data
• E. Yields faster deployment and execution times

正解：D

質問 # 46
A data engineer is testing a collection of mathematical functions, one of which calculates the area under a curve as described by another function.
Which kind of the test does the above line exemplify?

• A. functional
• B. Unit
• C. Integration
• D. Manual

正解：B

解説：
A unit test is designed to verify the correctness of a small, isolated piece of code, typically a single function. Testing a mathematical function that calculates the area under a curve is an example of a unit test because it is testing a specific, individual function to ensure it operates as expected.
Reference:
Software Testing Fundamentals: Unit Testing

質問 # 47
A data engineer has created a new cluster using shared access mode with default configurations. The data engineer needs to allow the development team access to view the driver logs if needed.
What are the minimal cluster permissions that allow the development team to accomplish this?

• A. CAN ATTACH TO
• B. CAN MANAGE
• C. CAN VIEW
• D. CAN RESTART

正解：C

解説：
Databricks provides different permission levels to control access to clusters. The correct minimal permission required for viewing driver logs is CAN VIEW.
Databricks Cluster Permission Levels:
* CAN ATTACH TO:
* Allows users to attach notebooks to a cluster but does not allow them to view logs.
* Not sufficient for viewing driver logs.
* CAN MANAGE:
* Grants full control over the cluster, including starting, stopping, and editing configurations.
* Too broad for this requirement.
* CAN VIEW (Correct Answer):
* Allows users to view cluster details, logs, and status but not modify any configurations.
* Minimal required permission for viewing logs.
* CAN RESTART:
* Grants permission to restart the cluster, but does not include log access.
* Not sufficient for viewing logs.
Conclusion:
The minimal permission needed to allow the development team to view driver logs is CAN VIEW.
References:
Databricks Cluster Permissions Documentation

質問 # 48
Which of the following is the correct statement for a session scoped temporary view?

• A. Temporary views can be still accessed even if cluster is restarted
• B. Temporary views are created in local_temp database
• C. Temporary views are lost once the notebook is detached and re-attached
• D. Temporary views stored in memory
• E. Temporary views can be still accessed even if the notebook is detached and attached

正解：C

解説：
Explanation
The answer is Temporary views are lost once the notebook is detached and attached There are two types of temporary views that can be created, Session scoped and Global
*A local/session scoped temporary view is only available with a spark session, so another notebook in the same cluster can not access it. if a notebook is detached and reattached local temporary view is lost.
*A global temporary view is available to all the notebooks in the cluster, if a cluster restarts global temporary view is lost.

質問 # 49
Each configuration below is identical to the extent that each cluster has 400 GB total of RAM, 160 total cores and only one Executor per VM.
Given a job with at least one wide transformation, which of the following cluster configurations will result in maximum performance?

• A. * Total VMs: 4
  * 100 GB per Executor
  * 40 Cores/Executor
• B. * Total VMs: 8
  * 50 GB per Executor
  * 20 Cores / Executor
• C. * Total VMs:2
  * 200 GB per Executor
  * 80 Cores / Executor
• D. * Total VMs; 1
  * 400 GB per Executor
  * 160 Cores / Executor

正解：B

解説：
This is the correct answer because it is the cluster configuration that will result in maximum performance for a job with at least one wide transformation. A wide transformation is a type of transformation that requires shuffling data across partitions, such as join, groupBy, or orderBy. Shuffling can be expensive and time- consuming, especially if there are too many or too few partitions. Therefore, it is important to choose a cluster configuration that can balance thetrade-off between parallelism and network overhead. In this case, having 8 VMs with 50 GB per executor and 20 cores per executor will create 8 partitions, each with enough memory and CPU resources to handle the shuffling efficiently. Having fewer VMs with more memory and cores per executor will create fewer partitions, which will reduce parallelism and increase the size of each shuffle block.
Having more VMs with less memory and cores per executor will create more partitions, which will increase parallelism but also increase the network overhead and the number of shuffle files. Verified References:
[Databricks Certified Data Engineer Professional], under "Performance Tuning" section; Databricks Documentation, under "Cluster configurations" section.

質問 # 50
A Delta Lake table representing metadata about content posts from users has the following schema:
* user_id LONG
* post_text STRING
* post_id STRING
* longitude FLOAT
* latitude FLOAT
* post_time TIMESTAMP
* date DATE
Based on the above schema, which column is a good candidate for partitioning the Delta Table?

• A. post_id
• B. user_id
• C. post_time
• D. date

正解：D

解説：
Partitioning a Delta Lake table is a strategy used to improve query performance by dividing the table into distinct segments based on the values of a specific column. This approach allows queries to scan only the relevant partitions, thereby reducing the amount of data read and enhancing performance.
Considerations for Choosing a Partition Column:
* Cardinality:Columns with high cardinality (i.e., a large number of unique values) are generally poor choices for partitioning. High cardinality can lead to a large number of small partitions, which can degrade performance.
* Query Patterns:The partition column should align with common query filters. If queries frequently filter data based on a particular column, partitioning by that column can be beneficial.
* Partition Size:Each partition should ideally contain at least 1 GB of data. This ensures that partitions are neither too small (leading to too many partitions) nor too large (negating the benefits of partitioning).
Evaluation of Columns:
* date:
* Cardinality:Typically low, especially if data spans over days, months, or years.
* Query Patterns:Many analytical queries filter data based on date ranges.
* Partition Size:Likely to meet the 1 GB threshold per partition, depending on data volume.
* user_id:
* Cardinality:High, as each user has a unique ID.
* Query Patterns:While some queries might filter by user_id, the high cardinality makes it unsuitable for partitioning.
* Partition Size:Partitions could be too small, leading to inefficiencies.
* post_id:
* Cardinality:Extremely high, with each post having a unique ID.
* Query Patterns:Unlikely to be used for filtering large datasets.
* Partition Size:Each partition would be very small, resulting in a large number of partitions.
* post_time:
* Cardinality:High, especially if it includes exact timestamps.
* Query Patterns:Queries might filter by time, but the high cardinality poses challenges.
* Partition Size:Similar to user_id, partitions could be too small.
Conclusion:
Given the considerations, the date column is the most suitable candidate for partitioning. It has low cardinality, aligns with common query patterns, and is likely to result in appropriately sized partitions.
References:
* Delta Lake Best Practices
* Partitioning in Delta Lake

質問 # 51
The data engineering team maintains a table of aggregate statistics through batch nightly updates. This includes total sales for the previous day alongside totals and averages for a variety of time periods including the 7 previous days, year-to-date, and quarter-to-date. This table is named store_saies_summary and the schema is as follows:
The table daily_store_sales contains all the information needed to update store_sales_summary. The schema for this table is:
store_id INT, sales_date DATE, total_sales FLOAT
If daily_store_sales is implemented as a Type 1 table and the total_sales column might be adjusted after manual data auditing, which approach is the safest to generate accurate reports in the store_sales_summary table?

• A. Use Structured Streaming to subscribe to the change data feed for daily_store_sales and apply changes to the aggregates in the store_sales_summary table with each update.
• B. Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and use upsert logic to update results in the store_sales_summary table.
• C. Implement the appropriate aggregate logic as a Structured Streaming read against the daily_store_sales table and use upsert logic to update results in the store_sales_summary table.
• D. Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and overwrite the store_sales_summary table with each Update.
• E. Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and append new rows nightly to the store_sales_summary table.

正解：A

解説：
The daily_store_sales table contains all the information needed to update store_sales_summary. The schema of the table is:
store_id INT, sales_date DATE, total_sales FLOAT
The daily_store_sales table is implemented as a Type 1 table, which means that old values are overwritten by new values and no history is maintained. The total_sales column might be adjusted after manual data auditing, which means that the data in the table may change over time.
The safest approach to generate accurate reports in the store_sales_summary table is to use Structured Streaming to subscribe to the change data feed for daily_store_sales and apply changes to the aggregates in the store_sales_summary table with each update. Structured Streaming is a scalable and fault-tolerant stream processing engine built on Spark SQL. Structured Streaming allows processing data streams as if they were tables or DataFrames, using familiar operations such as select, filter, groupBy, or join. Structured Streaming also supports output modes that specify how to write the results of a streaming query to a sink, such as append, update, or complete. Structured Streaming can handle both streaming and batch data sources in a unified manner.
The change data feed is a feature of Delta Lake that provides structured streaming sources that can subscribe to changes made to a Delta Lake table. The change data feed captures both data changes and schema changes as ordered events that can be processed by downstream applications or services. The change data feed can be configured with different options, such as starting from a specific version or timestamp, filtering by operation type or partition values, or excluding no-op changes.
By using Structured Streaming to subscribe to the change data feed for daily_store_sales, one can capture and process any changes made to the total_sales column due to manual data auditing. By applying these changes to the aggregates in the store_sales_summary table with each update, one can ensure that the reports are always consistent and accurate with the latest data. Verified References: [Databricks Certified Data Engineer Professional], under "Spark Core" section; Databricks Documentation, under "Structured Streaming" section; Databricks Documentation, under "Delta Change Data Feed" section.

質問 # 52
Which statement describes Delta Lake Auto Compaction?

• A. Before a Jobs cluster terminates, optimize is executed on all tables modified during the most recent job.
• B. Optimized writes use logical partitions instead of directory partitions; because partition boundaries are only represented in metadata, fewer small files are written.
• C. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 128 MB.
• D. Data is queued in a messaging bus instead of committing data directly to memory; all data is committed from the messaging bus in one batch once the job is complete.
• E. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 1 GB.

正解：C

解説：
Explanation
This is the correct answer because it describes the behavior of Delta Lake Auto Compaction, which is a feature that automatically optimizes the layout of Delta Lake tables by coalescing small files into larger ones. Auto Compaction runs as an asynchronous job after a write to a table has succeeded and checks if files within a partition can be further compacted. If yes, it runs an optimize job with a default target file size of 128 MB.
Auto Compaction only compacts files that have not been compacted previously. Verified References:
[Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "Auto Compaction for Delta Lake on Databricks" section.
"Auto compaction occurs after a write to a table has succeeded and runs synchronously on the cluster that has performed the write. Auto compaction only compacts files that haven't been compacted previously."
https://learn.microsoft.com/en-us/azure/databricks/delta/tune-file-size

質問 # 53
When evaluating the Ganglia Metrics for a given cluster with 3 executor nodes, which indicator would signal proper utilization of the VM's resources?

• A. Bytes Received never exceeds 80 million bytes per second
• B. Network I/O never spikes
• C. CPU Utilization is around 75%
• D. The five Minute Load Average remains consistent/flat
• E. Total Disk Space remains constant

正解：C

解説：
In the context of cluster performance and resource utilization, a CPU utilization rate of around 75% is generally considered a good indicator of efficient resource usage. This level of CPU utilization suggests that the cluster is being effectively used without being overburdened or underutilized.
* A consistent 75% CPU utilization indicates that the cluster's processing power is being effectively employed while leaving some headroom to handle spikes in workload or additional tasks without maxing out the CPU, which could lead to performance degradation.
* A five Minute Load Average that remains consistent/flat (Option A) might indicate underutilization or a bottleneck elsewhere.
* Monitoring network I/O (Options B and C) is important, but these metrics alone don't provide a complete picture of resource utilization efficiency.
* Total Disk Space (Option D) remaining constant is not necessarily an indicator of proper resource utilization, as it's more related to storage rather than computational efficiency.
References:
* Ganglia Monitoring System: Ganglia Documentation
* Databricks Documentation on Monitoring: Databricks Cluster Monitoring

質問 # 54
......

最新100%合格率保証付きの素晴らしいDatabricks-Certified-Professional-Data-Engineer試験問題PDF：https://www.goshiken.com/Databricks/Databricks-Certified-Professional-Data-Engineer-mondaishu.html

実践サンプルと問題集指導には2026年最新のDatabricks-Certified-Professional-Data-Engineer有効なテスト問題集：https://drive.google.com/open?id=1eVczsuD78TeOIIORZQr2in5FaHHlpy4M