Databricks Associate-Developer-Apache-Spark-3.5 Practice Test Pdf Exam Material [Q19-Q38]

QUESTION 19
A developer wants to test Spark Connect with an existing Spark application.
What are the two alternative ways the developer can start a local Spark Connect server without changing their existing application code? (Choose 2 answers)

QUESTION 20
A Spark engineer is troubleshooting a Spark application that has been encountering out-of-memory errors during execution. By reviewing the Spark driver logs, the engineer notices multiple “GC overhead limit exceeded” messages.
Which action should the engineer take to resolve this issue?

QUESTION 21
A DataFrame df has columns name, age, and salary. The developer needs to sort the DataFrame by age in ascending order and salary in descending order.
Which code snippet meets the requirement of the developer?

QUESTION 22
Which command overwrites an existing JSON file when writing a DataFrame?

QUESTION 23
47 of 55.
A data engineer has written the following code to join two DataFrames df1 and df2:
df1 = spark.read.csv(“sales_data.csv”)
df2 = spark.read.csv(“product_data.csv”)
df_joined = df1.join(df2, df1.product_id == df2.product_id)
The DataFrame df1 contains ~10 GB of sales data, and df2 contains ~8 MB of product data.
Which join strategy will Spark use?

QUESTION 24
What is the risk associated with this operation when converting a large Pandas API on Spark DataFrame back to a Pandas DataFrame?

QUESTION 25
26 of 55.
A data scientist at an e-commerce company is working with user data obtained from its subscriber database and has stored the data in a DataFrame df_user.
Before further processing, the data scientist wants to create another DataFrame df_user_non_pii and store only the non-PII columns.
The PII columns in df_user are name, email, and birthdate.
Which code snippet can be used to meet this requirement?

QUESTION 26
A data scientist wants each record in the DataFrame to contain:
The first attempt at the code does read the text files but each record contains a single line. This code is shown below:

The entire contents of a file
The full file path
The issue: reading line-by-line rather than full text per file.
Code:
corpus = spark.read.text(“/datasets/raw_txt/*”)
.select(‘*’, ‘_metadata.file_path’)
Which change will ensure one record per file?
Options:

QUESTION 27
A data engineer is running a batch processing job on a Spark cluster with the following configuration:
10 worker nodes
16 CPU cores per worker node
64 GB RAM per node
The data engineer wants to allocate four executors per node, each executor using four cores.
What is the total number of CPU cores used by the application?

QUESTION 28
In the code block below, aggDF contains aggregations on a streaming DataFrame:

Which output mode at line 3 ensures that the entire result table is written to the console during each trigger execution?

QUESTION 29
A data engineer needs to persist a file-based data source to a specific location. However, by default, Spark writes to the warehouse directory (e.g., /user/hive/warehouse). To override this, the engineer must explicitly define the file path.
Which line of code ensures the data is saved to a specific location?
Options:

QUESTION 30
8 of 55.
A data scientist at a large e-commerce company needs to process and analyze 2 TB of daily customer transaction data. The company wants to implement real-time fraud detection and personalized product recommendations.
Currently, the company uses a traditional relational database system, which struggles with the increasing data volume and velocity.
Which feature of Apache Spark effectively addresses this challenge?

QUESTION 31
A developer notices that all the post-shuffle partitions in a dataset are smaller than the value set for spark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold.
Which type of join will Adaptive Query Execution (AQE) choose in this case?

QUESTION 32
An engineer wants to join two DataFramesdf1anddf2on the respectiveemployee_idandemp_idcolumns:
df1:employee_id INT,name STRING
df2:emp_id INT,department STRING
The engineer uses:
result = df1.join(df2, df1.employee_id == df2.emp_id, how=’inner’)
What is the behaviour of the code snippet?

QUESTION 33
Given this code:

.withWatermark(“event_time”,”10 minutes”)
.groupBy(window(“event_time”,”15 minutes”))
.count()
What happens to data that arrives after the watermark threshold?
Options:

QUESTION 34
An MLOps engineer is building a Pandas UDF that applies a language model that translates English strings into Spanish. The initial code is loading the model on every call to the UDF, which is hurting the performance of the data pipeline.
The initial code is:

def in_spanish_inner(df: pd.Series) -> pd.Series:
model = get_translation_model(target_lang=’es’)
return df.apply(model)
in_spanish = sf.pandas_udf(in_spanish_inner, StringType())
How can the MLOps engineer change this code to reduce how many times the language model is loaded?

QUESTION 35
A data engineer needs to persist a file-based data source to a specific location. However, by default, Spark writes to the warehouse directory (e.g., /user/hive/warehouse). To override this, the engineer must explicitly define the file path.
Which line of code ensures the data is saved to a specific location?
Options:

QUESTION 36
Given this view definition:
df.createOrReplaceTempView(“users_vw”)
Which approach can be used to query the users_vw view after the session is terminated?
Options:

QUESTION 37
Given the code:

df = spark.read.csv(“large_dataset.csv”)
filtered_df = df.filter(col(“error_column”).contains(“error”))
mapped_df = filtered_df.select(split(col(“timestamp”),” “).getItem(0).alias(“date”), lit(1).alias(“count”)) reduced_df = mapped_df.groupBy(“date”).sum(“count”) reduced_df.count() reduced_df.show() At which point will Spark actually begin processing the data?

QUESTION 38
Given a DataFrame df that has 10 partitions, after running the code:
result = df.coalesce(20)
How many partitions will the result DataFrame have?