Free D-DS-FN-23 sample questions

Correct answer: Use a regression model to predict the missing dates based on other customer attributes like 'Join_Date' and 'Total_Purchases'.

Using a regression model (or another predictive imputation method) is the most robust approach. It leverages relationships with other variables to estimate the missing values, preserving the data's underlying structure better than simple mean/median imputation. Deleting 15% of the data would cause significant information loss. Replacing a date with the mean is statistically inappropriate for temporal data and could distort the distribution.

Correct answer: Customers who buy diapers are 3.5 times more likely to also buy beer compared to the average customer.

Lift measures how much more likely two items are to be purchased together than would be expected if they were statistically independent. A lift of 3.5 means the presence of diapers in a transaction makes the purchase of beer 3.5 times more likely than it would be in a random transaction. It quantifies the strength of the association beyond random chance.

Correct answer: To assign higher weights to terms that are frequent in a specific document but rare across the entire corpus of documents.

TF-IDF is a numerical statistic that reflects how important a word is to a document in a collection or corpus. It increases the weight of terms that appear frequently in a document (high Term Frequency) but penalizes terms that appear in many documents (low Inverse Document Frequency). This helps highlight terms that are characteristic of a specific document, making them more useful for classification.

Correct answer: Writing multiple MapReduce jobs and using Oozie to schedule and chain them together.

MapReduce is designed for large-scale data processing like this. For a multi-stage workflow, Oozie is the standard Hadoop workflow scheduler. It allows you to define a Directed Acyclic Graph (DAG) of actions, chaining multiple MapReduce jobs (or other actions like Pig or Hive scripts) together, handling dependencies and failures. Flume is for data ingestion, Sqoop for database transfer, and Zookeeper for coordination, none of which manage complex processing workflows.

Correct answer: False

This statement is false. 'Veracity' refers to the uncertainty, quality, and trustworthiness of the data. The speed at which data is generated and processed is referred to as 'Velocity'.

Correct answer: summary(product_cost)

The `summary()` function in R is specifically designed to provide a quick statistical summary of a variable. For a numerical vector, it returns the minimum, 1st quartile, median, mean, 3rd quartile, and maximum values. This single command gives a concise overview of central tendency (mean, median), dispersion (quartiles, min/max), and distribution shape.

Correct answers: Use a clear, simple chart, such as a bar chart, showing projected cost savings versus implementation costs over time.; Focus on a single, powerful message per slide, using annotations to highlight key financial figures.

Correct answer: The logistic regression model provides a better measure of discriminative power across all classification thresholds.

The AUC represents a model's ability to discriminate between positive and negative classes across all possible thresholds. A higher AUC (closer to 1.0) indicates better performance. An AUC of 0.85 means the logistic regression model is superior to the decision tree (AUC 0.78) in its ability to correctly rank a randomly chosen positive instance higher than a randomly chosen negative one. It does not directly state overall accuracy, which depends on a single chosen threshold.

Correct answer: Plotting the within-cluster sum of squares (WSS) for a range of K values and identifying the 'elbow' point.

The Elbow method is a common heuristic for finding the optimal number of clusters in K-means. It involves running the algorithm for a range of K values and plotting the WSS for each. The point where the rate of decrease in WSS sharply slows, forming an 'elbow' in the plot, is considered a good estimate for the optimal K. Gini impurity is for decision trees, and Lift charts are for classification models.

Correct answer: Apache Spark (specifically Spark Streaming)

Apache Spark, and particularly its Spark Streaming library, is designed for scalable, high-throughput, fault-tolerant processing of live data streams. It processes data in micro-batches, providing near real-time capabilities. While tools like Pig and Hive are excellent for batch processing, and HBase is a NoSQL database, Spark is the premier choice in the Hadoop ecosystem for real-time stream computation.