Free CT-AI sample questions

Correct answer: If a source test case with an applicant's income of $60,000 and 2 credit cards results in a risk score 'X', a follow-up test case with the same applicant but an income of $90,000 and 2 credit cards should result in a risk score less than or equal to 'X'.

This is the most effective Metamorphic Relation because it tests a fundamental logical property of a loan risk model without needing a precise expected output. The property is that, all other factors being equal, a higher income should not increase financial risk. This allows the team to verify the model's logical consistency even when the exact risk score calculation is unknown. The other options are less effective: a tiny income change may not trigger a score change, duplicating the test case only checks for determinism, and assuming proportionality is not a guaranteed property.

Correct answers: Analyze the model's predictions across different demographic subgroups (e.g., gender, ethnicity) and check for statistically significant differences in selection rates.; Review the training data for proxy variables, such as zip codes or attendance at specific universities, that strongly correlate with demographic groups.

This method, known as disparate impact analysis or statistical parity testing, is a primary technique for detecting outcome-based bias, regardless of the input features. It directly measures whether the model's decisions are fair in practice.

This is a critical step in identifying the root cause of hidden bias. Proxy variables are features that are not explicitly sensitive but act as stand-ins for protected characteristics, introducing bias through the training data itself.

Correct answer: The AI tool will not know the expected outcome for the generated tests, creating a significant test oracle problem.

This is the fundamental limitation of most AI-based test generation tools. While excellent at exploring the state space and generating novel inputs (the 'how to test'), they lack the domain-specific knowledge to determine what the correct output should be. This is known as the test oracle problem. The most common use for such tools is to find crashes or server errors (5xx), for which the oracle is simple, rather than to find subtle logical bugs.

Correct answer: Using a model-agnostic technique like LIME (Local Interpretable Model-Agnostic Explanations) to generate explanations for individual predictions by analyzing how small input perturbations affect the outcome.

For a complex black-box model like a DNN, direct inspection of weights is not human-interpretable. LIME is designed specifically for this use case: providing local, understandable explanations for individual predictions of any black-box model. This allows a clinician to ask 'Why did the model flag this specific patient?' and get an answer based on the most influential features (e.g., 'because of high heart rate and low blood pressure'), making it ideal for validating explainability in a clinical setting.

Correct answer: False

The statement is false. Neuron coverage is a very weak structural coverage criterion, analogous to statement coverage in traditional code. It only ensures that each neuron has produced an output above a certain threshold at least once. It does not test the complex interactions between neurons, the different activation ranges, or the combinatorial logic of the network. Therefore, it provides no guarantee that the model is free of logical flaws or defects.

Correct answer: Outsourcing the labeling to a crowdsourcing platform where each image is labeled by multiple (e.g., three) different workers, and using the majority vote as the final label.

This is a standard industry best practice for large-scale labeling tasks. Crowdsourcing is fast and cost-effective for a large dataset. The key to quality control in this approach is redundancy: having multiple independent workers label the same data item. A consensus mechanism, like a majority vote, is then used to establish a higher-confidence ground truth label, effectively filtering out random errors and individual worker biases. This balances speed, cost, and quality.

Correct answers: Introduce a small number of mislabeled images into the training dataset (e.g., images of 'Stop' signs labeled as 'Speed Limit 60') and observe if the trained model's accuracy degrades significantly or learns a backdoor.; Generate adversarial examples by applying subtle, almost imperceptible pixel-level noise to a valid image of a stop sign and checking if it causes a misclassification (e.g., to 'Green Light').

This activity directly simulates a data poisoning attack. By intentionally corrupting a portion of the training data, testers can evaluate the model's resilience and determine if it can be manipulated into making specific, incorrect classifications.

This describes the process of an adversarial attack. The goal is to create inputs that are visually indistinguishable from legitimate ones to a human but are specifically crafted to fool the model. Testing with such examples is crucial for assessing robustness against malicious attacks in production.

Correct answer: Unsupervised Learning (Association)

The scenario describes market basket analysis, which is a classic example of association rule mining. Since the goal is to discover hidden patterns and relationships in unlabeled transactional data, it falls under the category of Unsupervised Learning, and more specifically, Association. Classification and Regression are supervised techniques that require labeled data, and Reinforcement Learning involves an agent learning through trial and error with rewards, which is not applicable here.

Correct answer: It enables the safe, repeatable, and scalable testing of rare and dangerous edge-case scenarios (e.g., multiple drones converging, sudden obstacles).

The core value of simulation for autonomous systems is the ability to test high-risk scenarios without real-world consequences. A virtual environment allows testers to create and repeat dangerous edge cases (like near-misses or sensor failures) thousands of times to ensure robustness. This is impractical, unsafe, and prohibitively expensive to do in a physical environment. This makes simulation an indispensable tool for testing safety-critical AI systems.

Correct answer: Recall (Sensitivity)

The scenario explicitly states that the cost of a False Negative (missing a fraudulent transaction) is very high. Recall is calculated as TP / (TP + FN). To maximize Recall, the number of False Negatives must be minimized. Therefore, Recall is the most important metric to prioritize when the business cost of missing a positive case (in this case, fraud) is high. Precision would be prioritized if the cost of False Positives was the main concern.