ExamDumpster

Free CT-AI sample questions

Try these free CT-AI sample questions, each with its answer and an explanation. They cover machine learning data, model metrics and ways to test for bias. Use them to see how well you can apply test ideas to AI systems.

Try them in the simulator Same questions, with study, timed and flashcard modes.

Showing 10 of 20 free sample questions.

Question 1Choose one

A data scientist is building an image classification model. After training, they observe the following: - Training Accuracy: 99% - Validation Accuracy: 75% This significant gap between training and validation performance is a strong indicator of a common problem in machine learning. What is this problem, and what is the MOST likely cause?

Question 2Choose one

A test manager is creating a test plan for an AI system that provides real-time investment advice. The system is highly non-deterministic, and its complexity makes it a black box. The manager is concerned about regressions after weekly model updates. A previous version of the system with known, stable behavior is available. Which test technique is specifically designed to use a stable, previous version as a pseudo-oracle to check the behavior of a new version?

Question 3Choose one

During the data preparation phase for a credit scoring model, a data analyst uses a technique to rescale all numeric features, such as 'income' and 'age', so that they have a mean of zero and a standard deviation of one. What is this data pre-processing technique called, and what is its primary purpose?

Question 4Choose one

A large retail company has an AI model that predicts daily sales for its 1,000 stores. The model worked well for two years, but its accuracy has recently dropped significantly. A review reveals that since the model was deployed, customer shopping habits have changed due to new competitors and economic shifts. The model is now consistently underpredicting sales on weekends and overpredicting on weekdays. This phenomenon, where a model's performance degrades over time because the statistical properties of the target variable change, is known by a specific term. What is this term?

Question 5Choose one

A test engineer is working on a reinforcement learning (RL) agent designed to play a video game. The goal is to achieve the highest possible score. During testing, the engineer discovers that the agent found a bug in the game where it can repeatedly perform an action that gives a small number of points without advancing the game, allowing it to accumulate an infinitely high score without ever finishing the level. This behavior, where an AI system achieves its literal goal in an unintended and counterproductive way, is known as:

Question 6Choose 2

A project team is developing an AI system and is following the ML workflow. The team has just completed the 'Train the Model' step. According to the standard ML workflow, what are the next TWO immediate steps, performed iteratively, before the model is ready for independent testing against the holdout test dataset? (Select TWO) flowchart LR A[Data Prep] --> B[Train Model] B --> C{...} C --> D{...} D --> B C --> E[Test Model]

Question 7Choose one

A QA team is testing an AI-based system that uses a pre-trained model from a third-party vendor. The documentation for the pre-trained model is minimal. Which quality characteristic is most directly and negatively impacted by this lack of documentation, making it difficult for the QA team to assess risks like inherited bias or vulnerabilities?

Question 8Choose one

Which of the following describes a key difference between AI-based systems and conventional systems from a testing perspective?

Question 9Choose one

A team is testing an AI-powered chatbot. They notice that when users ask the same question multiple times with slightly different phrasing, the chatbot sometimes provides inconsistent or contradictory answers. This makes it difficult to write traditional, assertion-based automated tests. The test lead suggests a technique where they compare the chatbot's response to the response from a different, established chatbot service for the same input questions. What is this technique called?

Question 10Choose one

The F1-score is often used as a performance metric for classification models. It is calculated as the harmonic mean of two other fundamental metrics. What are these two metrics?

10 more free samples are waiting

Create a free account to unlock the whole CT-AI sample bank, or get full access to all 167 practice questions in the simulator.

Create account