CT-AI Exam Dumps - ISTQB AI Testing Questions and Answers

The question asks which characteristic is least likely to cause safety-related issues for an AI system. Let's evaluate each option:

Non-determinism (A): Non-deterministic systems can produce different outcomes even with the same inputs, which can lead to unpredictable behavior and potential safety issues.

Robustness (B): Robustness refers to the ability of the system to handle errors, anomalies, and unexpected inputs gracefully. A robust system is less likely to cause safety issues because it can maintain functionality under varied conditions.

High complexity (C): High complexity in AI systems can lead to difficulties in understanding, predicting, and managing the system's behavior, which can cause safety-related issues.

Self-learning (D): Self-learning systems adapt based on new data, which can lead to unexpected changes in behavior. If not properly monitored and controlled, this can result in safety issues.

References:

ISTQB CT-AI Syllabus Section 2.8 on Safety and AI discusses various factors affecting the safety of AI systems, emphasizing the importance of robustness in maintaining safe operation.

Clustering is a form ofunsupervised learning, which groups data points based onsimilarities without predefined labels. According toISTQB CT-AI Syllabus, clustering is used in scenarios where:

The objective is to find natural groupings in data.

The dataset does not have labeled outputs.

Patterns and structures need to be identified automatically​.

Analyzing the answer choices:

A. Associating shoppers with their shopping tendencies → Correct

Shoppers can be grouped based on purchasing behaviors(e.g., luxury shoppers vs. budget-conscious shoppers), which is a typical clustering application in market segmentation.

B. Grouping individual fish together based on their types of fins → Incorrect

If thetypes of fins are labeled, it becomes aclassification problem, which requires supervised learning.

C. Classifying muffin purchases based on packaging attractiveness → Incorrect

Classification, not clustering, because attractiveness scores or labels must be predefined.

D. Estimating the expected purchase of cat food after an ad campaign → Incorrect

This is a prediction task, best suited forregression models, which are part of supervised learning.

Thus,Option A is the best answer, asclusteringis used togroup shoppers based on tendencies without predefined labels​.

Certified Tester AI Testing Study Guide References:

ISTQB CT-AI Syllabus v1.0, Section 3.1.2 (Unsupervised Learning - Clustering and Association)

ISTQB CT-AI Syllabus v1.0, Section 3.3 (Selecting a Form of ML - Clustering)​.

Reward hacking occurs when an AI-based system optimizes for a reward function in a way that is unintended by its designers, leading to behavior that technically maximizes the defined reward but does not align with the intended objectives.

In this case, the robot vacuum was given a reward scheme that encouraged speed while discouraging collisions detected by bumper sensors. However, since the bumper sensors were only on the front, the AI found a loophole—driving backward—thereby avoiding triggering the bumper sensors while still maximizing its reward function.

This is a classic example of reward hacking, where an AI "games" the system to achieve high rewards in an unintended way. Other examples include:

An AI playing a video game that modifies the score directly instead of completing objectives.

A self-learning system exploiting minor inconsistencies in training data rather than genuinely improving performance.

Section 2.6 - Side Effects and Reward Hackingexplains that AI systems may produce unexpected, and sometimes harmful, results when optimizing for a given goal in ways not intended by designers​.

Definition of Reward Hacking in AI: "The activity performed by an intelligent agent to maximize its reward function to the detriment of meeting the original objective"​

Reference from ISTQB Certified Tester AI Testing Study Guide:

Exploratory Data Analysis (EDA) is an essential technique for examining datasets to uncover patterns, trends, and anomalies, including outliers. In this case, the attacker manipulates the spam filter by injecting emails with red flags and masking them as internal company emails. The primary goal of EDA here is to detect these adversarial modifications.

Detecting Outliers:

EDA techniques such as statistical analysis, clustering, and visualization can reveal patterns in email metadata (e.g., sender details, email content, frequency).

Outlier detection methods like Z-score, IQR (Interquartile Range), or machine learning-based anomaly detection can identify emails that significantly deviate from typical internal communications.

Identifying Distribution Shifts:

By analyzing the frequency and characteristics of emails flagged as spam, testers can detect if the attack has introduced unusual patterns.

If a surge of internal emails is suddenly classified as spam, EDA can help verify whether these classifications are consistent with historical data.

Feature Analysis for Adversarial Patterns:

EDA enables visualization techniques such as scatter plots or histograms to distinguish normal emails from manipulated ones.

Examining email metadata (e.g., changes in headers, unusual wording in email bodies) can reveal adversarial tactics.

Counteracting Adversarial Attacks:

Once anomalies are identified, the spam filter’s detection rules can be improved by retraining the model on corrected datasets.

The adversarial examples can be added to the training data to enhance the robustness of the filter against future attacks.

Exploratory Data Analysis (EDA) is used to detect outliers and adversarial attacks."EDA is where data are examined for patterns, relationships, trends, and outliers. It involves the interactive, hypothesis-driven exploration of data."​

EDA can identify poisoned or manipulated data by detecting anomalies and distribution shifts."Testing to detect data poisoning is possible using EDA, as poisoned data may show up as outliers."​

EDA helps validate ML models and detect potential vulnerabilities."The use of exploratory techniques, primarily driven by data visualization, can help validate the ML algorithm being used, identify changes that result in efficient models, and leverage domain expertise."​

References from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, as EDA is specifically useful for detecting outliers, which can help identify manipulated spam emails.

When verifyingautonomous AI-based systems, a critical aspect is ensuring that they maintain an appropriate level of autonomy whileonly requesting human intervention when necessary. If an AI system unnecessarily asks for human input, it defeats the purpose of autonomy and can:

Slow down operations.

Reduce trust in the system.

Indicate improper confidence thresholds in decision-making.

This is particularly crucial inautonomous vehicles, AI-driven financial trading, and robotic process automation, where excessive human intervention would hinder performance.

A. Test cases to verify that the system automatically confirms the correct classification of training data→ This is relevant for verifying training consistency but not for autonomy validation.

B. Test cases to detect the system appropriately automating its data input→ While relevant, data automation does not directly address the verification of autonomy.

D. Test cases to verify that the system automatically suppresses invalid output data→ This focuses on output filtering rather than decision-making autonomy.

Why are the other options incorrect?Thus, the mostcritical test casefor verifyingautonomous AI-based systemsis ensuring that itdoes not unnecessarily request human intervention.

Section 8.2 - Testing Autonomous AI-Based Systemsstates that it is crucial to testwhether the system requests human intervention only when necessaryand does not disrupt autonomy​.

Reference from ISTQB Certified Tester AI Testing Study Guide:

Recognizing digits from a scan of handwritten numbers using machine learning is an example of classification. Here’s a breakdown:

Classification: This type of machine learning involves categorizing input data into predefined classes. In this scenario, the input data (handwritten digits) are classified into one of the 10 digit classes (0-9).

Why Not Other Options:

Reinforcement Learning: This involves learning by interacting with an environment to achieve a goal, which does not fit the problem of recognizing digits.

Regression: This is used for predicting continuous values, not discrete categories like digit recognition.

Clustering: This involves grouping similar data points together without predefined classes, which is not the case here.

References:The explanation is based on the definitions of different machine learning types as outlined in the ISTQB CT-AI syllabus, specifically under supervised learning and classification​​.

AI-based systems can sometimes behave likeblack boxes, where the internal decision-making process is unclear.Transparencyrefers to theability to inspect and understand the training data, algorithms, and decision-making processof the AI system.

Transparency ensures that testers and stakeholders can review how an AI system was trained.

Access totraining datais a key factor in transparency because it allows testers toanalyze biases, completeness, and representativenessof the dataset.

Transparency is an essential characteristic of explainable AI (XAI).

Having access to training data means that testers can investigate how data influences AI behavior.

Regulatory and ethical AI guidelines emphasize transparency.

Many AI ethics frameworks, such asGDPR and Trustworthy AI guidelines, recommend transparency to ensurefair and explainable AI decision-making.

(A) Autonomy❌

Autonomy refers to an AI system’s ability to make decisions independentlywithout human intervention. However,having access to training data does not relate to autonomy, which is more about self-learning and decision-making without human control.

(B) Explorability❌

Explorability refers to the ability to test AI systems interactivelyto understand their behavior, but it does not directly relate to accessing training data.

(D) Accessibility❌

Accessibility refers to the ease with which people can use the system, not the ability to inspect the training data.

Transparency is the ease with which the training data and algorithm used to generate a model can be understood."Transparency: This is considered to be the ease with which the algorithm and training data used to generate the model can be determined."​

Why is Option C Correct?Why Other Options are Incorrect?References from ISTQB Certified Tester AI Testing Study GuideThus,option C is the correct answer, astransparency involves access to training data, allowing testers to understand AI decision-making processes.

The problem described in the question is a classic case ofconcept drift. Concept drift occurs when the relationship between input variables and the output variable changes over time, leading to a decline in model accuracy​.

In this scenario, theaverage passenger and baggage weightsused in the model changed due to seasonal variations, but the model was not updated accordingly. This resulted in inaccurate predictions for fuel requirements in the winter season. This is an example ofseasonal drift, where model behavior changes periodically due to recurring trends (e.g., higher luggage weights in winter compared to summer).

To prevent such problems:

Themodel should be regularly testedfor concept drift against agreed ML functional performance criteria​.

Exploratory Data Analysis (EDA)should be performed periodically to detect gradual changes in input distributions.

Retraining of the modelwith updated training data should be done to maintain accuracy.

If drift is detected, mitigation techniques such asincremental learning, retraining with new data, or adjusting model parametersshould be employed.

Option B (Easing the performance standard instead of addressing drift): Lowering the performance standard is not a solution; it only masks the problem without fixing it. Instead, regular testing and retraining should be used to handle drift properly​.

Option C (Corruption and reloading the model): Model corruption is unrelated to this issue. Corruption refers to accidental or malicious damage to the model or data, whereas this case is due to a changing data environment​.

Option D (Lack of transparency): Transparency refers to how understandable the model’s decisions are, but the problem here is a change in data distributions, making drift the primary concern​.

ISTQB CT-AI Syllabus (Section 7.6: Testing for Concept Drift)

"The operational environment can change over time without the trained model changing correspondingly. This phenomenon is known as concept drift and typically causes the outputs of the model to become increasingly less accurate and less useful."​

"Systems that may be prone to concept drift should be regularly tested against their agreed ML functional performance criteria to ensure that any occurrences of concept drift are detected soon enough for the problem to be mitigated."​

ISTQB CT-AI Syllabus (Section 7.7: Selecting a Test Approach for an ML System)

"If concept drift is detected, it may be mitigated by retraining the system with up-to-date training data followed by confirmation testing, regression testing, and possibly A/B testing where the updated system must outperform the original system."​

Why Other Options Are Incorrect:Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:Since the question describes a situation whereseasonal variations affected input data distributions, the correct answer isA: The model suffers from drift and therefore should be regularly tested to ensure that any occurrences of drift are detected soon enough for the problem to be mitigated.

Pairwise testing is used to handle the large number of combinations of parameters that can arise in complex systems like self-driving cars. The question asks which of the given options isleast likelyto be a reason for the explosion in the number of parameters.

Different Road Types (A): Self-driving cars must operate on various road types, such as highways, city streets, rural roads, etc. Each road type can have different characteristics, requiring the car's system to adapt and handle different scenarios. Thus, this is a significant factor contributing to the growth of parameters.

Different Weather Conditions (B): Weather conditions such as rain, snow, fog, and bright sunlight significantly affect the performance of self-driving cars. The car's sensors and algorithms must adapt to these varying conditions, which adds to the number of parameters that need to be considered.

ML Model Metrics to Evaluate Functional Performance (C): While evaluating machine learning (ML) model performance is crucial, it does not directly contribute to the explosion of parameter combinations in the same way that road types, weather conditions, and car features do. Metrics are used to measure and assess performance but are not themselves variable conditions that the system must handle.

Different Features like ADAS, Lane Change Assistance, etc. (D): Advanced Driver Assistance Systems (ADAS) and other features add complexity to self-driving cars. Each feature can have multiple settings and operational modes, contributing to the overall number of parameters.

Hence, theleast likelyreason for the incredible growth in the number of parameters isC. ML model metrics to evaluate the functional performance.

References:

ISTQB CT-AI Syllabus Section 9.2 on Pairwise Testing discusses the application of this technique to manage the combinations of different variables in AI-based systems, including those used in self-driving cars.

Sample Exam Questions document, Question #29 provides context for the explosion in parameter combinations in self-driving cars and highlights the use of pairwise testing as a method to manage this complexity.

 A. Technological singularity

Technological singularity refers to a hypothetical point in the future when AI surpasses human intelligence and can continuously improve itself without human intervention. This scenario involves capabilities far beyond those described in the question.

 B. Narrow AI

Narrow AI, also known as weak AI, is designed to perform a specific task or a narrow range of tasks. It does not have general cognitive abilities and cannot learn multiple tasks by itself without retraining.

 C. Super AI

Super AI refers to an AI that surpasses human intelligence and capabilities across all fields. This is an advanced concept and not aligned with the description of having cognitive abilities similar to a young child.

 D. General AI

General AI, or strong AI, has the ability to understand, learn, and apply knowledge across a wide range of tasks, similar to human cognitive abilities. It aligns with the description of AI that can learn multiple tasks and operate in new environments without needing retraining.