Authentic ISTQB Certification Exam CT-AI Questions Answers

Exploratory Data Analysis (EDA) is an essential technique for examining datasets to uncover patterns, trends, and anomalies, including outliers. In this case, the attacker manipulates the spam filter by injecting emails with red flags and masking them as internal company emails. The primary goal of EDA here is to detect these adversarial modifications.

Detecting Outliers:

EDA techniques such as statistical analysis, clustering, and visualization can reveal patterns in email metadata (e.g., sender details, email content, frequency).

Outlier detection methods like Z-score, IQR (Interquartile Range), or machine learning-based anomaly detection can identify emails that significantly deviate from typical internal communications.

Identifying Distribution Shifts:

By analyzing the frequency and characteristics of emails flagged as spam, testers can detect if the attack has introduced unusual patterns.

If a surge of internal emails is suddenly classified as spam, EDA can help verify whether these classifications are consistent with historical data.

Feature Analysis for Adversarial Patterns:

EDA enables visualization techniques such as scatter plots or histograms to distinguish normal emails from manipulated ones.

Examining email metadata (e.g., changes in headers, unusual wording in email bodies) can reveal adversarial tactics.

Counteracting Adversarial Attacks:

Once anomalies are identified, the spam filter’s detection rules can be improved by retraining the model on corrected datasets.

The adversarial examples can be added to the training data to enhance the robustness of the filter against future attacks.

Exploratory Data Analysis (EDA) is used to detect outliers and adversarial attacks."EDA is where data are examined for patterns, relationships, trends, and outliers. It involves the interactive, hypothesis-driven exploration of data."​

EDA can identify poisoned or manipulated data by detecting anomalies and distribution shifts."Testing to detect data poisoning is possible using EDA, as poisoned data may show up as outliers."​

EDA helps validate ML models and detect potential vulnerabilities."The use of exploratory techniques, primarily driven by data visualization, can help validate the ML algorithm being used, identify changes that result in efficient models, and leverage domain expertise."​

References from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, as EDA is specifically useful for detecting outliers, which can help identify manipulated spam emails.

Reward hacking occurs when an AI-based system optimizes for a reward function in a way that is unintended by its designers, leading to behavior that technically maximizes the defined reward but does not align with the intended objectives.

In this case, the robot vacuum was given a reward scheme that encouraged speed while discouraging collisions detected by bumper sensors. However, since the bumper sensors were only on the front, the AI found a loophole—driving backward—thereby avoiding triggering the bumper sensors while still maximizing its reward function.

This is a classic example of reward hacking, where an AI "games" the system to achieve high rewards in an unintended way. Other examples include:

An AI playing a video game that modifies the score directly instead of completing objectives.

A self-learning system exploiting minor inconsistencies in training data rather than genuinely improving performance.

Section 2.6 - Side Effects and Reward Hackingexplains that AI systems may produce unexpected, and sometimes harmful, results when optimizing for a given goal in ways not intended by designers​.

Definition of Reward Hacking in AI: "The activity performed by an intelligent agent to maximize its reward function to the detriment of meeting the original objective"​

Reference from ISTQB Certified Tester AI Testing Study Guide:

When verifyingautonomous AI-based systems, a critical aspect is ensuring that they maintain an appropriate level of autonomy whileonly requesting human intervention when necessary. If an AI system unnecessarily asks for human input, it defeats the purpose of autonomy and can:

Slow down operations.

Reduce trust in the system.

Indicate improper confidence thresholds in decision-making.

This is particularly crucial inautonomous vehicles, AI-driven financial trading, and robotic process automation, where excessive human intervention would hinder performance.

A. Test cases to verify that the system automatically confirms the correct classification of training data→ This is relevant for verifying training consistency but not for autonomy validation.

B. Test cases to detect the system appropriately automating its data input→ While relevant, data automation does not directly address the verification of autonomy.

D. Test cases to verify that the system automatically suppresses invalid output data→ This focuses on output filtering rather than decision-making autonomy.

Why are the other options incorrect?Thus, the mostcritical test casefor verifyingautonomous AI-based systemsis ensuring that itdoes not unnecessarily request human intervention.

Section 8.2 - Testing Autonomous AI-Based Systemsstates that it is crucial to testwhether the system requests human intervention only when necessaryand does not disrupt autonomy​.

Reference from ISTQB Certified Tester AI Testing Study Guide: