CT-AI Exam Dumps - ISTQB AI Testing Questions and Answers

Probabilistic and non-deterministic AI-based systemsdo not always produce the same output for identical inputs. This makes traditional testing approaches ineffective. Instead, the best approach is torun tests multiple times and analyze results statistically.

Statistical Validity:Running tests multiple times ensures that observed results are statistically significant. Instead of relying on a single test run,analyzing multiple iterations helps determine trends, probabilities, and outliers​.

Expected Result Tolerance:AI-based systems may produce different results within an acceptable range. Defining acceptable tolerances (e.g., "result must be within 2% of the optimal value") improves test effectiveness​.

A (Run Several Times for the Same Correct Result):AI systems are ofteninherently non-deterministicand may not return the exact same result every time. Expecting identical outputs contradicts the nature of these systems​.

B & C (Decomposing Tests into Data Ingestion Tests):While data ingestion quality is important, it does notdirectlysolve the issue of probabilistic test results. Statistical analysis is the key approach​.

ISTQB CT-AI Syllabus (Section 8.4: Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems)

"For probabilistic systems, running a test multiple times may be necessary to obtain a statistically valid test result."​.

"Where a single definitive output is not possible, results should be analyzed statistically rather than relying on individual test cases."​.

Why Other Options Are Incorrect:Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:Sinceprobabilistic AI systems do not always return the same result, the best approach is torun multiple test iterations and validate results statistically. Hence, thecorrect answer is D.

 A. This statement is incorrect. Current AI is true AI and there is no reason to believe that this fact will change over time.

AI is an evolving field, and the definition of what constitutes AI can change as technology advances.

 B. This statement is correct. In general, what is considered AI today may change over time.

The term AI is dynamic and has evolved over the years. What is considered AI today might be viewed as standard computing in the future. Historically, as technologies become mainstream, they often cease to be considered "AI".

 C. This statement is incorrect. What is considered AI today will continue to be AI even as technology evolves and changes.

This perspective does not account for the historical evolution of the definition of AI. As new technologies emerge, the boundaries of AI shift.

 D. This statement is correct. In general, today the term AI is utilized incorrectly.

While some may argue this, it is not a universal truth. The term AI encompasses a broad range of technologies and applications, and its usage is generally consistent with current technological capabilities.

A. Add 10% of the rows randomly and create another model and compare the R-Square scores of both the models.

Adding more data to the training set can affect the R-Square score, but it does not directly verify the correctness of the implementation.

B. Train various models by changing the order of input features and verify that the R-Square score of these models vary significantly.

Changing the order of input features should not significantly affect the R-Square score if the implementation is correct, but this approach is more about testing model robustness rather than correctness of the implementation.

C. Compare the R-Square score of the model obtained using two different implementations that utilize two different programming languages while using the same algorithm and the same training and testing data.

This approach directly compares the performance of two implementations of the same algorithm. If both implementations produce similar R-Square scores on the same training and testing data, it suggests that the new implementation "X" is correct.

D. Drop 10% of the rows randomly and create another model and compare the R-Square scores of both the models.

Dropping data can lead to variations in the R-Square score but does not directly verify the correctness of the implementation.

Therefore, optionCis the most appropriate strategy because it directly compares the performance of the new implementation "X" with another implementation using the same algorithm and datasets, which helps in verifying the correctness of the implementation​​.

Bias in an AI system occurs when the training data contains inherent prejudices that cause the model to make unfair predictions. Experience-based testing, particularlyExploratory Data Analysis (EDA), helps uncover these biases by analyzing patterns, distributions, and potential discriminatory factors in the training data​.

Option A:“Experience-based testing should be used to confirm that the training data set is operationally relevant. This can include applying exploratory data analysis (EDA) to check for bias within the training data set.”

This is the correct answer. EDA involves examining the dataset for bias, inconsistencies, or missing values, ensuring fairness in ML model predictions​.

Option B:“Back-to-back testing should be used to compare the model created using the training data set to another model created using the test data set. If the two models significantly differ, it will indicate there is bias in the original model.”

Back-to-back testing is used for regression testing and to compare versions of an AI system but is not primarily used to detect bias​.

Option C:“Acceptance testing should be used to make sure the algorithm is suitable for the customer. The team can re-work the acceptance criteria such that the algorithm is sure to correctly predict the remaining applicants that have been set aside for the validation data set ensuring no bias is present.”

Acceptance testing focuses on meeting predefined business requirements rather than detecting and mitigating bias​.

Option D:“A/B testing should be used to verify that the test data set does not detect any bias that might have been introduced by the original training data. If the two models significantly differ, it will indicate there is bias in the original model.”

A/B testing is used for evaluating variations of a model rather than for explicitly identifying bias​.

Bias Testing Methods:"AI-based systems should be tested for algorithmic bias, sample bias, and inappropriate bias. Experience-based testing and EDA are useful for detecting bias"​.

Exploratory Data Analysis (EDA):"EDA helps uncover potential bias in training data through visualization and statistical analysis"​.

Analysis of the Answer Options:ISTQB CT-AI Syllabus References:Thus,Option A is the best choice for detecting bias in the loan applicant model.