Appliances enhanced by artificial intelligence offer significant benefits to consumers. But for AI appliances to reach their true potential, consumers must have confidence that these devices reliably perform as promised. UL’s AI Algorithm Reproducibility Process Claim Verification program aims to strengthen confidence in the trustworthiness of a manufacturer’s claims regarding an appliance’s AI functionality. Appliances that can, for example, reduce energy consumption, alert owners to malfunctions such as leaks or pending maintenance needs or provide a smarter, easier way of managing routine tasks are increasingly popular in the market. Some smart devices carry out basic commands in response to data or human input. Appliances that use an AI algorithm to perform task can make more sophisticated decisions and can be trained to continue to advance their abilities. As an example, a simple smart appliance can be programmed to repeatedly perform an action based on the time of day or information such as the weather report. But an AI algorithm independently finds patterns or features in a dataset and uses that information to support the appliance functionality. Machine learning functions in much the same way, but where AI can handle unstructured and structured data, machine learning relies on structured data for training. Reproducibility is an important step toward determining the dependability of an algorithm used in products where that algorithm (machine learning or rule-based) provides critical functionality or is central to an important marketing claim. From the safety of autonomous driving, to appliances that rely on algorithms for special features that differentiate a product in a market, algorithms must be able to reliably produce, each and every time, an expected outcome when used as expected.
Assessing appliance algorithm repeatability
UL’s AI Algorithm Reproducibility Process Claim Verification program is an independent claim assessment focused on confirming that a specific algorithm is trustworthy – that it is capable of consistently producing a specified outcome. UL determines this by conducting an objective, science-based assessment that confirms the accuracy of a manufacturer’s marketing claim. UL’s Algorithm Reproducibility Process Claim Verification program involves an in-depth review of all aspects of the algorithm building and deployment processes, grounded on best-practices in predictive modeling and UL specific learnings. Assessment includes: detailed scoping of the application, description of the dataset, appropriateness of performance metrics, controls implemented in deployment, tracking of algorithm performance in production. The full UL Verification assessment includes technical analyses to check the manufacturer’s claims about the algorithm’s performance when used as intended. Products may perform differently when subjected to intentional or accidental misuse or exhibit unexpected failure modes during deployment. “As the global leader in safety science, testing and inspection – UL comments –, we have significant experience in helping manufacturers confidently bring innovative products to market and building consumer trust in new technologies. Appliance manufacturers can leverage UL’s experience in the AI space, and trusted reputation for scientific thoroughness, through the UL Algorithm Reproducibility Process Claim Verification program. This program is available to any company that has created an algorithm that is a critical component in determining the performance of a manufactured hardware product or system. After confirming that an algorithm meets the applicable requirements associated with that assessment, UL issues the Verified Mark for the Algorithm Reproducibility Process Claim. The UL Verified Mark delivers the confidence to consumers that a brand’s marketing claim is accurate, truthful, and credible”.
