Pitfalls and risks of generative AI in machine learning

Modern generative AI approaches—including large language models and foundation models more generally—are increasingly part of machine learning (ML) workflows. This brings new opportunities but also new ways in which things can go wrong, including challenges around evaluation, security, data provenance, technical debt, regulatory compliance, and hidden financial costs. This tutorial gives a concise overview of emerging pitfalls and risks and offers guidance on how to navigate them, with the aim of helping readers make informed decisions and avoid costly mistakes. Written in plain language and assuming no deep prior knowledge, it explores four key ways in which generative AI is currently applied in ML workflows: as a component of ML pipelines, as a designer of ML pipelines, as a synthesizer of data, and as an analyst.