.Attribute Medicine, Published online: 26 July 2024 doi:10.1038/ s41591-024-03181-6Although large foreign language models (LLMs) show assurance in controlled environments, a study currently reveals their restrictions in real-world medical requests and points the way towards robust assessment and also benchmarking just before clinical make use of.