Artificial intelligenceCybersecuritySoftware development
Code written with AI assistants contains more security vulnerabilities than code written without them.
Compares security weaknesses in code produced with and without AI assistance. Some controlled studies find more insecure solutions and misplaced confidence, while another user study finds no substantial increase in severe bugs. Generated-code benchmarks alone cannot establish the human comparison.
Current
Cases on this claim
Supporting
2
Disputing
1
3 cases from 1 contributor. These numbers count cases people have written. They are not votes, and Koinovera does not decide whether the claim is true.
Summary of the cases and their sourcesAI analysis2 sources added since it was written
AI summary
Written by AI from the published cases and the sources they cite. It describes what each side argues and what the readable sources show, and is not a ruling from Koinovera. The generation date does not guarantee that the claim wording or sources are unchanged. It only changes when someone asks for an update.
AI generatedCurrent evidence: Mostly supportive
An AI reading of the cases so far and the sources they cite. The label above rests on the sources alone, not on what the cases assert or how many take a side. It is not a verdict from Koinovera on whether the claim is true, and it may change as evidence is added or questioned.
A large-scale user study indicates that participants using an AI code assistant produced code with significantly more security vulnerabilities compared to those who did not use such tools. Additionally, users with AI assistance were more likely to perceive their code as secure, despite the increased presence of vulnerabilities. [1]
Do Users Write More Insecure Code with AI Assistants?
arXiv.org
View PDF
HTML (experimental)
Abstract:We conduct the first large-scale user study examining how users interact with an AI Code assistant to solve a variety of security related tasks across different programming languages. Overall, we find that participants who had access to an AI assistant based on OpenAI's codex-davinci-002 model wrote significantly less secure code than those without access. Additionally, participants with access to an AI assistant were more likely to believe they wrote secure code than those without access to the AI assistant. Furthermore, we find that participants who trusted the AI less and engaged more with the language and format of their prompts (e.g. re-phrasing, adjusting temperature) provided code with fewer security vulnerabilities. Finally, in order to bette
Every argument made about this claim, each with its own reasoning and sources.
Strongest orders cases by how readers have rated their reasoning, weighted by the standing of those readers and lowered when a case rests on poorly rated sources. Cases nobody has rated sit in the middle, newest first. It never counts how many cases or sources there are.