Security Code Review by LLMs: A Deep Dive into Responses

no code implementations • 29 Jan 2024 • Jiaxin Yu, Peng Liang, Yujia Fu, Amjed Tahir, Mojtaba Shahin, Chong Wang, Yangxiao Cai

To explore the challenges of applying LLMs in practical code review for security defect detection, this study compared the detection performance of three state-of-the-art LLMs (Gemini Pro, GPT-4, and GPT-3. 5) under five prompts on 549 code files that contain security defects from real-world code reviews.

Defect Detection

Copilot Refinement: Addressing Code Smells in Copilot-Generated Python Code

no code implementations • 25 Jan 2024 • Beiqi Zhang, Peng Liang, Qiong Feng, Yujia Fu, Zengyang Li

The results show that 8 out of 10 types of Python smells can be detected in Copilot-generated Python code, among which Multiply-Nested Container is the most common one.

Code Generation