Research

Browse Papers by Research Topic

Explore publications organized by research themes — from LLM benchmarking and formal verification to AI testing and software security.

▶ LLM Benchmark ▶ Formal Methods ▶ LLM for SE ▶ SE for AI ▶ Security
Topic
Showing 42 papers
ICML '26 2026
Position: Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility
Jialun Cao, Yuk-Kit Chan*, Zixuan Ling*, Wenxuan Wang†, Shuqing Li, Mingwei Liu, Ruixi Qiao, Yuting Han, Chaozheng Wang, Boxi Yu, Pinjia He, Shuai Wang, Zibin Zheng, Michael R. Lyu, Shing-Chi Cheung
Benchmark
Paper Media
ICML '26 2026
SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates on Test-based Benchmark
Boxi Yu, Yang Cao, Yuzhong Zhang, Liting Lin, Junjielong Xu, Zhiqing Zhong, Qinghua Xu, Guancheng Wang, Jialun Cao, Shing-Chi Cheung, Pinjia He, Lionel Briand
Benchmark
Paper
AAAI '25 2025
DomainEval: An Auto-Constructed Benchmark for Multi-Domain Code Generation
Qiming Zhu, Jialun Cao (Co-1st), Yaojie Lu, Hongyu Lin, Xianpei Han, Ben He, Le Sun, Shing-Chi Cheung
Benchmark
Paper Leaderboard Code
ACL '25 2025
CruxEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution
Ruiyang Xu, Jialun Cao (Co-1st), Yaojie Lu, Ming Wen, Hongyu Lin, Xianpei Han, Ben He, Shing-Chi Cheung, Le Sun
Benchmark
Paper Leaderboard Code
ASE '24 2024
JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models
Jialun Cao, Zhiyong Chen*, Jiarong Wu, Shing-chi Cheung, Chang Xu
Benchmark
Paper Leaderboard Code
ASE '24 2024
🏆 Distinguished Paper Award
Towards Understanding the Effectiveness of Large Language Models on Directed Test Input Generation
Zongze Jiang, Ming Wen, Jialun Cao, Xuanhua Shi, Hai Jin
Benchmark
Paper Code
Internetware '25 2025
CodeCleaner: Elevating Standards with A Robust Data Contamination Mitigation Toolkit
Jialun Cao, Songqiang Chen, Wuqi Zhang, Hau Ching Lo, Yeting Li, Shing-Chi Cheung
Benchmark
Paper Code
arXiv 2025
Concerned with Data Contamination? Assessing Countermeasures in Code Language Model
Jialun Cao, Wuqi Zhang, Shing-Chi Cheung
Benchmark
Paper
arXiv 2025
Isolating Language-Coding from Problem-Solving: Benchmarking LLMs with PseudoEval
Jiarong Wu, Songqiang Chen, Jialun Cao (Corresponding), Hau Ching Lo, Shing-Chi Cheung
Benchmark
Paper
arXiv 2025
FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
Dekun Dai, MingWei Liu, Anji Li, Jialun Cao, Yanlin Wang, Chong Wang, Xin Peng, Zibin Zheng
Benchmark
Paper
ACL '25 2025
From Informal to Formal — Incorporating and Evaluating LLMs on Natural Language Requirements to Verifiable Formal Proofs
Jialun Cao, Yaojie Lu, Meiziniu Li, Haoyang Ma, Haokun Li, Mengda He, Cheng Wen, Le Sun, Hongyu Zhang, Shengchao Qin, Shing-Chi Cheung, Cong Tian
10k+ HuggingFace downloads · 37k+ social media reads
Formal Methods
Paper HuggingFace Media
CAV '24 2024
Enchanting Program Specification Synthesis by Large Language Models using Static Analysis and Program Verification
Cheng Wen, Jialun Cao (Corresponding), Jie Su, Zhiwu Xu, Shengchao Qin, Mengda He, Haokun Li, Shing-Chi Cheung, Cong Tian
Formal Methods
Paper Homepage
FM '26 2026
ModelWisdom: An Integrated Toolkit for TLA+ Model Visualization, Digest and Repair
Zhiyong Chen, Jialun Cao, Chang Xu, Shing-Chi Cheung
Formal Methods
Paper Website
arXiv 2026
Can Large Language Models Model Programs Formally?
Zhiyong Chen, Jialun Cao, Jiarong Wu, Chang Xu, Shing-Chi Cheung
Formal Methods
Paper
arXiv 2026
LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation
Dong Xu, Jialun Cao, Guozhao Mo, Junjie Hu, Cheng Wen, Hongyu Lin, Xianpei Han, Shengchao Qin, Cong Tian, Shing-Chi Cheung, Le Sun, Yaojie Lu
Formal Methods
Paper HuggingFace
ICSE '26 2026
EmbedAgent: Benchmarking Large Language Models in Embedded System Development
Ruiyang Xu, Jialun Cao (Co-1st), Mingyuan Wu, Wenliang Zhong, Yaojie Lu, Ben He, Xianpei Han, Shing-Chi Cheung, Le Sun
LLM for SE
Paper
TOSEM '26 2026
Can Emulating Semantic Translation Help LLMs with Code Translation? A Study Based on Pseudocode
Songqiang Chen, Congying Xu, Jingyi Chen, Jialun Cao (Corresponding), Jiarong Wu, Shing-Chi Cheung
LLM for SE
Paper
ACL '26 2026
Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation
Qiming Zhu, Jialun Cao, Xuanang Chen, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, Shing-Chi Cheung
LLM for SE
Paper
FSE '26 2026
iCoRe: Iterative Correlation-Aware Retriever for Bug Reproduction
Junyi Wang, Jialun Cao, Zhongxin Liu
LLM for SE
Paper
TOSEM '26 2026
When Retrieval Augmentation Meets API Documentation: Can LLMs Code with Less-Common Libraries?
Jingyi Chen, Songqiang Chen, Jialun Cao (Corresponding), Jiasi Shen, Shing-Chi Cheung
LLM for SE
Paper
ASEJ '25 2025
A Study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair
Jialun Cao, Meiziniu Li, Ming Wen, Shing-chi Cheung
LLM for SE
Paper Official
ASE '26 2026
Doc2Feat-bench: Evaluating Documentation-Driven Feature Addition
Zhonghao Jiang, Le Deng, Jialun Cao, Michael Pradel, Zhongxin Liu
LLM for SE
Paper Leaderboard HuggingFace
arXiv 2025
What Builds Effective In-Context Examples for Code Generation?
Dongze Li, Songqiang Chen, Jialun Cao (Corresponding), Shing-Chi Cheung
LLM for SE
Paper
arXiv 2025
ReuseDroid: A VLM-empowered Android UI Test Migrator Boosted by Active Feedback
Xiaolei Li, Jialun Cao, Yepang Liu, Shing-Chi Cheung, Hailong Wang
LLM for SE
Paper
arXiv 2025
Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training
Yuanyi Wang, Yifan Yang, Su Lu, Yanggan Gu, Pengkai Wang, Wenjun Wang, Zhaoyi Yan, Congkai Xie, Jianmin Wu, Jialun Cao, Shing-Chi Cheung, Hongxia Yang
LLM for SE
Paper
ICSE '22 2022
DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programs
Jialun Cao, Meiziniu Li, Xiao Chen, Ming Wen, Yongqiang Tian, Bo Wu, Shing-chi Cheung
SE for AI
Paper Code
FSE '23 2023
Testing Coreference Resolution Systems without Labeled Test Sets
Jialun Cao, Yaojie Lu, Ming Wen, Shing-Chi Cheung
SE for AI
Paper Code
TOSEM '22 2022
SemMT: A Semantic-Based Testing Approach for Machine Translation Systems
Jialun Cao, Meiziniu Li, Yeting Li, Ming Wen, Shing-chi Cheung
SE for AI
Paper Code
ASE '24 2024
MR-Adopt: Automatic Deduction of Input Transformation Function for Metamorphic Testing
Congying Xu, Songqiang Chen, Jiarong Wu, Valerio Terragni, Shing-chi Cheung, Hengcheng Zhu, Jialun Cao (Corresponding)
SE for AI
Paper
FSE '25 2025
SemBIC: Semantic-aware Identification of Bug-inducing Commits
Xiao Chen, Hengcheng Zhu, Jialun Cao (Corresponding), Ming Wen, Shing-Chi Cheung
SE for AI
Paper
TOSEM '23 2023
COMET: Coverage-guided Model Generation For Deep Learning Library Testing
Meiziniu Li, Jialun Cao, Yongqiang Tian, Tsz On Li, Ming Wen, Shing-Chi Cheung
SE for AI
Paper Code
TOSEM '25 2025
Enhancing Differential Testing With LLMs For Testing Deep Learning Libraries
Meiziniu Li, Dongze Li, Jianmeng Liu, Jialun Cao, Yongqiang Tian, Shing-Chi Cheung
SE for AI
Paper
AAAI '25 2025
ICM-Assistant: Instruction-tuning Multimodal Large Language Models for Rule-based Explainable Image Content Moderation
Mengyang Wu, Yuzhi Zhao, Jialun Cao, Mingjie Xu, Zhongming Jiang, Xuehui Wang, Qinbin Li, Guangneng Hu, Shengchao Qin, Chi-Wing Fu
SE for AI
Paper
USENIX Sec '22 2022
RegexScalpel: Regular Expression Denial of Service (ReDoS) Defense by Localize-and-Fix
Yeting Li, Yecheng Sun, Zhiwu Xu, Jialun Cao, Yuekang Li, Rongchen Li, Haiming Chen, Shing-Chi Cheung, Yang Liu, Yang Xiao
Security
Paper
USENIX Sec '21 2021
ReDoSHunter: A Combined Static and Dynamic Approach for Regular Expression DoS Detection
Yeting Li, Zixuan Chen, Jialun Cao, Zhiwu Xu, Qiancheng Peng, Haiming Chen, Liyuan Chen, Shing-Chi Cheung
Security
Paper
TASE '24 2024
Fuzzing for Stateful Protocol Implementations: Are We There Yet?
Kunpeng Jian, Yanyan Zou, Yeting Li, Jialun Cao, Menghao Li, Jian Sun, Jingyi Shi, Wei Huo
Security
ASE '25 2025
Vulnerability-Affected Versions Identification: How Far Are We?
Xingchu Chen, Chengwei Liu, Jialun Cao, Yang Xiao, Xinyue Cai, Yeting Li, Jingyi Shi, Tianqi Sun, Haiming Chen, Wei Huo
Security
APSEC '24 2024
SDEFL: A Lightweight Fault Detection and Localization Method for Deep Neural Networks
Bo Yang, Jiawei Hu, Jialun Cao (Corresponding)
Security
FSE '23 2023
Understanding the Bug Characteristics and Fix Strategies of Federated Learning Systems
Xiaohu Du, Xiao Chen, Jialun Cao, Ming Wen, Shing-Chi Cheung, Hai Jin
LLM for SE
Paper
ICSE '21 2021
TransRegex: Multi-modal Regular Expression Synthesis by Generate-and-Repair
Yeting Li, Shuaimin Li, Zhiwu Xu, Jialun Cao, Zixuan Chen, Yun Hu, Haiming Chen, Shing-Chi Cheung
LLM for SE
Paper
ICCD '19 2019
A Learning-Based Framework for Automatic Parameterized Verification
Yongjian Li, Jialun Cao (1st student author), Jun Pang
Formal Methods
Paper
ASE '18 2018
L-CMP: An Automatic Learning-based Parameterized Verification Tool
Jialun Cao, Yongjian Li, Jun Pang
Formal Methods
Paper Code Video
Copyright © 2026 Jialun Cao · Homepage