LLM for Software Engineering

Featured Work

TOSEM '26

Can Emulating Semantic Translation Help LLMs with Code Translation?

Songqiang Chen, Congying Xu, Jingyi Chen, Jialun Cao (Corresponding), Jiarong Wu, Shing-Chi Cheung

Pseudocode-based translation emulates how humans perform semantic translation — first decomposing a program’s logic into pseudocode, then recomposing it in the target language. This approach complements direct translation, especially when moving from flexible to rigid languages or handling low-training-resource PLs.

Paper

💡 Pseudocode-based translation complements direct translation — combining both with test-based selection leverages their complementary strengths.

6 languages9,690 tasks

Java (Source)

Decompose

Pseudocode

Recompose

Python (Target)

Featured Work

ICSE '26

EmbedAgent: Benchmarking LLMs in Embedded System Development

Ruiyang Xu, Jialun Cao (Co-1st), Mingyuan Wu, Wenliang Zhong, Yaojie Lu, Ben He, Xianpei Han, Shing-Chi Cheung, Le Sun

Can LLMs handle the unique challenges of embedded development — register-level programming, hardware constraints, and real-time requirements? EmbedAgent puts them to the test.

Paper

💡 LLMs achieve 73.8% pass@1 on Pico but struggle with register-level programming on more complex boards — embedded development remains challenging.

126 cases9 components73.8% pass@1 (Pico)

User Prompt

"Initialize GPIO pin 5 as output for LED control, and set up UART at 115200 baud for serial communication on ESP32 (ESP-IDF)."

Featured Work

FSE '26

iCoRe: Iterative Correlation-Aware Retriever for Bug Reproduction

Junyi Wang, Jialun Cao, Zhongxin Liu

Generating a bug reproduction test (BRT) from an issue description requires finding the right code context iteratively. iCoRe correlates textual semantics with call-graph structure across retrieval rounds, closing a retrieval↔generation feedback loop that progressively converges on a working BRT.

Paper

💡 Iterative correlation-aware retrieval improves bug reproduction by 19.7%–31.7% over single-round retrieval methods.

SWT-bench Lite: 42.0% F2PTDD-bench Verified: 52.8%

📄

Bug Report

NullPointerException in UserService.getProfile() when session expires

Round 1

🔍

Initial Query

Extract key terms: UserService, getProfile, NullPointerException, session

📁

Retrieve Files

UserService.java, SessionManager.java (partial match)

🔗

Correlation Analysis

Detect dependency: UserService → SessionManager → CacheProvider

Round 2

🔄

Refined Query

Add correlated context: CacheProvider, session expiry handling

📁

Retrieve More

CacheProvider.java, SessionConfig.java — full context recovered

✔

Bug Reproduced

Complete test case generated with all necessary context files

Featured Work

ACL '26

Across Programming Language Silos: Cross-Lingual Retrieval-augmented Code Generation

Qiming Zhu, Jialun Cao, Xuanang Chen, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, Shing-Chi Cheung

Can retrieving code examples from other programming languages help generation? This study probes cross-lingual retrieval-augmented code generation across language silos, finding that transfer is real but non-trivial and unequal across language pairs.

Paper

💡 Cross-lingual code retrieval helps generation but the transfer is non-trivial and unequal across language pairs.

13 languages~14K instances

📚 Retrieved Examples (Cross-Lingual)

Java

Arrays.sort(arr); // built-in TimSort

C++

void quickSort(vector<int>& v, int lo, int hi) {
if (lo >= hi) return;
int pivot = partition(v, lo, hi);
quickSort(v, lo, pivot-1);
quickSort(v, pivot+1, hi);
}

JavaScript

arr.sort((a, b) => a - b);

Python

sorted_list = sorted(arr, key=lambda x: x)

➔

✍ Generated Code (Python)

Featured Work

TOSEM '26

When RAG Meets API Documentation: Can LLMs Code with Less-Common Libraries?

Jingyi Chen, Songqiang Chen, Jialun Cao, Jiasi Shen, Shing-Chi Cheung

Examines how RAG-enhanced LLMs utilize API documentation from less common libraries to improve code generation.

Paper

💡 Example code contributes the most — not descriptive texts or parameter lists — and RAG boosts performance by 83–220%.

API Docs

→

RAG Retrieval

→

LLM

→

Code

+83–220%

Featured Work

ASEJ '25

A Study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair

Jialun Cao, Meiziniu Li, Ming Wen, Shing-chi Cheung

Investigates ChatGPT's ability to repair bugs in deep learning programs through strategic prompt design and dialogue strategies.

Paper Official

💡 ChatGPT can detect bad code smell, API misuse, and deprecation in DL programs — but prior fault localization tools only achieve ~30% accuracy.

Zero-shot

→

Few-shot

→

Dialogue

Featured Work

ASE '26

Doc2Feat-bench: Evaluating Documentation-Driven Feature Addition

Zhonghao Jiang, Le Deng, Jialun Cao, Michael Pradel, Zhongxin Liu

Tests how well LLMs can implement software features described in natural language documentation rather than explicit code instructions.

Paper Leaderboard

💡 Top LLMs achieve only 28.07% task success rate — LLMs remain unready for fully natural language-driven development.

NL Description

→

LLM Agent

→

Result

28.07% — Not ready

Featured Work

arXiv

What Builds Effective In-Context Examples for Code Generation?

Dongze Li, Songqiang Chen, Jialun Cao, Shing-Chi Cheung

Investigates which code features (naming, formatting, structure) most influence effectiveness of in-context learning for code generation.

Paper

💡 Appropriate variable/function naming is crucial — their elimination causes up to 30 percentage points performance decrease.

Naming ★★★

Structure ★★

Formatting ★

Featured Work · Metamorphic Testing

ASE '24

MR-Adopt: Automatic Deduction of Input Transformation Function for Metamorphic Testing

Congying Xu, Songqiang Chen, Jiarong Wu, Valerio Terragni, Shing-chi Cheung, Hengcheng Zhu, Jialun Cao (Corresponding)

Uses LLMs to automatically generate input transformations for encoded metamorphic relations, making them reusable with new source inputs.

Paper

💡 Successfully produces transformations for 72% of encoded MRs, outperforming vanilla GPT-3.5 by 33.33% and boosting mutation scores by ~19%.

72% applicable+33.33% over GPT-3.5+19% mutation score

Encoded MR

→

LLM Deduction

→

Transformation Function

72% applicable

Featured Work · Bug-inducing Commits

FSE '25

SemBIC: Semantic-aware Identification of Bug-inducing Commits

Xiao Chen, Hengcheng Zhu, Jialun Cao (Corresponding), Ming Wen, Shing-Chi Cheung

Identifies bug-inducing commits using semantic analysis rather than relying solely on syntactic diff or traditional SZZ algorithms.

Paper

💡 Semantic-aware analysis significantly outperforms syntactic-only approaches in pinpointing the exact commit that introduced a bug.

Commit History

→

Semantic Analysis

→

Bug-inducing Commit

Featured Work · DL Library Testing

TOSEM '23

COMET: Coverage-guided Model Generation for DL Library Testing

Meiziniu Li, Jialun Cao, Yongqiang Tian, Tsz On Li, Ming Wen, Shing-Chi Cheung

Generates DL models guided by coverage feedback, exercising more layer inputs, parameter values, and layer sequences — surfacing 32 new bugs across 8 DL libraries including TensorFlow and MXNet.

Paper Code

💡 Coverage-guided model generation surfaces 32 new bugs across 8 DL libraries including TensorFlow and MXNet.

32 new bugs8 DL libraries69.7% layer-input cov.

Random Model

→

Coverage Feedback

→

Guided Generation

→

32 Bugs Found

Featured Work · Differential Testing

TOSEM '25

Enhancing Differential Testing with LLMs for Testing Deep Learning Libraries

Meiziniu Li, Dongze Li, Jianmeng Liu, Jialun Cao, Yongqiang Tian, Shing-Chi Cheung

DLLens uses LLMs to find equivalent API implementations across DL libraries and generate diverse test inputs to uncover bugs.

Paper

💡 Synthesized counterparts for 1.84× more APIs than prior methods and discovered 71 bugs (46 confirmed new, 10 already patched).

71 bugs1.84× more APIs46 confirmed new

TensorFlow API

↔

PyTorch API

→

Diff Test

→

71 Bugs