publications

2026

1. Mendel Gödel Machine: Recursive Self-Improving Coding Agents via Comparative Evolution

   Changzhi Liu, Yilun Liu, Sikuan Yan, Volker Tresp, and 1 more author

   arXiv preprint arXiv:2608.07645, 2026

   Abs PDF Code Website

   Self-improving coding agents that iteratively rewrite their own source code have demonstrated impressive performance on coding tasks. However, existing solutions generally derive self-modification from a single failure trajectory at a time, overlooking rich comparative signals available in the agent’s expanding archive of past attempts. According to Mendelian principles of controlled inheritance, we introduce Mendel Gödel Machine (MGM). In addition to the general single-trajectory clonal mutation, MGM includes two new types of self-modification that better utilizes evidences accumulated: the reaction-norm mutation edits an agent based on its trajectories on multiple tasks simultaneously, and the cross-lineage hybridization edits an agent using the trajectory of a reference agent from another lineage on the same task. Under an additive fitness landscape model, we prove theoretically and demonstrate via controlled surrogate simulation that the new strategies facilitate a faster and better convergence over single-trajectory baselines. Experiments on SWE-bench and Polyglot confirm MGM’s consistent improvement in performance, efficiency, and generalizability.

2. EMNLP 2026

   MemWM: Memory-Augmented Text-Based World Model

   Yujun Wang, Tao Zhang, Jinhe Bi, Aniri, Wenxuan Ye, Boliang Liu, Sikuan Yan, and 5 more authors

   In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP), Main Conference, 2026

   Abs Bib PDF

   World models are increasingly used to support planning in agents by predicting how environment states evolve in response to agent actions. Yet fluent next-state predictions can still omit task-critical facts, corrupt product attributes, or apply incorrect transition rules. To address such systematic prediction errors, we introduce MemWM, a memory-augmented text-based world model. MemWM uses world memory, a curated memory bank of transition rules, state caches, and hard-to-predict facts, to condition next-state imagination. We evaluate factual state preservation with Structured State Fidelity (SSF), which scores predicted states through benchmark-specific facts and fields. Compared with SFT, memory-augmented training improves SSF by up to 206.3%. In the full planning setting, we keep the policy model frozen and provide policy-side world skill: retrieved task-level skills and step-wise corrective guidance for action selection. Across ALFWorld, WebShop, and ScienceWorld, memory-augmented agents improve downstream success over an SFT-trained world-model agent, with up to a 65.4% relative gain. Sensitivity analyses further show that retrieved memory improves task success and efficiency under different memory and action-budget settings.

   @inproceedings{wang2026memwm,
     title = {MemWM: Memory-Augmented Text-Based World Model},
     author = {Wang, Yujun and Zhang, Tao and Bi, Jinhe and Aniri and Ye, Wenxuan and Liu, Boliang and Yan, Sikuan and Wang, Shuning and Zhou, Xuebing and Pirk, S{\"o}ren and Sch{\"u}tze, Hinrich and Ma, Yunpu},
     booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP), Main Conference},
     year = {2026},
   }

3. MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution

   Zefeng Wang, Minxi Yan, Jinhe Bi, Sikuan Yan, and 2 more authors

   arXiv preprint arXiv:2607.05297, 2026

   Abs PDF

   Recent LLM agents tackle increasingly long-horizon, open-ended tasks, and external skills, reusable procedural knowledge supplied to the agent, further extend this capability. However, a fixed, hand-authored skill is rarely optimal, and cannot adapt to the diversity of tasks an agent encounters. Self-improving agents address this by rewriting their own skill files from execution traces, yielding meaningful gains on challenging benchmarks. Yet such self-evolution remains non-recursive: it improves only the task skill while the improvement procedure is authored once and held fixed. We introduce MetaSkill-Evolve, a two-timescale framework that makes agentic skill improvement recursive: every branch carries both a task skill and a branch-local meta-skill whose five components parameterise the Analyzer, Retriever, Allocator, Proposer, and Evolver agents of the improvement pipeline. Task skills evolve on a fast loop while the meta-skill evolves on a slower one under the same pipeline applied to itself, with no additional model or objective. With all five pipeline agents sharing a single frozen backbone, MetaSkill-Evolve outperforms no-skill, static-skill, and single-level evolution baselines on three agentic benchmarks (OfficeQA, SealQA, ALFWorld), improving held-out test accuracy over the raw backbone by +23.54, +16.09, and +1.92 points respectively.

4. Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents

   Sikuan Yan, Ahmed Bahloul, Ercong Nie, Susanna Schwarzmann, and 3 more authors

   Under review, 2026

   Abs PDF Code

   Memory-augmented LLM agents enable interactions that extend beyond finite context windows by storing, updating, and reusing information across sessions. However, training such agents with reinforcement learning in multi-session environments is challenging because memory turns the agent’s past actions into part of its future environment. Once different rollouts write, update, or delete different memories, they no longer share the same intermediate memory state, making trajectory-level comparisons fundamentally unfair. This violates a key assumption behind group-relative methods such as GRPO, where rollouts are compared as if they were sampled from the same effective environment. Consequently, trajectory-level rewards provide noisy or biased credit signals for long-horizon memory operations. To address this challenge, we introduce Memory-R2, a training framework for long-horizon memory-augmented LLM agents. Its core algorithm, LoGo-GRPO, combines local and global group-relative optimization. The global objective preserves end-to-end learning from long-horizon trajectory-level rewards, while local rerollouts compare different memory-operation outcomes from the same intermediate memory state, yielding fairer group comparisons and more precise supervision for memory construction. Beyond credit assignment, Memory-R2 jointly optimizes memory formation and memory evolution with a shared-parameter co-learning design, where a fact extractor and a memory manager are instantiated from the same LLM backbone through role-specific prompts. To stabilize multi-step RL over long memory horizons, we adopt a progressive curriculum that increases the training horizon from 8 to 16 to 32 sessions. Together, these components provide an effective training paradigm for memory-augmented LLM agents in long-horizon multi-session settings.

5. EMNLP 2026 Main

   PyraVid: Hierarchical Multimodal Memory for Long-Horizon Video Reasoning

   Sikuan Yan, Sicheng Dong, Haotong Wang, Ercong Nie, and 7 more authors

   In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP), Main Conference, 2026

   Abs Bib PDF Code

   Memory has become an increasingly important component of agentic systems, as these systems are expected to reason over long-term experience. However, prior work has largely focused on unimodal memory, leaving multimodal memory relatively underexplored despite its central role in real-world applications. Compared with unimodal settings, multimodal memory introduces additional challenges, including heterogeneous input integration, person-centric information alignment, and evidence aggregation across different granularities. We present PyraVid, a hierarchical multimodal memory framework inspired by Event Segmentation Theory from cognitive science. PyraVid organizes long videos into a coarse-to-fine pyramid structure, enabling structured memory access and effective evidence aggregation. It further supports structure-guided memory expansion with pruning, allowing the retrieval of related events with strong causal connectivity but low semantic similarity while reducing noise. Experiments on multiple long-video understanding benchmarks show that PyraVid consistently improves performance across datasets, model scales, and question types, highlighting the effectiveness of hierarchical multimodal memory for long-horizon reasoning.

   @inproceedings{yan2026pyravid,
     title = {PyraVid: Hierarchical Multimodal Memory for Long-Horizon Video Reasoning},
     author = {Yan, Sikuan and Dong, Sicheng and Wang, Haotong and Nie, Ercong and Liu, Yilun and Bi, Jinhe and Xu, Yingjie and Schwarzmann, Susanna and Trivisonno, Riccardo and Tresp, Volker and Ma, Yunpu},
     booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP), Main Conference},
     year = {2026},
   }

6. ICML 2026

   EchoRL: Reinforcement Learning via Rollout Echoing

   Jinhe Bi, Aniri, Minglai Yang, Xingcheng Zhou, Wenke Huang, Sikuan Yan, and 6 more authors

   In Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026

   Abs Bib PDF

   Reinforcement Learning with Verifiable Rewards is an effective route for post-training to strengthen the reasoning capability of large language models. However, as training proceeds, the learning signal can collapse thus makes the training gain become marginal and ineffective. Specifically, a growing fraction of prompts’ rollouts become advantage-degenerated: all the self-generated rollouts show verified-success, making the standard deviation over their rewards be zero; accordingly each rollout’s advantage becomes degenerated (zero) as well. Given such rollouts’ advantages, the policy-gradient for model optimization eventually vanishes, capping the training performance. We argue that some of these rollouts still contain valuable learning signals but unfortunately omitted with the existing RLVR methods. In this paper, inspired through analyzing the entropy pattern behind golden trajectories produced by external expert models, we propose EchoRL for better exploiting the advantage-degenerated rollouts to further improve the training performance. EchoRL is a lightweight module that first identifies an EchoClip from verified-success rollouts based on their step-level entropy values, and then feeds this clip back as an auxiliary supervision signal in the RL objective. Extensive experiments across 10 benchmarks, 5 LLM backbones, and 4 popular RLVR post-training methods demonstrate that EchoRL consistently improves RLVR post-training with minimal overhead.

   @inproceedings{bi2026echorl,
     title = {EchoRL: Reinforcement Learning via Rollout Echoing},
     author = {Bi, Jinhe and Aniri and Yang, Minglai and Zhou, Xingcheng and Huang, Wenke and Yan, Sikuan and Wang, Yujun and Cao, Zixuan and F{\"a}rber, Michael and Xiao, Xun and Tresp, Volker and Ma, Yunpu},
     booktitle = {Proceedings of the 43rd International Conference on Machine Learning (ICML)},
     year = {2026},
   }

7. Reasoning Compression with Mixed-Policy Distillation

   Han Yang, Mingyan Wu, Bailan He, Zeyu Cao, Sikuan Yan, and 2 more authors

   Under review, 2026

   Abs PDF

   Reasoning-centric large language models (LLMs) achieve strong performance by generating intermediate reasoning trajectories, but often incur excessive token usage and high inference-time decoding cost. We observe that, when solving the same problems, larger reasoning models can often produce more concise traces, whereas smaller reasoning models tend to generate longer and more redundant trajectories. This is especially problematic in real-world deployment, where memory, latency, and serving-cost constraints often favor smaller models. Our observations suggest that reasoning compression can be transferred from large models to small ones rather than enforced through explicit length constraints. Based on this insight, we propose Mixed-Policy Distillation (MPD), a reasoning compression framework that transfers concise reasoning behavior from a larger-sized teacher to a smaller student by distilling teacher-compressed student trajectories. Unlike on-policy distillation, which aligns the student with teacher distributions over verbose student trajectories, or off-policy distillation, which relies on teacher-generated trajectories and may suffer from distribution mismatch, MPD combines the strengths of both. Given a student-sampled trajectory, the teacher rewrites it into a more concise reasoning trace, and the student is trained via KL-based alignment on the compressed trajectory. This preserves student-policy exploration while injecting teacher-guided compression. Experiments on Qwen3-1.7B show that MPD reduces token usage by up to 27.1% while improving performance across multiple reasoning benchmarks, demonstrating an effective approach to efficient small-model reasoning.

8. EMNLP 2026

   Routing-Free Mixture-of-Experts

   Yilun Liu, Jinru Han, Sikuan Yan, Volker Tresp, and 1 more author

   In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2026

   Abs Bib PDF

   Standard Mixture-of-Experts (MoE) models rely on centralized routing mechanisms that introduce rigid inductive biases. We propose Routing-Free MoE which eliminates any hard-coded centralized designs including external routers, Softmax, Top-K and load balancing, instead encapsulating all activation functionalities within individual experts and directly optimized through continuous gradient flow, enabling each expert to determine its activation entirely on its own. We introduce a unified adaptive load-balancing framework to simultaneously optimize both expert-balancing and token-balancing objectives through a configurable interpolation, allowing flexible and customizable resource allocation. Extensive experiments show that Routing-Free MoE can consistently outperform baselines with better scalability and robustness. We analyze its behavior in detail and offer insights that may facilitate future MoE design ad optimization.

   @inproceedings{liu2026routingfree,
     title = {Routing-Free Mixture-of-Experts},
     author = {Liu, Yilun and Han, Jinru and Yan, Sikuan and Tresp, Volker and Ma, Yunpu},
     booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
     year = {2026}
   }

9. ACL 2026 Main

   Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning

   Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, and 9 more authors

   In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL), Main Conference, 2026

   Abs Bib PDF

   Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of NLP tasks, but they remain fundamentally stateless, constrained by limited context windows that hinder long-horizon reasoning. Recent efforts to address this limitation often augment LLMs with an external memory bank, yet most existing pipelines are static and heuristic-driven, lacking a learned mechanism for deciding what to store, update, or retrieve. We present Memory-R1, a reinforcement learning (RL) framework that equips LLMs with the ability to actively manage and utilize external memory through two specialized agents: a Memory Manager that learns structured operations, including ADD, UPDATE, DELETE, and NOOP; and an Answer Agent that pre-selects and reasons over relevant entries. Both agents are fine-tuned with outcome-driven RL (PPO and GRPO), enabling adaptive memory management with minimal supervision. With only 152 training QA pairs, Memory-R1 outperforms strong baselines and generalizes across diverse question types, three benchmarks (LoCoMo, MSC, LongMemEval), and multiple model scales (3B-14B).

   @inproceedings{yan2026memoryr1,
     title = {Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning},
     author = {Yan, Sikuan and Yang, Xiufeng and Huang, Zuchao and Nie, Ercong and Ding, Zifeng and Li, Zonggen and Ma, Xiaowen and Bi, Jinhe and Kersting, Kristian and Pan, Jeff Z. and Sch{\"u}tze, Hinrich and Tresp, Volker and Ma, Yunpu},
     booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL), Main Conference},
     year = {2026},
   }

10. ACL 2026 Findings

    Self-Evolving Multi-Agent Systems via Textual Backpropagation

    Xiaowen Ma, Yunpu Ma, Chenyang Lin, Sikuan Yan, and 5 more authors

    In Findings of the Association for Computational Linguistics (ACL Findings), 2026

    Abs Bib PDF

    Leveraging multiple Large Language Models(LLMs) has proven effective for addressing complex, high-dimensional tasks, but current approaches often rely on static, manually engineered multi-agent configurations. To overcome these constraints, we present the Agentic Neural Network(ANN), a framework that conceptualizes multi-agent collaboration as a layered neural network architecture. In this design, each agent operates as a node, and each layer forms a cooperative "team" focused on a specific subtask. Agentic Neural Network follows a two-phase optimization strategy: (1) Forward Phase-Drawing inspiration from neural network forward passes, tasks are dynamically decomposed into subtasks, and cooperative agent teams with suitable aggregation methods are constructed layer by layer. (2) Backward Phase-Mirroring backpropagation, we refine both global and local collaboration through iterative feedback, allowing agents to self-evolve their roles, prompts, and coordination. This neuro-symbolic approach enables ANN to create new or specialized agent teams post-training, delivering notable gains in accuracy and adaptability. Across four benchmark datasets, ANN surpasses leading multi-agent baselines under the same configurations, showing consistent performance improvements. Our findings indicate that ANN provides a scalable, data-driven framework for multi-agent systems, combining the collaborative capabilities of LLMs with the efficiency and flexibility of neural network principles. We plan to open-source the entire framework.

    @inproceedings{ma2026selfevolving,
      title = {Self-Evolving Multi-Agent Systems via Textual Backpropagation},
      author = {Ma, Xiaowen and Ma, Yunpu and Lin, Chenyang and Yan, Sikuan and Bi, Jinhe and Cao, Zixuan and Tian, Yijun and Tresp, Volker and Sch{\"u}tze, Hinrich},
      booktitle = {Findings of the Association for Computational Linguistics (ACL Findings)},
      year = {2026},
    }

11. EMNLP 2026

    PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selection

    Jinhe Bi, Aniri, Zengjie Jin, Yifan Wang, Danqi Yan, Wenke Huang, Xiaowen Ma, Sikuan Yan, and 6 more authors

    In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2026

    Abs Bib PDF

    Leveraging multiple Large Language Models(LLMs) has proven effective for addressing complex, high-dimensional tasks, but current approaches often rely on static, manually engineered multi-agent configurations. To overcome these constraints, we present the Agentic Neural Network(ANN), a framework that conceptualizes multi-agent collaboration as a layered neural network architecture. In this design, each agent operates as a node, and each layer forms a cooperative "team" focused on a specific subtask. Agentic Neural Network follows a two-phase optimization strategy: (1) Forward Phase-Drawing inspiration from neural network forward passes, tasks are dynamically decomposed into subtasks, and cooperative agent teams with suitable aggregation methods are constructed layer by layer. (2) Backward Phase-Mirroring backpropagation, we refine both global and local collaboration through iterative feedback, allowing agents to self-evolve their roles, prompts, and coordination. This neuro-symbolic approach enables ANN to create new or specialized agent teams post-training, delivering notable gains in accuracy and adaptability. Across four benchmark datasets, ANN surpasses leading multi-agent baselines under the same configurations, showing consistent performance improvements. Our findings indicate that ANN provides a scalable, data-driven framework for multi-agent systems, combining the collaborative capabilities of LLMs with the efficiency and flexibility of neural network principles. We plan to open-source the entire framework.

    @inproceedings{bi2026prism,
      title = {PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selection},
      author = {Bi, Jinhe and Aniri and Jin, Zengjie and Wang, Yifan and Yan, Danqi and Huang, Wenke and Ma, Xiaowen and Yan, Sikuan and Hecker, Artur and Ye, Mang and Xiao, Xun and Sch{\"u}tze, Hinrich and Tresp, Volker and Ma, Yunpu},
      booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2026}
    }

2025

1. EMNLP 2025

   TCP: A Benchmark for Temporal Constraint-Based Planning

   Zifeng Ding, Sikuan Yan, Moy Yuan, Xianglong Hu, and 2 more authors

   In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025

   Abs Bib PDF

   Temporal reasoning and planning are essential capabilities for large language models (LLMs), yet most existing benchmarks evaluate them in isolation and under limited forms of complexity. To address this gap, we introduce the Temporal Constraint-based Planning (TCP) benchmark that jointly assesses both capabilities. Each instance in TCP features a naturalistic dialogue around a collaborative project, where diverse and interdependent temporal constraints are explicitly or implicitly expressed, and models must infer an optimal schedule that satisfies all constraints. To construct TCP, we generate abstract problem prototypes that are then paired with realistic scenarios from various domains and enriched into dialogues using an LLM. A human quality check is performed on a sampled subset to confirm the reliability of our benchmark. We evaluate state-of-the-art LLMs and find that even the strongest models may struggle with TCP, highlighting its difficulty and revealing limitations in LLMs’ temporal constraint-based planning abilities. We analyze underlying failure cases, open source our benchmark, and hope our findings can inspire future research.

   @inproceedings{ding2025tcp,
     title = {TCP: A Benchmark for Temporal Constraint-Based Planning},
     author = {Ding, Zifeng and Yan, Sikuan and Yuan, Moy and Hu, Xianglong and Lin, Fangru and Vlachos, Andreas},
     booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
     year = {2025},
   }