Biography

Zhiyuan Zhang (张之远) received the Bachelor’s degree of Science in Computer Science from Peking University in 2019 and received the Doctoral degree of Science in Computer Science from Peking University in 2024, supervised by Prof. Xu Sun (孙栩). His research interests include deep learning, natural language processing and the application to quantitative finance.

Research Interests

• Deep Learning
• Natural Language Processing
• Optimization, Robustness and Security
• Quantitative Finance

Education

• Sept. 2019 - Jul. 2024
  PhD student, majoring in Computer Software and Theory, at School of Computer Science, Peking University.
  Advisor: Xu Sun (孙栩). GPA: 3.85/4.00, Rank: 1.

• Sept. 2015 - Jul. 2019
  Bachelor, majoring in Science in Computer Science, at School of EECS, Peking University.
  GPA: 3.76/4.00, Rank: 6/204.

Selected Publications

Zhiyuan Zhang has 30 papers, including 16 first-author papers. The papers have been cited 1000+ times in Google Scholar up to 2024. Followings are selected publications (^#: Equal Contribution).

Links: [Google Scholar], [OpenReview], [DBLP], [Semantic Scholar].

• Fed-FA: Theoretically Modeling Client Data Divergence for Federated Language Backdoor Defense
  Zhiyuan Zhang, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun
  In NeurIPS 2023

• Incorporating Pre-trained Model Prompting in Multimodal Stock Volume Movement Prediction
  Ruibo Chen^#, Zhiyuan Zhang^#, Yi Liu, Ruihan Bao, Keiko Harimoto, Xu Sun
  In KDD-MLF 2023 (ML in Finance)

• ASAT: Adaptively Scaled Adversarial Training in Time Series
  Zhiyuan Zhang, Wei Li, Ruihan Bao, Keiko Harimoto, Yunfang Wu, Xu Sun
  In Neurocomputing 2023

• Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias
  Zhiyuan Zhang, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun
  In Findings of ACL 2023

• No Stock is an Island: Learning Internal and Relational Attributes of Stocks with Contrastive Learning
  Shicheng Li^#, Wei Li^#, Zhiyuan Zhang, Ruihan Bao, Keiko Harimoto, Xu Sun
  Comment: ⭐ Selected as the best paper in IJCAI-FinNLP 2022!
  In IJCAI-FinNLP 2022 (Financial Technology and NLP)

• How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data
  Zhiyuan Zhang, Lingjuan Lyu, Weiqiang Wang, Lichao Sun, Xu Sun
  Comment: Also related to our previous paper: Neural Network Surgery [pdf].
  In ICLR 2022

• GA-SAM: Gradient-Strength based Adaptive Sharpness-Aware Minimization for Improved Generalization
  Zhiyuan Zhang, Ruixuan Luo, Qi Su and Xu Sun
  Comment: The theoretical analysis of why and how the parameter robustness relates to the generalization ability via the distributional shift between training and test sets as a bridge. Also related to our previous papers: Parameter Corruption [pdf] and Adversarial Parameter Defense [pdf].
  In EMNLP 2022

• PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation
  Ruixuan Luo, Jingjing Xu, Yi Zhang, Zhiyuan Zhang, Xuancheng Ren, Xu Sun
  Comment: ⭐ A highly influential Chinese word segmentation open source toolkit: PKUSEG [GitHub].
  Preprint

• Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption
  Xu Sun^#, Zhiyuan Zhang^#, Xuancheng Ren, Ruixuan Luo, Liangyou Li
  Comment: The analysis of the robustness against parameter corruptions or perturbations.
  In AAAI 2021

• Building an Ellipsis-aware Chinese Dependency Treebank for Web Text
  Xuancheng Ren, Xu Sun, Ji Wen, Bingzhen Wei, Weidong Zhan, Zhiyuan Zhang
  Comment: Releasing a Chinese dependency treebank dataset [GitHub] of 319 weibos, containing 572 sentences with omissions restored and contexts reserved.
  In LREC 2018

Patents

• 一种模型训练方法及装置
  China Patent, No.: 202110221446.7, 2022
  Zhiyuan Zhang, Xu Sun, Ruixuan Luo, Bin He (Huawei)

• 一种模型的更新方法、装置及设备
  China Patent, No.: 202111094859.X, 2021
  Zhiyuan Zhang, Lingjuan Lyu, Weiqiang Wang (Ant Group)

• 一种模型训练方法及装置
  China Patent, No.: 202110475677.0, 2021
  Zhiyuan Zhang, Xuancheng Ren, Xu Sun, Bin He, Li Qian (Huawei)

Awards

Phd Student Period

• Peking University May Fourth Scholarship (北京大学五四奖学金), 2023
• Pacemaker to Merit Student (三好学生标兵), 2023
• Peking University President Scholarship (Highest Ph.D. scholarship, 北京大学校长奖学金), 2019, 2020, 2021
• Financial Technology and Natural Language Processing 2022: Best Paper Award, 2022
• Yang Fuqing & Wang Yangyuan Academician Scholarship, 2022
• Award for Scientific Research in Peking University, 2022
• Merit Student in Peking University (三好学生), 2021
• Huatai Securities Technology Scholarship, 2021
• Award for Academic Excellents in Peking University, 2020

Undergraduate Period

• Outstanding Graduates in Peking University (优秀毕业生), 2019
• Merit Student in Peking University (三好学生), 2016, 2017
• National Scholarship (国家奖学金), 2016

Courses and Math & Programming skills

Zhiyuan Zhang has a solid foundation in mathematical and programming.

Courses

He has scored 95+/100 in all the following courses:

• Mathmetical Courses: Calculus (高等数学上下), Probability & Statistics (概率统计), Set Theory & Graph Theory (集合论图论), Combinatorial Mathematics (组合数学), Number Theory (数论)
• Physics Courses: Mechanics (力学), Electromagnetism (电磁学)
• Programming Courses: Introduction to Computing (计算概论), Practice of Programming (程序设计实习), Data Structures & Algorithms (数据结构与算法), Algorithms Analysis & Design (算法分析与设计)

Undergraduate Period Math & Programming Awards

• Ranked 105 of about 50k (about 0.21%) in Preliminary Round of Alibaba Global Mathematics Competition (and entered the finals of the competition), 2018
• Second Prize in Peking University ACM, 2016

High school Period Math & Programming Awards

• First Prize in Chinese Mathematical Olympiad (in Provinces) (ranked 26 in Zhejiang Province, 全国高中数学联赛浙江赛区一等奖).
• First Prize in Ruida Cup High School Mathematics Competition (ranked 2, 睿达杯高中数学竞赛一等奖).
• Bronze Medal in Chinese Southeast Mathematical Olympiad (中国东南地区数学奥林匹克铜牌).
• Second Prize in Chinese Physics Olympiad (in Provinces) (全国中学生物理竞赛浙江赛区二等奖).
• Second Prize in National Olympiad in Informatics in Provinces (NOIP) (全国青少年信息学奥林匹克联赛浙江赛区二等奖).

Internships

• Research Intern at Tencent, WeChat AI, 2022 - 2024
• Research Intern at Ant Group, 2021
• Research Grant Support from Mizuho Securities, 2021 - 2024
• Research Grant Support from Huawei Noah’s Ark Lab, 2020 - 2021

Services

• Reviewer or Program Committee (PC) for AAAI, ACL, TACL, ECML, NLPCC, etc.
• Contributor for a paper list about robustness and security in deep learning on GitHub (1k+ stars at GitHub up to 2024: github.com/THUYimingLi/backdoor-learning-resources).
• Teaching Assistant (TA) for “Introduction to Natural Language Processing”, “Foundations of Computer Application”.

Invited Talks

• AI Time, 首场大模型专场, 四位讲者分享大模型研究畅聊ChatGPT: Mitigating Backdoors in Large Language Models (消除大模型中的后门), March 2023
• Shanghai AI Lab: A Study of Parameter Corruption of Neural Networks (神经网络参数扰动的研究), Aug. 2022
• Huawei - Peking University Academic Forum: Limitation and Improvement of Federated Aggregation Algorithm on Natural Language Processing Tasks (联邦聚合算法在自然语言处理任务上的局限性及改进), July 2022
• Beijing Language & Culture University: A Overview of Adversarial Attacks (对抗攻击综述报告), Dec. 2021

Full Paper List

Deep Learning: Optimization, Robustness and Security

• Enhancing Byzantine-Resistant Aggregations with Client Embedding
  Zhiyuan Zhang, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun
  In Findings of EMNLP 2024

• Fed-FA: Theoretically Modeling Client Data Divergence for Federated Language Backdoor Defense
  Zhiyuan Zhang, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun
  In NeurIPS 2023

• Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias
  Zhiyuan Zhang, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun
  In Findings of ACL 2023

• GA-SAM: Gradient-Strength based Adaptive Sharpness-Aware Minimization for Improved Generalization
  Zhiyuan Zhang, Ruixuan Luo, Qi Su and Xu Sun
  Comment: The theoretical analysis of why and how the parameter robustness relates to the generalization ability via the distributional shift between training and test sets as a bridge. Also related to our previous papers: Parameter Corruption [pdf] and Adversarial Parameter Defense [pdf].
  In EMNLP 2022

• Dim-Krum: Backdoor-Resistant Federated Learning for NLP with Dimension-wise Krum-Based Aggregation
  Zhiyuan Zhang, Qi Su and Xu Sun
  In Findings of EMNLP 2022

• Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models
  Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma, Chenguang Wang and Xu Sun
  In Findings of EMNLP 2022

• How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data
  Zhiyuan Zhang, Lingjuan Lyu, Weiqiang Wang, Lichao Sun, Xu Sun
  Comment: Also related to our previous paper: Neural Network Surgery [pdf].
  In ICLR 2022

• Adversarial Parameter Defense by Multi-step Risk Minimization
  Zhiyuan Zhang, Ruixuan Luo, Xuancheng Ren, Qi Su, Liangyou Li, Xu Sun
  Comment: The analysis of the generalization ability and the parameter robustness. A substantial journal extension of our previous conference paper: Parameter Corruption [pdf].
  In Neural Networks 2021

• Neural Network Surgery: Injecting Data Patterns into Pre-trained Models with Minimal Instance-wise Side Effects
  Zhiyuan Zhang, Xuancheng Ren, Qi Su, Xu Sun, Bin He
  Comment: Besides injecting data patterns with minimal instance-wise side effects, also about NLP backdoors for both benign and malicious purposes.
  In NAACL 2021

• Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption
  Xu Sun^#, Zhiyuan Zhang^#, Xuancheng Ren, Ruixuan Luo, Liangyou Li
  Comment: The analysis of the robustness against parameter corruption or perturbation.
  In AAAI 2021

• Memorized Sparse Backpropagation
  Zhiyuan Zhang, Pengcheng Yang, Xuancheng Ren, Qi Su, Xu Sun
  In Neurocomputing 2020

• Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks
  Sishuo Chen, Wenkai Yang, Zhiyuan Zhang, Xiaohan Bi and Xu Sun
  In Findings of EMNLP 2022

• Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models
  Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, Bin He
  In NAACL 2021

• Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning
  Runxin Xu^#, Fuli Luo^#, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, Fei Huang
  Comment: Besides the effective and generalizable fine-tuning approach, also related to the theoretical analysis of the generalization ability and the parameter robustness.
  In EMNLP 2021

• Understanding and Improving Layer Normalization
  Jingjing Xu, Xu Sun, Zhiyuan Zhang, Guangxiang Zhao, Junyang Lin
  In NeurIPS 2019

• Rethinking Skip Connection with Layer Normalization
  Fenglin Liu^#, Xuancheng Ren^#, Zhiyuan Zhang, Xu Sun, Yuexian Zou
  In COLING 2020

Natural Language Processing Applications

• PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation
  Ruixuan Luo, Jingjing Xu, Yi Zhang, Zhiyuan Zhang, Xuancheng Ren, Xu Sun
  Comment: ⭐ A highly influential Chinese word segmentation open source toolkit: PKUSEG [GitHub].
  Preprint

• Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models
  Zhiyuan Zhang, Xiaoqian Liu, Yi Zhang, Qi Su, Xu Sun and Bin He
  Comment: The full version: preprint [OpenReview].
  In Findings of EMNLP 2020 (short)

• Automatic Translating between Ancient Chinese and Contemporary Chinese with Limited Aligned Corpora
  Zhiyuan Zhang, Wei Li, Qi Su
  In NLPCC 2019 (short)

• Primal Meaning Recommendation for Chinese Expressions via Descriptions in On-line Encyclopedia
  Zhiyuan Zhang, Wei Li, Jingjing Xu, Xu Sun
  Preprint

• Building an Ellipsis-aware Chinese Dependency Treebank for Web Text
  Xuancheng Ren, Xu Sun, Ji Wen, Bingzhen Wei, Weidong Zhan, Zhiyuan Zhang
  Comment: Releasing a Chinese dependency treebank dataset [GitHub] of 319 weibos, containing 572 sentences with omissions restored and contexts reserved.
  In LREC 2018

• Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
  Guangxiang Zhao, Junyang Lin, Zhiyuan Zhang, Xuancheng Ren, Qi Su, Xu Sun
  Preprint

• MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
  Guangxiang Zhao, Xu Sun, Jingjing Xu, Zhiyuan Zhang, Liangchen Luo
  Preprint

Quantitative Finance and Time Series Analysis

• No Stock is an Island: Learning Internal and Relational Attributes of Stocks with Contrastive Learning
  Shicheng Li^#, Wei Li^#, Zhiyuan Zhang, Ruihan Bao, Keiko Harimoto, Xu Sun
  Comment: ⭐ Selected as the best paper in IJCAI-FinNLP 2022!
  In IJCAI-FinNLP 2022 (Financial Technology and NLP)

• ASAT: Adaptively Scaled Adversarial Training in Time Series
  Zhiyuan Zhang, Wei Li, Ruihan Bao, Keiko Harimoto, Yunfang Wu, Xu Sun
  In Neurocomputing 2022

• Incorporating Pre-trained Model Prompting in Multimodal Stock Volume Movement Prediction
  Ruibo Chen^#, Zhiyuan Zhang^#, Yi Liu, Ruihan Bao, Keiko Harimoto, Xu Sun
  In KDD-MLF 2023 (ML in Finance)

• Distributional Correlation–Aware Knowledge Distillation for Stock Trading Volume Prediction
  Lei Li, Zhiyuan Zhang, Ruihan Bao, Keiko Harimoto, Xu Sun
  In ECML-PKDD 2022

• Stock Trading Volume Prediction with Dual-Process Meta-Learning
  Ruibo Chen, Wei Li, Zhiyuan Zhang, Ruihan Bao, Keiko Harimoto, Xu Sun
  In ECML-PKDD 2022

• Incremental Stock Volume Prediction with Gradient Distillation and Diversified Memory Selection
  Shicheng Li, Zhiyuan Zhang, Ruihan Bao, Keiko Harimoto, Xu Sun
  In IJCAI-AI4TS 2022 (AI for Time Series Analysis)

• Learning Robust Representation for Clustering through Locality Preserving Variational Discriminative Network
  Ruixuan Luo, Wei Li, Zhiyuan Zhang, Ruihan Bao, Keiko Harimoto, Xu Sun
  In AAAI-RSEML 2021 (Robust, Secure, and Efficient ML)