AAAI 2021 Workshop: Towards Robust, Secure and Efficient Machine Learning (RSEML)

Overview

Machine learning technology has been improving with every passing day and has been extensively applied to nearly every corner of the society that offers substantial benefits to our daily lives. However, machine learning models face various threats. For example, it is known that machine learning models are vulnerable to adversarial samples. The existence of adversarial examples reveals that current machine learning models are vulnerable and can be easily fooled, leading to serious security concerns in machine learning systems such as autonomous driving vehicles or face recognition systems.

More recently, due to both data privacy requirements as specified in the European Union’s General Data Protection Regulation (GDPR), and the limitations of computation power, the training process of machine learning models has extended from centralized to decentralized (i.e. distributed or federated learning) where the model will suffer from even more threats. For example, in a federated learning setting, every client can perform various attacks such as backdoor attacks on the global model as clients have direct access to the global model. How to prevent privacy leaking during information exchange of a decentralized training method is also a critical issue.

At the same time, computation efficiency is a big concern for modern deep learning, both inference and training. For inference, people prefer inference on edge devices due to better privacy, but edge devices have very limited computational resource. For training, gradient or weight exchange is necessary for decentralized training, but such exchange requires communication, which may be slow. Furthermore, models that are robust to adversarial attacks usually require longer training time and orders of magnitude more computation FLOPs than normal networks.

This one-day workshop intends to bring experts from machine learning, security communities, and federated learning together to work more closely in addressing the posed concerns. Specifically, we seek to study threats and defenses to machine learning not only in a single node setting but also in a distributed setting. In summary, we seek solutions to achieve a wholistic solution for robust, secure and efficient machine learning.

Program Schedule

Time (PST)	Activity
15:45 – 16:00	Presenters to connect and test the system
16:00 – 16:05	Opening Remark by Prof. Qiang Yang [Video]
16:05 – 16:35	Keynote Session 1: Efficiency is the Key to Privacy (and Security) by Prof. Kurt Keutzer [PDF] [Video]
16:35 – 17:15	Technical Talks Session 1 (2 talks, 20 mins each including Q&A) Alberto Matachana, Kenneth Co, Luis Muñoz-González, David Martinez and Emil Lupu. Robustness and Transferability of Universal Attacks on Compressed Models [PDF] [Video] Yi Zhu, Yiwei Zhou and Menglin Xia. Generating Semantically Valid Adversarial Questions for TableQA [PDF] [Video]
17:15 – 17:20	Break (Presenters should connect and test the system)
17:20 – 17:50	Keynote Session 2: Vertical Federated Kernel Learning by Prof. Heng Huang [PDF] [Video]
17:50 – 18:30	Technical Talks Session 2 (2 talks, 20 mins each including Q&A) Shuhao Fu, Chulin Xie, Bo Li and Qifeng Chen. Attack-Resistant Federated Learning with Residual-based Reweighting [PDF] [Video] Chang Song, Elias Fallon and Hai Li. Improving Adversarial Robustness in Weight-quantized Neural Networks [PDF] [Video]
18:30 – 18:35	Break (Presenters should connect and test the system)
18:35 – 19:05	Keynote Session 3: On Private Prediction and Certified Removal by Dr. Laurens van der Maaten [Video]
19:05 – 19:45	Technical Talks Session 3 (2 talks, 20 mins each including Q&A) Xiaoyang Wang, Bo Li, Jacky Zhang, Bhavya Kailkhura and Klara Nahrstedt. Robusta: Robust AutoML for Feature Selection via Reinforcement Learning [PDF] [Video] Nasser Aldaghri, Hessam Mahdavifar and Ahmad Beirami. Coded Machine Unlearning [Video]
19:45 – 21:00	Poster Session All accepted papers
21:00 - 21:00	End of Workshop

Invited Speaker

Kurt Keutzer, University of California, Berkeley

Title: Efficiency is the Key to Privacy (and Security)
Time: 16:05-16:35 PST

Abstract: Given the numerous reported (and unreported) security breaches, as well as pervasive concerns around use of private data, local processing of personal data associated with speech, cameras, and language appears to be the only completely effective way to ensure security and privacy. However, performing these computations without access to the cloud requires significant improvements in the efficiency of Deep Learning at the edge. In this talk we will discuss our efforts to move one of the most computationally challenging problems, Natural Language Understanding, to the edge. We will also share some initial thoughts on addressing an even more challenging problem, recommendation systems, to mobile clients.

Biography: Kurt’s research at University of California, Berkeley, focuses on computational problems in Deep Learning. In particular, Kurt has worked to reduce the training time of ImageNet and BERT to minutes, and with the “Squeeze” family of Deep Neural nets, he has endeavored to develop a family of DNNs suitable for mobile and IOT applications. Before joining Berkeley as a Full Professor in 1998, Kurt was CTO and SVP at Synopsys. Kurt’s earlier contributions to Electronic Design Automation were recognized at the 50th Design Automation Conference where he was noted as a Top 10 most cited author, as an author of a Top 10 most cited paper, and as one of only three people to have won four Best Paper Awards at that conference. Kurt was named a Fellow of the IEEE in 1996.
Heng Huang, JD.com and University of Pittsburgh

Title: Vertical Federated Kernel Learning
Time: 17:20-17:50 PST

Abstract: Vertically partitioned data widely exist in the modern data mining and machine learning applications, where data are provided by multiple providers (companies, stakeholders, government departments, etc.) and each maintains the records of different feature sets with common entities. For example, a digital finance company, an E-commerce company, and a bank collect different information about the same person. The digital finance company has access to the online consumption, loan and repayment information. The E-commerce company has access to the online shopping information. The bank has customer information such as average monthly deposit, account balance. If the person submits a loan application to the digital finance company, the digital finance company might evaluate the credit risk to this financial loan by comprehensively utilizing the information stored in the three parts. However, direct access to the data in other providers or sharing of the data is often prohibited due to legal and commercial reasons. It is challenging to train these vertically partitioned data effectively and efficiently while keeping data privacy for traditional machine learning algorithms.

Vertical federated learning has been employed as good solution to address such situations. However, most existing vertical federated learning methods are linear models. To improve the prediction performance, we focus on nonlinear learning with kernels, and propose a federated doubly stochastic kernel learning (FDSKL) algorithm for vertically partitioned data. Specifically, we use random features to approximate the kernel mapping function and use doubly stochastic gradients to update the solutions, which are all computed federatedly without the disclosure of data. Importantly, we prove that FDSKL has a sublinear convergence rate, and can guarantee the data security under the semi-honest assumption. Extensive experimental results on a variety of benchmark datasets show that FDSKL is significantly faster than state-of-the-art federated learning methods when dealing with kernels, while retaining the similar generalization performance.

Biography: Dr. Heng Huang is John A. Jurenko Endowed Professor in Electrical and Computer Engineering at University of Pittsburgh, and is also a consulting researcher with JD Finance American Corporation. Dr. Huang received the PhD degree in Computer Science at Dartmouth College. His research areas include machine learning, big data mining, and biomedical data science. Dr. Huang has published more than 230 papers in top-tier conferences and many papers in premium journals, such as ICML, NeurIPS, KDD, RECOMB, ISMB, ICCV, CVPR, IJCAI, AAAI, TPAMI, JMLR, etc. Based on csrankings.org, for the last ten years, Dr. Huang is ranked 2nd among researchers in U.S. who published most top computer science conference papers. He is a Fellow of AIBME and serves as the Program Chair of ACM SIGKDD Conference 2020.
Laurens van der Maaten, Facebook AI Research (FAIR)

Title: On Private Prediction and Certified Removal
Time: 18:35-19:05 PST

Abstract: The talk will present two recent studies related to differential privacy in machine learning. The first half of the talk will present an empirical evaluation of techniques for differentially private prediction. In line with what theory suggests, we find that such private prediction techniques perform subpar in most learning settings. The results suggest that empirical work on differential privacy should focus on private training. The second half of the talk will discuss the problem of "removing" data from a trained machine learning model. Specifically, we propose a formulation for the removal problem that is based on differential privacy, called certified removal. We present an algorithm that performs certified removal from linear models efficiently, and show empirically that this certified-removal algorithm works well in practice.

Biography: Laurens van der Maaten is a Research Director at Facebook AI Research in New York. He leads FAIR's New York site. Prior, he worked as an Assistant Professor at Delft University of Technology (The Netherlands) and as a post-doctoral researcher at University of California, San Diego. He received his PhD from Tilburg University (The Netherlands) in 2009. With collaborators from Cornell University, he won the Best Paper Award at CVPR 2017. He is an editorial board member of IEEE Transactions on Pattern Analysis and Machine Intelligence and is regularly serving as area chair for the NeurIPS, ICML, and CVPR conferences. Laurens is interested in a variety of topics in machine learning and computer vision, including privacy, security, and robustness of machine-learning systems.

Topic of Interests

Topics including (but not limit to):

Theoretical contributions of adversarial machine learning.
Training data poisoning and adversarial learning.
Adversarial attacks (e.g. evasion) and defenses.
Secure machine learning.
Privacy-preserving machine learning.
Applications of privacy-preserving machine learning.
Privacy attacks such as membership inference, and model inversion.
Secure multi-party computation.
Model compression and efficiency improvement in both training and inference.
Efficiency improvement of information exchange in distributed training.
Model robustness against model compression.

Submission Guidelines

Submissions can be a full technical paper (up to 8 pages) or short paper (up to 4 pages) excluding references or supplementary materials.

Authors should only rely on the supplementary material to include minor details that do not fit in the main paper.

The submissions are anonymous for double-blind review.

The workshop will not have formal proceedings.

Please follow AAAI 2021 Latex style for paper format.

The final submission must be in PDF and please submit your papers to https://easychair.org/conferences/?conf=rseml2021