Previously: at West Lake, Hangzhou.

Xumiao Zhang
Alibaba Cloud

525 Almanor Avenue, Suite 400
Sunnyvale, CA 94085

Email: xumiao [at] umich.edu

I work at Alibaba Cloud as a researcher, tackling real-world challenges in large-scale cloud networks. I received Ph.D. in Computer Science and Engineering at the University of Michigan, Ann Arbor, where I was advised by Prof. Z. Morley Mao. Prior to UMich, I received my B.E. from the University of Science and Technology of China, under the supervision of Prof. Xiang-Yang Li. I was in the School of the Gifted Young and was enrolled in the Talent Program in Computer and Information Science and Technology.

I am broadly interested in computer networks and systems. My Ph.D. research strives to develop systems and networking support for characterizing and improving next-generation network infrastructures (e.g., 5G, LEO satellite, QUIC) and applications (e.g., connected and autonomous vehicles). I am currently working on bringing AI to network infrastructure.

What's New

June 2024: I joined Alibaba Cloud as a full-time researcher.
June 2024: I passed my oral defense and finally PhinisheD!
April 2024: OASIS received the Best Paper Award at MMSys 2024!
February 2024: I was invited to give a talk on LEO satellite networks at the IEEE VTS SAGWICS seminar.
February 2024: CloudEval-YAML was accepted by MLSys 2024!
January 2024: Our QUIC study was accepted (oral) by WWW 2024 and OASIS was accepted by ACM MMSys 2024!
January 2024: I joined Alibaba Cloud as a research intern!
November 2023: We released our CloudEval-YAML benchmark for code generation in cloud-native applications.
September 2023: Our measurement study on Starlink and cellular networks was accepted to ACM CoNEXT 2023.
June 2023: RAO was accepted to ACM MobiCom 2023.
June 2023: I served on the Program Committee for ACM S3 2023.
May 2022: I joined Alibaba as a research intern this summer.
April 2022: I served on the Program Committee for ACM S3 2022.
June 2021: Edge-assisted Multi-vehicle Perception was accepted to ACM MobiCom 2021.
May 2021: I served on the Artifact Evaluation Committee for ACM SIGCOMM 2021.
May 2021: Our paper on 5G network and power performance was accepted to ACM SIGCOMM 2021.
June 2020: I started my internship at General Motors, Warren.
March 2020: MPbond was accepted to ACM MobiSys 2020.
September 2019: I attended the POWDER-RENEW Mobile and Wireless Week and won first prize in the mini-project competition!
May 2019: I served on the Shadow Program Committee for ACM IMC 2019.
August 2018: I moved to Ann Arbor and started my Ph.D. journey at UMich!
July 2017: I started my internship supervised by Prof. Chunyi Peng at the Ohio State University.
September 2015: I was enrolled in the Talent Program in Computer and Information Science & Technology, USTC.
August 2014: I started my college life at the University of Science and Technology of China!

Selected Projects

QUIC Study: We systematically examine QUIC over fast Internet. We compare the UDP+QUIC+HTTP/3 stack with TCP+TLS+HTTP/2 and test with different web applications. We also perform an in-depth root cause analysis.
CloudEval-YAML: We build a practical, hand-curated benchmark for code generation in cloud-native applications. It features over 1000 problems with in-depth evaluations of popular LLMs in cloud settings.
Starlink-Cellular: We conduct a measurement study comparing the Starlink LEO satellite network with cellular networks and explore the potential for synergistic integration of two network types (e.g., using MPTCP).
RAO: RAO is a multi-vehicle collaboration system designed to address asynchronous sensor data and inaccurate occlusion localization. It efficiently merges asynchronous data from different vehicles with occlusion awareness, improving both the accuracy and coverage of vehicular cooperative perception.
EMP: EMP is an edge-assisted multi-vehicle collaboration system enabling scalable, adaptive, and efficient sensor data sharing to enhance the local processing (e.g., perception) of individual vehicles.
5G Measurement: We carry out the first comprehensive examination to understand the effects of 5G deployment strategies, radio bands, and protocol specific properties on network performance, power usage, and application QoE.

Publications (*: co-primary authors)

Full Papers

QUIC is not Quick Enough over Fast Internet WWW '24

Xumiao Zhang, Shuowei Jin, Yi He, Ahmad Hassan, Z. Morley Mao, Feng Qian, Zhi-Li Zhang
Proceedings of the ACM Web Conference 2024 (WWW '24)
Oral Presentation

QUIC is expected to be a game-changer in improving web application performance. In this paper, we conduct a systematic examination of QUIC's performance over high-speed networks. We find that over fast Internet, the UDP+QUIC+HTTP/3 stack suffers a data rate reduction of up to 45.2% compared to the TCP+TLS+HTTP/2 counterpart. Moreover, the performance gap between QUIC and HTTP/2 grows as the underlying bandwidth increases. We observe this issue on lightweight data transfer clients and major web browsers (Chrome, Edge, Firefox, Opera), on different hosts (desktop, mobile), and over diverse networks (wired broadband, cellular). It affects not only file transfers, but also various applications such as video streaming (up to 9.8% video bitrate reduction) and web browsing. Through rigorous packet trace analysis and kernel- and user-space profiling, we identify the root cause to be high receiver-side processing overhead, in particular, excessive data packets and QUIC's user-space ACKs. We make concrete recommendations for mitigating the observed performance issues.

@article{zhang2023quic, title={QUIC is not Quick Enough over Fast Internet}, author={Zhang, Xumiao and Jin, Shuowei and He, Yi and Hassan, Ahmad and Mao, Z Morley and Qian, Feng and Zhang, Zhi-Li}, journal={arXiv preprint arXiv:2310.09423}, year={2023} }

CloudEval-YAML: A Practical Benchmark for Cloud Configuration Generation MLSys '24

Yifei Xu*, Yuning Chen*, Xumiao Zhang*, Xianshang Lin, Pan Hu, Yunfei Ma, Songwu Lu, Wan Du, Z. Morley Mao, Ennan Zhai, Dennis Cai
Proceedings of the Seventh Annual Conference on Machine Learning and Systems (MLSys '24)

Among the thriving ecosystem of cloud computing and the proliferation of Large Language Model (LLM)-based code generation tools, there is a lack of benchmarking for code generation in cloud-native applications. In response to this need, we present CloudEval-YAML, a practical benchmark for cloud configuration generation. CloudEval-YAML tackles the diversity challenge by focusing on YAML, the de facto standard of numerous cloud-native tools. We develop the CloudEval-YAML benchmark with practicality in mind: the dataset consists of hand-written problems with unit tests targeting practical scenarios. We further enhanced the dataset to meet practical needs by rephrasing questions in a concise, abbreviated, and bilingual manner. The dataset consists of 1011 problems that take more than 1200 human hours to complete. To improve practicality during evaluation, we build a scalable evaluation platform for CloudEval-YAML that achieves a 20 times speedup over a single machine. To the best of our knowledge, the CloudEval-YAML dataset is the first hand-written dataset targeting cloud-native applications. We present an in-depth evaluation of 12 LLMs, leading to a deeper understanding of the problems and LLMs, as well as effective methods to improve task performance and reduce cost.

@article{xu2024cloudeval, title={A Practical Benchmark for Cloud Native YAML Configuration Generation}, author={Xu, Yifei and Chen, Yuning and Zhang, Xumiao and Lin, Xianshang and Hu, Pan and Ma, Yunfei and Lu, Songwu and Du, Wan and Mao, Z. Morley and Zhai, Ennan and Cai, Dennis}, journal={Proceedings of Machine Learning and Systems}, volume={6}, year={2024} }

Vulcan: Automatic Query Planning for Live ML Analytics NSDI '24

Yiwen Zhang, Xumiao Zhang, Ganesh Ananthanarayanan, Anand Iyer, Yuanchao Shu, Victor Bahl, Z. Morley Mao, Mosharaf Chowdhury
Proceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24)

Live ML analytics have gained increasing popularity with large-scale deployments due to recent evolution of ML technologies. To serve live ML queries, experts nowadays still need to perform manual query planning, which involves pipeline construction, query configuration, and pipeline placement across multiple edge tiers in a heterogeneous infrastructure. Finding the best query plan for a live ML query requires navigating a huge search space, calling for an efficient and systematic solution. In this paper, we propose Vulcan, a system that automatically generates query plans for live ML queries to optimize their accuracy, latency, and resource consumption. Based on the user query and performance requirements, Vulcan determines the best pipeline, placement, and query configuration for the query with low profiling cost; it also performs fast online adaptation after query deployment. Vulcan outperforms state-of-the-art ML analytics systems by 4.1x-30.1x in terms of search cost while delivering up to 3.3x better query latency.

@article{zhang2024vulcan, title={Vulcan: Automatic Query Planning for Live ML Analytics}, author={Zhang, Yiwen and Zhang, Xumiao and Ananthanarayanan, Ganesh and Iyer, Anand and Shu, Yuanchao and Bahl, Victor and Mao, Z. Morley and Chowdhury, Mosharaf}, year={2024} }

OASIS: Collaborative Neural-Enhanced Mobile Video Streaming MMSys '24

Shuowei Jin, Ruiyang Zhu, Ahmad Hassan, Xiao Zhu, Xumiao Zhang, Z. Morley Mao, Feng Qian, Zhi-Li Zhang
Proceedings of the 15th Conference on ACM Multimedia Systems (MMSys '24)
Best Paper Award

Neural-enhanced video streaming (e.g., super-resolution) is an ongoing revolution which can provide extremely high-quality video streaming services breaking the restriction of bandwidth. However, such enhancements require intense computation power that is not affordable for a single mobile device, which hinders their real-world deployment. To address the limitation, we propose OASIS, the first system that facilitates multiple users in close proximity to execute intense neural-enhanced video streaming in realtime. To this end, OASIS intelligently distributes computation tasks among multiple mobile devices, selects appropriate video bitrates and super-resolution models, and optimizes video chunk delivery. As a result, the expensive neural-enhanced streaming is done through distributed collaboration, achieving optimal quality of experience (QoE). We implement and evaluate OASIS on commodity smartphones from different vendors, under various network and computation conditions. Extensive experiments demonstrate the high efficiency of OASIS: it improves the video streaming QoE by 40%-200% and reduces each participant's energy consumption by 60% when the system scales up from a single device to six devices.

@article{jin2024oasis, title={OASIS: Collaborative Neural-Enhanced Mobile Video Streaming}, author={Jin, Shuowei and Zhu, Ruiyang and Hassan, Ahmad and Zhu, Xiao and Zhang, Xumiao and Mao, Z. Morley and Qian, Feng and Zhang, Zhi-Li}, year={2024} }

On Data Fabrication in Collaborative Vehicular Perception: Attacks and Countermeasures USENIX Security '24

Qingzhao Zhang, Shuowei Jin, Ruiyang Zhu, Jiachen Sun, Xumiao Zhang, Qi Alfred Chen, Z. Morley Mao
Proceedings of the 33rd USENIX Security Symposium (USENIX Security '24)

Collaborative perception, which greatly enhances the sensing capability of connected and autonomous vehicles (CAVs) by incorporating data from external resources, also brings forth potential security risks. CAVs' driving decisions rely on remote untrusted data, making them susceptible to attacks carried out by malicious participants in the collaborative perception system. However, security analysis and countermeasures for such threats are absent. To understand the impact of the vulnerability, we break the ground by proposing various real-time data fabrication attacks in which the attacker delivers crafted malicious data to victims in order to perturb their perception results, leading to hard brakes or increased collision risks. Our attacks demonstrate a high success rate of over 86% on high-fidelity simulated scenarios and are realizable in real-world experiments. To mitigate the vulnerability, we present a systematic anomaly detection approach that enables benign vehicles to jointly reveal malicious fabrication. It detects 91.5% of attacks with a false positive rate of 3% in simulated scenarios and significantly mitigates attack impacts in real-world scenarios.

@inproceedings{zhang2024on, title={On Data Fabrication in Collaborative Vehicular Perception: Attacks and Countermeasures}, author={Zhang, Qingzhao and Jin, Shuowei and Zhu, Ruiyang and Sun, Jiachen and Zhang, Xumiao and Chen, Qi Alfred and Mao, Z Morley}, booktitle={33nd USENIX Security Symposium (USENIX Security 24)}, year={2024} }

LEO Satellite vs. Cellular Networks: Exploring the Potential for Synergistic Integration CoNEXT '23

Bin Hu*, Xumiao Zhang*, Qixin Zhang, Nitin Varyani, Z. Morley Mao, Feng Qian, Zhi-Li Zhang
Proceedings of the 19th International Conference on emerging Networking EXperiments and Technologies (CoNEXT '23)

Low-Earth-Orbit (LEO) satellite networks, such as Starlink, are transforming global network connectivity by bringing Internet access to remote and underserved areas. However, the current coverage and performance of the LEO satellite network service compared with those of cellular networks are under-explored. In this work, we present a measurement study of the Starlink LEO satellite network in comparison with cellular networks, aiming to uncover the potential for synergistic integration. Through a large-scale data collection campaign and in-depth analysis, we (1) identify the performance characteristics of two Starlink configurations, (2) evaluate the coverage of the current Starlink deployment compared to major cellular carriers, and (3) investigate the potential benefits of enabling multipath using both LEO satellite and cellular networks.

@inproceedings{10.1145/3624354.3630588, author = {Hu, Bin and Zhang, Xumiao and Zhang, Qixin and Varyani, Nitin and Mao, Z. Morley and Qian, Feng and Zhang, Zhi-Li}, title = {LEO Satellite vs. Cellular Networks: Exploring the Potential for Synergistic Integration}, year = {2023}, isbn = {9798400704079}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3624354.3630588}, doi = {10.1145/3624354.3630588}, booktitle = {Companion of the 19th International Conference on Emerging Networking EXperiments and Technologies}, pages = {45–51}, numpages = {7}, keywords = {cellular network, low earth orbit, mptcp, satellite network, network measurement}, location = {, Paris, France, }, series = {CoNEXT 2023} }

Robust Real-time Multi-vehicle Collaboration on Asynchronous Sensors MobiCom '23

Qingzhao Zhang*, Xumiao Zhang*, Ruiyang Zhu*, Fan Bai, Mohammad Naserian, Z. Morley Mao
Proceedings of the 29th Annual International Conference on Mobile Computing and Networking (MobiCom '23)

Cooperative perception significantly enhances the perception performance of connected autonomous vehicles. Instead of purely relying on local sensors with limited range, it enables multiple vehicles and roadside infrastructures to share sensor data to perceive the environment collaboratively. Through our study, we realize that the performance of cooperative perception systems is limited in real-world deployment due to (1) out-of-sync sensor data during data fusion and (2) inaccurate localization of occluded areas. To address these challenges, we develop RAO, an innovative, effective, and lightweight cooperative perception system that merges asynchronous sensor data from different vehicles through our novel designs of motion-compensated occupancy flow prediction and on-demand data sharing, improving both the accuracy and coverage of the perception system. Our extensive evaluation, including real-world and emulation-based experiments, demonstrates that RAO outperforms state-of-the-art solutions by more than 34% in perception coverage and by up to 14% in perception accuracy, especially when asynchronous sensor data is present. RAO consistently performs well across a wide variety of map topologies and driving scenarios. RAO incurs negligible additional latency (8.5ms) and low data transmission overhead (10.9 KB per frame), making cooperative perception feasible.

@inproceedings{10.1145/3570361.3613271, author = {Zhang, Qingzhao and Zhang, Xumiao and Zhu, Ruiyang and Bai, Fan and Naserian, Mohammad and Mao, Z. Morley}, title = {Robust Real-Time Multi-Vehicle Collaboration on Asynchronous Sensors}, year = {2023}, isbn = {9781450399906}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3570361.3613271}, doi = {10.1145/3570361.3613271}, booktitle = {Proceedings of the 29th Annual International Conference on Mobile Computing and Networking}, articleno = {57}, numpages = {15}, keywords = {vehicular networks, LiDAR, cooperative perception, autonomous cars}, location = {Madrid, Spain}, series = {ACM MobiCom '23} }

A Cooperative Perception Environment for Traffic Operations and Control TRB '23

Hanlin Chen, Brian Liu, Xumiao Zhang, Feng Qian, Z. Morley Mao, Yiheng Feng

Existing data collection methods for traffic operations and control usually rely on infrastructure-based loop detectors or probe vehicle trajectories. Connected and automated vehicles (CAVs) not only can report data about themselves but also can provide the status of all detected surrounding vehicles. Integration of perception data from multiple CAVs as well as infrastructure sensors (e.g., LiDAR) can provide richer information even under a very low penetration rate. This paper aims to develop a cooperative data collection system, which integrates LiDAR point cloud data from both infrastructure and CAVs to create a cooperative perception environment for various transportation applications. The state-of-the-art 3D detection models are applied to detect vehicles in the merged point cloud. We test the proposed cooperative perception environment with the max pressure adaptive signal control model in a co-simulation platform with CARLA and SUMO. Results show that very low penetration rates of CAV plus an infrastructure sensor are sufficient to achieve comparable performance with 30% or higher penetration rates of connected vehicles (CV). We also show the equivalent CV penetration rate (E-CVPR) under different CAV penetration rates to demonstrate the data collection efficiency of the cooperative perception environment.

@article{chen2022cooperative, title={A Cooperative Perception Environment for Traffic Operations and Control}, author={Chen, Hanlin and Liu, Brian and Zhang, Xumiao and Qian, Feng and Mao, Z Morley and Feng, Yiheng}, journal={arXiv preprint arXiv:2208.02792}, year={2022} }

EMP: Edge-assisted Multi-vehicle Perception MobiCom '21

Xumiao Zhang, Anlan Zhang, Jiachen Sun, Xiao Zhu, Yihua Guo, Feng Qian, Z. Morley Mao
Proceedings of the 27th Annual International Conference on Mobile Computing and Networking (MobiCom '21)

Connected and Autonomous Vehicles (CAVs) heavily rely on 3D sensors such as LiDARs, radars, and stereo cameras. However, 3D sensors from a single vehicle suffer from two fundamental limitations: vulnerability to occlusion and loss of details on far-away objects. To overcome both limitations, in this paper, we design, implement, and evaluate EMP, a novel edge-assisted multi-vehicle perception system for CAVs. In EMP, multiple nearby CAVs share their raw sensor data with an edge server which then merges CAVs' individual views to form a more complete view with a higher resolution. The merged view can drastically enhance the perception quality of the participating CAVs. Our core methodological contribution is to make the sensor data sharing scalable, adaptive, and resource-efficient over oftentimes highly fluctuating wireless links through a series of novel algorithms, which are then integrated into a full-fledged cooperative sensing pipeline. Extensive evaluations demonstrate that EMP can achieve real-time processing at 24 FPS and end-to-end latency of 93 ms on average. EMP reduces the end-to-end latency by 49% to 65% compared to the traditional vehicle-to-vehicle (V2V) sharing approach without edge support. Our case studies show that cooperative sensing powered by EMP can detect hazards such as blind spots faster by 0.5 to 1.1 seconds, compared to a single vehicle's perception.

@inproceedings{10.1145/3447993.3483242, author = {Zhang, Xumiao and Zhang, Anlan and Sun, Jiachen and Zhu, Xiao and Guo, Y. Ethan and Qian, Feng and Mao, Z. Morley}, title = {EMP: Edge-Assisted Multi-Vehicle Perception}, year = {2021}, isbn = {9781450383424}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3447993.3483242}, doi = {10.1145/3447993.3483242}, booktitle = {Proceedings of the 27th Annual International Conference on Mobile Computing and Networking}, pages = {545–558}, numpages = {14}, keywords = {LiDAR, edge computing, autonomous cars, cooperative sensing}, location = {New Orleans, Louisiana}, series = {MobiCom '21} }

A Variegated Look at 5G in the Wild: Performance, Power, and QoE Implications SIGCOMM '21

Arvind Narayanan*, Xumiao Zhang*, Ruiyang Zhu, Ahmad Hassan, Shuowei Jin, Xiao Zhu, Xiaoxuan Zhang, Denis Rybkin, Zhengxuan Yang,
Z. Morley Mao, Feng Qian, Zhi-Li Zhang
Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication (SIGCOMM '21)

Motivated by the rapid deployment of 5G, we carry out an in-depth measurement study of the performance, power consumption, and application quality-of-experience (QoE) of commercial 5G networks in the wild. We examine different 5G carriers, deployment schemes (Non-Standalone, NSA vs. Standalone, SA), radio bands (mmWave and sub 6-GHz), protocol configurations (e.g. Radio Resource Control state transitions), mobility patterns (stationary, walking, driving), client devices (i.e. User Equipment), and upper-layer applications (file download, video streaming, and web browsing). Our findings reveal key characteristics of commercial 5G in terms of throughput, latency, handover behaviors, radio state transitions, and radio power consumption under the above diverse scenarios, with detailed comparisons to 4G/LTE networks. Furthermore, our study provides key insights into how upper-layer applications should best utilize 5G by balancing the critical tradeoff between performance and energy consumption, as well as by taking into account the availability of both network and computation resources. We have released the datasets and tools of our study at https://github.com/SIGCOMM21-5G/artifact.

@inproceedings{10.1145/3452296.3472923, author = {Narayanan, Arvind and Zhang, Xumiao and Zhu, Ruiyang and Hassan, Ahmad and Jin, Shuowei and Zhu, Xiao and Zhang, Xiaoxuan and Rybkin, Denis and Yang, Zhengxuan and Mao, Zhuoqing Morley and Qian, Feng and Zhang, Zhi-Li}, title = {A Variegated Look at 5G in the Wild: Performance, Power, and QoE Implications}, year = {2021}, isbn = {9781450383837}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3452296.3472923}, doi = {10.1145/3452296.3472923}, booktitle = {Proceedings of the 2021 ACM SIGCOMM 2021 Conference}, pages = {610–625}, numpages = {16}, keywords = {5G, mmWave, energy efficiency, power characteristics, dataset, network measurement, power model, video streaming, latency}, location = {Virtual Event, USA}, series = {SIGCOMM '21} }

MPBond: Efficient Network-level Collaboration Among Personal Mobile Devices MobiSys '20

Xiao Zhu, Jiachen Sun, Xumiao Zhang, Yihua Guo, Feng Qian, Z. Morley Mao
Proceedings of the 18th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys '20)

MPBond is an efficient system allowing multiple personal mobile devices to collaboratively fetch content from the Internet. For example, a smartwatch can assist its paired smartphone with downloading data. Inspired by the success of MPTCP, MPBond applies the concept of distributed multipath transport where multiple subflows can traverse different devices. We develop device/connection management schemes, a buffering strategy, a packet scheduling algorithm, and a policy framework tailored to MPBond’s architecture. We implement MPBond on commodity mobile devices such as Android smartphones and smartwatches. Our real-world evaluations using different workloads under various network conditions demonstrate the efficiency of MPBond. Compared to state-of-the-art collaboration frameworks, MPBond reduces file download time by 5% to 46%, and improves the video streaming bitrate by 2% to 118%. Meanwhile, it improves the energy efficiency by 10% to 57%.

@inproceedings{10.1145/3386901.3388943, author = {Zhu, Xiao and Sun, Jiachen and Zhang, Xumiao and Guo, Y. Ethan and Qian, Feng and Mao, Z. Morley}, title = {MPBond: Efficient Network-Level Collaboration among Personal Mobile Devices}, year = {2020}, isbn = {9781450379540}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3386901.3388943}, doi = {10.1145/3386901.3388943}, booktitle = {Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services}, pages = {364–376}, numpages = {13}, location = {Toronto, Ontario, Canada}, series = {MobiSys '20} }

Workshops, Posters, and Demos

CloudEval-YAML: A Realistic and Scalable Benchmark for Cloud Configuration Generation ML4Sys @ NeurIPS '23

Yifei Xu*, Yuning Chen*, Xumiao Zhang*, Xianshang Lin, Pan Hu, Yunfei Ma, Songwu Lu, Wan Du, Z. Morley Mao, Ennan Zhai, Dennis Cai
Workshop on ML for Systems at NeurIPS 2023

Among the thriving ecosystem of cloud computing and the proliferation of Large Language Model (LLM)-based code generation tools, there is a lack of benchmarking for code generation in cloud-native applications. In response to this need, we present CloudEval-YAML, a practical benchmark for cloud configuration generation. CloudEval-YAML tackles the diversity challenge by focusing on YAML, the de facto standard of numerous cloud-native tools. We develop the CloudEval-YAML benchmark with practicality in mind: the dataset consists of hand-written problems with unit tests targeting practical scenarios. To improve practicality during evaluation, we build a scalable evaluation platform for CloudEval-YAML that achieves a 20 times speedup over a single machine. To the best of our knowledge, the CloudEval-YAML dataset is the first hand-written dataset targeting cloud-native applications. We present an in-depth evaluation of 13 LLMs, leading to a deeper understanding of the problems and LLMs, as well as effective methods to improve task performance and reduce cost.

@article{xu2023cloudeval, title={CloudEval-YAML: A Realistic and Scalable Benchmark for Cloud Configuration Generation}, author={Xu, Yifei and Chen, Yuning and Zhang, Xumiao and Lin, Xianshang and Hu, Pan and Ma, Yunfei and Lu, Songwu and Du, Wan and Mao, Z. Morley and Zhai, Ennan and Cai, Dennis}, year={2023} }

Poster: QUIC is not Quick Enough over Fast Internet IMC '23

Xumiao Zhang, Shuowei Jin, Yi He, Ahmad Hassan, Z. Morley Mao, Feng Qian, Zhi-Li Zhang
Proceedings of the 23rd ACM Internet Measurement Conference (IMC '23)

QUIC is a multiplexed transport-layer protocol over UDP and comes with enforced encryption. It is expected to be a game-changer in improving web application performance. Together with the network layer and layers below, UDP, QUIC, and HTTP/3 form a new protocol stack for future network communication, whose current counterpart is TCP, TLS, and HTTP/2. In this study, to understand QUIC's performance over high-speed networks and its potential to replace the TCP stack, we carry out a series of experiments to compare the UDP+QUIC+HTTP/3 (QUIC) stack and the TCP+TLS+HTTP/2 (HTTP/2) stack. Preliminary measurements on file download reveal that QUIC suffers from a data rate reduction compared to HTTP/2 across different hosts.

@inproceedings{10.1145/3618257.3625002, author = {Zhang, Xumiao and Jin, Shuowei and He, Yi and Hassan, Ahmad and Mao, Z. Morley and Qian, Feng and Zhang, Zhi-Li}, title = {Poster: QUIC is Not Quick Enough over Fast Internet}, year = {2023}, isbn = {9798400703829}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3618257.3625002}, doi = {10.1145/3618257.3625002}, booktitle = {Proceedings of the 2023 ACM on Internet Measurement Conference}, pages = {730–731}, numpages = {2}, keywords = {http/3, quic, internet measurement, transport protocol}, location = {Montreal QC, Canada}, series = {IMC '23} }

Poster: Characterizing Performance and Power for mmWave 5G on Commodity Smartphones S3 '19

Xumiao Zhang, Xiao Zhu, Yihua Ethan Guo, Feng Qian, Z. Morley Mao
Proceedings of the 2019 on Wireless of the Students, by the Students, and for the Students Workshop (S3 '19)

During the first half of this year, three major operators in the US have announced their 5G deployment, which indicates the advent of next generation networks. To reduce the time to market, carriers utilize 5G NR for data plane operations while retaining the existing 4G infrastructure for control plane operations in what is called NSA (Non-Standalone) deployment mode defined as 5G system Phase I. EN-DC is a core technology of NSA 5G which supports the introduction of 5G services under 4G infrastructure. It enables a UE to connect to LTE and NR at the same time whereas the control plane connection is handled by LTE infrastructure. In this case, the UE will have only one single RRC state machine. Note that LTE RRC state machine contains RRC_CONNECTED and RRC_IDLE with different DRX settings while NR RRC has an additional state called RRC_INACTIVE. In this work, we aim to explore network and power characteristics for NSA 5G which introduces a surprisingly high data rate.

@inproceedings{10.1145/3349621.3355729, author = {Zhang, Xumiao and Zhu, Xiao and Guo, Yihua Ethan and Qian, Feng and Mao, Z. Morley}, title = {Poster: Characterizing Performance and Power for MmWave 5G on Commodity Smartphones}, year = {2019}, isbn = {9781450369299}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3349621.3355729}, doi = {10.1145/3349621.3355729}, booktitle = {Proceedings of the 2019 on Wireless of the Students, by the Students, and for the Students Workshop}, pages = {14}, numpages = {1}, keywords = {smartphone, nr, rrc state machine, lte, 5g, power model, 4g}, location = {Los Cabos, Mexico}, series = {S3'19} }

Demo: The Sound of Silence: End-to-End Sign Language Recognition Using SmartWatch MobiCom '17

Qian Dai, Jiahui Hou, Panlong Yang, Xiangyang Li, Fei Wang, Xumiao Zhang
Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking (MobiCom '17)

Sign Language is a natural and fully-fledged communication method for deaf and hearing-impaired people. In this demo, we propose the first SmartWatch-based American sign language (ASL) recognition system, which is more comfortable, portable and user-friendly and offers accessibility anytime, anywhere. This system is based on the intuitive idea that each sign has its specific motion pattern which can be transformed into unique gyroscope and accelerometer signals and then analyzed and learned by using Long-Short term memory recurrent neural network (LSTM-RNN) trained with connectionist temporal classification (CTC). In this way, signs and context information can be correctly recognized based on an off-the-shelf device (e.g. SmartWatch, Smartphone). The experiments show that, in the Known user split task, our system reaches an average word error rate of 7.29% to recognize 73 sentences formed by 103 ASL signs and achieves detection ratio up to 93.7% for a single sign. The result also shows our system has a good adaptation, even including new users, it can achieve an average word error rate of 21.6% at the sentence level and reach an average detection ratio of 79.4%. Moreover, our system performs real time ASL translation, outputting the speech within 1.69 seconds for a sentence of 12 signs in average.

@inproceedings{10.1145/3117811.3119853, author = {Dai, Qian and Hou, Jiahui and Yang, Panlong and Li, Xiangyang and Wang, Fei and Zhang, Xumiao}, title = {Demo: The Sound of Silence: End-to-End Sign Language Recognition Using SmartWatch}, year = {2017}, isbn = {9781450349161}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3117811.3119853}, doi = {10.1145/3117811.3119853}, booktitle = {Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking}, pages = {462–464}, numpages = {3}, keywords = {activity recognition, wearable computing, mobile sensing}, location = {Snowbird, Utah, USA}, series = {MobiCom '17} }

Invited Talks

February 2024: LEO Satellite vs. Cellular Networks: Exploring the Potential for Synergistic Integration, IEEE VTS SAGWICS
October 2022: EMP: Edge-assisted Multi-vehicle Perception, Guest Lecture for UMich EECS 589
October 2021: EMP: Edge-assisted Multi-vehicle Perception, Guest Lecture for UMN CSCI 8980 "Mobile Computing"

Teaching

Fall 2017: Fundamentals of Database Systems

Services

Program Committee: ACM S3 2023, ACM S3 2022
Journal Reviewer: IEEE/ACM ToN, IEEE TMC, IEEE JSAC, IEEE VTM, IEEE TVT
Conference Reviewer: ACM WWW 2024, IEEE INFOCOM 2024, IEEE INFOCOM 2023, IEEE VNC 2023, IEEE VNC 2021
Artifact Evaluation Committee: ACM SIGCOMM 2023, ACM CoNEXT 2023, ACM CoNEXT 2022, ACM SIGCOMM 2021
Shadow Program Committee: ACM IMC 2019
Volunteer: ACM MobiCom 2021, ACM TUR-C 2017

Awards

April 2024: Best Paper Award, MMSys 2024
2019 – 2023: Student (Travel) Grant, SIGCOMM 2023, MobiCom 2021, MobiSys 2021, SIGMETRICS 2021, HotMobile 2019
April 2018: Outstanding Graduate, Provincial Department of Education of Anhui
2015 – 2017: Outstanding Student Scholarship, USTC
January 2017: Honorable Mention of Mathematical Contest in Modeling 2017 (MCM)
June 2016: Second Prize of USTC Electronic Design Contest, Institute of Electronics, CAS
October 2015: Outstanding Student Leadership Award, USTC
May 2015: Social Responsibility Scholarship, USTC
September 2014: Outstanding Freshman Scholarship, USTC

You are the No. th vistor of my homepage.

Xumiao Zhang Alibaba Cloud