Aditya Soni

adityasoni25@gmail.com

Hey! I’m a Research Fellow at Microsoft Research in Bangalore, working with the M365 Research team.

I am interested in desigining principled reinforcement learning (RL) algorithms that can solve complex sequential-decision making problems. At Microsoft, my research spans systems like Microsoft Teams and Azure.

My current focus is on offline RL and challenges such as adapting policies to distribution shifts and non-stationarity in the environment. I am also very interested in exploring unsupervised RL, since the idea of training generalist policies that can adapt to diverse downstream tasks seems very exciting to me.

Previously, I interned at Microsoft Research, where I developed intelligent decision-making frameworks for improving sustainability in datacenters by optimizing CPU power consumption.

Before joining Microsoft, I did my bachelors in electrical engineering at Birla Institute of Technology and Science, Pilani.

I am applying to PhD programs to start in Fall 2025!

news

Nov 11, 2024	New paper on post-deployment policy adaptation to exogenous disturbances in environments.
Apr 27, 2024	Paper accepted at AIOps Workshop @ ASPLOS 2024 - Intelligent Overclocking for Improved Cloud Efficiency.

selected publications

Please see Google Scholar for a complete and up-to-date list of publications.

2024

Streetwise Agents: Empowering Offline RL Policies to Outsmart Exogenous Stochastic Disturbances in RTC

Aditya Soni, Mayukh Das, Anjaly Parayil, Supriyo Ghosh, Shivam Shandilya, Ching-An Cheng, Vishak Gopal, Sami Khairy, Gabriel Mittag, Yasaman Hosseinkashi, and Chetan Bansal

2024

Abs PDF

The difficulty of exploring and training online on real production systems limits the scope of real-time online data/feedback-driven decision making. The most feasible approach is to adopt offline reinforcement learning from limited trajectory samples. However, after deployment, such policies fail due to exogenous factors that temporarily or permanently disturb/alter the transition distribution of the assumed decision process structure induced by offline samples. This results in critical policy failures and generalization errors in sensitive domains like Real-Time Communication (RTC). We solve this crucial problem of identifying robust actions in presence of domain shifts due to unseen exogenous stochastic factors in the wild. As it is impossible to learn generalized offline policies within the support of offline data that are robust to these unseen exogenous disturbances, we propose a novel post-deployment shaping of policies (Streetwise), conditioned on real-time characterization of out-of-distribution sub-spaces. This leads to robust actions in bandwidth estimation (BWE) of network bottlenecks in RTC and in standard benchmarks. Our extensive experimental results on BWE and other standard offline RL benchmark environments demonstrate a significant improvement (≈ 18% on some scenarios) in final returns wrt. end-user metrics over state-of-the-art baselines.
Intelligent Overclocking for Improved Cloud Efficiency

Aditya Soni, Mayukh Das, Pulkit Misra, and Chetan Bansal

In AIOps ’24 workshop @ ASPLOS [5th International Workshop on Cloud Intelligence / AIOps], Apr 2024

Abs PDF

Overclocking computing nodes in datacenters is emerging as an area of great interest since it presents an opportunity to temporarily boost performance and handle transient demand surges more cost-effectively for cloud providers. However, indiscriminate overclocking increases risks like power capping events that negate performance gains, accelerated component wear, and power consumption spikes. This work proposes an intelligent overclocking orchestration framework that employs a holistic system continuously monitoring workload characteristics, resource utilization, and power telemetry across the infrastructure. It leverages advanced modeling techniques to anticipate demand and optimize overclocking decisions based on performance benefits and mitigation of associated risks through adaptive policies. Preliminary evaluations on production cloud workload traces demonstrate the efficacy of the intelligent overclocking system in reducing power capping incidents by up to 100 times compared to naive approaches. Furthermore, the work outlines ongoing research directions, including the application of reinforcement learning techniques to derive globally optimal overclocking policies while incorporating fairness and reliability objectives, and seamless extensibility to emerging accelerator technologies.