Aounon Kumar

AI/ML Researcher

I am a Postdoctoral Research Associate at Harvard University working in Trustworthy AI with Professor Himabindu Lakkaraju. My research focuses on the robustness, security, and reliability of machine learning (ML) models. It involves designing algorithms to defend models against adversarial inputs, for example, safeguarding large language models (LLMs) from prompts that circumvent safety guardrails. I have studied and contributed to model robustness in several machine learning domains including computer vision, reinforcement learning, and language modeling. My work has been accepted in prominent ML conferences such as ICML, ICLR and NeurIPS, and I am actively involved in collaborative projects within the academic community.

Media Coverage: My recent works on LLM robustness and reliability have been featured in major media and academic news outlets:

  1. Science News Magazine, D^3 Institute at Harvard. Work featured: Certifying LLM Safety against Adversarial Prompting.
  2. The New York Times. Work featured: Manipulating Large Language Models to Increase Product Visibility.
  3. The Washington Post, Bloomberg, Wired, New Scientist, The Register, TechSpot. Work featured: Can AI-Generated Text be Reliably Detected?.

Before joining Harvard, I completed my PhD at the University of Maryland in certified robustness in machine learning (see my dissertation here). I was fortunate to be advised by Professors Soheil Feizi and Tom Goldstein. During my PhD, I have spent time as an applied scientist intern at Amazon and a research intern at Nokia Bell Labs, where I worked on uncertainty estimation for human action recognition models and network security-related machine learning applications. I have also served as a reviewer for machine learning conferences such as ICML, ICLR and NeurIPS.

I did my undergraduate studies at IIT Mandi and my master’s at IIT Delhi, where I studied a wide range of topics in computer science such as machine learning, advanced algorithms, combinatorial optimization, complexity theory and cryptography. My master’s thesis was on the computational hardness of approximating the optimal solution of a variant of the k-center clustering problem.


News

   
Aug 30, 2024                       :page_facing_up: New pre-print on Manipulating Large Language Models to Increase Product Visibility! Covered by The New York Times.
   
Jul 10, 2024                       Accepted at COLM 2024 :llama:: Certifying LLM Safety against Adversarial Prompting. Covered by Science News Magazine.
   
Dec 19, 2023 Graduated from UMD! :man_student:


Selected Publications

See full list at Google Scholar.



COLM 2024 Certifying LLM Safety against Adversarial Prompting
Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Aaron Jiaxun Li, Soheil Feizi, Himabindu Lakkaraju
OpenReview, ArXiv, Code, PDF
Media Coverage: Science News Magazine, D^3 Institute at Harvard.


Preprint Manipulating Large Language Models to Increase Product Visibility
Aounon Kumar, Himabindu Lakkaraju
ArXiv, Code, PDF
Media Coverage: The New York Times.


TMLR 2024 Can AI-Generated Text be Reliably Detected?
Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, Soheil Feizi
OpenReview, ArXiv, Code, PDF
Media Coverage: The Washington Post, Bloomberg, Wired, New Scientist, The Register, TechSpot.


NeurIPS
D&B 2024
MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models
Tessa Han, Aounon Kumar, Chirag Agarwal, and Himabindu Lakkaraju
OpenReview, ArXiv, PDF


ICLR 2022 Policy Smoothing for Provably Robust Reinforcement Learning
Aounon Kumar, Alexander Levine, Soheil Feizi
OpenReview, ArXiv, Code, PDF


ICML 2020 Curse of Dimensionality on Randomized Smoothing for Certifiable Robustness
Aounon Kumar, Alexander Levine, Tom Goldstein, Soheil Feizi
ICML Proceedings, ArXiv, Code, PDF


Contact

Science and Engineering Complex :office:
150 Western Ave
Office #6220
Allston, MA 02134