May 1, 2021

May 1, 2021

RL-Based Sampling Schemes for Adaptive Quantitative Group Testing

RL-Based Sampling Schemes for Adaptive Quantitative Group Testing

RL-Based Sampling Schemes for Adaptive Quantitative Group Testing

HDSI Undergraduate Research Scholarship project, 2021. Mentored by Professor Tara Javidi. Presented to the California Alliance for Data Science Education, based at UC Berkeley.

HDSI Undergraduate Research Scholarship project, 2021. Mentored by Professor Tara Javidi. Presented to the California Alliance for Data Science Education, based at UC Berkeley.

HDSI Undergraduate Research Scholarship project, 2021. Mentored by Professor Tara Javidi. Presented to the California Alliance for Data Science Education, based at UC Berkeley.

Year

2021

Location

Halıcıoğlu Data Science Institute, UC San Diego

Category

Research

Duration

7 Months
Group Testing & Adaptive Methods
Group Testing & Adaptive Methods
Group Testing & Adaptive Methods

Group testing is a combinatorial strategy to identify a small number of "infected" individuals in a large population by grouping samples rather than testing each one individually.

  • Adaptive group testing refines test design based on prior results (Markov process), unlike non-adaptive methods where tests are predetermined.

  • The infected population is modeled as a sparse binary vector x of length N, with K ones.

  • Measurements are made using selected rows from the Walsh-Hadamard matrix W, which is orthogonal and binary-structured. Each test computes y_t = W_t x. The goal is to identify the positions of the 1’s in x with as few tests as possible.

Faster Ground-Truth Algorithm
Faster Ground-Truth Algorithm
Faster Ground-Truth Algorithm

We propose a faster, deterministic approach leveraging properties of the Walsh-Hadamard matrix:

  • Use initial log(N) rows to compute a scalar sum S of the infected indices.

  • Instead of solving a massive linear system (with up to nCk solutions), they reduce the problem to finding K values that sum to S, significantly speeding up computation.

  • Achieves better performance (e.g., works for K=5, N=32), while prior work was limited to K=2.

RL-Based Approach with Deep Q-Learning (DQN)
RL-Based Approach with Deep Q-Learning (DQN)
RL-Based Approach with Deep Q-Learning (DQN)

An RL agent is trained to choose matrix rows to include in the test set:

  • State: Current subset of sampled rows W_hat and corresponding measurements y.

  • Action: Pick a new row (without replacement) from W.

  • Reward: Encourages fewer candidate solutions (best case: 1), penalizes repetition or no improvement.

  • Training details: Uses experience replay, dynamic epsilon-greedy strategy, padded matrix representations, and custom episode termination conditions to cope with changing state/action space.

Results & Conclusion
Results & Conclusion
Results & Conclusion
  • Ground-truth approach showed significant improvements in speed and scalability.

  • DQN method struggled to converge efficiently—especially for larger N, K.

  • Training became unstable or ineffective as episode lengths and action/state space complexity increased.

  • Conclusion: While the deterministic method outperforms prior research, the RL approach requires major tuning and enhancements to match or surpass traditional algorithms.

  • More Works More Works

Let's Chat

BASED IN the Bay Area,

CALIFORNIA

AI Engineer
+ Musician and Artist

I would love to discuss anything ranging from Python development, LLMs and ML theory, EDM, sound design, to different art styles.

Let's Chat

BASED IN the Bay Area,

CALIFORNIA

AI Engineer
+ Musician and Artist

I would love to discuss anything ranging from Python development, LLMs and ML theory, EDM, sound design, to different art styles.

Let's Chat

I would love to discuss anything ranging from Python development, LLMs and ML theory, EDM, sound design, to different art styles.

Let's Chat

BASED IN the Bay Area,

CALIFORNIA

AI Engineer
+ Musician and Artist

I would love to discuss anything ranging from Python development, LLMs and ML theory, EDM, sound design, to different art styles.