Sherin Muckatira
Research Interests: Language Models, Pre-training, Generalization, LLM Evaluation, Mathematical Reasoning, Interpretability, Parameter-Efficient Fine-Tuning
I’m a Computer Science PhD candidate at the University of Massachusetts Lowell, advised by Prof. Anna Rumshisky. My research takes a data-centric approach to understanding language-model abilities: how training data can shape which capabilities emerge, how data exposure can reveal when models move from memorization toward generalization, and how evaluation data can uncover failures in LLMs that standard benchmarks miss. My work spans language-model pre-training, training dynamics, generalization, model analysis and interpretability, and LLM evaluation. I’m interested in Research Scientist and Applied Scientist roles focused on language models, reasoning, training, evaluation, and model understanding.
News
Research
My research takes a data-centric approach to understanding language-model abilities: how training data can shape which capabilities emerge, how data exposure can reveal generalization during pre-training, and how evaluation data can uncover capabilities and failures that final-answer benchmarks miss.
Eliciting abilities through pre-training data
I study how the structure of pre-training data affects which capabilities language models acquire. By simplifying the pre-training distribution, I investigate whether abilities typically associated with larger models—such as zero-shot in-context learning—can emerge at much smaller scales.
Understanding generalization during pre-training
I use data exposure during pre-training as a way to study when models move beyond what they have directly seen. By separating exposed and unexposed grammatical examples, I trace delayed generalization across training and analyze how model representations and attention change as this behavior emerges.
Evaluating reasoning beyond final answers
I develop process-aware evaluations for reasoning capabilities that are difficult to capture with standard final-answer benchmarks. Using the CROWDMATH program we develop an annotated dataset with collaborative research discussions and study whether language models can recognize partial progress, errors, corrections, and completed results in collaborative mathematical problem solving.
Efficient language-model pre-training
I have also worked on making large-scale language-model training more parameter-efficient. In ReLoRA, we introduce a method for training high-rank neural networks through a sequence of low-rank updates, showing that Transformer language models can achieve performance comparable to standard full-rank training while reducing memory use and improving training speed. The work explores whether techniques commonly used for parameter-efficient adaptation can also make pre-training more efficient.
Publications
Under Review, 2026
Under Review, 2026
Under Review, 2026
BMC bioinformatics, 2013
Education
Experience
- Study how language-model capabilities emerge during pre-training, with a focus on training dynamics, generalization, memorization, and learned representations.
- Pre-trained and analyzed Transformer language models across multiple scales to investigate when linguistic capabilities emerge and how they relate to data exposure.
- Developed data-centric evaluation methods for LLM reasoning, including benchmarks for mathematical problem solving and recognizing incremental progress toward solutions.
- Developing and testing efficient pre-training methods that reduce computational cost without sacrificing performance.
- Research spans LLM pre-training, evaluation, model interpretability, efficient training, and reasoning; published at NAACL, EMNLP, ICLR, and LREC-COLING.
- Developed multilingual evaluation methods for large language models, studying model quality beyond English.
- Built and trained LLM judge models for evaluating multi-turn conversations across languages.
- Built a synthetic-data generation pipeline for domains with limited and privacy-sensitive training data.
- Created structured training datasets and fine-tuned Mistral-7B-Instruct for document understanding and guideline identification.
- Implemented Gene expression pattern annotation using SIFT feature extraction on images in the Berkeley Drosophila Genome Project (BDGP).
- Constructed Codebooks using Bag of Words and Sparse Coding Approach.
- Developed firmware for the physical layer of Wireless LAN chips using the Wifi 802.11 protocol.
- Designed and implemented features such as Spectral Scan and Radar Detection.
- Developed signal processing applications for radio communication, focusing on transmit and receive chains on a Vector Signal Processor for Power Amplifier characterization.
- Implemented communication interfaces between host processors and co-processors to enhance functionality in Power Amplifier characterization applications.