Uncovering Geometric Primitives in Object Representations
Vision Models achieve remarkable accuracy in categorizing objects, yet it remains unclear if these successes are driven by superficial texture matching or a deeper structural understanding of the world. Determining whether a model’s internal representation of a „table“ is grounded in the abstract concept of a „rectangle“ is essential for developing AI that perceives the world through human-like geometric reasoning. This research introduces the concept of approximate probes to quantify the geometric transfer between pure mathematical ideals and complex real-world entities. These probes are then deployed across the layers of a pre-trained vision transformer to evaluate if the model successfully recognizes the underlying geometry of real-world photographs without ever being trained on them.
References
Tenney, I., Das, D., & Pavlick, E. (2019). BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 4593-4601).
Arnold, S., & Gröbner, R. (2026). Locating and Editing Figure-Ground Organization in Vision Transformers. arXiv preprint arXiv:2603.06407.
Mechanistic Interpretability of Syntactic Reduction
Natural language frequently employs abbreviated structures, such as verb contractions (e.g., isn’t) and the omission of subordinating conjunctions. This thesis aims to dissect the shared functional mechanisms behind morphological contractions and that-deletion within language models using logit lens and activation patching.
References
https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens
Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022). Locating and editing factual associations in gpt. Advances in neural information processing systems, 35, 17359-17372.
Arnold, S., & Gröbner, R. (2025,). Steering Prepositional Phrases in Language Models: A Case of with-headed Adjectival and Adverbial Complements in Gemma-2. In Proceedings of the 8th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP (pp. 69-78).