あなたの研究テーマについて教えてください。What is your research theme?
Title: Scalable Additive Smoothing Matrix: Integrating Transformers and CNNs
Summary: We analyze the information flow in Transformer architectures by contrasting their global attention with the local feature extraction of CNNs. To mitigate Transformers’ bias toward frequent co-occurrences, we propose an additive smoothing matrix that incorporates CNN-like local inductive biases, thereby enhancing spatial continuity. We validate our approach through attention dispersion analysis, the smoothing method, and comparisons of token representations shaped by local and global inductive biases.
This work tries to deepen the understanding of information flow in Transformer architectures and offers a simple mechanism to embed local inductive biases, bridging the gap between CNNs and Transformers. This contributes to advancing more robust and interpretable models for vision and multimodal learning.
なぜ博士課程へ進学しましたか？Why did you decide to pursue a doctoral program?
I have a strong passion for exploring the unknown, regardless of the topic. My resume reflects this curiosity. My undergraduate major was Biomedical Engineering, and I pursued a Master’s degree in Information and Communication Engineering. I also gained hands-on experience as an Embedded Engineer, working on hardware development. Currently, I have shifted my focus to Natural Language Processing. My constant curiosity about the unknown motivates me to persist in research, even when failures are a common part of the experimental process. The doctoral program offers me an excellent opportunity to explore a field in depth while simultaneously enhancing my cognitive capacity. This mutual reinforcement allows me to delve deeper into my research topics and continuously expand my understanding.
Graniteの支援で役立っている・助かっていることを教えてください。Please share what has been helpful or beneficial through the support provided by Granite.
1. The support has given me an opportunity to broaden my research scope. With large language models (LLMs) blurring the boundaries between modalities, the funding enables me to access resources beyond language, allowing me to explore and integrate other modalities into my research.
2. It has also provided valuable opportunities to communicate with leading researchers from around the world. This not only enhances my cognitive growth but also improves my communication effectiveness both of which are essential qualities for a researcher.
将来の目標、もしくはあなたが胸に秘めた野望があればぜひ教えてください。Please share your future goals or any ambitions you hold close to your heart.
My goal is to strengthen a systematic capability for (1) efficiently acquiring and understanding new knowledge, (2) identifying problems within information, and (3) developing effective solutions. This approach to processing information is not limited to my current research area; I also aim to apply it to hardware fields such as robotic arm control and drone control. With this mindset and experience, I hope to accelerate the discovery and resolution of challenges faced by my group, both in research and in development.
博士課程進学を考えている人へのメッセージをください。Please share a message for those considering pursuing a doctoral program.
I believe that genuine interest is the most important factor when considering a doctoral program. Interest helps us stay focused on the research process rather than being overly attached to the results. In reality, most experiments do not succeed on the first attempt, and failures are a natural part of the journey. If we only care about the outcomes, it is easy to lose patience and motivation over time. However, when driven by curiosity and passion, even setbacks can become valuable learning experiences. Therefore, before making a decision, I strongly recommend reflecting carefully on how much genuine interest you have in research.
