AI Research Scientist (Multimodal Foundation Models)

Quick Facts

• Focus area: multimodal foundation models—architecture, pre-training, and distillation for on-device intelligence

Description

Investigate how multimodal foundation models should be designed, trained, and distilled, with an emphasis on making capable models practical under on-device constraints (memory, latency, energy, and privacy). Develop and evaluate new architectures, pre-training objectives, data strategies, optimization methods, and teacher–student learning techniques to transfer capabilities from large models to smaller, efficient models. Co-develop frontier and efficient models for Apple silicon and incorporate deployment constraints throughout the research and evaluation pipeline.

Responsibilities

• Research model architecture, pre-training objectives, data mixtures, optimization, scaling, distillation, and evaluation across the foundation-model lifecycle
• Design architectures and teacher–student training methods that distill effectively while preserving reasoning and multimodal understanding
• Explore multimodal pre-training across language, images, video, audio, and sensor-derived representations
• Study compute- and data-optimal scaling, including curricula, tokenization, and training objectives
• Develop offline and on-policy distillation using supervision signals such as logits, representations, and rationales
• Evaluate models across language understanding, reasoning, instruction following, and multimodal capabilities

Requirements

• Hands-on experience designing, implementing, and running large-scale pre-training experiments for large language models
• Experience with LLM pre-training topics: architecture, training objectives, data mixtures, tokenization, curricula, scaling, and optimization
• Proficiency with PyTorch or JAX and distributed training systems
• Experience evaluating pre-trained models for language understanding, reasoning, instruction following, or multimodal capabilities
• Strong understanding of transformer-based architectures and approaches to efficient or scalable foundation-model training
• Master’s degree or equivalent practical experience in machine learning, computer science, or related technical field

Benefits

• Opportunity to shape new research directions and run ambitious experiments at scale
• Translate successful ideas into foundation-model technologies suitable for Apple products
• Potential for publications and open sourcing of selected models, research artifacts, evaluations, and tools