Senior Data Engineer

Are you passionate about shaping the future of AI and empowering millions of users to unlock their full potential? The M365 Copilot Hub and OneNote teams are leading an exciting transformation with Copilot Pages and Notebooks—intelligent, dynamic experiences infused with powerful AI that act as a true "second brain." Imagine effortlessly capturing ideas, intuitively understanding complex information, and seamlessly taking informed action. This is the heart of our mission.

As a Senior Data Engineer in Office Product Group, you will play a critical role in ensuring product telemetry is piped, aggregated, verified, and delivered to Data Scientists. Your work will span across multiple hosts and endpoints of Copilot Pages and Notebooks and scale to data coming in from around the world. The clarity driven by this data will enable us to have deep insight and understanding of how customers are using the product, influence decision making and shape the future of the product.

This opportunity will build your experience working directly with Software Engineers, Data Scientists & Product Managers, all working together to build an innovative product that delights users. The complexity and planet-scale nature of the data will challenge and increase your data pipeline design skills. The direct connection to both the customer and the building of the product will advance your skills in truly delivering product visions & missions.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Required Qualifications

• Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling or data engineering work
  • OR Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ year(s) experience in business analytics, data science, software development, or data engineering work
  • OR equivalent experience.
• 2+ years experience handling data under strict compliance, laws and regulations.
• 2+ years experience with large data sets spanning multiple architectures.
• 4+ years experience with C#, Python, SQL, Scala or other scripting languages for data processing and automation.

Other Requirements

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:

• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred Qualifications

• Experience with: PowerBI, Microsoft Fabric Data Platform, data warehousing (DW), real-time intelligence solutions, business intelligence (BI) tools, Microsoft Purview, and Azure databases such as SQL Database, Cosmos DB, or PostgreSQL.
• Experience with Kusto Databases and Kusto Query Language.
• Experience with operating under GDPR, DORA, CCPA and other regulations.
• Experience with Data Engineering for AI and Data Science.
• Expertise in distributed computing frameworks like Spark, Hadoop, or other cloud-based data solutions.
• Experience with on-premises enterprise data warehouse (EDW) appliance such as: Teradata, Netezza, or Exadata.
• Experience developing solutions in distributed cloud platforms such as Azure, AWS, or GCP, with a focus on data storage, processing, and security.

Data Engineering IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $158,400 - $258,000 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

Microsoft will accept applications and processes offers for these roles on an ongoing basis.

• You will coordinate with Data Scientists, Product Managers and Software Engineers to ensure that the production telemetry contains the information required to generate insights and understanding of how customers are using the product.
• You will build code to extract raw data, validate its quality, and apply your deep data knowledge to ensure the correct data is ingested across the pipeline. You’ll also create techniques to transform raw data into forms compatible with downstream data sources.
• You will identify opportunities to leverage and contribute to the development of data tools that are used to transform, manage, and access data. You’ll also write and validate code to test the storage and availability of data platforms so that they’re more resilient.
• You will write code to implement performance monitoring protocols across data pipelines, building visualizations and aggregations to monitor pipeline health. You’ll also implement solutions and self-healing processes that minimize points of failure across multiple product features.
• You will anticipate data governance needs, designing data modeling and handling procedures to ensure compliance with all applicable laws and policies. You’ll also govern data accessibility within your assigned pipelines.