Data Engineer

Job Description Must Have Technical/Functional Skills Must Have Skills Databricks (Notebooks, Delta Lake, Delta Live Tables, Unity Catalog) Python and PySpark Spark SQL and Apache Spark ETL/Data Pipeline Development Big Data technologies: Hadoop, Spark, Kafka, Hive, HDFS, Sqoop Strong SQL querying and data modeling Data ingestion, transformation, cleansing, and validation Cloud platforms (AWS/GCP) Data warehouse and large-scale data processing experience Experience with streaming data using Kafka and Spark Streaming Agile methodologies (Scrum, Kanban, SAFe) Preferred Skills Scala / Scala-Spark IBM DataStage, Tibco, or similar ETL tools AWS services: S3, Glue, Lambda, Kinesis/MSK BI Tools: Tableau, Power BI, Spotfire, OAC Databases: Teradata, DB2, BigQuery, Mainframe Git version control Data governance and retention modeling Roles & Responsibilities Design, develop, and maintain scalable data pipelines and ETL solutions. Build and optimize data ingestion frameworks for structured and unstructured data. Implement data quality, validation, standardization, and governance processes. Develop data infrastructure integrating multiple heterogeneous data sources. Support analytics, reporting, machine learning, and business intelligence initiatives. Work with Data Scientists and BI teams to prepare and curate data for analytics. Develop scalable solutions for Big Data processing and streaming workloads. Analyze data and provide actionable insights and recommendations to business stakeholders. Gather requirements from business teams and translate them into technical solutions. Lead and mentor junior data engineers. Collaborate across multiple projects as the Data Engineering SME. Drive adoption of Data Engineering best practices and standards. Salary Range-$100,000-$135,000 a year TCS Employee Benefits Summary Discretionary Annual Incentive. Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans. Family Support: Maternal & Parental Leaves. Insurance Options: Auto & Home Insurance, Identity Theft Protection. Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement Time Off: Vacation, Time Off, Sick Leave & Holidays. Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.