Lead Data Engineer

As the Lead Data Engineer at Insite AI, you will be responsible for leading the architectural design, development, and deployment of scalable data management systems in a high-performance, customer-driven environment. Insite AI's AI-based platform enables autonomous decision-making that creates a connected assortment strategy, category and pricing execution and trade promotion environment. Insite AI's team comes from leading CPGs and brings their experiences to build a fully customizable AI-based platform that gives CPGs and Consumer Brands the most “explain-ability”, foresight, and granularity that results in an unprecedented quality of decision-making. If you are passionate about leveraging the power of AI to help clients reach their full potential, this is the opportunity for you!

• Lead the development of highly available, fault-tolerant, and scalable data management systems that support various AI models and analytics applications
• Build and maintain data integration, ETL pipelines, and data warehouse architectures on cloud platforms like AWS, Azure, and GCP
• Define and implement data governance practices, systems, and processes for data discovery, lineage, metadata management, and data quality
• Work with cross-functional teams comprising Data Scientists, Product Managers, and Software Engineers to design, build, and deploy robust and scalable data pipelines that power real-time decision-making
• Provide technical mentorship and leadership to a team of data engineers, and drive innovation and best practices in the engineering organization
• Work closely with the customer success team to troubleshoot and resolve issues related to data quality, ingestion, and integration
• Evaluate and select technologies and tools that enhance the performance, scalability, and security of Insite AI's data infrastructure

Requirements

• Bachelor's or Master's degree in Computer Science, Software Engineering, or a related technical field
• 7+ years of experience in designing, building, and deploying large-scale data management systems in the cloud
• Strong proficiency in programming languages such as Python, Java, and/or Scala
• Deep understanding of big data technologies and platforms such as Hadoop, Spark, Kafka, and NoSQL databases
• Experience with cloud computing platforms such as AWS, Azure, and/or GCP
• Experience with data warehousing and ETL technologies such as Redshift, Snowflake, and Airflow
• Strong technical leadership skills and ability to mentor and guide team members
• Excellent communication skills and ability to work collaboratively in a cross-functional team environment.

Benefits

• Health Care Plan (Medical, Dental & Vision)
• Retirement Plan (401k, IRA)
• Life Insurance (Basic, Voluntary & AD&D)
• Paid Time Off (Vacation, Sick & Public Holidays)
• Remote - Work From Home
• Family Leave (Maternity, Paternity)
• Short Term & Long Term Disability
• Training & Development

Tags