Azure Data Engineer

We are a multinational team of individuals who believe that with the right knowledge and approach, technology is the answer to the challenges that business face today. We have been bringing this knowledge and approach to our customers since 2016, helping them translate technology into their success.

With Swiss roots and our own development team in Lima and throughout the region, we offer the best of both cultural areas: the talent and passion of Latin American professionals combined with Swiss organization skills and mindset.

Bertoni Solutions is looking for a Data Engineer to work 100% remotely for an USA client.

In this role, you will be responsible for developing data solutions within the Azure cloud environment, collaborating seamlessly with diverse teams across the organization to execute data strategy, design robust architecture, and establish effective governance. You must posses substantial expertise in Spark, Python, and valuable familiarity with Scala, along with a comprehensive grasp of Azure Data Factory (ADF) concepts; also, you must demostrate proficiency across a range of data access technologies, showcasing adeptness in strong SQL and PySpark-SQL.

This position is only available for applicants located/living in Latin America or North/Central America.

• Minimum of 6 years of experience.
• Hands-on experience using ADF to connect to different sources (RDBMS, Salesforce, flat files) and bring data into ADLS.
• Hands-on experience handling incremental load and full load approaches.
• Hands-on experience handling semi-structured data such as JSON.
• Hands-on experience using Databricks with delta lake concepts.
• Strong SQL skills with (Slow Change Types [SCD], Joins, Common Table Expressions [CTEs], Hierarchies, etc.)
• Hands-on experience with PySpark.
• Hands-on experience using Databricks Notebooks and optimization techniques.
• Experience with Azure Cloud Services for Data Engineering.
• Understanding of Data Layers: Raw, Refined and Curated Models.
• Experience with Databricks: programming and automating notebooks using ADF.
• Performance optimization of Spark or PySpark code, including techniques such as partitioning and data caching.
• Hands-on experience in creating a user-defined function (UDF) in Databricks and how it can be applied to a DataFrame.
• Management and operation of DataFrames for Data Engineering activities.
• Advanced English proficiency.
• Desirable: Scala.

• Excellent communication skills.
• Excellent Problem-solving skills.
• Critical thinking skills.
• Attention to detail.
• Decision-making.
• Teamwork.
• Adaptability.

Tags