Job Description

Databricks, Data Factory Role Big Data Engineer Responsibilities Implement Data integration and Data Warehouse based solutions using Big Data Technologies. Should be highly proficient in the use of Big Data / Open Source Technologies and standard techniques of Data Integration, Data Manipulation with hands-on contribution. Should be able to develop cost efficient and performant data pipelines in the cloud platform In-depth understanding of modern big data technology, including Data modeling and machine learning skills Knowledge of real time data streaming and aggregation architectural patterns and practice Essential Skills: 3+ Years hands on knowledge on SQL as well as SQL/NoSQL databases Proficient in programming languages such as Python and PySpark/Scala/Java Experience with Apache Spark in On-prem / Cloud (Databricks / EMR / DataProc HDInsight) Working knowledge of API development Experience in building data pipelines in ETL/ELT methods Experience with integration of data from multiple data sources Experience in one of the NoSQL databases, such as HBase, Cassandra, MongoDB Knowledge of real time/streaming/event based data engineering frameworks, such as Flume, Pub-Sub, Kinesis, Eventhub Experience with various messaging systems, such as Kafka or RabbitMQ Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming Working knowledge and experience in Big data services in one of the Cloud Provider will be good (AWS or Azure or GCP) The Role offers Opportunity to join a global team to do meaningful work that contributes to global strategy and individual development An outstanding opportunity to re-imagine, redesign, and apply technology to add value to the business and operations Gives an opportunity to showcase candidates' strong analytical skills and problem-solving ability Grow into a Technical architect role over a period