Senior Data Engineer- W2 Only (No H1b or OPT/CPT)

Job Description

Job Title: Senior Data Engineer

Location: Sunnyvale, CA- Hybrid

Duration: 8 months Contract to hire

Rate: $60-$70/hr on W2 Only

External Communities Job Description

Our customer, the world’s largest retailer, is looking for a Data Engineer to join their Data Ventures team on a 6 month contract-to-hire. This team is responsible for innovating to unlock the full value of client’s data by developing and productizing B2B data initiatives that empower merchants and suppliers to make better, faster decisions for the business. They are looking for a full-stack engineer with GCP experience to join their team as a leader in this initiative.

Enterprise Required Skills

* Sql, Spark, Hadoop, Big data, Scala, Hive, Cloud, Kafka, airflow, GCP, Python

Top Skills Details

7+ years of experience in:

* Excellent programming skills in Scala (Python is not preferred)
* Strong experience with Cloud (GCP environment)
* Experience with Spark/SQL/Hadoop
* Experience with Airflow

Responsibilities:

* Design, develop and build database to power Big Data analytical systems.
* Design data integration pipeline architecture and ensure successful creation of the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark, SQL, HQL and other technologies.
* Build robust and scalable applications using SQL, Scala and Spark.
* Create real time data streaming and processing using Kafka and/or Spark streaming.
* Work on creating data ingestion processes to maintain Global Data lake on Google cloud or Azure
* Engage with architects and senior technical leads to create and enhance complex software components.
* Design, configure and implement systems that can scale to process terabytes of data between heterogeneous systems on premise and cloud.
* Work with business customers, product managers and engineers to design feature-based solutions and implement them in an agile fashion.
* Develop proof-of-concept prototype with fast iteration and experimentation.
* Develop and maintain design documentation, test cases, performance and monitoring and performance evaluation using Git, Crontab, Putty, Jenkins, Maven, Confluence, ETL, Automic, Zookeeper, Cluster Manager
* Perform continuous integration and deployment using Jenkins and Git

Additional Skills & Qualifications

* GCP nice to have- primarily GCP environment: BQ, Dataproc, etc.