BlackRock

Lead Data Engineer

Posted on: 30 Nov 2021

Seattle, WA

Job Description

Description

About this role

BlackRock's Aladdin Client & Sales Solutions (ACSS) team is responsible for delivering tools, algorithms, and solutions to drive the efficiency of BlackRock's Client Business organizations globally and digitally transform how BlackRock's clients experience the firm. Do you have demonstrated success delivering impact and commercial outcomes? Then, we want you! We are passionate about delivering state of the art technology to our users in a scalable and resilient manner, coupled with a great user experience.

We are a team that recognizes strength comes from diversity and embrace technical expertise and curiosity equally. We are focused on delivering commercial outcomes and are passionate about building outstanding products.

Job Purpose/ Background

ACSS DaaP (Data as a Platform) team is looking for a development Leader to contribute into our data driven and computational platform. The data platform is compelling analytics offering, supported by high quality historical data and multiple heterogenous data storages.  This data is then transformed further to create analytical insights, which will be utilized by Data Science modelers and Application Developers to deliver actionable insights to sales and marketing teams.

The ideal candidate should be highly motivated to create, optimize, or redesign data pipelines and computation platform to support our next generation of products and facilitate deeper analysis. To lead in the crafting, implementation, and maintenance of this platform one would primarily need OLTP and OLAP knowledge along with Data Warehousing concepts and distributed computing experience, such as, Hadoop and Spark along with knowledge for data curation and analytical skills.

Scope of Role:

As Lead Data Engineer, you will:

Lead a global team with footprints in North America, Europe, and Asia

Lead and guide the development team on project execution to align with business vision and objectives while defining best practices and code standards

Share responsibility for architecting and implementing the next generation of Data Platform Architecture for Data Pipeline and Distributed Computation.

Collaborate with various partners to understand multi-functional requirements and convert them into reusable service components

Define and drive the data modeling solution for Data Warehousing and distributed data storage including best practices and design standards.

Core Responsibilities:

Participate in architecting, designing and implementation a scalable computation and data distribution platform.

Identify, design, and implement internal process improvements and share with the relevant technology organization.

Develop and mentor other team members in design and development while ensuring accurate development estimates on projects and tasks.

Establish and enforce CI/CD, code review, design standards, best development practices.

Motivate data engineers to deliver top quality, supportable products that continue to raise the bar higher.

Work with data scientists to develop data ready tools to support their job.

Identify, investigate, and resolve data discrepancies by finding the root cause of issues; work with partners across various cross-functional teams to prevent future occurrences.

Understand existing systems and resolve operations issues while working with other support staff located across the globe.

Automate manual ingestion processes and optimize data delivery ensuring SLAs are met. 

Design, maintain, and own the Data Infrastructure. Work with infrastructure teams on re-designing environment for greater scalability.

Be up to date with the latest tech trends in the big-data space and recommend them as needed.

You will be accountable for managing high-quality data exposed for internal and external consumption by downstream users and applications.

Qualifications:

10+ years of hands-on experience in Data Engineering or Software Engineering. 

Experience leading or mentoring a team and drive the execution. 

7+ years of experience with data transformation and computation leveraging OO languages with Python for data transformation (Core Python, Pandas and pySpark) or Java/J2EE/Spring architecture design and development 

5+ years experience with SPARK (pySpark), performance tuning and scaling. 

5+ years using distributed eco-systems like Hadoop, Hive, etc. Proficiency on bucketing, partitioning, tuning and different file storages (like S3) and formats (ORC, PARQUET & AVRO). 

Experience with design and actual implementation of Data Warehouse or OLAP system or Data Lake 

Experience with ETL tools and Workflow management systems like Airflow, Luigi, NiFi, Kilo. 

Extensive experience and advanced knowledge of SQL and relational databases (e.g., MS SQL Server, MySQL, Postgres), data modeling, stored procedures, and complex queries. . 

B.S. / M.S. degree in Computer Science, Engineering, or a related discipline. 

Huge Plus if you have:

Experience with stream-processing systems: Storm, Spark-Streaming, Kafka 

Experience with distributed databases and query engines like Snowflake or Presto is huge plus 

Experience with API development in Python or Java 

Experience with containerization architecture: Docker and Kubernetes 

Experience with cached databases (e.g., Ignite) 

Knowledge of any Graph Databases 

Experience with Scala 

Any experience with Cloud platform such as Azure, AWS or GCP is huge PLUS! 

BlackRock

New York, New York

BlackRock, Inc. is an American global investment management corporation based in New York City. Founded in 1988, initially as a risk management and fixed income institutional asset manager, BlackRock is the world's largest asset manager with $6.5 trillion in assets under management as of April 2019. BlackRock operates globally with 70 offices in 30 countries and clients in 100 countries. Due to its power and the sheer size and scope of its financial assets and activities, BlackRock has been called the world's largest shadow bank.

In May 2019, BlackRock received widespread criticism for the environmental impact of its holdings. It is counted among the top three shareholders in every oil “supermajor” except Total, and is among the top 10 shareholders in seven of the 10 biggest coal producers. In its 2018 annual letter to shareholders, chief executive Larry Fink said that his overriding duty is to make customers money, whatever the environmental consequences.

Similar Jobs