Apply Now »

Lead Data Engineer - Big Data/NoSQL

« Back to results

Ted Staeb Senior Associate

Phone Work 4157957320
Phone Fax

Job Info

Category Big Data/NoSQL
Employment Type Full-Time Employment
Compensation $0.00 - $0.00
Location United States, CA - 94941

Client Introduction

Our client is one of the top 3 job boards (late stage startup) in the US and they are aggressively expanding their team both in San Francisco and Mill Valley to join their core product team.

You’ll be part of a very fast-growing and rapidly innovating machine learning team which will be building our next generation of products. You will help define the ML roadmap and then be the owner of modeling and delivery of all Machine Learning products to support small businesses.

This includes a diverse range of needs covering areas like fraud prevention, paid advertising optimizations, and applicant delivery for job postings. A typical week would comprise brainstorming with product on next-gen features, prototyping models and improving model features, and collaborating with engineers to deploy ML pipelines and products.

The ideal candidate will be extremely curious, have a strong scientific background in statistics and machine learning, and be obsessed with results. You exercise very sound judgment and have the ability to balance sophistication with simplicity, scientific rigor with pragmatism, and agility with quality. All of this with the tightly coupled goals of helping small businesses recruit and grow.

In this role you work with a small, collaborative team of engineers, product managers, and designers – so excellent interpersonal and communication skills are also a must. And most importantly – we look for people that can prioritize, multi-task, and deliver – because it’s a lot more fun to get things done.

• All employees receive 3 paid volunteer days per year
• 100% company paid medical/dental/vision/life coverage; 80% dependent coverage
• Equity in a late stage startup backed by top-tier VCs
• Walking, running and biking trails steps away from the office
• Onsite gym and fitness classes
• Free catered lunch; new menu daily
• Paid holidays and flexible paid time off
• Your choice between Mac or PC
• Dog-friendly office (with dog-free zones if you are so inclined)
• Free parking

Job Description

We are looking for a talented engineer to join our growing data engineering team. The ideal candidate has significant experience in leading a small group of engineers that building scalable data platforms that enable business intelligence, analytics, data science, and data products. You must have strong, hands-on technical expertise in a variety of technologies and the proven ability to fashion robust, scalable solutions. You should have a passion for continuous improvement quality.

We embrace a wide variety of technologies and work very closely with data scientists and business stakeholders to deliver end to end solutions. If you are interested in a fast paced environment, the latest technologies, and fun data problems, come join us!

Job Responsibilities

• Design and develop big data applications using a variety of different technologies.
• Develop logical and physical data models for big data platforms.
• Automate workflows using Apache Airflow.
• Write data pipelines using Apache Hive, Apache Spark, Apache Kafka.
• Create solutions on AWS using services such as Kinesis, Lambda, and API Gateway.
• Provide ongoing maintenance and enhancements to existing systems, and participate in rotational on-call support.
• Learn our business domain and technology infrastructure quickly and share your knowledge freely and proactively with others in the team.
• Mentor junior engineers on the team
• Lead daily stand-ups and design reviews
• Groom and prioritize backlog using JIRA
• Act as the point of contact for your assigned business domain


• 7+ years of hands-on experience with developing data warehouse solutions and data products.
• 4+ years of hands-on experience developing a distributed data processing platform with Hadoop, Hive, Spark, Airflow, Kafka, etc.
• 2+ years of hands-on experience in modeling and designing schema for data lakes or for RDBMS platforms.
• Experience with programming languages: Python, Java, Scala, etc.
• Experience with scripting languages: Perl, Shell, etc.
• Practice working with, processing, and managing large data sets (multi TB/PB scale).
• Exposure to test driven development and automated testing frameworks.
• Background in Scrum/Agile development methodologies.
• Capable of delivering on multiple competing priorities with little supervision.

Nice to have
• Experience building machine learning pipelines or data products.
• Familiarity with AWS or GCS technologies.
• Be passionate about or have contributed to open sourced engineering projects in the past.

Required Experience

Required Education

Bachelor's Degree in computer science or equivalent experience.

Previous MonthNext Month