Principal Data Engineer
CAIS
This job is no longer accepting applications
See open jobs at CAIS.See open jobs similar to "Principal Data Engineer" Innovation Bay.CAIS is a fintech firm on a mission to build the first truly open marketplace for alternative investments, where financial advisors and asset managers can engage and transact directly on a massive scale. We are a rapidly growing, high-energy, and collaborative group looking for people with great attitudes, grit, and creative problem-solving skills. Join us in disrupting a multi-trillion-dollar industry.
We are seeking a Data Engineering Lead to join our Technology team. This high impact, hands-on role will be responsible for developing and maintaining the solution architecture that makes data a central part of CAIS. The Data Engineering Lead will collaborate with analytics and product teams to improve data models that feed business intelligence tools, increase data accessibility, and foster data-driven decision making across the firm. In addition, this role will work closely with our Product Engineering and Operations Engineering teams to provide data support on greenfield work.
Ideal candidates have the ability to design, track, and execute on a strategic vision that aligns key knowledge initiatives across all levels of the firm. You are a strong cross-functional communicator with a proven ability to solve open-ended problems in an elegant, scalable manner. They possess creativity, flexibility, and the drive to consistently deliver on mission critical projects with tight timelines and competing priorities.
Responsibilities
- Design, develop and automate secure, robust, and high-performing data platform infrastructure to drive CAIS business growth
- Use modern approaches to process data through to Snowflake using SQL and code-based ELT (and/or ETL)
- Develop production ready pipelines and robust enterprise class solutions
- Analyze production workloads and develop data workflow strategies to optimize Snowflake warehouse with scale and efficiency
- Design data platform capabilities, pipelines and models for optimal storage and retrieval that represent the product entities and meet business requirements
- Leverage best of breed technologies including open-source and cloud solutions to build sophisticated features that enhance platform capabilities and help support business analytics
- Build and support pipeline orchestration via Airflow
- Technical Leadership and Data Architecture (Snowflake preferred)
What We’re Looking For
- Engineering, Mathematics, or other technical degree(s)
- Understanding of AWS and how to create secure infrastructure and data pipelines
- Experience with AWS solutions such as Lambda, S3, and Kafka
- 7+ years of strong programming experience including Python / JVM languages with expertise in extracting and transforming data
- 10+ years of experience with SQL
- 10+ years of data warehousing experience with examples of complex ETL/ELT data workflows
- 7+ years of experience as a technical lead covering design, development and delivery.
- 7+ years of experience with object-oriented programming, strong ability to write easy-to-scale, high-quality code
- 7+ years of experience with schema design and dimensional data modeling
- Experience with Integration tools like Fivetran/Airbyte
- Experience with Kubernetes
- Experience with IaC (Terraform)
- Experience with or knowledge of Agile Software Development methodologies
- Experience deploying ML Pipelines / ML Ops (E.g. MLFlow)
Desirable Skills/Knowledge
- Prior experience working within Alternative Investments and/or Financial Services preferred
- Good knowledge of financial markets and financial instruments a plus
- AWS Certified Solution Architect / Associate or similar
- Experience with GitHub Actions a plus
- Experience in developing integrations with Tableau (or similar Business Intelligence tool) a plus
- REST API Development
- Distributed Computing
- Experience with Athena
- Experience with DBT
- Experience with functional programming
- JVM Stack (Java or Kotlin)
- Machine Learning
This job is no longer accepting applications
See open jobs at CAIS.See open jobs similar to "Principal Data Engineer" Innovation Bay.