Client: Our client is a Chicago-based registered investment advisor and the affiliate of London-based world’s top 10 investment companies.
Position overview: The project allows to work on different engineering solutions for all divisions of the investment company, including Data, Portfolio Management, Analysis, Operation etc. It's planned to adopt a wide array of technology and approaches from Open Source to tried-and-true enterprise platforms.
- Responsibilities: Build and support event-driven solutions using the Cloudera Data Platform (PaaS)
- Develop, test and maintain event-driven data architecture
- Develop data set processes for data modelling, mining, and production
- Recommend ways to improve data reliability, efficiency, and quality
- Implement large data migration efforts to modern architectures
- Document data flows, including system integrations and design patterns
- 3-5 years of hands-on experience with AWS Services such as S3, Glue, Lambda, RDS, Eventbridge, and other supporting AWS Serverless components
- Good knowledge of IAM principles of least access in AWS
- Good understanding of Infrastructure-as-code and hands-on experience with AWS Cloudformation
- Experience designing and developing ELT/ETL pipelines using Apache Spark and AWS Glue Streaming
- Experience working on Cloudera Data Platform (CDP)
- Good Knowledge of UC4, Apache Airflow and Apache Nifi
- Experience and understanding of standard SQL database products (Postgres, MySQL, SQL Server), and strong SQL and RDBMS knowledge
- Experience working within a set of data architecture principles
- Good knowledge of database design such as star Schema (Facts and Dimensions) and Data Lake Principles
- Basic understanding of AWS DevOps and CICD methodologies, being comfortable in automating Unit and Integration Test cases for Data Pipelines
- Knowledge of Batch and Streaming processes
- Experience working in projects based on Agile methodology
- Good spoken English
- Hands-on experience with AWS Codebuild and Codepipeline
- NoSQL knowledge
- Experience in Finance (Investment Management sector)