Lead Senior Data Engineer
5 Penn Plaza New York, NY 10001 US
Job Description
- Build and optimize data pipelines using the latest coding practices and industry standards, modern design patterns, and architectural principles.
- Conduct complex data analysis, report results, prepare data for prescriptive and predictive modeling, and integrate raw information from various sources.
- Develop and maintain process and data documentation.
- Monitor and manage data pipeline processes.
- Write unit, integration, and functional automation tests; research and resolve issues identified by quality assurance or product support, and develop solutions.
- Apply software development principles to data engineering challenges, including testing, documentation, modularity, DRY (Don't Repeat Yourself) principles, and CI/CD workflows.
Knowledge, Skills, and Abilities:
- Proficiency in programming languages such as Python, SQL, Java, and Shell scripting.
- Strong understanding of cloud computing platforms (e.g., AWS).
- Experience with data visualization tools (Looker, Mode, Sigma, ThoughtSpot, etc.) is a plus.
- In-depth knowledge of database security, backup, and recovery procedures.
- Excellent communication and collaboration skills.
- Exceptional problem-solving and troubleshooting abilities.
Education, Training, and Experience:
- Bachelor’s degree in computer science, information technology, or a related field (or equivalent experience).
- Minimum of 7 years of experience as a Data Engineer.
- Expert-level experience with Snowflake, dbt Cloud, SQL, Python, and Shell scripting.