Transport For NSW - Senior Data Engineer (AWS Big Data) (6 Months temporary Opportunity)
Data Scientist is needed in Sydney, Australia.
Client: Transport For NSW
Location: Sydney NSW, Australia
Contract: No Job Type Listed
Job Description
In this role, you will be responsible for working as part of a team delivering Data Architecture & Data Analytics. You will build data pipelines by integrating various data sources, maintaining, improving, cleaning, and manipulating data from divisional and other databases. Additionally, you will create and maintain optimal data pipelines from internal and external sources into a large cloud environment, implementing against reference designs to meet functional requirements.
Requirements
- Expertise in Cloud ETL development experience using various ETL tools and extensive knowledge in Data Warehousing principles.
- Design, develop, and implement ETL pipelines using cloud ETL tools (e.g. Matillion) to extract, transform, and load data into a cloud data warehouse (e.g. Snowflake).
- Design and maintain data models within Snowflake for query optimization and data analysis.
- Strong SQL skills to write efficient queries within Snowflake (SnowSQL).
- Develop and implement data quality checks and monitoring processes to ensure data accuracy and consistency.
- Utilize AWS S3 for data storage and retrieval in the data pipeline.
- Automate data pipeline workflows and scheduling using tools like cron or CloudWatch.
- Troubleshoot and resolve issues related to data pipelines, Snowflake, and AWS services.
- Collaborate with engineers, analysts, and stakeholders to understand data needs and deliver solutions.
- Document data pipelines and processes for clear communication and future reference.
- Experience deploying and maintaining data pipelines successfully in a production environment.
Other Information
- This is a full-time opportunity until Feb 2025 with potential for extensions.
- The role is based at Parramatta with flexible work practices, offering hybrid work options.
- Transport for NSW provides a safe, integrated, and efficient transport system, connecting people, communities, and industry daily.
- Support for people with disabilities is available throughout the recruitment process and at work.
- On Twine (contact details private)
No longer accepting applications
Don't miss out! Drop your email to get notified for Data Scientist roles
How It Works
Create your Profile for Free
Build your Twine portfolio to highlight your skills, and experience, while setting your desired pay rate.
Send Customized Pitches
Pitch for available jobs that match your skills, and access exclusive opportunities that are specifically tailored to your expertise.
Secure Payment Process
Clients make upfront Vault payments held by Twine, ensuring freelancer payment security, with 0% commission for freelancers.