We are currently working with a QLD State Government Department to assist in the recruitment of a hands-on Data Engineer with strong experience in Microsoft Fabric (or related tool set) and Azure to design, build, and operate secure, reliable data pipelines and products within the emerging Enterprise Data Platform (EDP).
Your Role
...
The Data Engineer will join the Performance Monitoring and Reporting Unit to support the delivery of a high-profile program. The role partners closely with data governance (including Microsoft Purview) to deliver privacy-by-design ingestion, high-quality curated datasets, and performant semantic models for analytics and reporting. You will build ingestion and transformation across Bronze/Silver/Gold layers, run and interpret Purview scans, apply classifications, and automate deployment and monitoring so our data products are repeatable, auditable, and cost-effective.
Key Responsibilities:
- Build and operate pipelines using Microsoft Fabric (Pipelines, Dataflows Gen2, Notebooks/Spark, Lakehouse/OneLake, SQL Endpoints) and relevant Azure services where required
- Implement medallion architecture (Bronze/Silver/Gold), dimensional/semantic models for Power BI, and reusable data products aligned to enterprise standards
- Execute and schedule metadata-only discovery in Microsoft Purview, apply classifications/sensitivity labels, register assets, and ensure lineage from source to insight
- Implement data quality checks, validation rules, and automated tests; maintain technical docs, runbooks, and data dictionaries
- Apply least-privilege access, managed identities, and masking/obfuscation where appropriate, ensuring separation of copy (ingestion) from use (purpose-based)
- Use Git-based workflows and Azure DevOps (or equivalent) to version, test, and deploy pipelines, notebooks, and models, enabling parameterisation and environment promotion
- Configure alerting, logging, and Operations dashboards; triage incidents; tune performance and storage; and optimise compute schedules
- Work with data stewards, custodians, analysts, and product owners to deliver prioritised use cases
About You
We are looking for an analytical, detail-oriented professional who is comfortable operating in a fast-moving program context.
Essential Skills and Experience:
- 5+ years in data engineering (or equivalent), delivering production-grade pipelines and curated datasets in enterprise environments
- Demonstrated experience with Microsoft Fabric (Pipelines/Dataflows Gen2/Notebooks, Lakehouse/OneLake, SQL Endpoints) and strong SQL and PySpark skills
- Practical experience with Azure data services (e.g., Azure Data Factory/Synapse/Databricks—exposure acceptable if primary is Fabric)
- Hands-on experience with Microsoft Purview: configuring/scheduling scans, managing collections, glossary/lineage, and applying classifications/sensitivity labels
- Familiarity with medallion architecture (Bronze/Silver/Gold), dimensional modelling, and serving to Power BI
- CI/CD experience with Azure DevOps (pipelines, environments, approvals); Infrastructure-as-Code exposure (Bicep/Terraform) is desirable
- Strong grasp of security and privacy fundamentals (RBAC, managed identity, data masking, audit logging) and data quality practices
- Experience working with governance teams, data stewards, and custodians, with clear written documentation and stakeholder communication skills
How to Apply: Click the Apply button or email to diane.sulayao@randstaddigital.com.au
At Randstad, we are passionate about providing equal employment opportunities and embracing diversity to the benefit of all. We actively encourage applications from any background.
