Optimize, operate at scale and enhance cloud-based data platform on Microsoft Azure (incl DataBricks)
Involve in cloud data platform enhancement project lifecycle with external vendor including design, development, testing, deployment, and documentation.
Work with Data Analyst to perform data preparation, cleansing, build optimized data models and visualization in Power BI
Perform proof of concept for cloud data products
Validate data workflows and integration points through rigorous testing (e.g., pipeline validation, schema checks, and performance benchmarking) to ensure solutions meet business and technical requirements.
Create operational playbooks and automation scripts to streamline deployment, monitoring, and troubleshooting, enabling efficient handover and long-term maintainability of data solutions.
Document data pipeline architecture, deployment processes, and operational runbooks to support effective troubleshooting and post-go-live maintenance by internal teams
Requirements:
Degree in Computer Science, Information Technology, Data Science or related field
At least 3 years of professional experience related to data platform
Experience working with large-scale data systems and cloud platforms in Databricks
Understanding of data lifecycle and governance principles (e.g., data quality, lineage, security, and compliance).
Experience in Python, SQL, Spark, Azure Data Factory, Event Hub / Kafka / Event Messaging, Azure Data Lake Storage Gen2, Databricks, Unity Catalog, Power BI
Experience in software development lifecycle (SDLC)
Strong sense of responsibility, self-motivated, committed, strong analytical and problem-solving skills, good interpersonal & communication skills, team player
Good command in spoken and written Chinese and English