Data Platform Engineer

Background

estie is building one of Japan’s largest office real estate data platforms and developing multiple products based on it. Because data itself is a powerful asset in our business, we place great importance on data engineering.

To enable multiple product teams to collect, process, and store data with high quality and speed—and to allow safe and efficient sharing of data across products—we are developing an internal data management infrastructure.

We are looking for data engineers who can leverage their expertise and experience in data engineering to build and continuously improve a highly advanced data foundation.

Responsibilities

You will be responsible for the development and operation of our data management platform as a specialist in the field of data engineering.

• Building and operating a data pipeline using a modern data stack, including Snowflake and dbt.
• Establishing guidelines and improving the developer experience for external teams, such as product development teams, working on the data management platform
• Designing and operating the security and governance of the data management platform.

Topics to tackle in the future

• Promoting the use of AI within the data management platform, including tools such as Snowflake Cortex AI
• Developing the data management platform to advance data democratization across the organization
• Introducing a data catalog to enable efficient access to the growing variety of data accompanying business expansion
• Building a Single Source of Truth data management platform that can be trusted and utilized company-wide
• Designing and constructing data structures that enable cross-organizational utilization of data from multiple products, along with implementing flexible access control and permission management

Technical Environment

• DWH: Snowflake
• ETL/ELT: Fivetran, dbt
• IaC: Terraform
• BI: Tableau, Streamlit in Snowflake
• Code Management: GitHub
• CI/CD: GitHub Actions
• Task Management: JIRA
• Communication Tools: Slack
• Other: GitHub Copilot

Requirements

• Practical experience in system development using cloud services
• Practical experience in at least one of the following areas:
  
  • Building and operating data infrastructure using languages such as SQL or Python
  • Building and operating infrastructure for web systems or batch processing systems

Preferred Skills

• Practical experience in data manipulation using advanced SQL
• Practical experience in designing and operating security and governance for data or infrastructure platforms
• Knowledge of and hands-on experience with Infrastructure as Code (IaC) tools such as Terraform
• Knowledge of and development experience with AWS
• Knowledge of and development experience with Snowflake
• Knowledge of RDBMS, including practical experience in table design and database tuning
• Experience using and selecting BI tools such as Tableau or Looker
• Experience using and building data transformation tools such as dbt