Senior Data Engineer (Digital Bank)

Overview

Under the mission of "Money Forward. Move your life forward," Money Forward aims to resolve the financial concerns and anxieties of individuals and businesses through the power of technology.
We have partnered with Sumitomo Mitsui Financial Group, Inc. and Sumitomo Mitsui Banking Corporation to establish a new company in preparation for the launch of a new digital bank.
We are currently seeking candidates for the position of Senior Data Engineer as part of this initiative.
*Based on the press release announced on April 16, 2025.

*This position involves employment with Money Forward, Inc., and a secondment to the new company (SMBC Money Forward Bank Preparatory Corporation). The evaluation system and employee benefits will follow the policies of Money Forward, Inc.

Responsibilities and Duties

• Design and implement data pipelines to ingest data from multiple source systems using Databricks native tools, as well as REST APIs
• Build and maintain Bronze/Silver/Gold layer transformations on Databricks ensuring data quality, consistency, and performance.
• Implement data quality checks and cross-system reconciliation logic.
• Develop and optimize SQL queries and transformations using dbt or similar tools.
• Design and implement data models for analytics and reporting use cases (ALM, ERM, regulatory reporting).
• Build REST APIs or data serving layers for downstream consumers.
• Participate in architecture decisions for data platform components.
• Write unit tests, integration tests, and data quality tests for pipelines.
• Monitor data pipeline performance, troubleshoot failures, and implement improvements.
• Optimize query performance through partitioning strategies, Z-ordering, and query tuning.
• Implement infrastructure as code for data platform components using Terraform.
• Set up CI/CD pipelines for automated testing and deployment of data pipelines.
• Mentor mid-level engineers and conduct code reviews.
• Contribute to documentation and best practices for the team.
• Collaborate with backend engineers to define API contracts and data schemas.
• Work with Technical Lead on platform design and technology selection decisions.
• Lead features and initiatives within the data platform.

Required Skills and Experience

• 5+ years of experience in data engineering with data focus or analytics engineering.
• Strong proficiency in SQL and Python.
• Hands-on experience building data pipelines using modern tools (Databricks, Spark, dbt, or similar).
• Experience with databricks development and with AWS cloud environments
• Strong understanding of data modeling techniques including dimensional modeling, data vault, or event-driven architectures.
• Experience with data quality validation and testing frameworks.
• Proven ability to debug and optimize slow queries and data processing jobs.
• Experience with version control (Git) and CI/CD pipelines.
• Understanding of data governance concepts: access control, audit logging, data lineage.
• Strong problem-solving skills and ability to work independently.
• Experience mentoring junior or mid-level engineers.
• Excellent communication skills for collaborating with cross-functional teams.
• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

Preferred Skills and Experience

While not specifically required, tell us if you have any of the following.

• Experience in financial services, fintech, or other regulated industries.
• Knowledge of banking domain concepts: core banking systems, payment processing, regulatory reporting, AML/transaction monitoring.
• Experience implementing data platforms that comply with regulatory requirements (FISC Security Guidelines, FSA/BOJ reporting, GDPR, APPI).
• Experience implementing cross-system reconciliation for financial data.
• Experience with performance tuning: partitioning strategies, query optimization, cost management.
• Experience building REST APIs with Python (FastAPI, Flask, or similar) for data serving.
• Knowledge of streaming data pipelines (Kafka, Kinesis, or similar).
• Experience with Terraform.
• Contributions to open-source data engineering projects.
• Experience with BI tools (QuickSight, Tableau, Looker, PowerBI).
• Experience leading technical initiatives from design through implementation.
• Track record of improving data platform performance or reducing costs (provide specific metrics).
• Experience in AI development and/or experience in using AI tools to improve development processes.
• Money Forward recently announced our AI Strategy roadmap which focuses on improving AI-driven operational efficiencies, as well as integrating AI agents into our products to deliver better value to our users. (More information here)

Language Requirements

• Japanese: Business Level (Fluent, capable of handling communication with clients in Japanese)
• English: TOEIC score of 700 or above
  
  (Note: If you have other qualifications or experiences demonstrating English proficiency, such as EIKEN Pre-1, EIKEN 2nd Grade (CSE score 1950+), TOEFL iBT 60+, IELTS 5.0+, or Cambridge FCE.), feel free to discuss with us)
  
  For those without a TOEIC 700+ equivalent score, they will be asked to take a designated test during the interview process (generally after the first interview).

Technology Stack

• Cloud Infrastructure:
  • AWS (primary cloud platform in Tokyo region)
  • S3 for data lake storage with VPC networking for secure connectivity
  • AWS IAM for security and access management
    
    
• Data Lakehouse Architecture:
  • Modern lakehouse architecture using Delta Lake for ACID transactions, time-travel, and schema evolution
  • Columnar storage formats (Parquet) optimized for analytics
  • Bronze/Silver/Gold medallion architecture for progressive data refinement
  • Partition strategies and Z-ordering for query performance
  • Unity Catalog for centralized governance and metadata management
    
    
• Orchestration & Processing:
  • Databricks Workflows for managed workflow orchestration
  • Distributed data processing with Apache Spark on Databricks clusters
  • Serverless compute and auto-scaling clusters for cost optimization
  • Streaming and batch ingestion patterns with Databricks AutoLoader
    
    
• Data Transformation:
  • dbt (data build tool) for SQL-based analytics engineering
  • Delta Live Tables for declarative ETL pipelines with built-in data quality
  • SQL and Python for data transformations
  • Incremental materialization strategies for efficiency
    
    
• Query & Analytics:
  • Databricks SQL for high-performance analytics queries
  • Serverless and auto-scaling SQL warehouses for variable workloads
  • Auto-scaling compute for variable workloads
  • Query result caching and optimization
  • REST APIs for data serving to downstream consumers
    
    
• Data Quality & Governance:
  • Automated data quality with Delta Live Tables expectations and Great Expectations
  • Cross-system reconciliation and validation logic
  • Fine-grained access control with column/row-level security using Unity Catalog
  • Automated data lineage tracking for regulatory compliance
  • Audit logging and 10-year data retention policies
    
    
• Business Intelligence:
  • Amazon QuickSight and/or Databricks SQL Dashboards
  • Integration with enterprise BI tools (Tableau, PowerBI, Looker)

Tools Used

• Version Control: GitHub
• CI/CD: GitHub Actions
• Infrastructure as Code: Terraform
• Monitoring: Databricks monitoring, AWS CloudWatch integration
• AI-Assisted Development: Claude Code, GitHub Copilot, ChatGPT

Development Structure

We operate in a small, agile team while collaborating closely with partners from the banking industry. The MIDAS team is growing rapidly, aiming for more than 10 data engineers within this year.

Work Environment

At Money Forward, we provide an environment where we can create world-class services together, and we are looking forward to welcoming you.

• Provided PC Specs: We provide PCs equipped with the latest CPUs (MacOS or Windows). Custom-made PCs tailored to business requirements and replacements with the latest OS are also possible.
• Money Forward Library: We have a library system where you can freely borrow books, ranging from technical books to management books. Desired books can be purchased at the company's expense.
• Referral Driven: We cover the cost of recruitment meals. There is a referral reward system.
• Conference Participation Support: The company partially covers participation in domestic and international conferences, such as RubyKaigi and Google I/O.