Software Engineer (data pipeline in charge)

What you work

You will lead the construction and operation of a data collection/analysis infrastructure for accounting and management data submitted by customers (mainly small and medium-sized enterprises and startups).

Specifically, you will be responsible for designing and developing data pipelines using Snowflake/dbt, as well as building operational flows. While we already have a data pipeline built in-house, migration is necessary to align with the current business situation.

We would like you to not only handle practical tasks but also proceed with the execution of business while discussing specific data processing challenges unique to our business domain with the CTO and data team leader.

• Smooth handling of differences in account items and data format
• Improvement of data processing real-time capability and stream processing 

How you work 

After joining Yoii, we will initially ask you to work on the data infrastructure, but depending on the situation of the product development, we also anticipate you working on Yoii Fuel and possibly future product development for new businesses.

The current development organization is structured as follows:

• Engineering Team: 1 Manager, 3 Engineers, 1 contractor
• Data Science Team: 1 Manager, 2 Data Scientists, 1 Data Analyst, 2 contractors

Both teams are composed of multinational members, so communication in the business is primarily in English (Meetings, Notion, Slack, etc.). When discussing requirements and specifications, there will also be exchanges in Japanese with members of the business side (mainly Risk Management member and CS member).

Since we are still an early-stage startup with a small number of members, you will have the opportunity to work on a variety of challenges. 

Requirements

Must have

• Experience in the development and operation of data pipelines
  • The length of practical experience is not specified, but the ability to learn and acquire necessary skills ad knowledges through study and research as needed
• Over 5 years of practical experience in system development involving coding
  • e.g., mobile apps, web apps, games, etc.
• Practical experience using SQL or equivalent knowledge
• Ability to code in basic Python

- Bachelor's degree in computer science or equivalent knowledge and experience

- English proficiency to communicate smoothly with both technical/non technical members

Nice to have

• Experience leading the design of data pipeline construction and improvement
• Proficiency in coding in statically typed languages 　
  • e.g., TypeScript, Go, Scala, Kotlin, Rust, etc. 
• Practical experience coding Python
• Practical experience in cloud environments such as AWS or GCP
• Development experience in distributed data processing using Apache Hadoop or Spark
• Dedication to competitive programming
• Practical experience in processing accounting data
• Japanese Proficiency