Senior Data Engineer

Description

Although the domestic manufacturing industry is estimated to be worth 180 trillion yen, many manufacturers and their suppliers are unable to fully demonstrate their development potential and technological capabilities because they are hindered by quotations and administrative tasks, as well as a lack of sales force and information.

CADDi’s mission is to remove those restrictions in various aspects of the industry and unleash each company’s potential to the fullest. We introduce technology to drive digital transformation in the industry so that all manufactures - from small factories and established leading manufactures to new start-ups - can thrive and add new value to the world. Will you bring your skill to create such a future with us?

Unleash the potential of manufacturing with data

At CADDi, we have a massive amount of technical drawings, corresponding quotations, along with data describing the required manufacturing processes. By processing these data into a form that is easier to utilize and quickly implementing the cycle of hypothesis testing, we aim to find answers to the problems faced by the manufacturing industry.

Besides what we already have, there is a lot more data out there that hasn’t been fully utilized. Assessing valuable data and preparing it for utilization is the key to taking the burden off this industry.

Responsibilities

• Design and implementation of data infrastructure
  • Manufacturing cost estimation data
  • Transaction performance data
  • Manufacturing control process data
  • Usage data for various products
• Analysis of base data to obtain hypotheses concerning business and operational improvement
• Build data processing pipelines in cooperation with algorithm designers

Development Environment

• Development language
  • Frontend: HTML, CSS, TypeScript
  • Backend: TypeScript, Rust, Kotlin
• Frameworks
  • Frontend: React, Apollo, Next.js, styled-components, WebGL, WebAssembly
  • Backend: Rust (tokio, tonic, diesel), Kotlin (Micronaut, Exposed), Node.js (express, NestJS)
• Infrastructure: GCP, GKE
• Database: PostgreSQL, Firestore
• API: GraphQL, REST, gRPC
• Authentication: Auth0
• Data Pipeline: GCP Vertex AI Pipeline, Dataflow
• Development tools: GitHub, CircleCI, Figma, Sentry, DataDog, Storybook
• Communication tools: Slack, Discord, JIRA, Miro

Qualifications

• An understanding our mission to unleash the potential of manufacturing
• 3+ years of hands-on experience with server-side technologies
• 3+ years of experience with data pipelines or data processing frameworks
• Experience using SQL for analysis
• General knowledge of Linux-based infrastructure and public cloud architecture
• Familiarity with container technologies such as Docker
• Experience operating container-related technologies such as Kubernetes, Tekton, Argo, etc.
• Experience with data pipeline technologies such as Airflow, Apache Beam, Spark, etc.
• Hands-on experience with declarative infrastructure-as-code technologies (terraform, puppet, ansible, etc)
• Experience using version control systems such as Git
• Proficiency in English

Preferred Qualifications

• General knowledge of computer networking
• Experience with cloud providers such as AWS, GCP, or Azure
• Experience monitoring data pipelines with modern solutions such as Datadog, New Relic, Grafana, etc
• Experience with production use cases of Elasticsearch
• Experience with kNN (ANN) frameworks (faiss, nmslib, etc) and associated database technologies (Milvus, Vald, etc)
• Experience with machine learning frameworks (Chainer, PyTorch, Tensorflow, etc)
• Experience in application development or analysis work with Rust, Python, MATLAB, or R

For overseas candidates

• Full Time Position In Japan Base
• Relocation support is available
• Visa support is available
• Japanese language educational support is available
• Remote working in Japan working hours (while waiting for Visa)
• We can guide you to the next step quickly if you could mention the following point in your application email.
  • Willing to relocate to Japan once visa is granted
  • Able to work full time JST 9:00-18:00～JST11:00-20:00 while waiting for visa