Software Engineer, Data Platform

Tokyo
Partial Remote
Full-time
May 13, 2022

The data platform software engineer in SmartNews plays a key role in accelerating products/business development. Great efforts are made on building a highly efficient and flexible data service for analytical and operational purposes.

To serve the internal users such as our analytics and product-dev teams, the mission of data engineers is to create high-level, easy-to-use data services for simplifying the accessing, integration, and consolidation of various data sets, and also building the platforms for executing tasks processing massive data in terms of TB per day.

Technology drives the growth of SmartNews, and thus we eagerly adopt cutting-edge technologies from the industry and academia, especially the open-source community.

Responsibilities

Design, develop, set up, and maintain new services, libraries, tools, frameworks for data processing or management, and investigate new algorithms to increase efficiency for Data Processing, such as ETL, Data Pipelines, OLAP DBMS, real-time messages and streams processing, data-sync between systems, etc.
Develop and operate in-housing data service offerings such as high performant K/V store, adopt/enhance open-source frameworks to meet SmartNews’ growing business needs
Develop tooling for system performance evaluation, monitoring, and tuning of the data processing procedures or platforms, get insights of efficiency and stabilizability and make continuous improvements, such as optimizing distributed query engines, computing resource management and isolation, multi-tier storage systems, etc.
Own and maintain the key data processing portfolios such as building and taking care of the environment, troubleshooting, and being responsible for incidents that occur during on-call periods. Work closely with data architecting/modeling roles to understand ways to implement the data service, and interact with the Site Reliability Engineering (SRE) team to deploy the environments and drive production excellence.
Devise system, tooling, and approaches for data privacy and security. Establish access control, create processes to handle sensitive data.
Diagnose and resolve complex technical challenges for data accessing or processing. Using elegant and systematic rather than ad-hoc methods to help other teams tune the performance and improve stability.

Requirements

Minimum Requirements

BS/MS degree in computer science or equivalent science/engineering degree with 5 or more years of experience
Strong Programming skills and experiences with a deep understanding of data structures and algorithms are required for building efficient and stable solutions
- Rich experiences with one or more programming languages such as Java, Scala, C++, or Python; familiar with agile development and manage testing skills
- Need certain knowledge on shell scripts and operating systems, especially on Linux
Good understanding of modern bigdata technologies and ecosystems
- Familiar with Hadoop, Spark, Hive, Presto, Redis, and Flink, be able to develop data processing programs with them in batch or streaming manner
- Familiar with modern data stores either RDBMS or NoSQL stores (such as MySQL, Cassandra or Druid, etc); have experience with developing application or function-extensions on such data stores
- Be able to implement and tune complicated heavy-lifting data flows (ETLs or pipelines), familiar with certain toolings
Capability of system design with good modularity and extensibility
- Familiar with system/module design methods and toolings such as UML
- Be able to draft the user-understandable blueprint and precise, detailed designs
Experience with building highly scalable distributed systems
- Able to design and implement distributed services with scalability and performance in mind
- Able to debug and troubleshoot performance and reliability problems

Preferred Experience

Experience with cloud-based architecture (e.g. Amazon Web Services)
Strong interest in high performance, high availability, high volume data processing
Good experience in data management, integration, security, and auditing

Benefits

Perks and Benefits at the SmartNews Tokyo Office

Support program for employees’ self-development and well-being
Healthy lunches are prepared on-site daily in our SmartKitchen, located in our West Office building. You can even see the cooks at work!
Need a pick-me-up? Stop by ChikyuCoffee, our in-house coffee shop located in both the East and West Office buildings, and get a cup of gourmet coffee, espresso-based beverage, or other seasonal drink!
SmartNews provides all healthcare and social insurance benefits required by Japanese labor law

Visa Sponsorship and Relocation Support

SmartNews can sponsor visas for candidates who meet eligibility requirements.

To support a smooth transition for candidates who are moving to Japan from overseas, SmartNews partners with a travel service provider.

APPLY NOW ➜

About SmartNews

Recently crowned Japan's latest tech unicorn, this company runs a wildly successful news service that’s powered by machine learning. It's an engineering-focused company (One of co-founders is an engineer) and they have offices in San Francisco, Palo Alto, Shanghai, Beijing that are apparently doing quite well too.

The mission of SmartNews is to discover and deliver quality stories to the world. Since its inception in 2012, the SmartNews app has been downloaded over 50 million downloads and maintained a monthly active user count of over 20 million.

In addition to being one of the most popular smartphone apps in Japan, SmartNews has worked hard on building their business in the US. They now have partnerships with over 300 major news publications in the US like CBS, Entertainment Weekly, Business Insider, National Geographic and more. They’ve had a great deal of success due to their ML-based sorting and recommendation systems.

SmartNews has a highly diverse development team with many world-class engineers and others working from their offices in Japan,China and the US. They maintain a very strict hiring bar for technical hires and an engineering-driven culture that makes them one of the strongest tech companies in Japan when it comes to technical skill level.

SmartNews is looking to hire the best people regardless of background, so they’re willing to hire people that don’t speak Japanese and (under normal circumstances) bring them to Japan from overseas if necessary. Everyone works together as one team regardless of location, and SmartNews provides everything their employees need to do their best work and maintains a Silicon Valley-inspired culture for engineers in Tokyo.

SmartNews's policies during COVID-19 are as follows:

All interviews are being conducted remotely
Employees are working remotely following the government’s instructions
They've given everyone some monetary incentive to support working from home

Japanese Language Policy
We do not require Japanese language ability in the working environment. We have a language support program if employees want to learn Japanese.
Visa Sponsorship
We support candidates to get a working visa from the start to the end (for JP office joiners).
Paid Leave
Our holidays shall include Saturdays, Sundays, national holidays, and other days designated by the Company.

Annual paid holidays
- 4-10 days of paid holiday will be granted depending on the month in which you join the Company.
- From the second year and onwards, paid holidays will be granted according to the number of years of your employment on April 1st of each year. The number of days granted shall be set forth in the Work Rules.
Other leaves
- Summer holidays
- Year-end and year-beginning holidays
- Special leave for congratulation or condolence (marriage or immediate family death)
- Sick/Injured Child Care Leave
- Family nursing care leave
- Maternity and paternity leave
Benefits
☕️ ChikyuCoffee - Delicious coffee provided by our Barista every day

🍹 Free soft drinks

🌽 SmartKitchen - Healthy organic lunch on a daily basis for free at our cafeteria

👩‍🎓 Foreign language development support

💜 Various social insurance benefits included

🏥 Health insurance

📓 Free book rent from our library collections

🌎 Translation/interpretation support (EN<>JP)
Remote Work
We accept work from home policy during natural disasters and emergency situations. Our company’s working policy during the pandemic follows the government’s recommendations.
Work style
Our engineering and product teams follow the discretionary labor system to better collaborate with members in the US and China.
Diversity
We support diversity in various ways and those are not simply reflected in the quantitative goals.

We have employees with 18 different nationalities. Groups of small gatherings with similar interests are actively communicating across divisions, departments, and regions. We have monthly Global All Hands to share important notes from each region and be aligned globally.

All employees in SmartNews have different personal and professional backgrounds and experiences.
Parents and Family
We are supporting our employees in various ways to help them manage their work better with family care.

We have 10 annual paid leaves (for a child) for employees with children until they enter highschool.

Working from home is encouraged when their family needs care. Employees can freely invite their family to the Smart Kitchen (our corporate kitchen where free lunches are served).

During the working from home period because of the pandemic, employees with small children autonomously held some online sessions for their children such as reading fairy tales and mutual online babysittings.