Software Engineer, Data Platform

About SmartNews

News should be for everyone. But not when it’s buried under cat videos—or when your internet is spotty—or when it’s in a filter bubble. That’s why we created SmartNews.

Our mission is to discover and deliver quality stories to the world. To accomplish this:

We are a machine learning company. News should be impartial, trending and trustworthy. Our algorithms evaluate millions of articles, social signals and human interactions to deliver the top 0.01% of stories that matter most, right now.

We love content. News should be sustainable. We support quality journalism and respect publishers by sharing article traffic and revenue.

Please note that we will only consider you for one application at a time.

Position

The data platform software engineer in SmartNews plays a key role in accelerating products/business development. Great efforts are made on building a highly efficient and flexible data service for analytical and operational purposes.

To serve the internal users such as our analytics and product-dev teams, the mission of data engineers is to create high-level, easy-to-use data services for simplifying the accessing, integration, and consolidation of various data sets, and also building the platforms for executing tasks processing massive data in terms of TB per day.

Technology drives the growth of SmartNews, and thus we eagerly adopt cutting-edge technologies from the industry and academia, especially the open-source community.

Responsibilities

• Develop and realize the overall data strategy for the company
• Develop and manage ETL/ELT pipelines and infrastructure of analytics databases
• Cooperate with the product team, data science team to design data features to enable product improvement and timely business KPI measurements 
• Determine the importance of each dataset and take appropriate measures to ensure data reliability and quality according to priority
• Improve the reliability and speed of queries for ever-increasing volumes of data
• Implement new types of data workflows
• Build dashboards to visualize key metric

Your qualification

• BS/MS degree in computer science or equivalent science/engineering degree
• Strong programming skills and experiences with a deep understanding of data structures and algorithms are required for building efficient and stable solutions
  • 3 years of rich experiences with Java or Python; familiar with agile development and manage testing skills
  • Need certain knowledge on shell scripts and operating systems, especially on Linux
• Good understanding of modern bigdata technologies and ecosystems
  • Strong experience with modern data stores either RDBMS or NoSQL stores (such as MySQL, Cassandra or Druid, etc); have experience with developing application or function-extensions on such data stores
  • Be able to implement and tune complicated heavy-lifting data flows (ETLs or pipelines), familiar with certain toolings
• Capability of system design with good modularity and extensibility
  • Familiar with system/module design methods and toolings such as UML
  • Be able to draft the user-understandable blueprint and precise, detailed designs
• Experience with building highly scalable distributed systems
  • Able to design and implement distributed services with scalability and performance in mind
  • Able to debug and troubleshoot performance and reliability problems
• English as the working language

Will be a plus

• 5 years of programming experience
• Have architected/implemented a data warehouse
• Have set up/implemented a data processing tool
• Experience with AWS (EC2, Kinesis, EMR, S3, etc.)
• Experience in development related to Spark, Hadoop, Flink, Clickhouse, Presto/Trino
• Experience in dashboard development with modern tools, such as Tableau or Superset
• Experience with infrastructure or data workflows for machine learning
• Knowledge of real-time streaming data systems
• Strong interest in new big data tools and technologies

What we offer

• Support program for employees’ self-development and well-being
• Healthy lunches are prepared on-site daily in our SmartKitchen, located in our West Office building. You can even see the cooks at work!
• Need a pick-me-up? Stop by ChikyuCoffee, our in-house coffee shop located in both the East and West Office buildings, and get a cup of gourmet coffee, espresso-based beverage, or other seasonal drink!
• SmartNews provides all healthcare and social insurance benefits required by Japanese labor law

Visa Sponsorship and Relocation Support

SmartNews can sponsor visas for candidates who meet eligibility requirements.

To support a smooth transition for candidates who are moving to Japan from overseas, SmartNews partners with a travel service provider.