From the left: Derek Szydlowksi, Lead Operations Developer; Andrea of Belvedere, VP of Engineering; Software Engineer Yosuke Tomita.
Gengo® is a translation platform providing all users with the ability to effortlessly understand and communicate information from around the world, unlimited by the boundaries of language.
In order to provide all customers with high-quality translations, Gengo accepts translation requests online or via an API, which it then opens up only to translators who have passed its strict translation tests, only 7% of whom are able to manage the feat.
The company boasts over 13,000 translators from 114 countries, enabling it to provide translation services in over 37 languages.
Translations in each language are conducted based on fixed price plans, and customers utilizing Gengo’s API are able to enjoy even more reasonable prices.
Along with translations for individual customers, the company also provides translation services geared toward businesses, including translating user reviews for TripAdvisor and Rakuten, subtitles for YouTube videos, and more.
Gengo, Inc. runs its “Gengo” system on the Amazon Web Service (abbreviated as AWS below), and has utilized PostgreSQL from an early stage as its base system database supporting the service.
The company has recently chosen to adopt pgpool-II as a way to load share in order to handle the increase in users it has experienced, as well as to realize the creation of a redundant structure capable of improving reliability.
Apparently, we originally used MySQL, but by the time we joined, the company had already switched over to PostgreSQL.
We heard that the reason for this switch was that PostgreSQL had improved to become much faster, and because it allowed the use of features such as schema partitioning.
We also think that the decision to go with PostgreSQL was a smart one.
Gengo processes approximately 30,000 transactions per day, including inserts and updates.
While we don’t have specific peak times, we do experience increased system load when receiving large orders from major clients through our API. In addition, as we have many translators located in the U.S. who check translation requests (jobs) when they wake up in the morning, we tend to experience increased load during this time as well.
We have implemented a variety of measures in order to handle increased transaction volumes since we first launched the service.
Capitalizing on the advantages of using the cloud-based AWS, we set up a database server exclusively for referencing, tried switching to servers with better performance, changing to an architecture that temporarily stores translation requests in a specialized queue, and tried a variety of other techniques to improve our system as well.
However, a time came when we suddenly experienced major trouble.
Our master database’s virtual sever image (AMI) collapsed for some reason, after which it would freeze once a certain period time had passed after restarting.
This resulted in the service being unavailable temporarily.
While this situation itself was quickly remedied, the risks of having a SPOF (Single Point of Failure) became quite clear.
Therefore, in order to eliminate this risk, we decided to adopt pgpool-II.
System structure before project began.
Our in-house engineers had handled all related matters until this point, but we felt that this project would take a fair amount of man-hours to complete.
There were also certain things that we didn’t understand about pgpool-II, and considering both the research and actual implementation, we realized it would take a great deal of time.
Since we’re also tasked with handling day-to-day operations, taking into account how such a project would affect our daily work, we decided to request support in adopting pgpool-II from specialists in PostgreSQL and pgpool-II.
While we did consider a few other companies besides SRA OSS, we decided to go with them as they had pgpool-II committers, and because they made us a proposal that we felt demonstrated proper technical understanding at the business level. We found their ability to quickly and accurately respond to our various questions at the business level particularly helpful.
Perhaps because our test period was relatively short, we did experience some unexpected trouble a few days before the release, but SRA OSS worked with us on-site to handle issues as they came up, and we were able to launch successfully.
We believe that this project proceeded very smoothly, considering the fact that it was extremely complex and difficult.
In terms of the materials they provided us with, they went beyond the simple creation of documents, preparing scripts written in consideration of actual operation and more.
While manual labor is prone to mistakes, by utilizing these scripts, we were able to proceed with tasks in an extremely smooth fashion.
As a result, everything was completed successfully, from running tests on a staging environment to crucial elements in implementing on the live environment.
While we view these components as firmly in the realm of SRA OSS’s know-how, we are very grateful for the fact that they were willing to disclose so much detailed information to us.
Adopting pgpool-II to eliminate SPOF
After hiring SRA OSS for this project, they’ve been kind enough to invite us to a variety of different PostgreSQL-related events.
Participating in these events, we were extremely surprised to realize just how actively the PostgreSQL community is working to add new features and more.
It’s extremely interesting to see just how fast PostgreSQL is evolving.
The new JSONB data type, which became available with the latest version of PostgreSQL, 9.4, has gained our attention as a data type able to store JSON-type data in binary format.
The Gengo system uses the JSON-format for data exchanges, so we felt that being able to handle the database using the same format would lead to an even more efficient system.
While we had considered this in the past as well, we decided not to adopt then as we were worried how a NoSQL database would handle transactions and other processes.
If the JSON-format data can be handled efficiently using PostgreSQL, then we think that provides us with a very attractive option.
We are considering automating operations in the future.
We want to automate the addition of servers and other such tasks required when load increases.
We also think that the number of transactions and volume of data will continue to increase.
Even if we use a number of high-performance machines, there will always be a limit somewhere.
This is why we are now considering what to do in order to scale-up when we reach these limits.
Recently, more and more database functionality is being provided in cloud-based services.
For instance, AWS has RDS for PostgreSQL.
Such services let users utilize databases very easily, letting us focus on application development.
However, we do also feel that losing complete touch with how database structures work is problematic from an engineering standpoint, and see balance as crucial.
While SRA OSS greatly assisted us on this project, they also allowed us to gain knowledge on how to create highly reliable systems by allowing our engineers to participate in the project as well.
We are confident that this know-how and experience will prove highly beneficial in improving our service quality going forward.
