More Insights Read more →
HP wanted to establish a presence at conferences in the open source technology realm. They sought after a technical and creative team to help them showcase their work with Open Source technologies. Read more →
LinkedIn is the number one social networking site for companies and professionals alike. As of June 2013, LinkedIn reported more than 225 million users in more than 200 countries and territories. The site is available in 19 languages and has 21.4 million unique U.S. visitors and 47.6 million globally each month. Forbes has named LinkedIn one of the fastest growing tech companies as of June 2013.
With such a large online presence, it is a huge benefit to know how many of those visitors are on mobile devices, what type of device is being used, and how many users are connecting via the LinkedIn mobile app.
LinkedIn needed to create superior quality of its applications in very rapid release cycle.
Before the Akvelon development team started automating tests for the LinkedIn application, testing was done manually, consuming large amounts of time and producing more errors.
Akvelon provided the most competitive fixed bid proposal for this project and was able to meet all of the project’s deadlines.
We have never seen better test results.
Manual testing is, in many cases, inconsistent due to various testing environments, complicated test preconditions, and the human factor. Given the complexity of the LinkedIn application for Android and iOS (being that they themselves are under constant development) it became extremely difficult for LinkedIn to ensure that new functionality does not break existing logic and that every release consistently provides common experiences for all platforms and user locales.
Akvelon worked with LinkedIn on test process automation through development of a modular, extendable, and reliable test framework based on industry standards like Robotium (Android), UIAutomation (iOS), and Hudson (continuous integration server). Akvelon’s main tasks were to extend test framework and develop test modules used as a building blocks to compose complicated test suites.
Test Framework is capable of automating UI tests for any Android and iOS application on real devices and on emulators. Testing can be fully automated by running tests with any Continuous Integration server.
Various basic and complex user actions can be automated: screen taps, scrolls, swipes, hardware buttons taps, screen orientation changes, editing text fields, etc.
Akvelon applied its best practices for the solution design and implementation:
* In-depth analysis of the business need based on historical data and brainstorming sessions with the company management
* Road-map arrangement: goals, deliverables, milestones, specifications, estimations, and communications
* Software design and development
* System deployment: testing, and solution feedback providing improvements to resolve additional queries
As a result, LinkedIn has received two powerful automated testing systems to check the LinkedIn mobile client application for both of the most popular mobile platforms – Android and iOS.
* By automating the test process for the LinkedIn Mobile application, LinkedIn was able to reduce the amount of hours spent testing
* As new features were added, regression testing took place automatically with minimal set-up time
* The Client was also able to minimize human error and limit environmental (Dev environment) impacts to the testing
* 75% of Android and 95% of iOS Test Cases were automated
Porto Seguro, one of Brazil’s largest auto and homeowner insurance companies, posted a challenge on Kaggle.com to build a model that predicts the probability that a driver will initiate an auto insurance claim within the next year. While Porto Seguro has used machine learning for the past 20 years, they’re looking to Kaggle’s machine learning community to explore new, more powerful methods. A more accurate prediction will allow them to further tailor their prices, and hopefully make auto insurance coverage more accessible to more drivers.
A group of two Akvelon machine learning engineers and a data scientist enlisted on Kaggle.com decided to compete side-by-side with more than 5,000 teams for the top positions in the leaderboard. The competition submissions are evaluated using Normalized Gini Coefficient. The Gini Coefficient ranges from approximately 0 for random guessing, to approximately 0.5 for a perfect score.
The insurance group provided the same data set to all competition participants as a raw data set with 893,000 rows of 59 distinct data points, with little or no specification as to what the data actually represents. The Akvelon team went through several different iterations to identify the meaning of each of those data points, to cleanse the data and to reduce the number of variables to a manageable subset. If the data model has too many data points with little or no impact on the output (event of the driver filing insurance claim) the resulting model becomes too complex, and has an adverse impact on the performance to generate the proper output in a timely manner.
The raw data set with 893,000 rows of 59 distinct data points
The work involved in crowdsourcing competitions requires lots of: “try, fail, fail fast then make some changes and try again, repeat the entire process over and over” approaches. For example, the team experimented with several prediction models such as: Random forest, Logistic regression, Extra tree classifier, KNN, Naïve Bayes, Adaboost, Xgboost, Lightgmb, Catboost, Neural networks.
‘Competitions requires lots of: “try, fail, fail fast then make some changes and try again, repeat the entire process over and over” approaches’
One of the chosen approaches was the t-SNE (t-distributed stochastic neighbor embedding) which is a nonlinear dimensionality reduction technique that is particularly well-suited for embedding high-dimensional data into a space of two or three dimensions, which can then be visualized in a scatter plot. Specifically, it models each high-dimensional object by a two or three dimensional point in such a way that similar objects are modeled by nearby points and dissimilar objects are modeled by distant points. The image below shows the result of this analysis.
t-SNE result (yellow dots corresponds to 1, purple – 0)
The most promising alternatives were XGBoost and LightGBM which placed the team onto the top half of the competition leaderboard with a score of ~0.282. The top ranked team had a score of 0.290. Akvelon ‘s team score was 97.2% of the top ranked team. (The higher the score the better, with the maximum value of 0.5). The next step was to adjust the data model parameters such as (depth of the tree, number of estimators, size of the train and validation sets). Those experimentations raised the score to about ~0.284. With a model that was promising, it was the right time to go back and revisit the groomed data set and do a better approach in choosing which data point to include and which one to replace.
Further research and investigations helped the Akvelon team improve the models. Some of the improvements involved including a Neural Network which raised the score to ~0.285, placing the team in the top 2% of the entire competition.
After some additional tweaks, adjustments and learning from the past Kaggle winner teams approaches, the Akvelon team ranked in the top 1% with the score of 0.287 against the leader having 0.291 (98.6% from the top scoring team).
At the final stage of the competition, the organizers used 70% of the complete data set, unseen yet by any of the competitors. Each teams result against this new data set will determine the final score in the Leaderboard.
Porto Seguro Machine Learning Kaggle Competition Final Standings
The Akvelon Team placed 22nd out of more than 5,000 teams in the final standings after applying the complete data set. Finishing the competition in the top 0.5%.
Participating in the Porto Seguro Safe Driver prediction competition increased our expertise in the Machine Learning field. The trial and error portion of this project helped us refine data quickly, focusing on accuracy and details in every model. The data also provided a real world data set for a real world issue, so our models could be applied to different situations and everyday occurrences.
Finishing in the top 0.5% is a proud accomplishment when competing against more than 5,000 other teams.
If you would like to learn more about our Machine Learning capabilities and solutions, contact us today!
The Internet Age has brought many discussions around the role of human capital and technology. The real answer lies in a mix of innovations in tech and human creativity. Read more.
We’re proud to be collaborating with Microsoft to implement the newly released Sports Performance Platform to professional sports teams around the globe. Read more.
New technology was presented at Microsoft Data Insights Summit which will help in player injury prevention through through the use of data and visualization. Read more.
Sorry, the comment form is closed at this time.
We make big software happen. Contact Us