Today’s fast-paced business environment makes CIOs and IT leaders constantly boost time to value and speed up software release cycles. As mentioned in Gartner’s press release, businesses are expected to deliver digital dividends as never before using every possible digital channel and asset. Outsourcing qualified engineers is just half the battle to optimize development costs. Finding teams that master AI-powered tools to increase their productivity is equally important. Akvelon’s engineers have recently spent over 8 000 hours investigating GitHub Copilot’s efficiency under real-life conditions and come to valuable data-driven conclusions.
The GitHub Copilot team has stated that their tool boosts productivity and reduces time for routine tasks. Akvelon couldn’t stay aside from exploring how true these statements are for engineers’ regular workflow. So we’ve launched a massive investigation project where 30 experienced developers have been using GitHub Copilot for their various day-to-day activities.
Further come the survey results from our GitHub Copilot application and quite unique insights on how the use of the tool affects the delivery time of real-life projects.
- What Can GitHub Copilot Do to Accelerate Project Development
- Unlocking GitHub Copilot’s Power: Whom It Benefits the Most
- How GitHub Copilot boosts productivity: Akvelon’s results vs GitHub stats
- What Are the Possible Concerns Associated with Using GitHub Copilot?
- Why Should CIOs Seek Teams That Embrace GitHub Copilot
What Can GitHub Copilot Do To Accelerate Project Development
GitHub Copilot, like a proper second pilot, picks up on the context of the code, and instantly suggests corresponding code lines and completes functions. It is trained to adapt to the libraries in use, giving developers an advantage in their day-to-day routine. At least these are the benefits stated by the GitHub Copilot creators.
GitHub Copilot survey participants
Programming languages used for testing GitHub Copilot
Unlocking GitHub Copilot’s Power: Whom It Benefits the Most
The technologies and specifics of the projects that Akvelon worked on at the time of the survey dictated the conditions for our experiment. Also, the seniority level, the skillset, and the goals of the participants varied, which influenced their success in using the tool.
Despite the fact we had some limitations, for example, in programming languages applied, we’ve challenged GitHub Copilot in a way. We tested the tool for front-end and back-end programming and even data processing.
Our observations revealed that GitHub Copilot brought the most value to those struggling with routine tasks and repetitive code patterns for both: front-end and back-end parts. Developers got proper code pieces suggested automatically based on what they've already written. GitHub Copilot also eased coding, completing methods and functions for developers. Finally, the process of creating unit tests got simplified to a noticeable extent.
When survey participants face more complex tasks like writing algorithms, it seems GitHub still lacks training for this kind of work. Sometimes the tool makes false suggestions which takes extra time for an engineer to validate the provided code.
GitHub Copilot may not fully meet all user expectations at present. However, the tool is continuously advancing and progressing as it faces more challenges, resulting in increased efficiency.
How GitHub Copilot Boosts Productivity: Akvelon’s Results vs GitHub Stats
GitHub Copilot promises to bring the spark to the development process that engineers miss due to routine boilerplate code writing. As mentioned earlier, the tool helps streamline typical code writing and increases productivity completing functions for the developers. Within our GitHub Copilot survey, we’ve been checking the tool for code writing quality and productivity boost.
Let’s first see what survey results say about the relevancy of GitHub Copilot’s suggestions.
According to most survey participants, the suggestions provided by GitHub Copilot were quite accurate. Developers were able to incorporate these suggestions into their code right away. However, for more unique or specialized tasks, additional manual validation was needed to ensure accuracy.
Next, we received feedback from the survey respondents regarding the increase in coding speed. This aspect has the potential to impact the overall productivity of developers.
Most respondents reported a coding speed improvement of up to 25%, while nearly 14% experienced an even greater increase of up to 50%.
The majority of the survey participants (81%) have witnessed that their productivity grew while they were using GitHub Copilot for coding and testing. The average time reduction rate for completing tasks with GitHub Copilot was 16% according to our survey results. Yet some developers have experienced a boost of up to 50%.
Survey Update: More GitHub Copilot Efficiency Zones Unlocked
After conducting our initial GitHub Copilot efficiency survey, we further investigated how this tool can enhance productivity across different roles within software development teams. The findings demonstrated GitHub Copilot's effectiveness beyond just code writing and unit test creation, enabling us to expand the survey with more insights.
The tool significantly speeds up code refactoring and maintenance, aids in script development and automation within DevOps workflows, facilitates project documentation creation, and assists developers in getting up to speed with new libraries and languages during onboarding.
What Are the Possible Concerns Associated with Using GitHub Copilot?
GitHub Copilot is still quite fresh on the market, and its ability to provide value hugely varies on the engineers’ experience, language in use, and project specifics. The tool significantly reduces the need for manual effort for typical programmer tasks, streamlining coding and bug fixing. But there are particular cases when GitHub Copilot can rather interfere with the job to be done smoothly than help.
Let’s take a closer look at the key concerns associated with using GitHub Copilot and how we address them at Akvelon.
GitHub Copilot may give false code suggestions that mislead developers
Let's be real, among early GitHub Copilot adopters, there have been instances where the suggested code options may not align perfectly with the purpose of the solution. For example, GitHub Copilot may not efficiently assist in utilizing newly-arrived libraries and frameworks. The tool was trained on a predefined set of source code patterns, and it requires further training to pick up on the latest libraries and stay up to date.
Based on the findings from the Akvelon survey, one-third of participants reported that GitHub Copilot's suggestions had up to 80% of accuracy. But in case developers know exactly what solution they’re going for, even 50% of accuracy from the tool may be helpful. Then developers browse through all given recommendations and pick up the optimal one. According to our survey, engineers with more than a year of experience can identify issues or errors in the code, choosing the best way to go for their domain and context.
The key message here is not to treat GitHub Copilot as a miracle worker who delivers perfectly sculptured end results, but rather pick the most suitable perks out of many options. It’s still on developers to leverage their expertise and judgment to avoid misleading suggestions and detect truly valuable input.
The same principle applies when using GitHub Copilot to enhance code quality. According to our survey, a majority of participants noticed a slightly positive impact on their code.
Approximately 7% noted a slightly negative effect which might be related to the fact that they haven’t had enough time to master the tool. It’s important to understand that the negative figure can diminish as engineers gain more practice with GitHub Copilot and their overall professional experience grows.
Security vulnerabilities in the code suggestions
Just like GitHub Copilot users may face trouble with non-optimal code suggestions, they may run upon code that contains security vulnerabilities. As previously mentioned, it is important to review the suggested code before using it, the same applies to security matters. We insist that only experienced programmers capable of addressing vulnerability concerns should rely on GitHub Copilot code suggestions. However, it's worth noting that security issues are not exclusive to AI-generated code.
Additionally, GitHub Copilot continually updates its algorithms with the recent launch of an AI-driven vulnerability prevention system that actively detects and blocks insecure coding patterns in real time. This ongoing effort helps to mitigate security risks associated with the suggestions provided by GitHub Copilot.
GitHub Copilot raising potential legal issues for business
GitHub Copilot was trained on open-source repositories. Many users felt concerned that the tool would provide code suggestions that violate copyrights putting companies at risk if they integrate someone’s code into their production software. However, GitHub Copilot does not extract code from repositories to present it as its own suggestions.
Furthermore, GitHub Copilot users can disable suggestions that match the code found in public repositories, mitigating any copyright-related risks.
Just a reminder, you shouldn’t expect from GitHub Copilot to generate complex algorithms independently. It rather helps to complete common functions and methods, reducing repetitive tasks.
Why Should CIOs Seek Teams That Embrace GitHub Copilot
Leveraging cutting-edge technologies and approaches is crucial for businesses to stay competitive. Teams with a certain level of expertise with AI tools already explore GitHub Copilot efficiency and reach higher productivity while delivering better customer experiences as a result. At this point, even a 10% increase in productivity can impact the speed of deployment in the long run, which only deepens our passion for cutting-edge tools!
By partnering with Akvelon, you’ll find a pioneering services provider who already knows how to use benefits from AI safely in your projects. We’ll help you deliver features faster using our expertise with GitHub Copilot.
This article was written by:
Ilya Polishchuk
Director of Engineering
