AI can benefit all stages of product development, from concept creation to design, prototyping, testing, and final production. It can help with market research and analysis, improve the user interface and functionality of products, and even contribute to code writing.
Since we set out on our AI-turbocharged journey at Akvelon, we've been using the latest AI tools and technologies, such as GitHub Copilot, Copilot Chat, and ChatGPT, to find ways to be more efficient, perform better, and use resources more effectively when building software products. In this material we describe our way to streamline and simplify API testing using AI.
While all stages of product development are important and can have a big impact on the final product, software testing is the essential step that ensures that the product meets technical and market requirements and performs as expected.
The core aspects that product testing covers include:
Locating defects and bugs
Ensuring requirement compliance
Assessing product quality and performance
Verifying security of the product
Evaluating usability, scalability, and compatibility
Maintaining regulatory compliance
If utilized correctly, AI has great potential to reduce the need for human involvement in the testing process and accelerate most of the aspects listed above.
The Current Landscape of Testing
Modern applications are no longer monolithic. They typically consist of multiple independent services linked at runtime that communicate with each other through APIs. Inefficient APIs can lead to integration challenges, compromised security, and poor software quality, resulting in user frustration and financial losses.
Therefore, API testing is essential to ensure that the software performs properly, offers the expected functionality, and maintains security and reliability.
The core advantages of API testing that contribute to higher efficiency, reduced expenses, and accelerated software quality:
Both test stability and test coverage remain foundational to ensuring a software product’s reliability and functionality, and navigating these obstacles requires much time and manual effort unless the process is optimized.
Before delving into Akvelon's solution for ensuring product quality, let's discuss the challenges common specifically for API testing.
Challenges Specific to API Testing
The Potential of AI in API Testing
As we already mentioned, test stability and comprehensive test coverage are the keys to stable, durable, and secure software. Addressing these requirements often involves significant resources, especially in the time and effort needed from test engineers. The application of AI in API testing can help companies optimize resource usage most efficiently.
One of the key benefits of employing AI for testing is its ability to automate the creation of test cases and scenarios, thereby freeing testers to concentrate on other tasks, such as test execution. Additionally, AI can identify potential test cases that may have been overlooked by human testers, enhancing overall test case coverage and reducing the risk of missing critical defects.
In summary, teams can enhance their testing efficiency, including test coverage, speed, and accuracy, using AI for:
Identifying new test cases
Creating test scenarios
Generating test data
Introducing the AI-powered API Testing Tool
Akvelon has embraced the benefits of AI-driven automation for efficiency enhancement. We've developed an AI-powered API Testing Tool that leverages LLM and Swagger API descriptions to automate the creation of API tests for web apps. Our solution helps address the critical challenge of test coverage, not just suggesting diverse positive and negative test scenarios but generating edge cases.
As a result of our effort, Akvelon’s AI-powered API Testing Tool can reduce the time it takes a mid-level SDET to cover one API endpoint with 20 tests by 6x.
The key functionalities of the AI-powered API Testing Tool include:
Automated test case creation for APIs
Automatic generation of test scripts based on the developed test cases
Recommendations for both positive and negative test scenarios
Discovery of edge cases to ensure comprehensive API testing
Generation of corresponding test scripts for a wide range of response codes
How AI-powered API Testing Tool works
We can break down the workflow of the solution to generate API tests into 4 steps:
1. We begin with integrating your application's API into our AI-powered Testing Tool. This can be achieved by either:
a) Providing the URL to your Swagger JSON file as an argument
b) Directly pasting the content of the JSON file
2. Our solution then uses its AI capabilities to generate prompts and detailed test cases for each of your APIs.
3. For the next step, test engineers can adjust the automatically generated test cases or add new ones depending on the project's specific needs.
4. Lastly, based on the achieved test cases and prompts, our AI-powered Testing Tool produces the final tests ready for execution.
Benefits of Using Our Tool Over Traditional Testing Methods
Harnessing the power of artificial intelligence, our cutting-edge AI-powered API Testing Tool revolutionizes traditional testing methods and brings unparalleled advantages to businesses. Dive into its features and see how it stands out:
1. Streamlined test generation and efficiency
Our solution is incredibly efficient, generating 20 tests for a single API endpoint in just 1 minute and 38 seconds, a process that takes a mid-level SDET engineer up to 20 hours.
AI-powered Testing Tool's efficiency advantage grows exponentially with larger projects. For instance, testing 50 endpoints with 20 tests takes a mid-level SDET specialist 600 hours, while the tool accomplishes the same task in just 13.6 hours.
3. Cost saving
LLM offers a cost-effective solution for API testing. At just $0.064 per endpoint or a total of $3.2 for 50 endpoints, businesses can achieve significant savings compared to the traditional costs of SDET labor.
4. High-quality test coverage
Quality isn't sacrificed for speed. Our tool generates tests and then employs AI-generated test cases, ensuring diverse and comprehensive API test coverage.
AI-powered Testing Tool can be tailored to a project's unique needs, allowing the AI to learn from previous tests. This ensures consistency in test stylization and approach, aligning with a project's specific requirements.
Concerns When Incorporating AI in API Testing
The application of AI in API testing is associated with several common concerns, like a possible lack of traceability or transparency of AI-generated test cases, a potential for AI to be biased and generate biased test cases, or the complexity of implementing AI in API testing.
In response to these concerns, we made sure our AI-powered Testing Tool provides clear traceability to requirements and related artifacts. Leveraging GenAI models trained on diverse datasets, we minimize the risk of bias. Furthermore, our approach emphasizes the importance of skilled test engineers who can adeptly review and, when necessary, modify the AI-generated test cases.
Security and Confidentiality Concerns
Knowing that data security is one of the top priorities for our customers when implementing our API testing solution, we've ensured a robust safeguarding process. Since our testing solution only requires the Swagger API description, there's no risk of Personal Identifiable Information (PII) exposure. The API Testing Tool strictly processes only the metadata from the JSON file, ensuring that all client-specific data stays pristine and unaltered.
For enhanced data protection, we offer the capability to generate test cases using a local LLM model instead of relying on external OpenAI LLMs. By utilizing the local LLM, clients can operate the AI-powered API Testing Tool entirely in-house, ensuring data remains within organizational boundaries.
Dependence on AI and potential for overlooking human intuition and expertise
While AI-powered testing tools and other solutions for product development can be extremely advanced, it's important to remember that no tool is infallible. Blending AI capabilities with human intuition and expertise is the way to go. While AI can automate tasks like test case generation, human testers offer deep knowledge of the APIs and can identify issues AI still might overlook. Their creative thinking can also craft unique test cases in addition to the AI's scope. Merging AI's efficiency with human insights ensures a thorough, reliable, and all-encompassing API testing process.
As technology rapidly advances, testing methods must evolve to keep pace. Traditional testing approaches no longer serve the complex and distributed nature of modern software. Adapting technologies like AI in software testing becomes essential for ensuring the quality and reliability of software while addressing the challenges of new technologies. Also, with AI-powered testing tools, companies can reduce the cost of testing, streamline the process, and deliver enhanced customer experience.
Our AI-powered API Testing Tool will help organizations cut operational costs and save SDET time for tasks other than test generation. What's most important, the solution will help ensure comprehensive test coverage and test stability with regard to security and integrity of the companies data.
Director of Engineering at Akvelon