Adding the Sealights agent to your Application Under Test will have varying levels of impact on application performance. This page offers benchmarking data so that our customers can assess the impact of each integration.
AWS Lambda - Sealights Layer
AWS Lambda pricing is based on number of invocations, time of execution and memory used. The following tables detail some approximate performance related statistics to help assess the cost and performance impact of this. Note that AWS Lambda allocates CPU power in proportion to memory so results will vary widely.
The benchmarking data below is sectioned by each of the languages for which Sealights provides a Lambda Solution.
SL Java Lambda Layer
These benchmarks were collected against two differently sized Lambda projects:
Short-lived execution with fewer methods executed.
Long-lived execution involving CPU-intensive use of the same methods.
Each Lambda was tested under these scenarios:
No Sealights Layer (baseline): Execution of the Lambda function alone
Lambda with Sealights layer: Execution of the Lambda function with the Sealights layer applied
No Active Test Execution (Standby Mode): Execution of the Lambda function with the Sealights layer applied , but without an active/open test execution.
Table 1. Average execution times
Handler execution time | Metric | No Sealights Layer | Lambda with Sealights layer | No Active Test Execution (Standby Mode) |
|---|---|---|---|---|
Short-Lived | AVG time [ms] | 45.8 | 81.6 | 73.2 |
Long-Lived | AVG time [ms] | 4,750.3 | 5,380.6 | 4,852.85 |
Table 2. Init time and memory consumption
Handler execution time | Metric | No Sealights Layer | Lambda with Sealights layer | No Active Test Execution (Standby Mode) |
Short-Lived | AVG memory usage [MB] | 107.2 | 239.3 | 198.1 |
Long-Lived | AVG memory usage [MB] | 124.7 | 246.6 | 213.5 |
Recommendations
When creating the Lambda function, ensure that the allocated memory is appropriate for your use case. As a starting point, consider doubling the memory for Lambdas using the Sealights Java Layer.
The Lambda Layer does not impact ephemeral storage, so there is no need to increase it when using the Sealights Layer.
Increase the Lambda timeout to suit your use case, as the initial execution and initialization may take longer.
SL NodeJS Lambda Layer
These benchmarks were collected against two Lambda functions:
Short-lived execution (
ping, with baseline execution time 167ms)Long-lived execution (
getComments, with baseline execution time 3411.46ms)
Each Lambda was tested under these scenarios:
No Sealights Layer (baseline): Execution of the Lambda function alone
Lambda with Sealights layer: Execution of the Lambda function with the Sealights layer applied
No Active Test Execution (Standby Mode): Execution of the Lambda function with the Sealights layer applied , but without an active/open test execution.
Each Lambda was tested with resource allocation of:
128MB RAM
1024MB RAM
Metrics Collected:
Absolute Time Impact – the number of ms added to the Lambda function under the different testing scenarios
Memory Utilization – the memory used by the Lambda function under the different testing scenarios
Table1: Average Execution Times (128MB RAM)
Handler execution time | Metric | No Sealights Layer | Lambda with Sealights layer | No Active Test Execution (Standby Mode) |
Short-Lived | AVG time [ms] | 332 | 1,473 | 324 |
Long-Lived | AVG time [ms] | 4,082 | 6,189 | 4,829 |
Table2: Average Memory Consumption (128MB RAM)
Handler execution time | Metric | No Sealights Layer | Lambda with Sealights layer | No Active Test Execution (Standby Mode) |
Short-Lived | AVG memory usage [MB] | 91 | 113 | 108 |
Long-Lived | AVG memory usage [MB] | 107 | 114 | 113 |
Table: Average Execution Times (1024MB RAM)
Handler execution time | Metric | No Sealights Layer | Lambda with Sealights layer | No Active Test Execution (Standby Mode) |
Short-Lived | AVG time [ms] | 52 | 224 | 86.6 |
Long-Lived | AVG time [ms] | 573 | 1064 | 796 |
Table: Average Memory Consumption (1024MB RAM)
Handler execution time | Metric | No Sealights Layer | Lambda with Sealights layer | No Active Test Execution (Standby Mode) |
Short-Lived | AVG memory usage [MB] | 92 | 136 | 117 |
Long-Lived | AVG memory usage [MB] | 100 | 170 | 143 |