Performance Impact

Owned by Samantha Gray
Last updated: Dec 19, 2024
3 min read

Adding the Sealights agent to your Application Under Test will have varying levels of impact on application performance. This page offers benchmarking data so that our customers can assess the impact of each integration.

 

AWS Lambda - Sealights Layer

AWS Lambda pricing is based on number of invocations, time of execution and memory used. The following tables detail some approximate performance related statistics to help assess the cost and performance impact of this. Note that AWS Lambda allocates CPU power in proportion to memory so results will vary widely.

The benchmarking data below is sectioned by each of the languages for which Sealights provides a Lambda Solution.

SL Java Lambda Layer

These benchmarks were collected against two differently sized Lambda projects:

  • Short-lived execution with fewer methods executed. 

  • Long-lived execution involving CPU-intensive use of the same methods. 

Each Lambda was tested under these scenarios: 

  1. No Sealights Layer (baseline): Execution of the Lambda function alone

  2. Lambda with Sealights layer: Execution of the Lambda function with the Sealights layer applied 

  3. No Active Test Execution (Standby Mode): Execution of the Lambda function with the Sealights layer applied , but without an active/open test execution.

 

Table 1. Average execution times 

Handler execution time 

Metric 

No Sealights Layer 
(baseline) 

Lambda with Sealights layer 

No Active Test Execution (Standby Mode)

Handler execution time 

Metric 

No Sealights Layer 
(baseline) 

Lambda with Sealights layer 

No Active Test Execution (Standby Mode)

Short-Lived

AVG time [ms] 

45.8 

81.6 

73.2 

Long-Lived 

AVG time [ms] 

4,750.3 

5,380.6 

4,852.85 

 Table 2. Init time and memory consumption 

Handler execution time 

Metric 

No Sealights Layer 
(baseline) 

Lambda with Sealights layer 

No Active Test Execution (Standby Mode)

Short-Lived

AVG memory usage [MB] 

107.2 

239.3 

198.1 

Long-Lived 

AVG memory usage [MB] 

124.7 

246.6 

213.5 

 

Recommendations

  • When creating the Lambda function, ensure that the allocated memory is appropriate for your use case. As a starting point, consider doubling the memory for Lambdas using the Sealights Java Layer. 

  • The Lambda Layer does not impact ephemeral storage, so there is no need to increase it when using the Sealights Layer. 

  • Increase the Lambda timeout to suit your use case, as the initial execution and initialization may take longer. 

 

SL NodeJS Lambda Layer

These benchmarks were collected against two Lambda functions:

  • Short-lived execution (ping, with baseline execution time 167ms)

  • Long-lived execution (getComments, with baseline execution time 3411.46ms)

Each Lambda was tested under these scenarios: 

  1. No Sealights Layer (baseline): Execution of the Lambda function alone

  2. Lambda with Sealights layer: Execution of the Lambda function with the Sealights layer applied 

  3. No Active Test Execution (Standby Mode): Execution of the Lambda function with the Sealights layer applied , but without an active/open test execution.

Each Lambda was tested with resource allocation of:

  • 128MB RAM

  • 1024MB RAM

Metrics Collected:

  • Absolute Time Impact – the number of ms added to the Lambda function under the different testing scenarios

  • Memory Utilization – the memory used by the Lambda function under the different testing scenarios

 

Table1: Average Execution Times (128MB RAM) 

Handler execution time 

Metric 

No Sealights Layer
(baseline) 

Lambda with Sealights layer 

No Active Test Execution (Standby Mode) 

Short-Lived 

AVG time [ms] 

332 

1,473 

324 

Long-Lived 

AVG time [ms] 

4,082 

6,189 

4,829 

Table2: Average Memory Consumption (128MB RAM) 

Handler execution time 

Metric 

No Sealights Layer
(baseline) 

Lambda with Sealights layer 

No Active Test Execution (Standby Mode) 

Short-Lived 

AVG memory usage [MB] 

  91 

  113 

108 

Long-Lived 

AVG memory usage [MB] 

  107 

  114 

  113 

Table: Average Execution Times (1024MB RAM) 

Handler execution time 

Metric 

No Sealights Layer
(baseline) 

Lambda with Sealights layer 

No Active Test Execution (Standby Mode) 

Short-Lived 

AVG time [ms] 

52 

224 

86.6 

Long-Lived 

AVG time [ms] 

573 

1064 

796 

Table: Average Memory Consumption (1024MB RAM) 

Handler execution time 

Metric 

No Sealights Layer
(baseline) 

Lambda with Sealights layer 

No Active Test Execution (Standby Mode) 

Short-Lived 

AVG memory usage [MB] 

92  

136 

117 

Long-Lived 

AVG memory usage [MB] 

100 

170 

143 

Recommendations

  1. Optimize RAM allocation to at least 512-1024MB for layered executions to minimize performance penalties. 

  2. Avoid using a remote collector outside the VPC, especially when network latency is a concern. 

  3. Use additional layers selectively to balance runtime performance and required functionality.