Algemeen AWS Improve testing15 april 2016

AWS Lambda part 2: API Gateway playing tricks?

Last week we looked at how AWS Lambda behaves with one user at a frequent request rate (see first post). We figured that AWS Lambda will create a pool of running instances. When an event triggers a lambda function, AWS Lambda will look in the pool to see if there is a free instance. If that is not the case, AWS will launch a new instance. This week we will increase the experiment with more users to see what happens. We will use the same code as in the previous post and we will run our tests from the same EC2 instance ‘m4.xlarge’ as last week.

50 users per second

Today we repeat the same test as last week, but we increase the number of users to see how AWS Lambda will react. We executed a few runs with 50 users. The results below show the results from a test where only hot instances of the lambda function were used. We’ll explain later why this is an interesting test session.

statistics hot containers

The statistics show a great overall response time, as can be expected from such a small function. The Max response time is quite high, but that can even be explained by a network hickup. We were surprised to see the response time over the whole run. See below:

run diagram hot instances

We saw several spikes in the response times. At that time we didn’t now that all functions were hot, so we figured this was due the creation of new JVM containers. When we looked at the output of our test, we saw that the containers were still hot and that now new containers were made. We were only using 4 instances! This lead to the conclusion that AWS Lambda had no problems whatsoever with our test and that API gateway or something else was causing the spikes. We decided to create a new API endpoint with a mock endpoint as integration type, which returns an empty HTTP 200 response. This way we can see how API Gateway handles our requests.

When running the same test as before, but with mock integration instead of a Lambda function, we saw largely the same results. This substantiated our earlier finding that API Gateway was causing the spikes.

mock
mock stats

We ran different kind of setups of Gatling to see if Gatling did some strange calls to give these results. It turned out that in every test we saw these spikes.

Next time

Next time we would like to eliminate API Gateway from the test and call the Lambda function directly. Stay tuned for the next blog or feel free to contact us via mail: Martijn van de grift (martijn.van.de.grift@trivento.nl) or Jeroen Gordijn (jeroen.gordijn@trivento.nl)
This blog is a co-creation of Martijn van de Grift and Jeroen Gordijn.

Martijn van der Grift