Vladimir Dyuzhev, author of MockMotor

Vladimir Dyuzhev
MockMotor Creator

How to Record HTTP Traffic with MockMotor

MockMotor can act as a simple MITM to record and mock the traffic. Tweet This

MockMotor saves you from tedious manual work.

One of the ways it does that is the automatic creation of mocks for passing traffic.

Forward Functionality

Forwarding is a less often used functionality in MockMotor.

A forward response is not producing a reply payload on its own. Instead, it forwards the request to an external URL, i.e. to a real backend.

In the request flow, MockMotor forwards URI, parameters, payload and headers down to the backend. That makes most authentications schemes (e.g. Basic HTTP or JWT) work without any special configuration.

In the response flow, MockMotor sends the response payload and response headers provided by the backend, to the caller.

Forwarding Usefulness

Forwarding has a few uses:

  • Mocking only a selected few operations, relegating the rest to the real backend.
  • Modifying the real responses before sending them back to the consumer.
  • Recording the traffic.
  • In this post, I’m going to use the forwarding to create a mock version of a live service.

    Record Functionality

    Forward responses by default only send the traffic to a live backend.

    However, when the Record is enabled, the response is not only sent back to the consumer but is also recorded as a fully configured mock.

    Recorded Mock’s Match Parameters

    The recorded mock uses as many of the request parameters as possible to provide the exact match:

  • SOAP Action or First Element Name
  • Relative URI
  • Names and values of segments in the URI
  • Names and values of HTTP query parameters
  • Recorded Mock’s Values

    The mock also saves the response values:

  • Response HTTP Status
  • Response payload
  • Response time (as a delay)
  • Content-Type
  • Request payload (to use in Debug mode)
  • Recording Yandex Weather API

    The Yandex Weather API is widely used by applications that deal with weather in Russia and its neighbours. Its input is a HTTP request having two query parameters: lat for latitude and lon for longitude:

    https://api.weather.yandex.ru/v1/forecast?lat=59.939015&lon=30.315804
    

    I consider this a HTTP service and not a REST one. For a modern truly honestly REST service I’d expect the parameters be in the URI segments, like this:

    https://api.weather.yandex.ru/v1/forecast/lat/59.939015/lon/30.315804

    However, this distinction is debatable. We’re not here to have an ivory-tower discussion, so let’s just mock what we’ve got, ok?

    The response contains a massive amount of information about the weather in the requested geographical point.

    However, in a test mode, it only allows 5000 requests a day. It seems like a lot, but if you generate maps with weather markers, 5000 requests can all be used up before lunch.

    The solution is, of course, to mock the responses.

    The number of locations used in test scenarios is limited. We can mock them all and then execute tens of thousands of requests - without any scowl from Yandex.

    Lets Be Lazy

    Let’s create the mocks the most effortless way - make MockMotor record the responses for us.

    Create a Service

    First, we need to create a test service that mocks the Yandex Weather API. I just did that:

    As you can see, its relative URI is /yandex-weather.

    Create a Forward Response

    How a forwarding mock should look?

  • It should match any HTTP operation.
  • It should use Javascript.
  • It should have Yandex API URL set in the Forward field.
  • It should have Record enabled.
  • In the forwarding response, we shouldn't have any of response payload, HTTP status, content-type, attachments or delay set. Currently, they are ignored anyway, but in future they can be used for altering the response properties, so better clear those fields.

    Below is the forward response. (Unrelated properties are hidden.)

    Note the Forward URL field that contains the target system hostname. Anything in request path after the mock service URI http://127.0.0.1:7080/ExamplesForBlog/yandex-weather is appended to the forward URL. That includes relative path and query parameters.

    E.g. this URL:

    http://127.0.0.1:7080/ExamplesForBlog/yandex-weather/v1/forecast?lat=59.939015&lon=30.315804
    

    becomes, when MockMotor calls Yandex, this:

    https://api.weather.yandex.ru/v1/forecast?lat=59.939015&lon=30.315804
    

    Note also that the Record is set to Record and Activate. The new mock then is placed above the forwarding mock, and begins handling the matching traffic immediately.

    When the Record is set to Just Record, the newly created mock is placed below the forwarding mock.

    This is done for cases when the same operation is called multiple times during a flow and produces different responses. To make sure the new mock doesn’t break the current test flow it is placed below the forwarding mock where the new requests won’t reach it.

    When the flow is completed, you need to review the recorded mock and move it to its proper place in the list manually.

    Execute a Request for Weather

    Now, let’s execute our first request.

    Since we do not have any local mocks yet, the request will be handled by the forwarding mock. It sends the request to the actual Yandex backend. When Yandex responds, MockMotor records the response properties as a new mock.

    Lets find the weather in St. Petersburg. This city is my childhood home incidently, and also a home of Russian Tsars, the birthplace of 1917 revolution and a location of lots of other interesting things, but it is often rainy here so it worth to check.

    The latitude and longitude of St. Petersburg is 59.939012,30.315804. (To be precise, this is a location of the Palace Square - the very heart of SPb).

    GET http://127.0.0.1:7080/ExamplesForBlog/yandex-weather/v1/forecast?&lat=59.939012&lon=30.315804 HTTP/1.1
    Accept-Encoding: gzip,deflate
    X-Yandex-API-Key: 3ae58b34-****-****-****-da111b3d6fe7
    Host: 127.0.0.1:7080
    Connection: Keep-Alive
    User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
    

    Oh great, we’ve got a response! Lots of data in it (2 degrees Celsius - cold, as usual):

    {
       "now": 1553914074,
       "now_dt": "2019-03-30T02:47:54.740Z",
       "info":    {
          "f": true,
          "n": true,
          "nr": true,
          "ns": true,
          "nsr": true,
          "p": true,
          "lat": 59.939012,
          "lon": 30.315804,
          "tzinfo":       {
             "name": "Europe/Moscow",
             "abbr": "MSK",
             "offset": 10800,
             "dst": false
          },
          "def_pressure_mm": 759,
          "def_pressure_pa": 1012,
          "_h": false,
          "url": "https://yandex.ru/pogoda/?lat=59.939012&lon=30.315804"
       },
       "fact":    {
          "temp": 2,
          "feels_like": -2,
          "temp_water": 0,
        ...
    

    MockMotor should’ve recorded the response as a mock. Let’s confirm that.

    Review the Recorded Response

    Let’s take take a look at the service’s responses again. You can see the newly recorded mock:

    It is placed ahead of the forward mock so any new matched traffic is handled by mock. That is, if we have another request with the same lat and lon, it is responded by the mock already.

    You can notice that MockMotor recorded:

  • HTTP operation GET
  • Relative URI /v1/forecast
  • Query parameter lon value
  • Query parameter lat value
  • Response time 1367ms
  • In fact, we should probably reduce the precision of the match by lat and lon. Something along the lines:

    Math.floor(parseFloat(http.parameters.lon)*10000)==303158 && ...

    After all, the weather on the other side of Winter Palace is not different from this side. But for simplicity of this tutorial, I’ll leave the match as is.

    Looking inside of the recorded response, you can see it has the response payload, HTTP status and content type recorded as well:

    Test It

    Let’s test our new mock. We execute the same request for Palace Square once again. Now it should be replied from mock, and not eat into our daily call limit.

    And indeed, you can see X-MockMotor-Delay: 1367 header in the response, telling us that the response was generated locally.

    You can also see that the call count next to the mock response has incremented:

    Repeat

    To add more locations, just call the mock service with more combinations of lat and lon.

    After running for some time, the mock service handles any predefined test locations from your test scenarios, not leaking any calls to the actual Yandex service.

    You can see the complete service on the demo MockMotor instance.

    Please Share

    Was this post useful? Then please share! Tweet This