2020-09-11 Pivoting to LT
I realized that the competition in the development mock service space is too tight. Everyone and their mum are making mock service one way or another.
However, MockMotor has competitive advantages in one particular area: load testing.
It is fast. It can handle the PROD-like load.
It is accounts-based. Load tests require thousands of accounts to properly account for caching effects.
It is not aimed at developers, but more at testers.
(The “aimed at testers” part, admittedly, requires more work.)
The LT tests also more expensive (they require LT-grade backends, which are running on expensive hardware) and more critical. In other words, LT is where they have budgets.
It makes sense to leave the development mocks to Postman and others, and explore the LT mocks space, where MockMotor has started.
2020-08-17 Losing Stats Readings
Running a test on a beta cluster in Bell, I noticed two annoying things.
Firstly, some of the readings from clone instances are missed. On the graph, it looks like the clone suddenly drops its CPU, memory and volume to zero. That is obviously not true - just the statistics entry wasn't received in time.
Secondly, the live counters seem to freeze from time to time. While it could be due to the VM configuration issue we have with those boxes, I suspected it might be related to stats collection.
I reviewed the code and immediately found a few places where:
- Stats were collected using a great deal of resources (number of XML and JSON responses, number of currently logged-in users) and not used.
- Stats lock was taken too early.
- Stats lock was not required at all - this is the live counters call, BTW.
I also updated the clone stats logic so that the clone reading was always placed in its proper place in the list, and not always to the most recent cell.
In my test instance, I don't see the missing stats anymore. I need to set up a regular cloned performance test to watch for issues like that.
2020-08-16 Turn to Load Testing
The mock services field is crowded. Everyone offers mock service products in one form or another.
However, most of those products are SaaS and REST only.
Both of these properties are a problem for enterprise and a stand out opportunity for MockMotor.
SaaS mock products don't let a company run a load test with production-like volumes. No matter how good instances the mock service uses on Amazon, the bandwidth between the data centers can't compare to the 1Gbit (or more) network the company has internally.
REST-only may have sense ten years from now, but today many of enterprise services are SOAP. Excluding them is a showstopper.
MockMotor, on the other hand, is an on-premises product and supports SOAP natively.
It becomes clear that MockMotor positioning should shift towards on-premises load testing - i.e. where those countless SaaS REST-only services can't compete.
Starting today, I'm going to update all content on the site to emphasize the load testing capabilities.
I will also plan and execute (regularly) a set of load testing-aiming tests to confirm and measure MockMotor's capabilities.
I listened to the “Everyone Hates Marketers” podcast episode with Kevin Roger.
He said that most performant jokes among the standup comedians are built by the “Identity-Struggle-Discovery-Surprise” formula.
Identity: “I'm in Walmart with my kids.” The listeners immediately understand the situation and the persona in it. Struggle: “My kids are being assholes.” The listeners been there and done that, so they understand the struggle, too. Discovery: “And I see that fly swatter on the shelf.” There is something that promises the end to the struggle. Surprise: “I swat my kids with it, and then realize I don't have kids!” Funny.
According to Kevin, the same formula applies to positioning in business. Instead of surprise, use whatever result you or your customers want. As he described his own positioning (in my own, very simplified reading of it): “I was a standup comic for years. I was broke. Then I found this magic formula. Join my membership site, and you'll get the business results you want!”
Sounds very convincing.
So I went and updated my own About page. It now reads:
I'm building web services-based enterprise systems for some 25 years for clients small and large.
I was always straggling to test the web services to the point where I felt 100% confident in their functionality and performance.
Years ago, I started creating a basic mock server to help my clients and me to cover the scenarios that are hard or incorrect to test locally.
Today, this mock service has grown into MockMotor - a product that can support most complex functional and performance testing needs. Use MockMotor to confidently deliver APIs into production, reduce dependencies on partner teams, perform PROD-like load tests and verify corner cases.
Sounds much more focused than the one that I had before.