This post was last updated on 11/21/23.
As Kicksaw’s CTO, Tim Sabat, once said, a good test suite is like a warm blanket — it keeps your code cozy and covered. What do we mean by this? Keep reading to find out.
The landscape of automated testing
One of the biggest challenges when testing is covering code that interacts with an external service. This is where the domain of unit tests ends and that of integration tests begins, but the latter is multitudes more complicated than the former. Once it’s time to test anything that makes a network call, the bar is raised.
Two common approaches are:
- Patching and mocking code that makes a network call
- Configuring resources to exist in a certain state before testing the code
This is a lot of complexity to add either way, but it’s especially complicated to run tests that allow write operations to external services. You have to have scripts that set everything up before the test and tear it all back down afterwards. In almost all cases, a team will opt to mock the code which makes an API call. The advantages of mocking api calls are compelling:
- No setup/clean up
- Tests run just as fast as they would in a unit test (since there’s no network activity)
This can save you an exorbitant amount of time in both development and test execution time, but the approach is not without its downsides:
- You’re not testing any of the code that’s being patched
- Mocking can quickly become complicated
Introducing simple-mockforce
Some services are so commonly interacted with that it’s worth writing a domain-specific library for testing them. Inspired by moto, we here at Kicksaw have produced something quite domain-specific with simple-mockforce, a companion package for simple-salesforce that enables testing for code that interacts with Salesforce’s REST API.
No API calls were made to Salesforce in the execution of this test. In the above code, we create an object. Afterwards, we get the object and check that its contents are as expected. The state persists despite the calls to Salesforce being intercepted; the write operations affect the read operations, but no server is involved.
This approach is made possible by patching all API calls to the Salesforce API using the responses library, where they’re instead fed to an in-memory, virtual Salesforce organization that exists throughout the duration of a test. The code for it can be seen here.
This virtual instance is anything but a real Salesforce organization. It enforces almost none of the server-side validation you’d get when interacting with a real org, and it only supports the most basic of SOQL queries, but it does enable you to write tests for code that hits the Salesforce API. Instead of the mocking happening on your end, on the client level, it’s happening at the destination level; the server itself is mocked. And as long as you don’t throw millions of records at it, your code won’t be able to tell the difference between it and a real Salesforce organization.
This has allowed us to reach 100% or near 100% code coverage in some of our integrations here at Kicksaw. Our development projects range anywhere from simple to complex, but a common goal throughout is that we need to move large amounts of data into Salesforce, sometimes with non-trivial business logic in between.
Being able to execute our simple-salesforce code as-is in our tests has done a lot for the integrity of our projects. Since we’re patching the HTTP calls and not any of the simple-salesforce code, every line of both our own code and the library’s code is executed. If you’ve ever accidentally patched over code that contained a bug where the two meet, you’ll appreciate how much better this approach is.
Challenges
The biggest challenge with creating a virtual Salesforce organization has without a doubt been supporting SOQL and the /query endpoint. In order to do so, we took an example of a SQL parser from pyparsing‘s codebase, and, using pyparsing, created a rough cut of a python-based SOQL parser.
If you’re not familiar with what SOQL is, it’s a query language specific to Salesforce which looks a lot like SQL, but it only supports read operations. In addition to that, it also has some Salesforce-specific tokens that would never fly in SQL, and the way it handles relations is completely unrecognizable to any other relational database. For example, instead of joins, you access related data via object dot-notation.
This part of the project isn’t close to finished, though, and has a long way to go before it can fulfill all the needs of the simple-salesforce community. The SOQL parser is in the earliest phase software can be in; think pre, pre-Alpha.
Future plans
Beyond support for more complex SOQL queries, there’s a few extra things we’d like to accomplish with this library.
As already mentioned, the virtual Salesforce organization which simple-mockforce provides offers nothing in the way of server-side validation, since it’s just a Python class and not an actual server. In the future, it would be great if simple-mockforce read in the XML of an organization (see the Metadata API) and used it to mimic some of the server-side behavior you’d get when interacting with a real instance.
For example, currently this code will work.
But we have no way of telling the virtual instance if CustomObject__c is a defined object or not, or if CustomField__c is a valid field for said object; the object will simply be created and added to the org’s in-memory state.
To improve the library further, simple-mockforce should support virtualizing a specific organization. If you push a non-existent object to a real Salesforce organization, the API will tell you so, and it would be great if the virtual organization did that too.
Contributors wanted!
To summarize, simple-mockforce and the python-soql-parser library which it relies on are still nascent packages. At Kicksaw, we’ve found immeasurable value in open-source libraries such as simple-salesforce, and we want to contribute back to the community which has enabled us to crank out over 30 integrations for our clients in the last year. To do so, we thought we’d make the simple-mockforce package open-source, because even though it currently suits our needs, someone else in the community might find a lot of value by simply adding a new feature to our already existing codebase; we hope to be here to help if that happens!
Working with Salesforce can be a challenge, as can testing code that works with Salesforce. The goal of simple-mockforce is to at least address the latter. While there’s still a long way to go, Kicksaw is already using this library in the integrations we build for our clients, and we’re proud of the test coverage it gives us — it truly does keep our code cozy and well-covered. If you’re reading this and you’re interested in contributing, please open an issue or a PR! Every contribution helps bring us closer to a 1.0.0 release.