Lessons learned on our journey to SOC 2 compliance
Shortcut’s Security Posture
At Shortcut, security is a top priority. Each quarter, we have ongoing security work that is everyone’s responsibility. A large portion of such security work comes from our Platform squad - the team responsible for standing up and maintaining our cloud infrastructure and tooling that facilitates and hosts the everyday work of developers (think DevOps). Some security features the Platform squad here at Shortcut aims to provide are:
- Secure networking configuration
- Encryption by default
- Auditing and logging
- Continuous backup strategies
Additionally, every team is responsible for implementing additional security best practices more specific to their stack such as vulnerability monitoring, patching, and responding to security incidents.
Pursuing SOC 2 Compliance
While we maintain a strong security posture, it was important for us to prove to our customers that we do everything we claim to do. This led us to pursue a SOC 2 Type II report that would provide evidence of our compliance with the security practices I listed above and much more. So how does SOC 2 help us do this?
As an organization, we continuously assert that we abide by certain policies, procedures, and operational controls that mitigate risk and enhance security. During a SOC 2 audit, a CPA firm conducts a review of such policies, procedures, and controls to verify that not only do we have them in place, but they are also designed and operating effectively. Once the audit period has concluded a SOC 2 Type II report presents an auditor’s attestation of what policies, procedures, and operational controls Shortcut, as an organization, has established and adheres to for some period of time.
The Audit Itself
Navigating SOC 2 can be tricky for two primary reasons. One reason is that there is no list of requirements that everyone must follow. Instead, there are numerous pieces of criteria that can be selected from in order to demonstrate that your organization mitigates such risk. Obviously, the more you can prove the better, but not all the criteria apply to everyone. The second reason is that the audit process itself is long, typically 5 days. Auditors have to walk through huge amounts of evidence to prove you are compliant with the practices, procedures, and controls you claim to implement. This is on top of the amount of time it takes for you to gather all that evidence and implement any additional controls you want to be included in the report.
To mitigate the unknowns and the time it took to gather evidence, we contracted with an organization named Vanta that help automate a lot of the process. Their platform integrates with a lot of our tooling such as AWS, Github, and Google. It aggregates that evidence and presents it in a succinct format to auditors themselves so we didn’t need to be present for a lot of the evidence handling. On top of that, it monitored a lot of our policies, procedures, and controls that we claimed to do. Such as whether MFA was enabled on AWS or whether we had vulnerability monitoring in place. The platform would also suggest additional controls we could implement or provide evidence for. While not all of them applied to us, we saw the list of options available to us for what evidence we could provide that would be relevant to our SOC2 audit and report.
Shortcut (of course)
Of course, we used Shortcut to track our ongoing work. The work affected more than just one team so the cross-collaboration was crucial. It was a huge lift to get all the necessary evidence and also implement additional controls we wanted in time for the audit. We created an Epic in Shortcut to track our progress, putting all relevant stories in there so that teams could see what was assigned to who.
On top of that, Shortcut itself provided many auditing features that auditors wanted to see as evidence. Features such as the Github integration, story labels, story activity, and more allowed auditors to see all the evidence they needed about incidents, escalations, and security incidents just by looking at a single story.
Here is how we prepared for the audit. In general, all of it took about 3 months. While it sounds short, it was due to the fact that we had very few additional controls to implement because a lot of it was already in place. A majority of our time was spent gathering the appropriate evidence.
Phase 1 - Policy Crafting
Document all the policies, procedures, and operational controls we implement at Shortcut.
Phase 2 - Inventory + Vulnerability Monitoring
List everything we are responsible for managing and operating. This includes all AWS services, Github repos, office physical hardware, employee laptops, etc. Additionally, where applicable implement vulnerability monitoring where not already in place.
Phase 3 - Vendor Assessment
Run through a list of all our vendors and conduct a risk assessment for each one as well as contingency plans for when they are unavailable.
Phase 4 - Dry Runs
Run a disaster recovery drill to simulate what happens when our primary AWS region goes down. Second, conduct a risk analysis of all possible risks posed to our organization.
Phase 5 - Evidence Gathering
For every policy, procedure, control we claim to implement, document and gather evidence to deliver to auditors. While Vanta covered a lot of it, there was still a huge chunk of evidence specific to us that we had to gather evidence for.
Finally, here's a non-exhaustive list of the things we took away from this process.
Put security before compliance.
Compliance is just checking the boxes. Looking for that list of boxes to check won’t necessarily making your organization secure. It could leave you in a position very far from a secure posture. We found that by putting security first, compliance naturally followed.
Make the process transparent to the entire organization.
Individuals at every level will often be the agents of implementing necessary controls to be compliant. When it comes to things like background checks and monitoring employee laptops, we wanted individuals to be on board with the degree to which we implemented those. In the end, we found that folks were more than obliged to help increase the security of our organization when they understood the need for it.
Don’t rush the process.
If your organization isn’t ready to be compliant, you might think you need to cut some corners to get there. When it comes to security, those are the corners you don’t want to cut. So if you need more time, take it.
Don’t let perfection be the enemy of good.
A mature security posture is never going to be perfect. The list of possible SOC 2 controls you can provide evidence for is endless, and I’m talking about a hypothetical list here because a real list doesn’t really exist and it is ever evolving anyway. Since the security landscape is in a perpetual state of change and so is your product, you will never have enough people nor time to implement every possible risk mitigation and security best practice there is. This is why we practice security at depth but at a certain point, all levels of depth just become tedious.
Security and compliance can be at odds with development.
This is just a fact and trying to change it won’t do either security or development any good. Baking security in at every layer takes extra time. Doing the necessary checks to ensure you maintain compliance also takes time. There are a ton of tools out there to automate both security and compliance -- you can significantly reduce the degree to which these fields are at odds. But any attempt to reduce security in order to speed up development obviously leaves you in a more vulnerable position.