Patching Strategies: A Director and Senior Executive perspective
By Dr Sally Ernst and Mark Stanhope
Patching as a concept is simply keeping software up-to-date. Well, it sounds simple but actually the process of a good patching regime and the extent of places software exists make it a little more complex.
The key thing for directors to understand is that an appropriate patching regime is required as part of any good governance to protect their organisational assets and sensitive information. Not only is it a compliance requirement, it is expected as part of any normal operating environment by businesses, governments, banks and insurers.
Here is a director, senior executive, business owner level synopsis of:
- what an appropriate patching regime entails,
- why it protects your organisation,
- how things can go wrong and how to prevent those risks
- some things you can measure and ask.
Software exists in multiple places in any organisation, home and personal networks, including phones, routers, servers, webservers personal and other devices, and within devices to control other devices (eg. camera within a phone, controller within an MRI, manufacturing machine, lift, water or sewerage facility, and so on). This software is prone to ‘vulnerabilities’. Vulnerabilities are exactly that – points of weakness that an attacker can use to get into your systems and networks, resulting in access to your organisational assets and sensitive information. Vulnerabilities may exist in software you or your organisation uses, however until it is publicised it is not as big an issue for the community in general. If it’s not publically known but is used by an attacker, it is called a ‘Zero Day’ attack. However, once a vulnerability does become public, it also alerts attacks across the world that the vulnerability exists and they start building tools (which are often distributed for free) to attack individuals and organisations at scale.An important prevention strategy for these attacks, therefore, is a patching regime that updates software to ‘patch’ the vulnerability in the software.
- What your organisation’s risk appetite is
- What to include in your patching regime
- This requires an organisation, and/or individual, to know:
- What the process will be for each of regular patches and accelerated patches, including how patches are:
- assured to be from a trusted source
- identified and tracked
- categorised as regular or accelerated
- tested and staged
- as a total regime, assured as being performed correctly
- Who is responsible for the patching process
Regular versus emergency patches
An organisation, to have an effective patching strategy, needs to agree its risk appetite to vulnerability management, ie:
Do you need to apply patches as quickly as possible because you have a big presence and are likely to be a target?
What parts of the organisation are in which category of importance?
To help assess this risk, the NIST (National Institute of Standards and Technology) is an internationally recognised standards body that, among other things:
- labels vulnerabilities with a CVE (Common Vulnerabilities and Exposures) number, and
- rates with a CVSS (Common Vulnerability Scoring System) number out of 10 (with 10 being most critical) the impact of software vulnerabilities
In the case of Shellshock (CVE-2014-6271) for example, the CVSS rating was 10 and an incomplete patch release resulted in a further vulnerability (CVE-2014-7169 ), also rated 10, to track the completion of the patch.
When a CVSS rating is 5 (eg. Heartbleed) or higher, your organisation may have agreed the patch needs to be accelerated beyond that which you may normally have as part of your patching regime.
Note that with Shellshock attacker tools had been released and attacks were being attempted within one day of the vulnerability being publicised,and within two day attacks were becoming successful at scale.
By using the CVSS scoring system, then, it is possible to determine an approach that effectively manages the risks of taking patches early and managing the risks that an exploitable vulnerability presents to the organisation.
NIST’s vulnerability search engine immediately comes up with the additional vulnerability, the severity ratings (in this case highest possible) and locations you can click through for further information.
You now know this vulnerability is a baddie, and can ask your tech team intelligent questions about what they are doing to ensure your organisational assets and sensitive information are being protected from it.
Testing and staging patches
Testing and staging is important because a bad patch could wipe your business out.
However, the testing and staging process needs to be within a reasonable period. You are now balancing your risk with being attacked because of not closing off a vulnerability, with the risk of something going wrong by applying a patch that does something nasty to your systems. The organisation needs to have process and procedures, then, to ensure that you can patch and test quickly to avoid breaking your environment.
An important part of the testing process is ensuring the patched system still performs the functions the business needs, as it did before. This type of testing needs to be documented and is called a ‘regression plan’.
Staging and canary systems
Staging involves determining which machines need to be patched and in what order. For example,
After the testing process is complete, a ‘Canary Group’ – a small subset of machines – may be selected patch first and early and can provide early warning of a patch that is going to cause issues with your environment.
A sample patching regime
- Receive patches from trusted source (eg. ensure the patch matches any SHA1 hash checks)
- Risk assess the vulnerabilities and patches and allocate to the routine patching process or the accelerated one
- For routine and accelerated patches
- Apply patches to test environments
- Conduct documented business regression tests
- If successful, apply the patches to the canary systems.
- For routine patches only,
- wait 1 week before starting the patching of the first 25% of the environment,
- continue patching on a weekly basis.
- which will unfortunately bring you around to the next monthly patch cycle, but it will protect against a “bad” patch taking your environment down.
- For accelerated patches only,
- determine how long you have to apply these patches
- divide the organisation into manageable sections
- apply the patch.
What if a patch doesn’t exist or can’t be applied?
If it turns out that the patch does bad things to your environment or a patch doesn’t exist, considerations include:
- Compensating controls, which are alternative methods – additional security mechanisms – to limiting the risk of a vulnerability being exploited
- Updating legacy systems, as at some point, the balance of risk and cost of compensating controls may tip toward replacing a system (eg. upgrading from windows xp to windows 7)
- Where patches are still in development, a combination of compensating controls and tracking the patch release will be needed.
If you are a victim of a zero day attack, you can report it to:
- Your security provider
- The CERT (Computer Emergency Response Team) in your country
Patching software means keeping software up-to-date to protect against vulnerabilities that leave your organisational assets and sensitive information open to attack. A timely patching regime is expected as part of any normal operating environment and needs to consider:
- the entire ecosystem data exists in;
- the importance of that data;
- how and where it is stored and transmitted; and,
- in that context what software needs then to be included and how it is to be treated as part of the process.
- Any changes in context and/or the data, systems and software
- Routine versus emergency patching.
- NIST’s NVD (National Vulnerabilities Database) provides the internationally accepted vulnerability labelling (CVE) and severity rating standard (CVSS).
- Testing and staging, including:
- source of patch
- a documented regression plan
- Compensating controls and replacing legacy systems need to be assessed where patching regimes fail
- Metrics can be monitored such as:
- adherence to agreed times to patch
- vulnerable systems: vulnerabilities due to non-existent patches or patches that can’t be applied
- the cost of compensating controls on, versus upgrading or replacing, vulnerable systems that are of high consequence.
About the Authors
Dr Sally Ernst
Dr Sally Ernst, co-founder UK and Australian Cyber Security Networks (www.CSNs.co), has over 15 years leadership experience in British and Asia-Pacific tech companies. She has held Board positions, contributed to government-led forums, and invested in startups. Sally’s Doctorate specialises in tech intrapreneurship in a radical innovation context and she continues to be involved in industry research. Sally regularly speaks on cyber security from a business perspective.
Mark is the Head of Operations and Governance for Mobile Payments, a real time payment service that allows individuals to push payments without having to exchange sensitive information, available day and night, 365 days per year, supporting the demands of personal and business customers; and, Senior Executive Secretary CSFI-CWD (EMEA Region) for The Cyber Security Forum Initiative (CSFI), a non-profit organization providing Cyber Warfare awareness, guidance, and security solutions through collaboration, education, volunteer work, and training to assist the US Government, US Military, Commercial Interests, and International Partners. With over 25 years of experience, Mark was previously the Senior Manager for e-Crime at Lloyds Banking Group and has also held senior information security management roles at BACs and Voca.