CISSP-Study-Resources

CISSP Resources

View project on GitHub

Domain-6 Security Assessment and Testing

  • Security assessment and testing programs are an important mechanism for validating the on-going effectiveness of security controls
    • they include a variety of tools, such as vulnerability assessments, penetration tests, software testing, audits, and other control validation
  • Every org should have a security assessment and testing program defined and operational
  • Security assessments: comprehensive reviews of the security of a system, application, or other tested environment
    • during a security assessment, a trained information security professional performs a risk assessment that identifies vulnerabilities in the tested environment that may allow a compromise and makes recommendations for remediation, as needed
    • a security assessment includes the use of security testing tools, but go beyond scanning and manual penetration tests
    • the main work product of a security assessment is normally an assessment report addressed to management that contains the results of the assessment in nontechnical language and concludes with specific recommendations for improving the security of the tested environment
  • An organization’s audit strategy will depend on its size, industry, financial status and other factors
    • a small non-profit, a small private company and a small public company will have different requirements and goals for their audit strategies
    • the audit strategy should be assessed and tested regularly to ensure that the organization is not doing a disservice to itself with the current strategy
    • there are three types of audit strategies: internal, external, and third-party
  • Software testing verifies that code functions as designed and doesn’t contain security flaws
  • Security management needs to perform a variety of activities to properly oversee the information security program
    • Log reviews, especially for admin activities, ensure that systems are not misused
    • Account management reviews ensure that only authorized users have access to information and systems
    • Backup verification ensures that the org’s data protection process is working properly
    • Key performance and risk indicators provide a high-level view of security program effectiveness
  • Artifact: piece of evidence such as text, or a reference to a resource which is submitted in response to a question
  • Assessment: testing or evaluation of controls to understand which are implemented correctly, operating as intended and producing the desired outcome in meeting the security or privacy requirements of a system or org
  • Audit: process of reviewing a system for compliance against a standard or baseline (e.g. audit of security controls, baselines, financial records) can be formal and independent, or informal/internal
  • Chaos Engineering: discipline of experiments on a software system in production to build confidence in the system’s capabilities to withstand turbulent/unexpected conditions
  • Code testing suite: usually used to validate function, statement, branch and condition coverage
  • Compliance Calendar: tracks an org’s audits, assessments, required filings, due dates and related
  • Compliance Tests: an evaluation that determines if an org’s controls are being applied according to management policies and procedures
  • Penetration Testing/Ethical Penentration Testing: security testing and assessment where testers actively attempt to circumvent/defaut a system’s security features; typically constrained by contracts to stay within specified Rules of Engagement (RoE)
  • Examination: process of reviewing/inspecting/observing/studying/analyzing specs/mechanisms/activities to understand, clarify, or obtain evidence
  • Findings: results created by the application of an assessment procedure
  • Functional order of controls: deter, deny, detect, delay, deterimine, and decide
  • Fuzzing: uses modified inputs to test sofware performance under unexpected circumstances; mutation fuzzing modifies known inputs to generate synthetic inputs that may trigger unexpected behavior; generational fuzzing develops inputs based on models of expected inputs to perform the same task
  • IAM system: identity and access management system combines lifecycle management and monitoring tools to ensure that identity and authorization are properly handled throughout an org
  • ITSM: IT Service Management tools include change management and associated approval tracking
  • Judgement Sampling: AKA purposive or authoritative sampling, a non-probability sampling technique where members are chosen only on the basis of the researcher’s knowledge and judgement
  • Misue Case Testing: testing strategy from a hostile actor’s point of view, attempting to lead to integrity failures, malfunctions, or other security or safety compromises
  • Mutation testing: mutation testing modifies a program in small ways and then tests that mutant to determine if it behaves as it should or if it fails; technique is used to design and test software through mutation
  • Plan of Action and Milestones (POA&M): a document indentifying tasks to be accomplished, including details, resources, milestones, and completion target dates
  • RUM: real user monitoring is a passive monitoring technique that records user interation with an app or system to ensure performance and proper app behavior; often used as a predeploymment process using the actual user interface
  • RoE: Rules of Engagement, set of rules/constraints/boundaries that establish limits of participant activity; in ethical pen testing, an RoE defines the scope of testing, and to establish liabilty limits for both testers and the sponsoring org or system owners
  • SCF: Script Check Engine is designed to make scripts interoperable with security policy definitions
  • Statistical Sampling: process of selecting subsets of examples from a population with the objective of estimating properties of the total population
  • Substantive Test: testing technique used by an auditor to obtain the audit evidence in order to support the auditor’s opinion
  • Testing: process of exersizing one or more assessment objects (activities or mechanisms) under specified conditions to compare actual to expected behaior
  • Trust Services Criteria (TSC): used by an auditor when evaluating the suitability of the design and operating effectiveness of controls relevant to the security, availabiliity, or processing integrity of information and systems or the confidentiality or privacy of the info processed by the entity

6.1 Design and validate assessment, test, and audit strategies (OSG-9 Chpt 15)

  • Security audits occur when a third-party performs an assessment of the security controls protecting an org’s information assets
  • 6.1.1 Internal
    • An organization’s security staff can perform security tests and assessments, and the results are meant for internal use only, designed to evaluate controls with an eye toward finding potential improvements
    • An internal audit strategy should be aligned to the organization’s business and day-to-day operations
      • e.g. a publicly traded company will have a more rigorous internal auditing strategy than a privately held company
    • Designing the audit strategy should include laying out applicable regulatory requirements and compliance goals
    • Internal audits are performed by an organization’s internal audit staff and are typically intended for internal audiences, and management use
  • 6.1.2 External
    • An external audit strategy should complement the internal strategy, providing regular checks to ensure that procedures are being followed and the organization is meeting its compliance goals
    • External audits are performed by an outside auditing firm
      • these audits have a high degree of external validity because the auditors performing the assessment theoretically have no conflict of interest with the org itself
      • audits by these firms are generally considered acceptable by most investors and governing bodies
      • third-party audit reporting is generally intended for the org’s governing body
  • 6.1.3 Third-party
    • Third-party audits are conducted by, or on behalf of, another org
    • In the case of a third-party audit, the org initiating the audit generally selects the auditors and designs the scope of the audit
    • The statement on Standards for Attestation Engagements document 18 (SSAE 18), titled Reporting on Controls, provides a common standard to be used by auditors performing assessments of service orgs with the intent of allowing the org to conduct external assessments, instead of multiple third-party assessments, and then sharing the resulting report with customers and potential customers
      • outside of the US, similar engagements are conducted under the International Standard for Attestation Engagements (ISAE) 3402, Assurance Reports on Controls at a Service Organization
    • SSAE 18 and ISAE 3402 engagements are commonly referred to as a service organization controls (SOC) audits
    • Three forms of SOC audits:
      • SOC 1 Engagements: assess the organization’s controls that might impact the accuracy of financial reporting
      • SOC 2 Engagements: assess the organization’s controls that affect the security (confidentiality, integrity, and availability) and privacy of information stored in a system
        • SOC 2 audit results are confidential and are usually only shared outside an org under an NDA
      • SOC 3 Engagements: assess the organization’s controls that affect the security (confidentiality, integrity, and availability) and privacy information stored in a system
        • however, SOC3 audit results are intended for public disclosure
    • Two types of SOC reports:
      • Type I Reports: provide the auditor’s opinion on the description provided by management and the suitability of the design of the controls
        • type I reports also cover only a specific point in time, rather than an extended period
        • think of Type I report as more of a documentation review
      • Type II Reports: go further and also provide the auditor’s opinion on the operating effectiveness of the controls
        • the auditor actually confirms the controls are functioning properly
        • Type II reports also cover an extended period of time, at least 6 months
        • think of Type II report as similar to a traditional audit; the auditor is checking the paperwork, and verifying the controls are functioning properly
      • Type II reports are considered much more reliable than Type I reports (Type I reports simply take the service orgs word that the controls are implemented as described)

6.2 Conduct security control testing (OSG-9 Chpt 15)

  • Security control testing can include testing of the physical facility, logical systems and applications; common testing methods:
  • 6.2.1 Vulnerability assessment
    • Vulnerabilities: weaknesses in systems and security controls that might be exploited by a threat
    • Vulnerability assessments: examining systems for these weaknesses
    • The goal of a vulnerability assessment is to identify elements in an environment that are not adequately protected – and not necessarily from a technical perspective; you can also assess the vulnerability of physical security or the external reliance on power, for instance
      • can include personnel testing, physical testing, system and network testing, and other facilities tests
    • Vulnerability assessments are some of the most important testing tools in the information security professional’s toolkit
    • Security Content Automation Protocol (SCAP): provides a common framework for discussion and facilitation of automation of interactions between different security systems (sponsored by NIST)
      • SCAP components related to vulnerability assessments:
        • Common Vulnerabilities and Exposures (CVE): provides a naming system for describing security vulnerabilities
        • Common Vulnerability Scoring Systems (CVSS): provides a standardized scoring system for describing the severity of security vulnerabilities; it includes metrics and calc tools for exploitability, impact, how mature exploit code is, and how vulnerabilities can be remediated, and a means to score vulns against users’ unqiue requirements
        • Common Configuration Enumeration (CCE): provides a naming system for system config issues
        • Common Platform Enumeration (CPE): provides a naming system for operating systems, applications, and devices
        • Extensible Configuration Checklist Description Format (XCCDF): provides a language for specifying security checklists
        • Open Vulnerability and Assessment Language (OVAL): provides a language for describing security testing procedures; used to describe the security condition of a system
    • Vulnerability scans automatically probe systems, applications, and networks looking for weaknesses that could be exploited by an attacker
      • flaws may include missing patches, misconfigurations, or faulty code
    • Four main categories of vulnerability scans:
      • network discovery scans
      • network vulnerability scans
      • web application vulnerability scans
      • database vulnerability scans
    • Authenticated scans: (AKA credentialed security scan) involves conducting vulnerability assessments and security checks on a network, system, or application using valid credentials; this approach enables the scanner to simulate the actions of an authenticated user, allowing it to access deeper layers of the target system, gather more information, and provide a more accurate assessment of vulnerabilities; often uses a read-only account to access configuration files
  • 6.2.2 Penetration testing
    • Penetration tests goes beyond vulnerability testing techniques because it actually attempts to exploit systems
    • Vulnerability management programs take the results of the tests as inputs and then implement a risk management process for identfied vulnerabilities
    • NIST defines the penetration testing process as consisting of four phases:
      • planning: includes agreement on the scope of the test and the rules of engagement
        • ensures that both the testing team and management are in agreement about the nature of the test and that it is explicitly authorized
      • information gathering and discovery: uses manual and automated tools to collect information about the target environment
        • basic reconnaissance (website mapping)
        • network discovery
        • testers probe for system weaknesses using network, web and db vuln scans
      • attack: seeks to use manual and automated exploit tools to attempt to defeat system security
        • step where pen testing goes beyond vuln scanning as vuln scans don’t attempt to actually exploit detected vulns
      • reporting: summarizes the results of the pen testing and makes recommendations for improvements to system security
    • tests are normally categorized into three groups:
      • white-box penetration test:
        • provides the attackers with detailed information about the systems they target
        • this bypasses many of the reconnaissance steps that normally precede attacks, shortening the time of the attack and increasing the likelihood that it will find security flaws
        • these tests are sometimes called “known environment” tests
        • in white-box testing, the tester has access to the source code and performss testing from a developer’s perspective
      • gray-box penetration test:
        • AKA partial knowledge tests, these are sometimes chosen to balance the advantages and disadvantages of white- and black-box penetration tests
        • this is particularly common when black-box results are desired but costs or time constraints mean that some knowledge is needed to complete the testing
        • these tests are sometimes called “partially known environment” tests
        • in gray-box testing, the tester evaluates software from a user perspective but has access to the source code
      • black-box penetration test:
        • does not provide attackers with any information prior to the attack
        • this simulates an external attacker trying to gain access to information about the business and technical environment before engaging in an attack
        • these tests are sometimes called “unknown environment” tests
  • 6.2.3 Log reviews
    • Security Information and Event Management (SIEM): packages that collect information using the syslog functionality present in many devices, operating systems, and applications
    • Admins may choose to deploy logging policies through Windows Group Policy Objects (GPOs)
    • Logging systems should also make use of the Network Time Protocol (NTP) to ensure that clocks are synchronized on systems sending log entries to the SIEM as well as the SIEM itself, ensuring info from multiple sources have a consistent timeline
    • Information security managers should also periodically conduct log reviews, particularly for sensitive functions, to ensure that privileged users are not abusing their privileges
    • Network flow (NetFlow) logs are particularly useful when investigating security incidents
  • 6.2.4 Synthetic transactions
    • Synthetic transactions: scripted transactions with known expected results
    • Synthetic monitoring: uses emulated or recorded transactions to monitor for performance changes in response time, functionality, or other performance monitors
    • Dynamic testing may include the use of synthetic transactions to verify system performance; synthetic transactions are run against code and compare out to expected state
  • 6.2.5 Code review and testing
    • Code review and testing is “one of the most critical components of a software testing program”
    • These procedures provide third-party reviews of the work performed by developers before moving code into a production environment, possibly discovering security, performance, or reliability flaws in apps before they go live and negatively impact business operations
    • In code review, AKA peer review, developers other than the one who wrote the code review it for defects; code review can be a formal or informal validation process
    • Fagan inspections: the most formal code review process follows six steps: 1) planning 2) overview 3) preparation 4) inspection 5) rework 6) follow-up
      • Entry criteria are the criteria or requirements which must be met to enter a specific process
      • Exit criteria are the criteria or requirements which must be met to complete a specific process
    • Static application security testing (SAST): evaluates the security of software without running it by analyzing either the source code or the compiled application; code reviews are an example of static app security testing
    • Dynamic application security testing (DAST): evaluates the security of software in a runtime environment and is often the only option for organizations deploying applications written by someone else
  • 6.2.6 Misuse case testing
    • Misuse case testing: AKA abuse case testing - used by software testers to evaluate the vulnerability of their software to known risks; focuses on behaviors that are not what the org desires or that are counter to the proper function of a system/app
    • In misuse case testing, testers first enumerate the known misuse cases, then attempt to exploit those use cases with manual or automated attack techniques
  • 6.2.7 Test coverage analysis
    • A test coverage analysis is used to estimate the degree of testing conducted against new software; to provide insight into how well testing covered the use cases that an app is being tested for
    • Test coverage: number of use cases tested / total number of use cases
      • requires enumerating possible use cases (which is a difficult task), and anyone using test coverage calcs to understand the process used to develop the input values
    • Five common criteria used for test coverage analysis:
      • branch coverage: has every IF statement been executed under all IF and ELSE conditions?
      • condition coverage: has every logical test in the code been executed under all sets of inputs?
      • functional coverage: has every function in the code been called and returned results?
      • loop coverage: has every loop in the code been executed under conditions that cause code execution multiple times, only once, and not at all?
      • statement coverage: has every line of code been executed during the test?
    • Test coverage report: measures how many of the test cases have been completed; is used to provide test metrics when using test cases
  • 6.2.8 Interface testing
    • Interface testing assesses the performance of modules against the interface specs to ensure that they will work together properly when all the development efforts are complete
    • Interface testing essentially assesses the interaction between components and users with API testing, user interface testing, and physical interface testing
      • Three types of interfaces should be tested:
        • application programming interfaces (APIs): offer a standardized way for code modules to interact and may be exposed to the outside world through web services
          • should test APIs to ensure they enforce all security requirements
        • user interfaces (UIs): examples include graphical user interfaces (GUIs) and command-line interfaces
          • UIs provide end users with the ability to interact with the software, and tests should include reviews of all UIs
        • physical interfaces: exist in some apps that manipulate machinery, logic controllers, or other objects
          • software testers should pay careful attention to physical interfaces because of the potential consequences if they fail
      • Also see OWASP API security top 10
  • 6.2.9 Breach attack simulations
    • Breach and attack simulation (BAS): platforms that seek to automate some aspects of penetration testing
    • The BAS platform is not actually waging attacks, but conducting automated testing of security controls to identify deficencies
    • A BAS system combines red team (attack) and blue team (defense) techniques together with automation to simulate advanced persistent threats (and other advanced threat actors) running against the environment
    • Designed to inject threat indicators onto systems and networks in an effort to trigger other security controls (e.g. place a suspicious file on a server)
      • detection and prevention controls should immediately detect and/or block this traffic as potentially malicious
    • See:
      • OWASP Web Security Testing Guide
      • OSSTMM (Open Source Security Testing Methodology Manual)
      • NIST 800-115
      • FedRAMP Penetration Test Guidance
      • PCI DSS Information Supplemental on Penetration Testing
  • 6.2.10 Compliance checks
    • Orgs should create and maintain compliance plans documenting each of their regulatory obligations and map those to the specific security controls designed to satisfy each objective
    • Compliance checks are an important part of security testing and assessment programs for regulated firms: these checks verify that all of the controls listed in a compliance plan are functioning properly and are effectively meeting regulatory requirements

6.3 Collect security process data (e.g. technical and administrative) (OSG-9 Chpts 15,18)

  • Many components of the information security program generate data that is curcial to security assessment processes; these components include:
    • Account management process
    • Management review and approval
    • Key performance and risk indicators
    • Backup verification data
    • Data generated by disaster recovery and business continuity programs
  • 6.3.1 Account management
    • Preferred attacker techniques for obtaining privilege user access include:
      • compromising an existing privileged account: mitigated through use of strong authentication (strong passwords and multifactor), and by admins use of privileged accounts only for specific tasks
      • privelege escalation of a regular account or creation of a new account: these approaches can be mitigated by paying attention to the creation, modification, and use of user accounts
  • 6.3.2 Management review and approval
    • Account management reviews ensure that users only retain authorized permissions and that unauthorized modifications do not occur
    • Full review of accounts: time-consuming to review all, and often done only for highly privileged accounts
    • Organizations that don’t have time to conduct a full review process may use sampling, but only if sampling is truely random
    • Adding accounts: should be a well-defined process, and users should sign an AUP
    • Adding, removing, and modifying accounts and permissions should be carefully controlled and documented
    • Accounts that are no longer needed should be suspended
    • ISO 9000 standards use a Plan-Do-Check-Act loop
      • plan: foundation of everything in the ISMS, determines goals and drives policies
      • do: security operations
      • check: security assessment and testing (this objective)
      • act: formally do the management review
  • 6.3.3 Key performance and risk indicators
    • Key Performance Indicators (KPIs): measures that provide significance of showing the performance of an ISMS compared to stated goals
    • Choose the factors that can show the state of security
    • Define baselines for some (or better yet all) of the factors
    • Develop a plan for periodically capturing factor values (use automation!)
    • Analyze and interpret the data and report the results
    • Key metrics or KPIs that should be monitored by security managers may vary from org to org, but could include:
      • number of open vulns
      • time to resolve vulns
      • vulnerability/defect recurrence
      • number of compromised accounts
      • number of software flaws detected in pre-production scanning
      • repeat audit findings
      • user attempts to visit known malicious sites
    • Develop a dashboard of metrics and track them
  • 6.3.4 Backup verification data
    • Managers should periodically inspect the results of backups to verify that the process functions effectively and meets the organization’s data protection needs
      • this might include reviewing logs, inspecting hash values, or requesting an actual restore of a system or file
  • 6.3.5 Training and awareness
    • Training and awareness programs play a crucial role in preparing an organization’s workforce to support information security programs
    • They educate employees about current threats and advise them on best practices for protecting information and systems under their care from attacks
    • Program should begin with initial training designed to provide foundation knowledge to employees who are joining the org or moving to a new role; the initial training should be tailored to an individual’s role
    • Training and awareness should continue to take place throughout the year, reminding employees of their responsibilities and updating them on changes to the organization’s operating environment and threat landscape
    • Use phishing simulations to evaluate the effectiveness of their security awareness programs
  • 6.3.6 Disaster Recover (DR) and Business Continuity (BC)
    • Business Continuity (BC): the processes used by an organization to ensure, holistically, that its vital business processes remain unaffected or can be quickly restored following a serious incident
    • Disaster Recovery (DR): is a subset of BC, that focuses on restoring information systems after a disaster
    • These processes need to be periodically accessed, and regular testing of disaster recovery and business continuity controls provide organizations with the assurance they are effectively protected against disruptions to business ops
    • Protection of life is of the utmost importance and should be dealt with first before attempting to save material things

6.4 Analyze test output and generate report (OSG-9 Chpt 15)

  • Step 1: review and understand the data
    • The goal of the analysis process is to proceed logically from facts to actionable info
    • A list of vulns and policy exceptions is of little value to business leaders unless it’s used in context, so once all results have been analyzed, you’re ready to start writing the official report
  • Step 2: determine the business impact of those facts
    • Ask “so what?”
  • Step 3: determine what is actionable
    • The analysis process leads to valuable results only if they are actionable
  • 6.4.1 Remediation
    • Rather than software defects, most vulnerabilities in average orgs come from misconfigured systems, inadequate policies, unsound business processes, or unaware staff
    • Vuln remediation should include all stakeholders, not just IT
  • 6.4.2 Exception handling
    • Exception handling: the process of handling unexpected activity, since software should never depend on users behaving properly
      • “expect the unexpected”, gracefully handle invalid input and improperly sequenced activity etc
    • Sometimes vulns can’t be patched in a timely manner (e.g. medical devices needing re-accreditation) and the solution is to implement compensitory controls, document the exception and decision, and revisit
      • compensitory controls: measures taken to address any weaknesses of existing controls or to compensate for the inability to meet specific security requirements due to various different constraints
      • e.g. micro-segmentation of device, access restrictions, monitoring etc
    • Exception handling may be required due to system crash as the result of patching (requiring roll-back)
  • 6.4.3 Ethical disclosure
    • While conducting security testing, cybersecurity pros may discover previously undiscovered vulns (perhaps implementing compensating controls to correct) that they may be unable to correct
    • Ethical disclosure: the idea that security pros who detect a vuln have a responsibility to report it to the vendor, providing them with enough time to patch or remediate
      • the disclosure should be made privately to the vendor providing reasonable amount of time to correct
      • if the vuln is not corrected, then public disclosure of the vuln is warrented, such that other professionals can make informed decisions about future use of the product(s)

6.5 Conduct or facilitate security audits (OSG-9 Chpt 15)

  • 6.5.1 Internal
    • Having an internal team conduct security audits has several advantages:
      • understanding the internal environment reduces time
      • an internal team can delve into all parts of systems, because they have insider knowledge
      • internal auditors can be more agile in adapting to changing needs, rescheduling failed assessment components quickly
    • Disadvantages of using an internal team to conduct security audits:
      • the team may have limited exposure to new/other methodologies (e.g. the team may have depth but not breadth of experience and knowledge)
      • potential conflicts of interest (e.g. reluctance to throw other teams under the bus and accurately report their findings)
      • audit team members may start with an agenda (say to secure funding) and overstate faults, or have interpersonal motives
  • 6.5.2 External
    • An external audit (sometimes called a second-party audit) is one conducted by (or on behalf of) a business partner
    • External audits are tied to contracts; by definition, an external audit should be scoped to include only the contractual obligations of an organization
  • 6.5.3 Third-party
    • Third-party audits are often needed to demonstrate compliance with some government regulation or industry standard
    • Advantages of having a third-party audit an organization:
      • they likely have breadth of experience auditing many types of systems, across many types of organizations
      • they are not affected by internal dynamics or org politics
    • Disadvantage of using a third-party auditor:
      • cost: third-party auditors are going to be much more costly than internal teams; this means that the organization is not likely to conduct audits as frequently
      • internal resources are still required to assist or accompany auditors, to answer questions and guide