Test plan covers all what’s of the test cycles.

• Test Environment Requirements
• Human Resource Requirements
• Applicable test classes

• Test Schedule
• Test Tools Requirements
• Environment Constraints
• Dev build acceptance criteria
• Priority & Severity definition
• Test Metrices
• Milestones
• Test Sign off Criteria
• Test Results Reporting
• Escalation and Reporting Matrix
• Test Automation
• Team Structure
• Customer Structure

Test Strategy  follows the test plan and covers all the How’s of testing

•  How various test environments would be build
•  How dev builds going to be deployed
•  Product  Internationalization Testing Strategy
•  Bugs Categorization as per priority and severity
•  Test Automation strategy
•  Test process implementation

Traceability matrix A traceability matrix is a table that correlates requirements and design documents to test documents. It is used to change tests when the source documents are changed, or to verify that the test results are correct.

Test case  A set of test data and test programs (test scripts) and their expected results. A test case validates one or more system requirements and generates a pass or fail.

Test harness The software, tools, samples of data input and output, and configurations are all referred to collectively as a test harness.

Security development lifecycle (SDL) supplements traditonal SDLC like waterfall, iterative, Agile and can also be represented by SD3 – Secure by design, secure by development, and secure by deployment . In SDL, we begin with secure design – designing the new product to be secure. That is, ensuring that the product does not jeopardize the confidentiality, integrity, or availability (CIA) of either the services offered or assets accessed by the software. Secure design occurs during the product design phase of the SDLC.

Secure coding ensures that engineers do not introduce software vulnerabilities into the code base. Software vulnerabilities are software issues with security implications, such as allowing an attacker to deny access to a service or asset. Secure coding occurs during the implementation phase of the SDLC.

Security testing serves two purposes, first to ensure that the software does not violate the CIA of services or assets, and second to ensure that security vulnerabilities are not present. Security testing is done when functionality, usability, performance, and other testing occurs.

During the deployment phase of the software, engineers address any security issues that are found.

Secure by Design- There are many useful secure design principles, including the principles of least privilege, separation of duties and rights, defense in depth, attack surface reduction (ASR), basic privacy, and threat modeling.

• Attack Surface Reduction: The attack surface of a piece of software is the parts of the program (code and functionality) that are exposed to a human user or another program. The concept of ASR (attack surface reduction) is to minimize the attack surface while still satisfying the functional requirements of the product. Secure coding will reduce but not eliminate all vulnerabilities in your program; by reducing the attack surface, you minimize the number of vulnerabilities that the attacker can discover and attempt to exploit. The attack surface is defined by the set of interfaces, or entry points, to the program. Attack surface analysis (ASA) is the process of identifying and understanding all of the entry points that make up the attack surface, and is done by enumerating all of the interfaces, protocols and code execution paths. Another important element of ASA is understanding the trust levels required to access each entry point.For each entry point, you must consider the importance of the feature that it enables. For features that are not important to a vast majority of the users, turn the feature off, disable it by default, or don’t even install it by default; force the users that really want or need it to take explicit action to obtain that feature. This way, any vulnerabilities related to that feature will affect a very small percentage of the product’s user base.Next consider which classes of users need that feature, and then restrict its use to those classes. For example, don’t default to making the feature remotely accessible, don’t default to allowing anonymous access, don’t default to running with more privilege than is needed, etc. A significant aspect of ASR is restricting who has access to a product feature, and how they can obtain and use that access.
• Privacy and Security while related, are not the same thing- but they go hand in hand when creating trust with customers.Privacy is about control and choice regarding a user’s personal information. Security is about protecting assets from threats.Obviously a security breach can result in a loss of privacy, but it is possible for a secure system to cause the loss of privacy without a breach.Consider this secure, but privacy violating scenario: Securely storing personably identifiable information and then sending that info using a securely encrypted communication channel to third parties without notification or consent of the user may be securely implemented but obviously doesn’t take into consideration the rights of the user- some rights may have legal implications!
• Threat Modelling One of the most effective techniques for developing secure products is to understand the threats to the product, its users, and its environment. This is achieved via a process/technique known as threat modeling. Threat modeling occurs in the design phase of the development lifecycle and has the goal of helping you to understand the potential security threats to the system, determine risk, and establish appropriate mitigations. The process is ever-evolving as lessons are learned about how to make it less burdensome, less subjective, and more beneficial. At a high level, Threat Modeling consists of a number of activities done during the design phase of product development. These activities begin by envisioning the product as it will be used by typical users in a typical environment, and then identifying all of the potential threats to the product and to assets accessed via the product. During this process all security related assumptions and external dependencies are documented, as are the “external security notes” – notes to help users and administrators understand the security boundaries of the product under development.Next you build the threat model by creating a number of Data Flow Diagrams (DFDs), which model the trust boundaries of the application and its components and the flow of data between the application and its environment as well as the flow of data between components within the application. The first and highest level DFD is the context diagram which models the system and its interaction with external entities. In this diagram, the product is represented as a multi-process or complex process. Now with the context diagram as a starting point, you drill down into each complex process, creating a new DFD for it, repeating this process until there are no complex processes remaining that have not been modeled by a DFD.Now that you have a completed model you determine the types of threats to the product (from the attacker’s perspective) and list all of the DFD elements. The DFD elements represent the things that need to be protected from attack.Knowing what needs to be protected, and how it will be attacked, enables you to choose appropriate mitigations for each threat. Note that we are still in the design stage of product development! We are now designing security controls into the product based upon the most likely threats; the most cost effective time to deal with such issues.

The above diagram represent a DFD. With the following conventions

DFA Elements:- Circle represents the processes, rectangles represents external entities, parallel lines represents data stores, and arrowed lines represents data flows

Using STRIDE taxonomy one can identify the type of threats at various DFD elements. STRIDE stands for S- Spoofing; T- Tampering; R- Repudition; I- Information disclosure; D- Denial of Services; E- Elevation of Privilege. The below given matrix identified STRIDE for the above given DFD.

Mitigation techniques for each type of  threat in STRIDE

Secure by Development- Some of the common programming security vulnerabilities added to the software during development phase -

1. Buffer overflow
2. Integer arithmetic errors
3. SQL Injection
4. Cross Scripting
5. Cryptography

Common programming error can be taken care by introducing “Standard Annotation Language (SAL)  meta language that helps static analysis tools … find bugs – including security bugs in the development phase. 

The main benefit of SAL is that you can find more bugs with just a little bit of upfront work.  Process of adding SAL annotations to existing code can also find bugs as the developer questions the assumptions previously made about how the function being annotated works. By this I mean that as a developer adds annotations to a function, she must think about how the function works in more detail than simply assuming it was written correctly. This process finds assumption issues.Any bugs found in SAL annotated functions tend to be real bugs, not false positives, which has the benefit of speedier bug triage and code fixes.

“Finally, SAL is highly leveraged; when you annotate a function, any code that calls that function will get the benefit of the annotation.”

Secure by Testing-“Fuzzing” is a testing methodology that uncovers security issues due to inputs that have not been properly validated. The two most common are “smart fuzzing” and “dumb fuzzing.” Smart fuzzing uses knowledge of legal inputs and varies specific values based upon knowledge of the inputs and how they are processed, while dumb fuzzing uses completely random values. Smart fuzzing requires more work on the part of the tester, but that work is rewarded with better test coverage.One useful variation is “targeted fuzzing” which is fuzzing with sets of specific input values instead of random values.

The way to fuzz test is to: 1.Identify all entry points 2.Determine the valid inputs to that entry point 3.Create a collection of valid inputs 4.Create invalid inputs by modifying each of the valid inputs 5.Feed each of the generated invalid inputs to the program.

Much of this process can be automated, so use or build your own tools to make this process more efficient.

Secure by Deployment- This is the phase where the product is been deployed for the end customer as per the deployment document. Product needs to be deployed with the minimum required network privilege,open ports and protocols that are necessary for functioning of the software.

Agile methods generally promote project management process that encourage frequent inspection and adaptation, self organized cross functional teams, thin processes. In Agile methods cost  and resources are constraints wherein requirements are trade off. 

Benefits derived from agile software development are customer focus- customers are involved in early stage of software development which enable them to provide their feedback in early stage of product development, risk surface early on, adaptive development approach help in fine tune product been developed as per the changing market and business scenarios.

Agile methods breaks the tasks into smaller unit of work with mimimal planning, typically with 4-8 weeks cycle, go through full development cycle starting with planning, requirments, design, develop, test and demo to the stakeholders to get feedback.

Agile methods emphasize face to face communication over the written document within colocated teams. Teams working in different locations need to have good communication medium into the system such as video conferencing, Instant messaging, Email, voice etc.

Execution of projects in geo distributed team become complicated and needs to be planned well for effective execution.

Some of noted challenges while executing project with multi location teams

Good Practices which can help to overall challenges faced with multi location teams.

Agile methods
Some of the well-known agile software development methods:

Agile practices
-Test Driven Development (TDD)
-Continuous Integration
-Pair Programming

Test Automation is a very critical phase in the overall product/software development cycle which has been developed in short iterations. This phase deals with identification and automation of the areas which would enable software development in short time intervals.

Depending upon the nature of application been developed, one can identify test classes suitable for test automation which would help in reducing the overall software development life cycle. 

Some of the potential classes for test automation.

- Build Verification Testing(BVTs)-  Deals with identification of minimum  required test cases to valid the build before deploying in the test environment. Potential candidates for BVTs test are the ones which help in validating complete flow of the feature.

-Feature Verification Testing(FVTs)- FVTs deals with identification of the test cases which will help in calling out if the feature is been implemented in completeness.

-User Interface Testing (UI)-  Test cases for testing UI for multilingual interface, and multi devices

-Business layer Testing- Test cases which enables testing of business layer in isolation.

-Browser Compatibility testing- Test cases identified for testing application compatibility for the identified browsers.

-Deployment Automation- Packaging all the identified test automation classes along with the build to be deployed in the environment and to generate build execution result reports identifying the success/failure of the overall build and classes.

High percentage of test automation also helps business in quick go/no go decision by looking into the metrics been derived by test automation.  – Code Coverage, – Test Coverage, -Comparing results of multiple test runs to check on stability of the developed product, coding standards enforcement metrics results, test cases effectiveness- no of bugs found by automation.

Some of the known tools which enable in doing test automation – VSTS, SilkTest, TestComplete, winrunner.

1. Cloudburst (negative): The failure of a cloud computing environment due to the inability to handle a spike in demand.
2. Cloudburst (positive): The dynamic deployment of a software application that runs on internal organizational compute resources to a public cloud to address a spike in demand.

Cloudstorming: The act of connecting multiple cloud computing environments.
Vertical Cloud: A cloud computing environment optimized for use in a particular vertical — i.e., industry — or application use case.
Private Cloud: A cloud computing-like environment within the boundaries of an organization and typically for its exclusive usage.
Internal Cloud: A cloud computing-like environment within the boundaries of an organization and typically available for exclusive use by said organization.
Hybrid Cloud: A computing environment combining both private (internal) and public (external) cloud computing environments. May either be on a continuous basis or in the form of a ‘cloudburst’.
Cloudware: A general term referring to a variety of software, typically at the infrastructure level, that enables building, deploying, running or managing applications in a cloud computing environment.
External Cloud: A cloud computing environment that is external to the boundaries of the organization. Although it often is, an external cloud is not necessarily a public cloud. Some external clouds make their cloud infrastructure available to specific other organizations and not to the public at-large.
Public Cloud: A cloud computing environment that is open for use to the general public, whether individuals, corporations or other types of organizations. Amazon Web Services are an example of a public cloud.

Cloud Provider: An organization that makes a cloud computing environment available to others, such as an external or public cloud.
Cloud Enabler: A general term that refers to organizations (typically vendors) who are not cloud providers per se, but make available technology, such as cloudware, that enables cloud computing

Cloud-Oriented Architecture (COA): An architecture for IT infrastructure and software applications that is optimized for use in cloud computing environments. The term is not yet in wide use, and as is the case for the term “cloud computing” itself, there is no common or generally accepted definition or specific description of a cloud-oriented architecture.
Cloud Service Architecture (CSA): A term coined by Jeff Barr, chief evangelist at Amazon Web Services. The term describes an architecture in which applications and application components act as services on the cloud, which serve other applications within the same cloud environment.
Virtual Private Cloud (VPC): A term coined by Reuven Cohen, CEO and founder of Enomaly. The term describes a concept that is similar to, and derived from, the familiar concept of a Virtual Private Network (VPN), but applied to cloud computing. It is the notion of turning a public cloud into a virtual private cloud, particularly in terms of security and the ability to create a VPC across components that are both within the cloud and external to it.
Cloud Portability: The ability to move applications (and often their associated data) across cloud computing environments from different cloud providers, as well as across private or internal cloud and public or external clouds.