TODO.org

<2014-06-30 Mån> [0/2]

• [ ] Fixed documentation.
• [ ] Use click instead of docopt
  • [ ] This will change the structure
  • [ ] Tests will need to be updated
  • [ ] New possibility to do tests in a different way?
• [ ] How to push a tag?

<2014-06-29 Sön> [5/5]

• [X] Fixed som PEP issues.
• [X] Convert to a package
• [X] Refactor test scripts
• [X] Update README.md
• [X] Update DESCRIPTION.rst

<2014-06-29 Sön>

Use git flow approach. A simple reminder is

http://jeffkreeftmeijer.com/2010/why-arent-you-using-git-flow/

<2014-05-31 Lör>

Refactoring according to TODO.

<2014-05-30 Fre>

Refactor code to also work with python2.7

Bump version to 1.20 and push these changes

<2014-03-09 Sön>

Plan and follow through some ToDo items

Start using some simple org-mode TODO commands.

<2014-03-08 Sat>

PEP8 checking

Different scores need refactoring [1/4]

• [ ] Use DRY on vulnerability.py and cvss.py to factor out data

related to scores.

• Currently quite a lot of repeated data. For example see BASE_SCORE and base_metrics()
• One problem is the mix of basic data and information used for the interactive user interface.
• The mix of lists and tuples when initializing all Metrics works, but is not very clear and a bit diffcult to change because of duplication and repetition.

• [X] First: move them all to the same file and make that work
• [ ] Second: rearrange the data so that the info is more clearly structured.
• [ ] DRY enough?

Refactor cvss.py [0/5]

• [ ] Should contain only the logic for the command line syntax. The rest should be factored out into other files.
• [ ] Change from docopt to click for more precision
• [ ] Reading and processing input interactively
• [ ] Display calculated scores
• [ ] Setup and creation of a VulnerabilityVector
• [ ] Setup and creation of the CommonVulnerabilityScore

<2014-03-07 Fri>

Should transform this to a packgage with cvss.py [4/4]

• [X] Why a package? So I can distribute it via PyPi
• [X] cvss.py the main program
• [X] The examples should no longer need soft links to run
• [X] Should be possible to package cvss.py as a stand alone program

test_vulnerability.py och vulnerability.md should move [3/3]

• [X] Moved to directory examples
• [X] How should the doctests import?
• [X] How can these imports be made location independent?

README.md needs to be updated to reflect new structure

Scripts in tests need to be refactored [8/8]

In particular they should be made more location independent.

• [X] test_uc01.sh
• [X] test_uc02.sh
• [X] test_uc03.sh
• [X] test_uc04.sh
• [X] test_uc05.sh
• [X] test_uc06.sh
• [X] run_all_tests.sh
• [X] run_doctests.sh

General clean-up and restructuring

Commit the changes that were made in november.

Completed, but maybe not in a perfect way. Still need to understand how git flow och github interacts.

2013-11-14

Extract VulnerabilityVector from cvss.py

Break the dependency on cvss_210 used for checking.

Tested. DONE.

2013-11-12

With the command line interface stablized, focus should now shift to ensure that the code better reflects the problem domain. A vulnerability vector and the corresponding score algorithm are the central items. Interactive input of a vector and output in different formats should use these abstractions.

So, the vulnerability is the container and the CommonVulnerabilityScore is the algorithm that caculates the score given a vulnerability vector.

DRY and SHY and Tell the other guy.

2013-11-12

base_metrics() and ilk functions should be classes

Extract these to metric.py (pdding, prepare factory, select, read_and_set).

display and output should move to separate class or file

refactor main program

More functions. Keep knowledge about docopt in main program.

vulnerability_vector could also be a class

Done, but need to be moved inte separate file.

2013-11-09

UseCases incoporated.

Installation instructions updated.

2013-11-08

Next step? Would like to have better interactive features. Use cmd?

Less code in main program. Should read more easily.

parse a vector interactively and return a score

when complete otherwise allow for interactive input

Start using cmd

Convert the main-loop to use cmd.Cmd when interactive. Print and vector as data is completed.

score base temporal environmental caclulate scores and print vectors

base_parameters args* temporal_parameters args environmental_parameters check args use acceptable ones and loop over remaining

rst output

Generate verbose output in a RST format.

plain text output

2013-11-07

Refactor using cmd and docopt

Refactor cli (cli2)

Simplify! Tested. DONE.

Normalize output

Print scores and vectors. Tested. DONE.

Parse vectors and calculate scores

Score. Use default values when none available.

Extract score_from function. Tested, DONE.

2013-11-06

Refactor using cmd and docopt (cmd_and_docopt)

Extract functions from main program

Tested. DONE.

Start using docopt for basic command line control

Add -v –verbose to print tables. Default is to print scores and vectors. Tested. DONE.

Add more command line options

Tested. DONE.

2013-11-02

Command line interafce (cli)

Create a parser for the available metrics. The output should be a sequence of selected values that are ordered in the same way as the available metrics.

Reorganize source files, the main program should reside in cvss.py

Tested. Done.

A cli with 14 is a bit much. Do the interactive version first.

Interactive implemented. Tested. Done.

Create a public project on github.

Done.

2013-11-02

cvss_refactor_3

Template patterna

The CommonVulnerabilityScore should be the base and the cvssv210 should be the concrete class.

Done. Changed names and added version property. Tested.

cvss_refactor_2

Add function to make addressing with short names possible.

Make it private. Tested. Done.

Extract formulas into a separate class

Tested. Done.

2013-11-01

cvss_refactor_1

DRY when it it comes to initial metric values and metric short names. As the metrics are prepared the mapping could be constructed.

Metric changed. CommonVulnerabilityScore changed. No more DRY. Added short_name in input data with Metric.short_name solved this. Tested. Done.

metric_refactor_2

cvs_factory and helper functions should be simpler

Magic number should be removed. Helpers do too much. Let them just encapsulate the values. Pad selected and loop using zip an enumerate.

Keep initial data in helper functions.

Change factory to refelct this. Done. Tested.

Refactor cvs_factory

No good calling pop on selected. Extracted helper functions. Moved cvs_factory and helper functions to cvss.py

Refactor CommonVulnerabilityScore ctor

Moved the mapping to a helper function.

2013-10-31

CommonVulnerabilityScore

Should use the template pattern for the different scores and behave like a dict that has a fixed set of keys with a Metric value. How should these keys be initialized and where? DRY.

metric_refactor_1

A Metric has a name, a string value and a float value and can can be in one of several different states. These states are set when the Metric is constructed. Once initilized, different states can be selected but otherwise the Metric can not be changed.

Indexing use string key

Change so that indexing is done by a string key that is one of the allowed values. It is not robust to have the order of intialization controlling how we set the value. Use OrdereDict?

Tested. Done.

**

metric_refactor_2

cvs_factory and helper functions should be simpler

Magic number should be removed. Helpers do too much. Let them just encapsulate the values. Pad selected and loop using zip an enumerate.

2013-10-30

Need to decide on clearer way to store Metrics and their values.

Done, for now.

Recreate the tests from the examples in the specification.

Done.

CommonVulnerabilityScore is constructed with a set of Metrics.

It can calculate the different scores and the respective vectors.

These have been implemented in a first version. Looking at how MetricVAlue and a Metric is constructed it is not good that the formulas use index values that must be given on construction. This mapping should be moved to the class that uses the index values. The problem is that the metric short name is used inside Metric. To make this happen do the following:

Create a mapping in CommonVulnerabilityScore use it internally.

Created mapping from Metric to short name for matric. Tested. Done.

Remove the need for short_name in Metric.

Removed short_name in Metric. Tested. Done

Change interface of Metric.

Fixed. Tested. Done.

Parsing a command line or interactive input is a different matter.

Need a factory for CommonVulnerabilityScore to ease the pain of building. Let it take a number of Metrics with their respective MetricValues. Fix the at a later stage.

cvs_factory

First version in place. Added a default test. Tested. Done.

cvs_factory refactored

Group into three groups. Loop. Tested. Done. Not too happy about this one.

cvs_factory refactored

Does not look right. Need a simpler way. I want it to be easier to set the actual value of a metric. Is there a better data structure for this. Maybe dict or namedtuple?

cvs_factory refactored II

Function moved over to metric.py. Less coupling. Tested. Done.

cvs_factory refactored III

Metric refactored. Find a way to make selections in a cvs. Tested. All tests changed. Done.

What should the data look like to build the Metrics?

Use namedtuple for MetricValue and or Metric?

update tests

2013-10-29

CVSS, how does it work?

Base metrics are assigned values, the base equation calculates a score ranging from 0 to 10, and a vector is created. The vector facilitates the “open” nature of the framework. It is a text string that contains the values assigned to each metric, and it is used to communicate exactly how the score for each vulnerability is derived.

The base score can be refined by assigning values to the temporal and environmental metrics. This is useful in order to provide additional context for a vulnerability by more accurately reflecting the risk posed by the vulnerability to a user’s environment.

For a temporal score, the temporal equation will combine the temporal metrics with the base score to produce a temporal score ranging from 0 to 10. Similarly, if an environmental score is needed, the environmental equation will combine the environmental metrics with the temporal score to produce an environmental score ranging from 0 to 10.

Base, Temporal, Environmental Vectors

Each metric in the vector consists of the abbreviated metric name, followed by a “:” (colon), then the abbreviated metric value. The vector lists these metrics in a predetermined order, using the “/” (slash) character to separate the metrics. If a temporal or environmental metric is not to be used, it is given a value of “ND” (not defined). The base, temporal, and environmental vectors are shown below in Table 13.

Metric Value Description Base AV:[L,A,N]/AC:[H,M,L]/Au:[M,S,N]/C:[N,P,C]/I:[N,P,C]/A:[N,P,C] Temporal E:[U,POC,F,H,ND]/RL:[OF,TF,W,U,ND]/RC:[UC,UR,C,ND] Environmental CDP:[N,L,LM,MH,H,ND]/TD:[N,L,M,H,ND]/CR:[L,M,H,ND]/ IR:[L,M,H,ND]/AR:[L,M,H,ND]

Table 13: Base, Temporal and Environmental Vectors

For example, a vulnerability with base metric values of “Access Vector: Low, Access Complexity: Medium, Authentication: None, Confidentiality Impact: None, Integrity Impact: Partial, Availability Impact: Complete” would have the following base vector: “AV:L/AC:M/Au:N/C:N/I:P/A:C.”

Equations

Scoring equations and algorithms for the base, temporal and environmental metric groups are described below. Further discussion of the origin and testing of these equations is available at http://www.first.org/cvss. There are three set of Equations:

Base Equation

The base equation is the foundation of CVSS scoring. The base equation (formula version 2.10) is:

BaseScore = round_to_1_decimal(

(0.6*Impact + 0.4*Exploitability - 1.5)* f(Impact) )

Impact = 10.41*(1-(1-ConfImpact)*(1-IntegImpact)*(1-AvailImpact))

Exploitability = 20 * AccessVector * AccessComplexity * Authentication

f(Impact)= 0 if Impact=0, 1.176 otherwise

AccessVector = case AccessVector of

requires local access: 0.395 adjacent network accessible: 0.646 network accessible: 1.0

AccessComplexity = case AccessComplexity of

high: 0.35 medium: 0.61 low: 0.71

Authentication = case Authentication of

requires multiple instances of authentication: 0.45 requires single instance of authentication: 0.56 requires no authentication: 0.704

ConfImpact = case ConfidentialityImpact of

none: 0.0 partial: 0.275 complete: 0.660

IntegImpact = case IntegrityImpact of

none: 0.0 partial: 0.275 complete: 0.660

AvailImpact = case AvailabilityImpact of

none: 0.0 partial: 0.275 complete: 0.660

Temporal Equation

If employed, the temporal equation will combine the temporal metrics with the base score to produce a temporal score ranging from 0 to 10. Further, the temporal score will produce a temporal score no higher than the base score, and no greater than 33% lower than the base score. The temporal equation is:

TemporalScore = round_to_1_decimal(BaseScore*Exploitability

*RemediationLevel*ReportConfidence)

Exploitability = case Exploitability of

unproven: 0.85 proof-of-concept: 0.9 functional: 0.95 high: 1.00 not defined: 1.00

RemediationLevel = case RemediationLevel of

official-fix: 0.87 temporary-fix: 0.90 workaround: 0.95 unavailable: 1.00 not defined: 1.00

ReportConfidence = case ReportConfidence of

unconfirmed: 0.90 uncorroborated: 0.95 confirmed: 1.00 not defined: 1.00

Environmental Equation

If employed, the environmental equation will combine the environmental metrics with the temporal score to produce an environmental score ranging from 0 to 10. Further, this equation will produce a score no higher than the temporal score. The environmental equation is:

EnvironmentalScore = round_to_1_decimal(

(AdjustedTemporal+ (10-AdjustedTemporal)*CollateralDamagePotential)*TargetDistribution )

AdjustedTemporal = TemporalScore recomputed with the BaseScores Impact

sub-equation replaced with the AdjustedImpact equation

AdjustedImpact = min(10,10.41*(1-(1-ConfImpact*ConfReq)*(1-IntegImpact*IntegReq)

*(1-AvailImpact*AvailReq)))

CollateralDamagePotential = case CollateralDamagePotential of

none: 0 low: 0.1 low-medium: 0.3 medium-high: 0.4 high: 0.5 not defined: 0

TargetDistribution = case TargetDistribution of

none: 0 low: 0.25 medium: 0.75 high: 1.00 not defined: 1.00

ConfReq = case ConfReq of

low: 0.5 medium: 1.0 high: 1.51 not defined: 1.0

IntegReq = case IntegReq of

low: 0.5 medium: 1.0 high: 1.51 not defined: 1.0

AvailReq = case AvailReq of

low: 0.5 medium: 1.0 high: 1.51 not defined: 1.0

First Action Plan

Program

The program reads an number of key-value pairs and from these calculates a score based on the values read and their respective weight. It also prints a vulnerability vector.

How to test

In http://www.first.org/cvss/cvss-guide.html there are a number of examples. That can be used to verify.

At http://nvd.nist.gov/cvss.cfm?calculator&adv&version=2 we can calculate scores using a web interface and get a number of vectors.

At http://jvnrss.ise.chuo-u.ac.jp/jtg/cvss/en/CVSSv2.html there is another calculator. Can be sued to compare.

Next step

Rewrite code to use python3. Add tests and a command line interface using standard python modules. When this is done add an interactive mode.

Integartion with emacs

Use pymacs or write your own elsip interactive function.

Alternatives

docopt, 7 Python libs.

First reading

No documentation and a number of global variables. Appear to break the DRY-principle in several places.

A set of global variables (acc_vec, …) where the order seems to matter according to a comment. Why?

Three functions and a very long main function. Hard to modify and extend.

is_valid_input : mixing all metric_value abbreviations in one big if. Cute and brittle. Adding a metric means that the function has to change.

cvss_score : input parameter ib not used? Formula unclear and obfuscated.

find_risk : not scalable and not very pythonic.

main : just too long. Separate presenation, calculation and logic.

no tests : nada, niente, rien…