diff --git a/leapp/reporting/__init__.py b/leapp/reporting/__init__.py index 7b87a3867..7a0e2233b 100644 --- a/leapp/reporting/__init__.py +++ b/leapp/reporting/__init__.py @@ -125,6 +125,9 @@ class Groups(BaseListPrimitive): name = 'groups' INHIBITOR = 'inhibitor' + # This is used for framework error reports + # Reports indicating failure in actors should use the FAILURE group + _ERROR = 'error' FAILURE = 'failure' ACCESSIBILITY = 'accessibility' AUTHENTICATION = 'authentication' @@ -188,6 +191,10 @@ def __init__(self, *args, **kwargs): # Groups that match _DEPRECATION_FLAGS will be shown as flags, the rest as tags _DEPRECATION_FLAGS = [Groups.INHIBITOR, Groups.FAILURE] +# NOTE(mmatuska): this a temporary solution until Groups._ERROR in the json report-schema +# is altered to account for this addition of Groups_ERROR +_UPCOMING_DEPRECATION_FLAGS = [Groups._ERROR] + class Key(BasePrimitive): """Stable identifier for report entries.""" @@ -378,12 +385,22 @@ def _create_report_object(entries): _sanitize_entries(entries) for entry in entries: entry.apply(report) + if Groups.INHIBITOR in report.get('groups', []): + if Groups._ERROR in report.get('groups', []): + # *error != inhibitor* + msg = ('Only one of Groups._ERROR and Groups.INHIBITOR can be set at once.' + ' Maybe you were looking for Groups.FAILURE?') + raise ValueError(msg) + # Before removing Flags, the original inhibitor detection worked # by checking the `flags` field; keep the `flags` field until we drop Flags completely # Currently we know that 'flags' does not exist otherwise so it's # safe to just set it report['flags'] = [Groups.INHIBITOR] + if Groups._ERROR in report.get('groups', []): + # see above for explanation + report['flags'] = [Groups._ERROR] return Report(report=report) @@ -431,6 +448,12 @@ def create_report_from_error(error_dict): Convert error json to report json """ error = ErrorModel.create(error_dict) - entries = [Title(error.message), Summary(error.details or ""), Severity('high'), Audience('sysadmin')] + entries = [ + Title(error.message), + Summary(error.details or ""), + Severity('high'), + Audience('sysadmin'), + Groups([Groups._ERROR]) + ] report = _create_report_object(entries) return json.loads(report.dump()['report']) diff --git a/leapp/utils/output.py b/leapp/utils/output.py index ae24a1d99..4bca48127 100644 --- a/leapp/utils/output.py +++ b/leapp/utils/output.py @@ -42,6 +42,19 @@ def pretty_block(text, target, end_text=None, color=Color.bold, width=60): target.write('\n') +def _print_errors_summary(errors): + width = 4 + len(str(len(errors))) + for position, error in enumerate(errors, start=1): + model = ErrorModel.create(json.loads(error['message']['data'])) + + error_message = model.message + if six.PY2: + error_message = model.message.encode('utf-8', 'xmlcharrefreplace') + + sys.stdout.write("{idx:{width}}. Actor: {actor}\n{spacer} Message: {message}\n".format( + idx=position, width=width, spacer=" " * width, message=error_message, actor=model.actor)) + + def print_error(error): model = ErrorModel.create(json.loads(error['message']['data'])) @@ -67,19 +80,6 @@ def _print_report_titles(reports): print('{idx:{width}}. {title}'.format(idx=position, width=width, title=report['title'])) -def report_inhibitors(context_id): - # The following imports are required to be here to avoid import loop problems - from leapp.utils.report import fetch_upgrade_report_messages, is_inhibitor # noqa; pylint: disable=import-outside-toplevel - reports = fetch_upgrade_report_messages(context_id) - inhibitors = [report for report in reports if is_inhibitor(report)] - if inhibitors: - text = 'UPGRADE INHIBITED' - with pretty_block(text=text, end_text=text, color=Color.red, target=sys.stdout): - print('Upgrade has been inhibited due to the following problems:') - _print_report_titles(inhibitors) - print('Consult the pre-upgrade report for details and possible remediation.') - - def report_deprecations(context_id, start=None): deprecations = get_audit_entry(event='deprecation', context=context_id) if start: @@ -110,25 +110,41 @@ def report_errors(errors): print_error(error) -def _filter_reports(reports, severity=None, is_inhibitor=False): +def _filter_reports(reports, severity=None, has_flag=None): # The following imports are required to be here to avoid import loop problems - from leapp.utils.report import is_inhibitor as isinhibitor # noqa; pylint: disable=import-outside-toplevel - if not severity: - return [report for report in reports if isinhibitor(report) == is_inhibitor] - return [report for report in reports if report['severity'] == severity and isinhibitor(report) == is_inhibitor] + from leapp.reporting import Groups # noqa; pylint: disable=import-outside-toplevel + from leapp.utils.report import has_flag_group # noqa; pylint: disable=import-outside-toplevel + + def is_ordinary_report(report): + return not has_flag_group(report, Groups._ERROR) and not has_flag_group(report, Groups.INHIBITOR) + + result = reports + if has_flag is False: + result = [rep for rep in result if is_ordinary_report(rep)] + elif has_flag is not None: + result = [rep for rep in result if has_flag_group(rep, has_flag)] + if severity: + result = [rep for rep in result if rep['severity'] == severity] + + return result def _print_reports_summary(reports): # The following imports are required to be here to avoid import loop problems - from leapp.reporting import Severity # noqa; pylint: disable=import-outside-toplevel + from leapp.reporting import Groups, Severity # noqa; pylint: disable=import-outside-toplevel + + errors = _filter_reports(reports, has_flag=Groups._ERROR) + inhibitors = _filter_reports(reports, has_flag=Groups.INHIBITOR) + + ordinary = _filter_reports(reports, has_flag=False) - inhibitors = _filter_reports(reports, is_inhibitor=True) - high = _filter_reports(reports, Severity.HIGH) - medium = _filter_reports(reports, Severity.MEDIUM) - low = _filter_reports(reports, Severity.LOW) - info = _filter_reports(reports, Severity.INFO) + high = _filter_reports(ordinary, Severity.HIGH) + medium = _filter_reports(ordinary, Severity.MEDIUM) + low = _filter_reports(ordinary, Severity.LOW) + info = _filter_reports(ordinary, Severity.INFO) print('Reports summary:') + print(' Errors: {:5}'.format(len(errors))) print(' Inhibitors: {:5}'.format(len(inhibitors))) print(' HIGH severity reports: {:5}'.format(len(high))) print(' MEDIUM severity reports: {:5}'.format(len(medium))) @@ -136,36 +152,53 @@ def _print_reports_summary(reports): print(' INFO severity reports: {:5}'.format(len(info))) -def report_info(context_id, report_paths, log_paths, answerfile=None, fail=False): +# NOTE(mmatuska): it would be nicer to get the errors from reports, +# however the reports don't have the information about which actor raised the error :( +def report_info(context_id, report_paths, log_paths, answerfile=None, fail=False, errors=None): report_paths = [report_paths] if not isinstance(report_paths, list) else report_paths log_paths = [log_paths] if not isinstance(log_paths, list) else log_paths if log_paths: - sys.stdout.write("\n") + if not errors: # there is already a newline after the errors section + sys.stdout.write("\n") for log_path in log_paths: sys.stdout.write("Debug output written to {path}\n".format(path=log_path)) if report_paths: # The following imports are required to be here to avoid import loop problems from leapp.reporting import Severity # noqa; pylint: disable=import-outside-toplevel - from leapp.utils.report import fetch_upgrade_report_messages # noqa; pylint: disable=import-outside-toplevel + from leapp.utils.report import fetch_upgrade_report_messages, Groups # noqa; pylint: disable=import-outside-toplevel + reports = fetch_upgrade_report_messages(context_id) - high = _filter_reports(reports, Severity.HIGH) - medium = _filter_reports(reports, Severity.MEDIUM) + inhibitors = _filter_reports(reports, has_flag=Groups.INHIBITOR) + + ordinary = _filter_reports(reports, has_flag=False) + high = _filter_reports(ordinary, Severity.HIGH) + medium = _filter_reports(ordinary, Severity.MEDIUM) color = Color.green if medium or high: color = Color.yellow if fail: - color = Color.bold + color = Color.red + + with pretty_block("REPORT OVERVIEW", target=sys.stdout, color=color): + if errors: + print('Following errors occurred and the upgrade cannot continue:') + _print_errors_summary(errors) + sys.stdout.write('\n') + + if inhibitors: + print('Upgrade has been inhibited due to the following problems:') + _print_report_titles(inhibitors) + sys.stdout.write('\n') - with pretty_block("REPORT", target=sys.stdout, color=color): if high or medium: print('HIGH and MEDIUM severity reports:') _print_report_titles(high + medium) + sys.stdout.write('\n') - sys.stdout.write('\n') _print_reports_summary(reports) print( diff --git a/leapp/utils/report.py b/leapp/utils/report.py index d24a17740..2db0e55f9 100644 --- a/leapp/utils/report.py +++ b/leapp/utils/report.py @@ -7,6 +7,7 @@ from leapp.reporting import ( _DEPRECATION_FLAGS, + _UPCOMING_DEPRECATION_FLAGS, Groups, Remediation, Severity, @@ -51,7 +52,7 @@ def fetch_upgrade_report_messages(context_id): """ :param context_id: ID to identify the needed messages :type context_id: str - :return: All upgrade messages of type "Report" withing the given context + :return: All upgrade messages of type "Report" within the given context """ report_msgs = get_messages(names=['Report', 'ErrorModel'], context=context_id) or [] @@ -94,17 +95,22 @@ def fetch_upgrade_report_messages(context_id): def is_inhibitor(message): + return has_flag_group(message, Groups.INHIBITOR) + + +def has_flag_group(message, group): """ - A helper function to check if a message is inhibiting upgrade. - It takes care of possible differences in report schema, like tags/flags -> groups merge in 1.2.0 + A helper to check if a message has Group which would have been a Flag before the Tags/Flags merge in 1.2.0 """ - # NOTE(ivasilev) As groups and flags are mutual exclusive, let's not bother with report_schema version + # NOTE(borrowed from ivasilev) As groups and flags are mutual exclusive, let's not bother with report_schema version # and just check both fields for inhibitor presence - return Groups.INHIBITOR in message.get('groups', message.get('flags', [])) + return group in message.get('groups', message.get('flags', [])) def importance(message): - if is_inhibitor(message): + if has_flag_group(message, Groups._ERROR): + return -1 + if has_flag_group(message, Groups.INHIBITOR): return 0 return SEVERITY_LEVELS.get(message['severity'], 99) @@ -115,8 +121,13 @@ def generate_report_file(messages_to_report, context, path, report_schema='1.1.0 if path.endswith(".txt"): with io.open(path, 'w', encoding='utf-8') as f: for message in sorted(messages_to_report, key=importance): - f.write(u'Risk Factor: {}{}\n'.format(message['severity'], - ' (inhibitor)' if is_inhibitor(message) else '')) + flag = '' + if has_flag_group(message, Groups._ERROR): + flag = '(error)' + elif has_flag_group(message, Groups.INHIBITOR): + flag = '(inhibitor)' + + f.write(u'Risk Factor: {} {}\n'.format(message['severity'], flag)) f.write(u'Title: {}\n'.format(message['title'])) f.write(u'Summary: {}\n'.format(message['summary'])) remediation = Remediation.from_dict(message.get('detail', {})) @@ -136,12 +147,13 @@ def generate_report_file(messages_to_report, context, path, report_schema='1.1.0 f.write(u'-' * 40 + '\n') elif path.endswith(".json"): with io.open(path, 'w', encoding='utf-8') as f: - # Here all possible convertions will take place + # Here all possible conversions will take place if report_schema_tuple < (1, 1, 0): # report-schema 1.0.0 doesn't have a stable report key # copy list of messages here not to mess the initial structure for possible further invocations messages_to_report = list(messages_to_report) for m in messages_to_report: + # FIXME(mmatuska): This modifies the messages as list() does shallow copy, more occurrences below m.pop('key') if report_schema_tuple < (1, 2, 0): # groups were introduced in 1.2.0, before that there was a tags/flags split @@ -149,7 +161,10 @@ def generate_report_file(messages_to_report, context, path, report_schema='1.1.0 for msg in messages_to_report: groups = msg.pop('groups', []) msg['flags'] = [g for g in groups if g in _DEPRECATION_FLAGS] - msg['tags'] = [g for g in groups if g not in _DEPRECATION_FLAGS] + msg['tags'] = [ + g for g in groups + if g not in _DEPRECATION_FLAGS and g not in _UPCOMING_DEPRECATION_FLAGS + ] if report_schema_tuple == (1, 2, 0): # DEPRECATED: flags, tags # NOTE(pstodulk): This is a temporary solution to ensure that diff --git a/packaging/leapp.spec b/packaging/leapp.spec index a5936e16d..118e253a3 100644 --- a/packaging/leapp.spec +++ b/packaging/leapp.spec @@ -13,7 +13,7 @@ # This is kind of help for more flexible development of leapp repository, # so people do not have to wait for new official release of leapp to ensure # it is installed/used the compatible one. -%global framework_version 4.0 +%global framework_version 5.0 # IMPORTANT: everytime the requirements are changed, increment number by one # - same for Provides in deps subpackage