[16.0][MIG][spec_driven_model]

.pre-commit-config.yaml

@@ -3,6 +3,7 @@ exclude: |
   # NOT INSTALLABLE ADDONS
   # END NOT INSTALLABLE ADDONS
   ^l10n_br_nfe_spec/models/v4_0/|  # (don't reformat generated code)
+  ^spec_driven_model/tests/|  # (tests include generated code)
   # Files and folders generated by bots, to avoid loops
   ^setup/|/static/description/index\.html$|
   # We don't want to mess with tool-generated files

oca_dependencies.txt

@@ -1,3 +1,4 @@
 web
 account-payment
 account-reconcile
+odoo-test-helper

setup/spec_driven_model/odoo/addons/spec_driven_model

@@ -0,0 +1 @@
+../../../../spec_driven_model

setup/spec_driven_model/setup.py

@@ -0,0 +1,6 @@
+import setuptools
+
+setuptools.setup(
+    setup_requires=['setuptools-odoo'],
+    odoo_addon=True,
+)

spec_driven_model/README.rst

@@ -0,0 +1,160 @@
+=================
+Spec Driven Model
+=================
+
+.. 
+   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+   !! This file is generated by oca-gen-addon-readme !!
+   !! changes will be overwritten.                   !!
+   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+   !! source digest: sha256:7a60762120de2d0aa75c133393ab92467fdfd07bc140b199f292a250b0ffe461
+   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+
+.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
+    :target: https://odoo-community.org/page/development-status
+    :alt: Beta
+.. |badge2| image:: https://img.shields.io/badge/licence-LGPL--3-blue.png
+    :target: http://www.gnu.org/licenses/lgpl-3.0-standalone.html
+    :alt: License: LGPL-3
+.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fl10n--brazil-lightgray.png?logo=github
+    :target: https://github.com/OCA/l10n-brazil/tree/16.0/spec_driven_model
+    :alt: OCA/l10n-brazil
+.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
+    :target: https://translation.odoo-community.org/projects/l10n-brazil-16-0/l10n-brazil-16-0-spec_driven_model
+    :alt: Translate me on Weblate
+.. |badge5| image:: https://img.shields.io/badge/runboat-Try%20me-875A7B.png
+    :target: https://runboat.odoo-community.org/builds?repo=OCA/l10n-brazil&target_branch=16.0
+    :alt: Try me on Runboat
+
+|badge1| |badge2| |badge3| |badge4| |badge5|
+
+Intro
+~~~~~
+
+This module is a databinding framework for Odoo and XML data: it allows to go from XML to Odoo objects back and forth. This module started with the `GenerateDS <https://www.davekuhlman.org/generateDS.html>`_  pure Python databinding framework and is now being migrated to xsdata. So a good starting point is to read `the xsdata documentation here <https://xsdata.readthedocs.io/>`_
+
+But what if instead of only generating Python structures from XML files you could actually generate full blown Odoo objects or serialize Odoo objects back to XML? This is what this module is for!
+
+First you should generate xsdata Python binding libraries you would generate for your specific XSD grammar, the Brazilian Electronic Invoicing for instance, or UBL.
+
+Second you should generate Odoo abstract mixins for all these pure Python bindings. This can be achieved using `xsdata-odoo <https://github.com/akretion/xsdata-odoo>`_. An example is OCA/l10n-brazil/l10n_br_nfe_spec for the Brazilian Electronic Invoicing.
+
+SpecModel
+~~~~~~~~~
+
+Now that you have generated these Odoo abstract bindings you should tell Odoo how to use them. For instance you may want that your electronic invoice abstract model matches the Odoo `res.partner` object. This is fairly easy, you mostly need to define an override like::
+
+
+  from odoo.addons.spec_driven_model.models import spec_models
+
+
+  class ResPartner(spec_models.SpecModel):
+      _inherit = [
+          'res.partner',
+          'partner.binding.mixin',
+      ]
+
+Notice you should inherit from `spec_models.SpecModel` and not the usual `models.Model`.
+
+**Field mapping**: You can then define two ways mapping between fields by overriding fields from Odoo or from the binding and using `_compute=` , `_inverse=` or simply `related=`.
+
+**Relational fields**: simple fields are easily mapped this way. However what about relational fields? In your XSD schema, your electronic invoice is related to the `partner.binding.mixin` not to an Odoo `res.partner`. Don't worry, when `SpecModel` classes are instanciated for all relational fields, we look if their comodel have been injected into some existing Odoo model and if so we remap them to the proper Odoo model.
+
+**Field prefixes**: to avoid field collision between the Odoo fields and the XSD fields, the XSD fields are prefixed with the name of the schema and a few digits representing the schema version (typically 2 digits). So if your schema get a minor version upgrade, the same fields and classes are used. For a major upgrade however new fields and classes may be used so data of several major versions could co-exist inside your Odoo database.
+
+
+StackedModel
+~~~~~~~~~~~~
+
+Sadly real life XML is a bit more complex than that. Often XML structures are deeply nested just because it makes it easier for XSD schemas to validate them! for instance an electronic invoice line can be a nested structure with lots of tax details and product details. In a relational model like Odoo however you often want flatter data structures. This is where `StackedModel` comes to the rescue! It inherits from `SpecModel` and when you inherit from `StackedModel` you can inherit from all the generated mixins corresponding to the nested XML tags below some tag (here `invoice.line.binding.mixin`). All the fields corresponding to these XML tag attributes will be collected in your model and the XML parsing and serialization will happen as expected::
+
+
+  from odoo.addons.spec_driven_model.models import spec_models
+
+
+  class InvoiceLine(spec_models.StackedModel):
+      _inherit = [
+          'account.move.line',
+          'invoice.line.binding.mixin',
+      ]
+      _stacked = 'invoice.line.binding.mixin'
+
+All many2one fields that are required in the XSD (xsd_required=True) will get their model stacked automatically and recursively. You can force non required many2one fields to be stacked using the `_force_stack_paths` attribute. On the contrary, you can avoid some required many2one fields to be stacked using the `stack_skip` attribute.
+
+
+Hooks
+~~~~~
+
+Because XSD schemas can define lot's of different models, spec_driven_model comes with handy hooks that will automatically make all XSD mixins turn into concrete Odoo model (eg with a table) if you didn't inject them into existing Odoo models.
+
+**Table of contents**
+
+.. contents::
+   :local:
+
+Usage
+=====
+
+See my detailed OCA Days explanations here:
+https://www.youtube.com/watch?v=6gFOe7Wh8uA
+
+You are also encouraged to look at the tests directory which features a full blown example from the famous PurchaseOrder.xsd from Microsoft tutorials.
+
+Known issues / Roadmap
+======================
+
+Migrate from generateDS to xsdata; see the xsdata Pull Requests in the repo.
+
+Bug Tracker
+===========
+
+Bugs are tracked on `GitHub Issues <https://github.com/OCA/l10n-brazil/issues>`_.
+In case of trouble, please check there if your issue has already been reported.
+If you spotted it first, help us to smash it by providing a detailed and welcomed
+`feedback <https://github.com/OCA/l10n-brazil/issues/new?body=module:%20spec_driven_model%0Aversion:%2016.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.
+
+Do not contact contributors directly about support or help with technical issues.
+
+Credits
+=======
+
+Authors
+~~~~~~~
+
+* Akretion
+
+Contributors
+~~~~~~~~~~~~
+
+* `AKRETION <https://akretion.com/pt-BR/>`_:
+
+  * Raphaël Valyi <[email protected]>
+
+* `KMEE <https://kmee.com.br>`_:
+
+  * Gabriel Cardoso de Faria <[email protected]>
+
+Maintainers
+~~~~~~~~~~~
+
+This module is maintained by the OCA.
+
+.. image:: https://odoo-community.org/logo.png
+   :alt: Odoo Community Association
+   :target: https://odoo-community.org
+
+OCA, or the Odoo Community Association, is a nonprofit organization whose
+mission is to support the collaborative development of Odoo features and
+promote its widespread use.
+
+.. |maintainer-rvalyi| image:: https://github.com/rvalyi.png?size=40px
+    :target: https://github.com/rvalyi
+    :alt: rvalyi
+
+Current `maintainer <https://odoo-community.org/page/maintainer-role>`__:
+
+|maintainer-rvalyi| 
+
+This module is part of the `OCA/l10n-brazil <https://github.com/OCA/l10n-brazil/tree/16.0/spec_driven_model>`_ project on GitHub.
+
+You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

spec_driven_model/__init__.py

@@ -0,0 +1 @@
+from . import models

spec_driven_model/__manifest__.py

@@ -0,0 +1,17 @@
+# Copyright 2019 Akretion
+# License LGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
+
+{
+    "name": "Spec Driven Model",
+    "summary": """
+        Tools for specifications driven mixins (from xsd for instance)""",
+    "version": "16.0.1.0.0",
+    "maintainers": ["rvalyi"],
+    "license": "LGPL-3",
+    "author": "Akretion,Odoo Community Association (OCA)",
+    "website": "https://github.com/OCA/l10n-brazil",
+    "depends": [],
+    "data": [],
+    "demo": [],
+    "development_status": "Beta",
+}

spec_driven_model/hooks.py

@@ -0,0 +1,192 @@
+# Copyright (C) 2019-TODAY - Raphaël Valyi Akretion
+# License LGPL-3.0 or later (https://www.gnu.org/licenses/lgpl-3.0.en.html).
+
+import inspect
+import logging
+import sys
+
+from odoo import SUPERUSER_ID, api, models
+
+from .models.spec_models import SpecModel, StackedModel
+
+_logger = logging.getLogger(__name__)
+
+
+def post_init_hook(cr, registry, module_name, spec_module):
+    """
+    Automatically generate access rules for spec models
+    """
+    env = api.Environment(cr, SUPERUSER_ID, {})
+    remaining_models = get_remaining_spec_models(cr, registry, module_name, spec_module)
+    fields = [
+        "id",
+        "name",
+        "model_id/id",
+        "group_id/id",
+        "perm_read",
+        "perm_write",
+        "perm_create",
+        "perm_unlink",
+    ]
+    access_data = []
+    for model in remaining_models:
+        underline_name = model.replace(".", "_")
+        model_id = "%s_spec.model_%s" % (
+            module_name,
+            underline_name,
+        )
+        access_data.append(
+            [
+                "access_%s_user" % (underline_name,),
+                underline_name,
+                model_id,
+                "%s.group_user" % (module_name,),
+                "1",
+                "0",
+                "0",
+                "0",
+            ]
+        )
+        access_data.append(
+            [
+                "access_%s_manager" % (underline_name,),
+                underline_name,
+                model_id,
+                "%s.group_manager" % (module_name,),
+                "1",
+                "1",
+                "1",
+                "1",
+            ]
+        )
+    env["ir.model.access"].load(fields, access_data)
+
+
+def get_remaining_spec_models(cr, registry, module_name, spec_module):
+    """
+    Figure out the list of spec models not injected into existing
+    Odoo models.
+    """
+    cr.execute(
+        """select ir_model.model from ir_model_data
+               join ir_model on res_id=ir_model.id
+               where ir_model_data.model='ir.model'
+               and module=%s;""",
+        (module_name,),
+    )
+    module_models = [
+        i[0]
+        for i in cr.fetchall()
+        if registry.get(i[0]) and not registry[i[0]]._abstract
+    ]
+
+    injected_models = set()
+    for model in module_models:
+        base_class = registry[model]
+        # 1st classic Odoo classes
+        if hasattr(base_class, "_inherit"):
+            injected_models.add(base_class._name)
+            for cls in base_class.mro():
+                if hasattr(cls, "_inherit") and cls._inherit:
+                    if isinstance(cls._inherit, list):
+                        inherit_list = cls._inherit
+                    else:
+                        inherit_list = [cls._inherit]
+                    for inherit in inherit_list:
+                        if inherit.startswith("spec.mixin."):
+                            injected_models.add(cls._name)
+
+    # visit_stack will now need the associated spec classes
+    injected_classes = set()
+    remaining_models = set()
+
+    for m in injected_models:
+        c = SpecModel._odoo_name_to_class(m, spec_module)
+        if c is not None:
+            injected_classes.add(c)
+
+    for model in module_models:
+        base_class = registry[model]
+        # 2nd StackedModel classes, that we will visit
+        if hasattr(base_class, "_stacked"):
+            node = SpecModel._odoo_name_to_class(base_class._stacked, spec_module)
+
+            env = api.Environment(cr, SUPERUSER_ID, {})
+            for (
+                _kind,
+                klass,
+                _path,
+                _field_path,
+                _child_concrete,
+            ) in base_class._visit_stack(env, node):
+                injected_classes.add(klass)
+
+    all_spec_models = {
+        c._name
+        for name, c in inspect.getmembers(sys.modules[spec_module], inspect.isclass)
+        if c._name in registry
+    }
+
+    remaining_models = remaining_models.union(
+        {i for i in all_spec_models if i not in [c._name for c in injected_classes]}
+    )
+    return remaining_models
+
+
+def register_hook(env, module_name, spec_module, force=False):
+    """
+    Called by Model#_register_hook once all modules are loaded.
+    Here we take all spec models that are not injected in existing concrete
+    Odoo models and we make them concrete automatically with
+    their _auto_init method that will create their SQL DDL structure.
+    """
+    load_key = "_%s_loaded" % (spec_module,)
+    if hasattr(env.registry, load_key) and not force:  # already done for registry
+        return
+    setattr(env.registry, load_key, True)
+
+    remaining_models = get_remaining_spec_models(
+        env.cr, env.registry, module_name, spec_module
+    )
+    for name in remaining_models:
+        spec_class = StackedModel._odoo_name_to_class(name, spec_module)
+        spec_class._module = "fiscal"  # TODO use python_module ?
+        fields = env[spec_class._name].fields_get_keys()
+        rec_name = next(
+            filter(
+                lambda x: (
+                    x.startswith(env[spec_class._name]._field_prefix)
+                    and "_choice" not in x
+                ),
+                fields,
+            )
+        )
+        inherit = list(spec_class._inherit) + ["spec.mixin"]
+        c = type(
+            name,
+            (SpecModel, spec_class),
+            {
+                "_name": name,
+                "_inherit": inherit,
+                "_original_module": "fiscal",
+                "_odoo_module": module_name,
+                "_spec_module": spec_module,
+                "_rec_name": rec_name,
+                "_module": module_name,
+            },
+        )
+        models.MetaModel.module_to_models[module_name] += [c]
+
+        # now we init these models properly
+        # a bit like odoo.modules.loading#load_module_graph would do.
+        c._build_model(env.registry, env.cr)
+
+        env[name]._prepare_setup()
+        env[name]._setup_base()
+        env[name]._setup_fields()
+        env[name]._setup_complete()
+
+    hook_key = "_%s_need_hook" % (module_name,)
+    if hasattr(env.registry, hook_key) and getattr(env.registry, hook_key):
+        env.registry.init_models(env.cr, remaining_models, {"module": module_name})
+        setattr(env.registry, hook_key, False)