Editing Transformation rule meta data#1501

#1501

birds_nest/pybirdai/templates/pybirdai/bird_diffs_and_corrections.html

@@ -24,11 +24,11 @@ <h1 style="text-align: center;">BIRD Export Diffs and Corrections</h1>
         <span>Export Database to CSV Files</span>
     </a>
     <a href="{% url 'pybirdai:import_members_from_csv' %}" class="task-item">
-        <img src="{% static 'images/import_csv.webp' %}" alt="Import Members">
+        <img src="{% static 'images/import_members.webp' %}" alt="Import Members">
         <span>Import Members from CSV File</span>
     </a>
     <a href="{% url 'pybirdai:import_variables_from_csv' %}" class="task-item">
-        <img src="{% static 'images/import_csv.webp' %}" alt="Import Variables">
+        <img src="{% static 'images/import_variables.webp' %}" alt="Import Variables">
         <span>Import Variables from CSV File</span>
     </a>
     <a href="{% url 'pybirdai:convert_ldm_to_sdd_hierarchies' %}" class="task-item">

birds_nest/pybirdai/templates/pybirdai/create_transformation_rules_in_python.html

@@ -22,14 +22,6 @@ <h1 style="text-align: center;">Create Python Transformation Rules from SMCubes<
 <div class="task-grid">
 
 
-    <a href="{% url 'pybirdai:run_create_executable_filters' %}" class="task-item">
-        <img src="{% static 'images/executable.webp' %}" alt="executable">
-        <span>Create Executable Filters (Fast in Memory)</span>
-    </a>
-    <a href="{% url 'pybirdai:run_create_python_joins' %}" class="task-item">
-        <img src="{% static 'images/executable.webp' %}" alt="executable">
-        <span>Create Executable Joins (Fast in Memory)</span>
-    </a>
     <a href="{% url 'pybirdai:run_create_executable_filters_from_db' %}" class="task-item">
         <img src="{% static 'images/executable.webp' %}" alt="executable">
         <span>Create Executable Filters (Slower from DB)</span>
@@ -38,6 +30,15 @@ <h1 style="text-align: center;">Create Python Transformation Rules from SMCubes<
         <img src="{% static 'images/executable.webp' %}" alt="executable">
         <span>Create Executable Joins (Slower from DB)</span>
     </a>
+    <a href="{% url 'pybirdai:run_create_executable_filters' %}" class="task-item">
+        <img src="{% static 'images/executable.webp' %}" alt="executable">
+        <span>Create Executable Filters (Fast in Memory, Experimental)</span>
+    </a>
+    <a href="{% url 'pybirdai:run_create_python_joins' %}" class="task-item">
+        <img src="{% static 'images/executable.webp' %}" alt="executable">
+        <span>Create Executable Joins (Fast in Memory, Experimental)</span>
+    </a>
+
     <a href="{% url 'pybirdai:derivation_transformation_rules' %}" class="task-item">
         <img src="{% static 'images/formula.webp' %}" alt="formula">
         <span>Change Psuedocode for Derivation Transformation Rules</span>

birds_nest/pybirdai/templates/pybirdai/derivation_transformation_rules.html

@@ -7,9 +7,10 @@ <h1>Change Psuedocode for Derivation Transformation Rules</h1>
 
 <p> PyBIRD AI generates all the code for calling the generation transformation rules and executable derivation rules.<br><br>
     For each derivation rule, We have a clear location for where the code should live, users need to do the last step of translating the psuedocode into actual python code.<br><br>
-    This <a href="https://youtu.be/MY5drY3RlJs">video</a>   explains how to change the psuedocode for derivation transformation rules into executable Python code.<br><br>
+    This <a href="https://youtu.be/MY5drY3RlJs">video</a>   explains how to change the psuedocode for derivation transformation rules into executable Python code. 
+    Note that the video shows adding GRSS_CRRYNG_AMNT to the ABSTRCT_INSTRMNT_RL in the Enriched Input Layer, but we recommend using the LDM.<br><br>
 
-    The example code used for the GRSS_CRRYNG_AMNT derived attribute is available  <a href="https://github.com/regcommunity/FreeBIRD/blob/main/birds_nest/pybirdai/bird_data_model.py#L645">here</a><br><br>
+    The example code used for the GRSS_CRRYNG_AMNT derived attribute to be added to the FNNCL_ASST_INSTRMNT_DRVD_DT table in the LDM is available  <a href="https://github.com/regcommunity/FreeBIRD_LDM/blob/main/birds_nest/pybirdai/bird_data_model.py#L9805">here</a><br><br>
     If you copy this function be sure to copy from the very start of the line above @property , because Python  is very sensitive to the difference between tabs and spaces (and so are Python programmers!)
 
 

birds_nest/pybirdai/templates/pybirdai/import_data_model_artefacts.html

@@ -20,24 +20,22 @@
 <h1 style="text-align: center;">Import Data Model Artefacts (Choose One) </h1>
 <div class="task-grid">
 
-    <a href="{% url 'pybirdai:upload_sqldev_eil_files' %}" class="task-item">
-        <img src="{% static 'images/upload-sqldev-eil-files.webp' %}" alt="upload-sqldev-eil-files">
-        <span>Upload SQLDEveloper EIL Files (Recommended)</span>
-    </a>
     <a href="{% url 'pybirdai:upload_sqldev_eldm_files' %}" class="task-item">
         <img src="{% static 'images/import_sqldev_eldm_files.webp' %}" alt="import_sqldev_eldm_files">
-        <span>Import SQLDEveloper ELDM Files </span>
+        <span>Import SQLDEveloper ELDM Files (Recommended) </span>
     </a>
+    <a href="{% url 'pybirdai:upload_sqldev_eil_files' %}" class="task-item">
+        <img src="{% static 'images/upload-sqldev-eil-files.webp' %}" alt="upload-sqldev-eil-files">
+        <span>Upload SQLDEveloper EIL Files </span>
+    </a>    
     <a href="{% url 'pybirdai:upload_sqldev_eil_files' %}" class="task-item">
         <img src="{% static 'images/import_bird_eil_datamodel.webp' %}" alt="import_bird_eil_datamodel">
         <span>Import Technical export file of BIRD EIL datamodel(Not supported yet)</span>
-    </a>
-
+    </a>    
     <a href="{% url 'pybirdai:upload_sqldev_eil_files' %}" class="task-item">
         <img src="{% static 'images/import_bird_eldm_datamodel.webp' %}" alt="import_bird_eldm_datamodel">
         <span>Import Technical export file of BIRD ELDM datamodel(Not supported yet)</span>
     </a>
-
     <a href="{% url 'pybirdai:create_bird_database' %}" class="task-item">
         <img src="{% static 'images/back_arrow.jpg' %}" alt="back_arrow">
         <span>Back to the Create BIRD Database Page</span>

birds_nest/pybirdai/templates/pybirdai/import_variables.html

@@ -10,6 +10,7 @@ <h1>Import Variables from CSV File</h1>
 
     <p>Upload a CSV file containing variable data to import into the database. The CSV file should follow the same structure as the variable.csv export.</p>
 
+    <p>If you are looking for the single INSTRMNT_TYP Variable to enrich the BIRD LDM then you can find it <a href="https://github.com/regcommunity/FreeBIRD_LDM/blob/main/initial_correction/variable_typ_instrmnt.csv">here</a>.</p>
     <form method="post" id="importForm" enctype="multipart/form-data">
         {% csrf_token %}
         <div class="form-group">

birds_nest/pybirdai/templates/pybirdai/insert_data_into_bird_database.html

@@ -15,7 +15,7 @@ <h1>Insert Data into BIRD  Database</h1>
     Second you can use the command line terminal to insert data into the database and run pre-prepared sql script to insert rows into the BIRD database.
     <br><br>
     Soon we will make a video showing how to do this, but for now we list the steps below.<br><br>
-    To do this first  stop the application if it is running by doing CTRL-C in the terminal, then get the script and store it in the birds_nest directory (not birds_nest/birds_nest) , there is an example available at <a href="https://github.com/regcommunity/FreeBIRD/blob/main/birds_nest/sqlite_inserts.sql">sqlite_inserts.sql</a>
+    To do this first  stop the application if it is running by doing CTRL-C in the terminal, then get the script and store it in the birds_nest directory (not birds_nest/birds_nest) , there is an example available at <a href="https://github.com/regcommunity/FreeBIRD_LDM/blob/main/birds_nest/sql_inserts.sql">sqlite_inserts.sql</a>
     <br><br>
     Then go to the terminal, making sure you are in the birds_nest directory and that you have sqlite installed and available in your path. If you are using github codespaces you will find that sqlite is already installed and available in your path, 
     otherwise you may need to follow the instructions in https://www.sqlite.org/index.html . Note that sqlite also provides a useful tool called sqldiff, it is good to install this also if you want to use it.<br><br>

birds_nest/pybirdai/templates/pybirdai/manual_edits.html

@@ -17,10 +17,10 @@ <h1>Make Final Manual Edits to Generated Python Code</h1>
     Although PyBIRD AI generates about 95% of the code we need for generation rules, we need to complete the last 5% manually <br><br>
     This is because we have flexibility in how we define the Joins of tables on the input layer data <br><br>
     Do the edits to the generated python joins code as required <b>(This step is to be documented further)</b><br><br> 
-    For now, as an example you can find the class  F_05_01_REF_FINREP_3_0_Loans_and_advances_Table in  pybirdai/process_steps/filter_code/F_05_01_REF_FINREP_3_0_logic.py and replace it with the code   <a href="https://github.com/regcommunity/FreeBIRD/blob/main/birds_nest/pybirdai/process_steps/filter_code/F_05_01_REF_FINREP_3_0_logic.py#L142">here</a> <br><br>
+    For now, as an example you can find the class  F_05_01_REF_FINREP_3_0_Other_Loans_Table in  pybirdai/process_steps/filter_code/F_05_01_REF_FINREP_3_0_logic.py and replace it with the code   <a href="https://github.com/regcommunity/FreeBIRD_LDM/blob/main/birds_nest/pybirdai/process_steps/filter_code/F_05_01_REF_FINREP_3_0_logic.py#L166">here</a> <br><br>
 
     If we have not yet refined the joins metadata (see seperate tutorial), then you will need to manually fix the code where carrying amount is being created twice in some function defintions. To fix this for the Finrep F05.01 report you should find the  
-    class Loans_and_advances(F_05_01_REF_FINREP_3_0_Base): in pybirdai/process_steps/filter_code/F_05_01_REF_FINREP_3_0_logic.py and replace it with this class which has CRRYING_AMNT described only once    <a href="https://github.com/regcommunity/FreeBIRD/blob/main/birds_nest/pybirdai/process_steps/filter_code/F_05_01_REF_FINREP_3_0_logic.py#L98">here</a> <br><br>
+    class Other_loans(F_05_01_REF_FINREP_3_0_Base): in pybirdai/process_steps/filter_code/F_05_01_REF_FINREP_3_0_logic.py and replace it with this class which has RPYMNT_RGHTS defaulted   <a href="https://github.com/regcommunity/FreeBIRD_LDM/blob/main/birds_nest/pybirdai/process_steps/filter_code/F_05_01_REF_FINREP_3_0_logic.py#L104">here</a> <br><br>
 
     This will allow you later to run the transformations for the  F05.01 report.<br><br>
 

birds_nest/pybirdai/templates/pybirdai/populate_bird_metadata_database.html

@@ -36,6 +36,10 @@ <h1 style="text-align: center;">Populate BIRD Metadata Database</h1>
         <img src="{% static 'images/report_templates.webp' %}" alt="Non Ref Info icon">
         <span>Populate BIRD Metadata Database with BIRD Report Templates</span>
     </a>
+    <a href="{% url 'pybirdai:import_variables_from_csv' %}" class="task-item">
+        <img src="{% static 'images/import_variables.webp' %}" alt="Import Variables">
+        <span>If using LDM import TYP_INSTRMNT Variable from CSV File</span>
+    </a>
     <a href="{% url 'pybirdai:home' %}" class="task-item">
         <img src="{% static 'images/back_arrow.jpg' %}" alt="back_arrow">
         <span>Back to the PyBIRD AI Home Page</span>

birds_nest/pybirdai/templates/pybirdai/upload_joins_configuration.html

@@ -5,9 +5,9 @@ <h1>Upload Join Configuration Files</h1>
 
 <a href="{% url 'pybirdai:create_transformation_rules_configuration' %}">Back to Create Transformations Rules Configuration</a>
 
-<p>Currently there are 4 configuration files, and ready made versions are available <a href="https://github.com/regcommunity/FreeBIRD/tree/main/birds_nest/resources/joins_configuration">here</a> <br><br> Theses files are in_scope_reports_FINREP_REF.csv , join_for_product_ldm_definitions_FINREP_REF.csv , join_for_product_il_definitions_FINREP_REF.csv , And join_for_product_to_reference_category_FINREP_REF.csv
+<p>Currently there are 3 configuration files, and ready made versions are available <a href="https://github.com/regcommunity/FreeBIRD_LDM/tree/main/joins_configuration">here</a> <br><br> Theses files are in_scope_reports_FINREP_REF.csv , join_for_product_ldm_definitions_FINREP_REF.csv , join_for_product_il_definitions_FINREP_REF.csv , And join_for_product_to_reference_category_FINREP_REF.csv
 
-Documentation on creating these files is to follow, basically they list the reports in scope, and show which tables in BIRD input layer or LDM are used for each products used in reports (like debt securities used in Finrep) . <br><br>
+Documentation on creating these files is to follow, basically they list the reports in scope, and show which tables are joined togther in the BIRD LDM  for each product used in reports (like debt securities used in Finrep) . We could also create a configuration file to show which BIRD IL tables are used for each product used in reports. <br><br>
     and then upload them by clicking the link below: <br><br>
 
 </p>