Documenting and removing partially the need for Node.js in local Pipe…

.github/workflows/tests_01.yml

@@ -22,7 +22,7 @@ jobs:
         BIGML_EXTERNAL_CONN_DB: ${{ secrets.BIGML_EXTERNAL_CONN_DB }}
         BIGML_EXTERNAL_CONN_USER: ${{ secrets.BIGML_EXTERNAL_CONN_USER }}
         BIGML_EXTERNAL_CONN_PWD: ${{ secrets.BIGML_EXTERNAL_CONN_PWD }}
-        BIGML_DELTA: 5
+        BIGML_DELTA: ${{ vars.BIGML_DELTA }}
 
     steps:
     - name: Install packages

.github/workflows/tests_05.yml

@@ -22,7 +22,7 @@ jobs:
         BIGML_EXTERNAL_CONN_DB: ${{ secrets.BIGML_EXTERNAL_CONN_DB }}
         BIGML_EXTERNAL_CONN_USER: ${{ secrets.BIGML_EXTERNAL_CONN_USER }}
         BIGML_EXTERNAL_CONN_PWD: ${{ secrets.BIGML_EXTERNAL_CONN_PWD }}
-        BIGML_DELTA: 5
+        BIGML_DELTA: ${{ vars.BIGML_DELTA }}
     steps:
     - name: Install packages
       uses: actions/checkout@v3

.github/workflows/tests_22.yml

@@ -22,7 +22,7 @@ jobs:
         BIGML_EXTERNAL_CONN_DB: ${{ secrets.BIGML_EXTERNAL_CONN_DB }}
         BIGML_EXTERNAL_CONN_USER: ${{ secrets.BIGML_EXTERNAL_CONN_USER }}
         BIGML_EXTERNAL_CONN_PWD: ${{ secrets.BIGML_EXTERNAL_CONN_PWD }}
-        BIGML_DELTA: 10
+        BIGML_DELTA: ${{ vars.BIGML_DELTA }}
 
     steps:
     - name: Install packages

.github/workflows/tests_23.yml

@@ -22,7 +22,7 @@ jobs:
         BIGML_EXTERNAL_CONN_DB: ${{ secrets.BIGML_EXTERNAL_CONN_DB }}
         BIGML_EXTERNAL_CONN_USER: ${{ secrets.BIGML_EXTERNAL_CONN_USER }}
         BIGML_EXTERNAL_CONN_PWD: ${{ secrets.BIGML_EXTERNAL_CONN_PWD }}
-        BIGML_DELTA: 5
+        BIGML_DELTA: ${{ vars.BIGML_DELTA }}
 
     steps:
     - name: Install packages

.github/workflows/tests_36.yml

@@ -22,7 +22,7 @@ jobs:
         BIGML_EXTERNAL_CONN_DB: ${{ secrets.BIGML_EXTERNAL_CONN_DB }}
         BIGML_EXTERNAL_CONN_USER: ${{ secrets.BIGML_EXTERNAL_CONN_USER }}
         BIGML_EXTERNAL_CONN_PWD: ${{ secrets.BIGML_EXTERNAL_CONN_PWD }}
-        BIGML_DELTA: 5
+        BIGML_DELTA: ${{ vars.BIGML_DELTA }}
 
     steps:
     - name: Install packages

HISTORY.rst

@@ -3,6 +3,11 @@
 History
 -------
 
+9.8.0.dev1 (2024-02-28)
+-----------------------
+
+- Documenting and removing partially the need for Node.js in Pipelines.
+
 9.8.0.dev (2024-02-19)
 ----------------------
 

bigml/dataset.py

@@ -21,6 +21,7 @@
 import os
 import logging
 import warnings
+import subprocess
 
 from bigml.fields import Fields, sorted_headers, get_new_fields
 from bigml.api import get_api_connection, get_dataset_id, get_status
@@ -30,6 +31,13 @@
 from bigml.flatline import Flatline
 from bigml.featurizer import Featurizer
 
+process = subprocess.Popen(['node -v'], stdout=subprocess.PIPE, shell=True)
+out = process.stdout.read()
+FLATLINE_READY = out.startswith(b"v")
+if FLATLINE_READY:
+    from bigml.flatline import Flatline
+
+
 #pylint: disable=locally-disabled,bare-except,ungrouped-imports
 try:
     # avoiding tensorflow info logging
@@ -211,6 +219,10 @@ def transform(self, input_data_list):
         rows = [self._input_array(input_data) for input_data in
             input_data_list]
         if self.transformations:
+            if not FLATLINE_READY:
+                raise ValueError("Nodejs should be installed to handle this"
+                                 " dataset's transformations. Please, check"
+                                 " the bindings documentation for details.")
             out_headers, out_arrays = self._transform(rows)
             rows = [dict(zip(out_headers, row)) for row
                 in out_arrays]

bigml/tests/test_35_d_compare_predictions.py

@@ -70,18 +70,18 @@ def test_scenario2(self):
                    "input_data", "forecast", "model_conf"]
         examples = [
 
-            ['data/grades.csv', '30', '30', '120',
+            ['data/grades.csv', '30', '30', '300',
              '{"000005": {"horizon": 5}}',
              '{"000005": [{"point_forecast": [73.96192, 74.04106, 74.12029, '
              '74.1996, 74.27899], "model": "M,M,N"}]}',
              '{"objective_fields": ["000001", "000005"], "period": 12}'],
-            ['data/grades.csv', '30', '30', '120',
+            ['data/grades.csv', '30', '30', '300',
              '{"000005": {"horizon": 5, "ets_models": {"names": ["M,N,A"], '
              '"criterion": "aic", "limit": 3}}}',
              '{"000005": [{"point_forecast":  [67.43222, 68.24468, '
              '64.14437, 67.5662, 67.79028], "model": "M,N,A"}]}',
              '{"objective_fields": ["000001", "000005"], "period": 12}'],
-            ['data/grades.csv', '30', '30', '120',
+            ['data/grades.csv', '30', '30', '300',
              '{"000005": {"horizon": 5, "ets_models": {"names": ["A,A,A"], '
              '"criterion": "aic", "limit": 3}}}',
              '{"000005": [{"point_forecast": [74.73553, 71.6163, 71.90264, '

bigml/version.py

@@ -1 +1 @@
-__version__ = '9.8.0.dev'
+__version__ = '9.8.0.dev1'

docs/index.rst

@@ -77,6 +77,11 @@ The bindings will also use ``simplejson`` if you happen to have it
 installed, but that is optional: we fall back to Python's built-in JSON
 libraries is ``simplejson`` is not found.
 
+`Node.js <https://nodejs.org/en>`_ is not installed by default, but will be
+needed for `Local Pipelines <local_resources.html#local-pipelines` to work
+when datasets containing new added features are part of the transformation
+workflow.
+
 The bindings provide support to use the ``BigML`` platform to create, update,
 get and delete resources, but also to produce local predictions using the
 models created in ``BigML``. Most of them will be actionable with the basic

docs/local_resources.rst

@@ -2218,7 +2218,9 @@ the existing BigML objects and create the prediction pipeline.
 The first obvious goal that we may have is reproducing the same feature
 extraction and transformations that were used when training our data to create
 our model. That is achieved by using a ``BMLPipeline`` object built
-on the training dataset.
+on the training dataset. Note that, if your datasets contain features derived
+from the original fields in your data, ``Nodejs`` has to be previously
+installed for the transformations to work locally.
 
 .. code-block:: python
 
@@ -2489,7 +2491,6 @@ and libraries. A new data transformer can be created by deriving the
 to cover the particulars of the functions to be used in the generation of
 new fields.
 
-
 Local Evaluations
 -----------------