v2024.11.14.2

Changelog:

1. SpectraKeras:
   • Prediction shows not only the class of material, but also its name. The name of each predicted material is taken from the file that was generated using DataMakers, which has to have the name: AAA_table_names.h5.
   • Web-version now accepts both single and multi-file (batch).
   • Fix typo that led to crash in CNN
   • Fixed deprecated call in batch mode.
   • Fixed a few bugs detected within converting to C with Cython
   • Improved display of model used depending on framework
   • Minor cleanup and typo fixes
   • Updated web version for latest models
2. Experimental
   • New experimental web version fo SpectraKeras_CNN that uses pyscript as backend. Not operational yet until support for tflite_runtime is added.
3. Utilities
   • DownloadExtractZip is now available in two versions for Raman and XRD.
   • ConvMineralListCSVtoH5: New. Converts CSV files with lists of names of materials or the file names, that was generated using DataMaker.
   • GetSpectraNames: New. Get name of material from h5 formatted file generated using ConvMineralListCSVtoH5.

v2024.10.08.1

Changelog:

1. SpectraKeras:
   • Updated TF model definitions to current TF recommendations.
   • Keras v3 models are now saved with a .keras extension.
   • Added compatibility with TensorFlow v2.17.0+ and Keras v3, including conversion to .tflite.
   • Remove double saving models during training.
   • Fix loading of models during quantization for TF<2.16.0
   • Fix compatibility in opening/saving models in keras 3.
   • Fix discrepancy in visualization of accuracy/mae between keras 2 and 3.
   • Show Keras version at the end of run (alongside TF version).
2. Utilities
   • Multi2SingleLine: Now accepts either files or directly text as input.
   • ConvertToTFLiteK3: Updated for compatibility with TensorFlow v2.17.0+ and Keras v3.
   • ValidFileMaker: Added support both csv and txt. Bug fixes.
   • *DataMakers: Fix crash on any of the DataMakers when data begins with value "0.000".

v2024.03.08.1

Changelog:

1. SpectraKeras
   • Added support for TF 2.16.1. While keras v3 is the default in TF 2.16.1 (as opposed tov2 in previous releases), models created with keras v3 are incompatible with previous versions of TF. Spectrakeras has now a new flag kerasVersion in the SpectraKeras_XXX.ini, with default using keras v2. Models trained and saved with TF 2.15.0 and earlier as well as models trained and saved in TF2.16.1 in keras v2 mode are compatible. Models trained and saved in TF2.16.1 in keras v3 mode (kerasVersion=3inSpectraKeras_XXX.ini) are not backward compatible. If backward compatibility is a priority, keras v2 mode should be used.
   • A new flag kerasVersion = 2inSpectraKeras_XXX.ini. If using ini files from previous versions of SpectraKeras, please add the flag in the System section for correct operation.
   • Improved performance in reading inference files (especially during batch processing).
   • SpectraKeras-web: Initial release of a web based version that relies on the same python3 backend through PHP. This repo provides the index.php needed to run, not models, etc. This method is absolutely not scalable (compared to tensorflow-js), as the prediction is done server-side. It is only to allow web-access to SpectraKeras for limited personal use.
2. Utilities
   • LabelFinder.py, RangeToDataMaker.py: code cleanup.
   • ConvertToTFLiteK2.py: Revised for compatibility with TF 2.16.1 using 'keras v2` and models saved with previous versions of TF.
   • ConvertToTFLiteK3.py: Revised for compatibility with TF 2.16.1 using 'keras v3`.
3. Other
   *
4. Known issues

v2023.12.22.3

Changelog:

1. Introduced new versioning system consistent with common practices.
2. SpectraKeras
   • Added support for TF 2.15.0 and newer, and Python 3.11.
   • Models are now natively saved using the high-level format .keras, rather then the legacy .h5. Models saved with previous versions in the .h5 format can be seamlessly open by the new version.
   • Improved visualization of quantization operations.
3. Utilities
   • New: AddMixHighFreqMaker.py: Data augmentation via data average of high frequency datapoints.
   • New:ThresholdCrossValidMaker.py: Choose randomly selected datapoints only from classes with multiple members (high frequency)
   • New:ShowClasses.py: Show unique classes in training/validation dataset.
   • RruffDataMaker.py, GenericDataMaker.py,XmuDataMaker.py: Set boundaries to minimum of spectra, rather than zero. Fix setting boundaries to actual minima.
   • RandomCrossValidMaker.py: Increase granularity in percentage while naming training/test files. Show unique classes in validation set, and check for classes with multiple members (ask to save files if that is the case). Added flag to allow fully random seeding of selection for split. Code cleanup.
   • MixMakerRruff.py: Make plotting of the mixture optional. Fixed deprecated call.
   • PlotRruffSpectraRound.py,PlotRruffSpectra.py: Fixed bug and deprecated call.
   • PlotMultiASCII.py: Fix bugs and remove legend in plot.
4. Other
   • Folder cleanup
5. Known issues
   • Inference using Coral EdgeTPU requires Tflite-runtime 2.11.1 or lower (on Linux). This can be compiled from TensorFlow source by following directions listed here. This will be fixed in the version of SpectraKeras (and EdgeTPU libraries), planned along the release of TF 2.16.0 in 2024.

20230710a

Changelog:
Next release

1. SpectraKeras
   • Supports TF 2.12.0 and newer. TF 2.13.x recommended.
   • Removed deprecated optimization flag for tflite
   • Disable saving NN diagram to prevent crash for incompatible libpng.
2. Utilities
   • CleanSplilneData: NEW - Remove repeated data, renormalize and spline (optional) ASCII files
   • PlotASCIIData: NEW - Plot ASCII Data - single file
   • PlotMultiASCII: NEW - Plot multiple ASCII data with renormalization
   • ReadRruff: Revised file management and bug fixes.
   • GenericDataMaker: Added parameter for file separator in determining classification class

20230401a

Changelog:
This release supports Tensorflow >= 2.12. Previous versions are no longer supported

1. SpectraKeras
   • Support for new TF 2.11 optimizer (removed deprecated optimizers)
   • Updated keras saving methods to the new keras.saving class (TF >=2.12)
   • Added support for Python 3.11.

20221103a

Changelog:
This release supports Tensorflow <= 2.11.x. This will be the last release supporting TF <=2.11.x.

1. SpectraKeras
   • This interim version supports both TF 2.10 and TF 2.11. If TF 2.11 is detected, the new optimizer introduced in TF2.11 is used, otherwise it will fall back to the legacy version.
2. Utilities
   • Updated Multi2SingleLine

20221026b

Changelog:
This will be the last release supporting TensorFlow <= 2.10.x. Future releases will require TensorFlow => 2.11.x.

1. SpectraKeras
   • Added backend support for softmax accuracy determination.
2. Utilities:
   • Updated calls for deprecated numpy functions (numpy >1.20)

20210713a

Changelog:

1. SpectraKeras

   • Models are now saved with .h5 extension, rather than non-standard .hd5. If you have previously trained models saved in .hd5, just rename the extension as .h5. No change in functionality besides the change in extension. This is needed for better handling within TF.
   • Added ability to save model with max val_accuracy, accuracy (classifier), or min val_mae or mae. When saveBestModel flag is set to False the model saved is from the last epoch during the training run.
   • These additions breaks the configuration .ini file. Add the following new options under Parameters in the ini file to fix it.
     stopAtBest = False
saveBestModel = False
metricBestModelR = val_mae
metricBestModelC = val_accuracy
   • Fixed crashing bug when plotting weights for NN with specific length
   • Several improvements in the output, including an improved summary of the parameters used.
   • Updated documentation

2. Utilities:

   • SaveExcelCSV: Save Excel files as CSV or HDF h5(sheets within xlsx are saved as individual files).

20210510a

Changelog:

1. SpectraKeras
   • Removed support for tensorflow 1.x. Minimum version of TF:
     • TF 2.0 if using TensorFlow proper
     • TF 2.3 if using quantization for TFLite and Coral.
   • Added new setup-tflite.py to create whl packages for system using only tflite-runtime.
2. Utilities:
   • GenericDataMaker.py: Generic Training Datamaker, customizable for different file formats in the defParam class.