v0.4 🔥 New models

1. applied the SAITS embedding strategy to models Crossformer, PatchTST, DLinear, ETSformer, FEDformer, Informer, and Autoformer to make them applicable to POTS data as imputation methods;
2. fixed a bug in USGAN loss function;
3. gathered several Transformer embedding methods into the packagepypots.nn.modules.transformer.embedding;
4. added the attribute best_epoch for NN models to record the best epoch num and log it after model training;
5. made the self-attention operator replaceable in the class MultiHeadAttention for Transformer models;
6. renamed the argument d_inner of all models in previous versions into d_ffn. This is for unified argument naming and easier understanding;
7. removed deprecated functions save_model() and load_model() in all NN model classes, which are now replaced by save()and load();

What's Changed

• Removing deprecated functions by @WenjieDu in #318
• Add Autoformer as an imputation model by @WenjieDu in #320
• Removing deprecated save_model and load_model, adding the imputation model Autoformer by @WenjieDu in #321
• Simplify MultiHeadAttention by @WenjieDu in #322
• Add PatchTST as an imputation model by @WenjieDu in #323
• Renaming d_inner into d_ffn by @WenjieDu in #325
• Adding PatchTST, renaming d_innner into d_ffn, and refactoring Autofomer by @WenjieDu in #326
• Add DLinear as an imputation model by @WenjieDu in #327
• Add ETSformer as an imputation model by @WenjieDu in #328
• Add Crossformer as an imputation model by @WenjieDu in #329
• Add FEDformer as an imputation model by @WenjieDu in #330
• Add Crossformer, Autoformer, PatchTST, DLinear, ETSformer, FEDformer as imputation models by @WenjieDu in #331
• Refactor embedding package, remove the unused part in Autoformer, and update the docs by @WenjieDu in #332
• Make the self-attention operator replaceable in Transformer by @WenjieDu in #334
• Add informer as an imputation model by @WenjieDu in #335
• Speed up testing procedure by @WenjieDu in #336
• Add Informer, speed up CI testing, and make self-attention operator replaceable by @WenjieDu in #337
• debug USGAN by @AugustJW in #339
• Fix USGAN loss function, and update the docs by @WenjieDu in #340
• Add the attribute best_epoch to record the best epoch num by @WenjieDu in #342
• Apply SAITS embedding strategy to new added models by @WenjieDu in #343
• Release v0.4, apply SAITS embedding strategy to the newly added models, and update README by @WenjieDu in #344

Full Changelog: v0.3.2...v0.4

v0.3.2 🐞 Bugfix

1. fixed an issue that stopped us from running Raindrop on multiple CUDA devices;
2. added Mean and Median as naive imputation methods;

What's Changed

• Refactor LOCF, fix Raindrop on multiple cuda devices, and update docs by @WenjieDu in #308
• Remind how to display the figs rather than invoking plt.show() by @WenjieDu in #310
• Update the docs and requirements by @WenjieDu in #311
• Fixing some bugs, updating the docs and requirements by @WenjieDu in #312
• Make CI workflows only test with Python v3.7 and v3.11 by @WenjieDu in #313
• Update the docs and release v0.3.2 by @WenjieDu in #314
• Add mean and median as imputation methods, and update docs by @WenjieDu in #317

Full Changelog: v0.3.1...v0.3.2

v0.3.1

A bug in the calculation of the delta matrix (time-decay matrix) discussed in #294 gets fixed in this update.

What's Changed

• Update logo URLs by @WenjieDu in #293
• Fixing the issue in delta calculation by @WenjieDu in #297
• Fixing the issue in time-decay matrix calculation and simplify the code by @WenjieDu in #298
• Roll back the delta calculation of M-RNN to the same with GRU-D by @WenjieDu in #300

Full Changelog: v0.3...v0.3.1

v0.3 coming with new features 😎

Happy New Year, dear friends! 🥳

New features and updated APIs in PyPOTS are brought to you here! In v0.3, we

1. added TimesNet as an imputation model;
2. simplified the structure of val_set. In previous versions, you had to give indicating_mask in the dictionary val_set that tells PyPOTS to use which values to validate the model. Now you only need to give X_ori (i.e. X_intact before) and X, both leaving their missing data as NaNs. PyPOTS will handle everything left to evaluate the model for you;
3. enabled PyPOTS to tune hyperparameters for external models (implemented with the PyPOTS framework but haven't been integrated into PyPOTS);
4. updated the package pypots.data.saving. Separated the functions for pickle saving and h5py saving, and added load_dict_from_h5 that can inverse (deserialize) the process of save_dict_into_h5;
5. fixed some bugs (#255, #263, #266, #280, #282, #286, #289);

What's Changed

• Code refactor by @WenjieDu in #251
• Adding TimesNet as an imputation model by @WenjieDu in #252
• Adding TimesNet, refactoring code, and updating docs by @WenjieDu in #253
• Fixing CSDI gtmask bug by @WenjieDu in #255
• Fixing CSDI gt_mask issue, and setting a fixed random seed for testing cases by @WenjieDu in #256
• Making CSDI return all n_sampling_times imputation samples by @WenjieDu in #258
• Adding get_random_seed(), and adding func calc_quantile_crps() by @WenjieDu in #260
• Making CSDI val process same as the original by @WenjieDu in #262
• Fix missing argument attn_dropout in imputation Transformer by @WenjieDu in #263
• Adding visualization functions by @AugustJW in #267
• Add cluster plotting functions in pypots.utils.visualization by @vemuribv in #182
• Fixing unstable nonstationary norm, adding utils.visual, and doing some code refactoring by @WenjieDu in #266
• Updating package pypots.data.saving by @WenjieDu in #268
• Enabling to tune hyperparameters for outside models implemented with PyPOTS framework by @WenjieDu in #269
• Simplifying the structure of val_set, and using a consistent strategy when lazy-loading val_set by @WenjieDu in #272
• Renaming X_intact into X_ori, and adding matplotlib as a dependency by @WenjieDu in #274
• Simplifying val_set, renaming X_intact, and adding unit tests for the visual package by @WenjieDu in #275
• Update GP-VAE by @WenjieDu in #277
• Updating GP-VAE, adding load_dict_from_h5, etc. by @WenjieDu in #278
• Adding _check_inputs() for error calculation functions by @WenjieDu in #279
• Fixing CSDI, adding placeholder for epoch num in logging by @WenjieDu in #280
• Fixing the infinite loop in LOCF by @WenjieDu in #282
• Update docs by @WenjieDu in #285
• Updating docs, fixing CSDI&LOCF&MRNN, and adding the strategy to save all models by @WenjieDu in #284
• Making PyPOTS able to save all models during training, checking if d_model=n_heads*d_k for SAITS and Transformer by @WenjieDu in #287
• Fixing MRNN by @WenjieDu in #286
• Fix issues in MRNN and update the hyperparameter tuning functionality by @WenjieDu in #288
• Fixing the type error of random_seed in pypots.cli.tuning and updating the docs by @WenjieDu in #289
• Updating load_dict_from_h5() by @WenjieDu in #290

Full Changelog: v0.2.1...v0.3

v0.2.1

Here are updates,

1. for missing values after LOCF imputation (that are missing since the first step hence LOCF doesn't work), we added more options to handle them. Please refer to the argument first_step_imputation in LOCF docs. The default option is "zero" in previous versions, but we've changed it to "backward" which is more reasonable;
2. enabled SAITS to return latent attention weights from blocks in predict() for advanced analysis e.g. in #178;
3. renamed model saving and loading functions save_model() and load_model() into save() and load();

What's Changed

• Check if X_intact contains missing data for imputation models, check and list mismatched hyperparameters in the tuning mode by @WenjieDu in #234
• Make SAITS return attention weights in predict() by @WenjieDu in #239
• Adding other options for the first step imputation in LOCF by @WenjieDu in #240
• Fixing the problem about staling issues by @WenjieDu in #244
• Testing with Python 3.11 and support it by @WenjieDu in #246
• Rename save_model() and load_model() into save() and load() by @WenjieDu in #247
• Refactoring save_model() and load_model(), and updating docs by @WenjieDu in #249

Full Changelog: v0.2...v0.2.1

PyPOTS v0.2 🤗

In PyPOTS v0.2 this new version, we

1. enabled hyperparameter tuning for all NN algorithms;
2. fixed a bug in the updating strategy of term F in CRLI;
3. replaced the license GPL-v3 with BSD-3-Clause that has less constraints;
4. announced PyPOTS ecosystem;

What's Changed

• Adding tsdb and pygrinder into the docs by @WenjieDu in #212
• Fixing disappeared TSDB and PyGrinder by @WenjieDu in #213
• Replacing PyCorruptor with PyGrinder by @WenjieDu in #215
• Merge the docs of PyPOTS ecosystem, and replace pycorruptor with pygrinder in pypots by @WenjieDu in #216
• Clone TSDB and PyGrinder repos to use their latest code and docs by @WenjieDu in #217
• Install from source code to use the latest docs of TSDB and PyGrinder by @WenjieDu in #219
• Install from TSDB and PyGrinder repos to use their latest docs by @WenjieDu in #220
• Enable hyperparameter tuning with NNI framework by @WenjieDu in #221
• Fixing dependency error in testing_CI workflow by @WenjieDu in #223
• Enable hyperparameter tuning with NNI, fix dependency error in testing_CI, and update docs by @WenjieDu in #224
• Fix the bug in the updating strategy of term F in CRLI by @WenjieDu in #226
• Apply BSD-3 license, and update docs by @WenjieDu in #228
• Fix a bug in CRLI, switch to BSD-3 license by @WenjieDu in #229
• Update version limitations on dependencies, and install dependencies in PyPI publishing workflow by @WenjieDu in #230

Full Changelog: v0.1.4...v0.2

v0.1.4

In this new version, we made the following changes:

1. added the imputation model CSDI;
2. added the unified method predict() for all models to run inference on the given test set;
3. enabled clustering algorithms to select the best model on the validation set;
4. fixed the bug that GP-VAE failed to run on CUDA devices;
5. made SAITS to use customized loss function specified by users;

What's Changed

• Add the method predict() for all models by @WenjieDu in #199
• Refactor algorithms' module structure, enable customized loss function in SAITS, enable GP-VAE to run on CUDA, etc. by @WenjieDu in #201
• Merge dev into main by @WenjieDu in #202
• Make clustering algorithms to select the best model according to the loss on a given validation set by @WenjieDu in #204
• Fixing failed CI testing due to dependency installation error by @WenjieDu in #206
• Adding the model CSDI by @WenjieDu in #208
• Refactoring, and updating the docs by @WenjieDu in #209
• Adding CSDI, updating the docs by @WenjieDu in #210

Full Changelog: v0.1.3...v0.1.4

v0.1.3

In this release, we mainly

1. added new models US-GAN and GP-VAE for the imputation task on POTS data;
2. appended util functions save_dict_into_h5() to help users directly save a dataset from a Python Dictionary into an HDF5 file that can be used by models in PyPOTS with lazy-loading strategy to save RAM and CUDA memory;
3. added internal clustering validation measures to help evaluate clustering models;
4. enabled PyPOTS clustering models to return their latent representation for advanced analysis, e.g. internal clustering validation;
5. added learning-rate schedulers for optimizers;

👍 Kudos to our new contributors, @AugustJW and @vemuribv! The above points 1 and 3 are implemented by them that enrich this release and make it happen here.

😎 More awesome features are on the road. Stay tuned or join us to make them happen!

What's Changed

• Adding the package to store frequently-used modules; by @WenjieDu in #170
• feat: add Transformer encoder, decoder, and decoder_layer; by @WenjieDu in #172
• Adding the package pypots/modules, and appending modules of vanilla Transformer by @WenjieDu in #173
• Add internal clustering validation measures to pypots.utils.metrics by @vemuribv in #179
• Add save_dict_into_h5() and sliding_window(), add .gitignore, and update the docs by @WenjieDu in #180
• Merge dev into pr/176 by @WenjieDu in #181
• Fixing failed doc build by @WenjieDu in #186
• Fixing the failed workflow Autoreply to Issues Opened for PyPOTS and update the PR template by @WenjieDu in #184
• Merge dev into main by @WenjieDu in #188
• add models GP-VAE/USGAN by @AugustJW in #176
• Refactor testing cases by @WenjieDu in #189
• Add models US-GAN and GP-VAE, update docs, refactor testing cases, add cal_internal_cluster_validation_metrics() by @WenjieDu in #190
• Extract latent from clustering models by @WenjieDu in #192
• Add internal and external cluster validation funcs, and enable CRLI and VaDER to return latent for advanced analysis by @WenjieDu in #193
• Add learning-rate schedulers by @WenjieDu in #195
• Add learning-rate schedulers for optimizers, and by @WenjieDu in #196
• Releasing v0.1.3; by @WenjieDu in #197

New Contributors

• @vemuribv made their first contribution in #179
• @AugustJW made their first contribution in #176

Full Changelog: v0.1.2...v0.1.3

v0.1.2

In this new version, we mainly fix some bugs, update the docs, and update TSDB changed APIs to keep compatible with it.

What's Changed

• Update logging, and fix the error in the usage example by @WenjieDu in #131
• Fix single-item device list throwing an error by @MaciejSkrabski in #133
• Fix the single-item device list error, update docs by @WenjieDu in #134
• Specifying the version of protobuf<=4.21.12 to fix issue #135 by @WenjieDu in #136
• Update docs and citation info by @WenjieDu in #140
• Fix daily testing by @WenjieDu in #143
• Fixing failed daily testing with MacOS python 3.7, and update docs by @WenjieDu in #144
• Update docs by @WenjieDu in #148
• Fixing the bug in raindrop when running on multiple devices by @WenjieDu in #149
• Updating docs, and fixing the bug in raindrop when running on multiple devices by @WenjieDu in #150
• Fixing the invalid badge of PyPI downloads by @WenjieDu in #152
• Updating stale workflow, and updating docs by @WenjieDu in #154
• Update actions/stale@v8, and fix docs typo by @WenjieDu in #156
• fix: the type hint for function mcar() in comments by @yhzhu99 in #157
• Update the greeting workflow, and update the docs by @WenjieDu in #159
• Updating workflows and docs by @WenjieDu in #162
• Fixing the bug caused by changed tsdb api, updating the docs by @WenjieDu in #166
• Updating docs and releasing v0.1.2 by @WenjieDu in #167

New Contributors

• @yhzhu99 made their first contribution in #157

Full Changelog: v0.1.1...v0.1.2

v0.1.1 is here! And we bring you parallel training on multiple GPUs 🚀

In this new version, we

1. enable PyPOTS to run on multiple CUDA devices to train models in parallel. NOW you can further speed up your model training in PyPOTS 🥳;
2. add the imputation model M-RNN;
3. add the optimizer Adadelta;

What's Changed

• Use relative imports replace absolute ones by @WenjieDu in #117
• Add optimizer Adadelta, and fix issue #116 by @WenjieDu in #118
• Docs auto param type hints by @WenjieDu in #119
• Use sphinx_autodoc_typehints to auto gene type hints for params, and improve docs by @WenjieDu in #120
• Enable parallel training on multi GPUs by @WenjieDu in #122
• skip multi-gpu test if not multi-gpu host by @MaciejSkrabski in #124
• Enable models to run on multiple CUDA devices by @WenjieDu in #125
• Add model MRNN by @WenjieDu in #126
• Add model M-RNN, and update the docs by @WenjieDu in #127
• Fix VaDER failed testing and MRNN failed training on multiple devices by @WenjieDu in #128

Full Changelog: v0.1.0...v0.1.1