Releases: ONSdigital/research-and-development
0.5.3
What's Changed
- RDRP-581 - MoR Trimming Bug to deal with nulls by @dpurches in #160
- 567 NI staging by @AnneONS in #157
- 580 export SIC, FTE, CD by @jwestw in #159
- RDRP-555: Top up the postcodes to 8 digit before validation by @JenCheshire in #161
- RDRP-582: Manual Trimming Issue by @coatet in #163
Full Changelog: 0.5.1-alpha-release...0.5.3-alpha-release
Version 0.5.1
What's Changed
- Remove unnecessary mapper and duplicate output #151
- Remove duplication of the status filtered output #151
- Fix conditional paths for construction #151
- Bug fix for TAU output weighted values #154
- Update MoR trimming to not use nulls #153
- 566 & 569 headcount and 604 bugfixes #155
- Remove instance 0 from outputs #156
- Postcode top-up for TAU #158
- Fix constructed records missing from SAS and TAU outputs #158
Version 0.5.0
Includes the following new features:
556 manual trimming output
519 numeric PG codes are updated for SAS GB output
Nulls are replaced with zeros in clear target variables
501 Manual imputation trimming is implemented
Output bus are fixed:
#532 postcode mapper should be read in though not validated when reading from feather
#536 and #543 filter status in weigthed_df to include imputed statuses
#542 remove records that don't do R&D from Tau and Sas (but leave for long and short forms)
#546 remove references to 4xx cols in all outputs (FTE data was missing in Tau and Sas)
#547 check filtering by status within output functions
548 Mean of ratios bug fix
100 Construction is implemented. Additions and amendments are disabled
541 Postcodes are corrected to improve the linking to region codes
537 Short form expansion fall-back
v0.4.0 - Fixes to run in DAP
What's Changed
- RDRP-535: Save records with invalid statuses for QA by @JenCheshire in #134
- Merge pull request #134 from ONSdigital/RDRP-538 by @dpurches in #137
- 522 fte short forms by @AnneONS in #132
- v_0.4.0 - New version by @dpurches in #138
Full Changelog: 0.3.6-alpha-release...0.4.0-alpha-release
0.3.6-alpha-release: Merge pull request #131 from ONSdigital/RDRP-531
Rdrp-531: fix outputs
0.3.5-alpha-release
Merge pull request #125 from ONSdigital/505_fix_hdfs_path FIX: 505 fix hdfs path
0.3.4 Imputation bug fixes
This branch implements several fixes for the imputation module:
- Expansion imputation (step 4 TMI long forms): where there are zeros in the target variables, the breakdown cols were nulls. This is fixed by removing division by zero and entering zeros for the breakdown cols instead.
- MoR carry forwards: removed "postcode only" rows from the backdata before merging, also resetting the index at the end of the process.
Census imputation for short forms: methodology advised that "P" selection entries should not after all be included in the calculation of values to impute. - Short form expansion imputation: removed division by zero error to ensure breakdown columns are not null.
- Short form expansion imputation: include trimmed short form rows in the calculations and in the evaluations
- Also staging_main has been re-structured, and the postcode evaluation is now contained in the feather file.
What's Changed
- Carry forwards bugfix: update CF join conditions by @AnneONS in #118
- RDRP-469: QA Output - Total Intram by Civil or Defence by @JenCheshire in #119
- RDRP-489, RDRP-490, RDRP-491: Construction module by @coatet in #115
- Increment version to 0.3.3 by @coatet in #120
- 486 more imputation fixes by @AnneONS in #121
- 498 postcode in json block by @jwestw in #122
Full Changelog: 0.3.1-alpha-release...0.3.4-alpha-release
0.3.1-alpha-release: Merge pull request #116 from ONSdigital/495-export-config
v0.3.0
This release includes:
- Expansion imputation
- Further outputs
- Post code fix
What's Changed
- Rdrp 467 by @zorge69 in #110
- 324 gb sas by @zorge69 in #105
- Draft: RDRP-488: Stage updated snapshot file by @coatet in #112
- RDRP-463: Improve postcode validation by @JenCheshire in #107
- Sf expansion 481 by @AnneONS in #109
- 482 numeric pg by @zorge69 in #104
- Rdrp 468 intram region by @zorge69 in #113
Full Changelog: 0.2.7-alpha-release...0.3.0-alpha-release
Imputation through the whole pipeline
The results of TMI imputation (long forms only) now feed through to the outputs.
Outlier-detection and Estimation modules have been updated to deal with short forms in long-form format.
Output schemas have been moved from the outputs folder to the config folder.