0.5.3

What's Changed

• RDRP-581 - MoR Trimming Bug to deal with nulls by @dpurches in #160
• 567 NI staging by @AnneONS in #157
• 580 export SIC, FTE, CD by @jwestw in #159
• RDRP-555: Top up the postcodes to 8 digit before validation by @JenCheshire in #161
• RDRP-582: Manual Trimming Issue by @coatet in #163

Full Changelog: 0.5.1-alpha-release...0.5.3-alpha-release

Version 0.5.1

What's Changed

• Remove unnecessary mapper and duplicate output #151
• Remove duplication of the status filtered output #151
• Fix conditional paths for construction #151
• Bug fix for TAU output weighted values #154
• Update MoR trimming to not use nulls #153
• 566 & 569 headcount and 604 bugfixes #155
• Remove instance 0 from outputs #156
• Postcode top-up for TAU #158
• Fix constructed records missing from SAS and TAU outputs #158

Version 0.5.0

Includes the following new features:
556 manual trimming output
519 numeric PG codes are updated for SAS GB output
Nulls are replaced with zeros in clear target variables
501 Manual imputation trimming is implemented
Output bus are fixed:
#532 postcode mapper should be read in though not validated when reading from feather
#536 and #543 filter status in weigthed_df to include imputed statuses
#542 remove records that don't do R&D from Tau and Sas (but leave for long and short forms)
#546 remove references to 4xx cols in all outputs (FTE data was missing in Tau and Sas)
#547 check filtering by status within output functions
548 Mean of ratios bug fix
100 Construction is implemented. Additions and amendments are disabled
541 Postcodes are corrected to improve the linking to region codes
537 Short form expansion fall-back

v0.4.0 - Fixes to run in DAP

What's Changed

• RDRP-535: Save records with invalid statuses for QA by @JenCheshire in #134
• Merge pull request #134 from ONSdigital/RDRP-538 by @dpurches in #137
• 522 fte short forms by @AnneONS in #132
• v_0.4.0 - New version by @dpurches in #138

Full Changelog: 0.3.6-alpha-release...0.4.0-alpha-release

0.3.6-alpha-release: Merge pull request #131 from ONSdigital/RDRP-531

Rdrp-531: fix outputs

0.3.5-alpha-release

Merge pull request #125 from ONSdigital/505_fix_hdfs_path

FIX: 505 fix hdfs path

0.3.4 Imputation bug fixes

This branch implements several fixes for the imputation module:

• Expansion imputation (step 4 TMI long forms): where there are zeros in the target variables, the breakdown cols were nulls. This is fixed by removing division by zero and entering zeros for the breakdown cols instead.
• MoR carry forwards: removed "postcode only" rows from the backdata before merging, also resetting the index at the end of the process.
  Census imputation for short forms: methodology advised that "P" selection entries should not after all be included in the calculation of values to impute.
• Short form expansion imputation: removed division by zero error to ensure breakdown columns are not null.
• Short form expansion imputation: include trimmed short form rows in the calculations and in the evaluations
• Also staging_main has been re-structured, and the postcode evaluation is now contained in the feather file.

What's Changed

• Carry forwards bugfix: update CF join conditions by @AnneONS in #118
• RDRP-469: QA Output - Total Intram by Civil or Defence by @JenCheshire in #119
• RDRP-489, RDRP-490, RDRP-491: Construction module by @coatet in #115
• Increment version to 0.3.3 by @coatet in #120
• 486 more imputation fixes by @AnneONS in #121
• 498 postcode in json block by @jwestw in #122

Full Changelog: 0.3.1-alpha-release...0.3.4-alpha-release

0.3.1-alpha-release: Merge pull request #116 from ONSdigital/495-export-config

495 export config

v0.3.0

This release includes:

• Expansion imputation
• Further outputs
• Post code fix

What's Changed

• Rdrp 467 by @zorge69 in #110
• 324 gb sas by @zorge69 in #105
• Draft: RDRP-488: Stage updated snapshot file by @coatet in #112
• RDRP-463: Improve postcode validation by @JenCheshire in #107
• Sf expansion 481 by @AnneONS in #109
• 482 numeric pg by @zorge69 in #104
• Rdrp 468 intram region by @zorge69 in #113

Full Changelog: 0.2.7-alpha-release...0.3.0-alpha-release

Imputation through the whole pipeline

The results of TMI imputation (long forms only) now feed through to the outputs.

Outlier-detection and Estimation modules have been updated to deal with short forms in long-form format.

Output schemas have been moved from the outputs folder to the config folder.