Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PIMS-1327: Data fixes after seeding Postgres #2191

Merged
merged 17 commits into from
Feb 22, 2024
Merged

PIMS-1327: Data fixes after seeding Postgres #2191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
f425508
adjusting evaluation keys and disabling agency seeding
LawrenceLau2020 Feb 15, 2024
f4baed0
adjusting fiscal key seed
LawrenceLau2020 Feb 15, 2024
28afb1f
adding latest tasks.json ETL task for DBeaver
LawrenceLau2020 Feb 15, 2024
1f83d4a
Merge branch 'main' into PIMS-1327
LawrenceLau2020 Feb 15, 2024
9d14543
adjusting ETL for users and adding project related tables
LawrenceLau2020 Feb 16, 2024
3e21ef5
adding migration for nullable columns adding ETL for project tables
LawrenceLau2020 Feb 16, 2024
9f4907d
adding ETL for project_properties
LawrenceLau2020 Feb 16, 2024
0d672e7
lint fix
LawrenceLau2020 Feb 16, 2024
43e88a1
Merge branch 'main' into PIMS-1327
LawrenceLau2020 Feb 16, 2024
6216d4f
added more nullable changes and the final tasks.json
LawrenceLau2020 Feb 16, 2024
2eaf796
cleaning up the tasks.json to only have the PIMS ETL task
LawrenceLau2020 Feb 16, 2024
19f3cfd
lint fix
LawrenceLau2020 Feb 16, 2024
1f8f577
Merge branch 'main' into PIMS-1327
LawrenceLau2020 Feb 16, 2024
a5d9e73
Merge branch 'main' into PIMS-1327
LawrenceLau2020 Feb 21, 2024
7d89e30
Merge branch 'main' into PIMS-1327
LawrenceLau2020 Feb 21, 2024
d6d8f10
updated DBeaver task and nullable columns
LawrenceLau2020 Feb 21, 2024
d1770d6
Merge branch 'main' into PIMS-1327
LawrenceLau2020 Feb 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions express-api/src/typeorm/Entities/NotificationQueue.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,15 +51,15 @@ export class NotificationQueue extends BaseEntity {
Tag: string;

// Project Relation
@Column({ name: 'project_id', type: 'int' })
@Column({ name: 'project_id', type: 'int', nullable: true })
ProjectId: number;

@ManyToOne(() => Project, (Project) => Project.Id)
@JoinColumn({ name: 'project_id' })
Project: Project;

// Agency Relation
@Column({ name: 'to_agency_id', type: 'int' })
@Column({ name: 'to_agency_id', type: 'int', nullable: true })
ToAgencyId: number;

@ManyToOne(() => Agency, (Agency) => Agency.Id)
Expand Down
2 changes: 1 addition & 1 deletion express-api/src/typeorm/Entities/ProjectAgencyResponse.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ export class ProjectAgencyResponse extends BaseEntity {
OfferAmount: number;

// Notification Relation
@Column({ name: 'notification_id', type: 'int' })
@Column({ name: 'notification_id', type: 'int', nullable: true })
NotificationId: number;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason this column is never used, as I checked in production to confirm there are no references to the Notification Id. Possibly be cause an agency could have more than one notification associated to it.


@ManyToOne(() => NotificationQueue, (Notification) => Notification.Id, { nullable: true })
Expand Down
4 changes: 2 additions & 2 deletions express-api/src/typeorm/Entities/ProjectProperty.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,15 @@ export class ProjectProperty extends BaseEntity {
PropertyType: PropertyType;

// Parcel Relation
@Column({ name: 'parcel_id', type: 'int' })
@Column({ name: 'parcel_id', type: 'int', nullable: true })
ParcelId: number;

@ManyToOne(() => Parcel, (Parcel) => Parcel.Id)
@JoinColumn({ name: 'parcel_id' })
Parcel: Parcel;

// Building Relation
@Column({ name: 'building_id', type: 'int' })
@Column({ name: 'building_id', type: 'int', nullable: true })
BuildingId: number;

@ManyToOne(() => Building, (Building) => Building.Id)
Expand Down
2 changes: 1 addition & 1 deletion express-api/src/typeorm/Entities/ProjectReport.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ export class ProjectReport extends BaseEntity {
@Column({ type: 'character varying', length: 250, nullable: true })
Name: string;

@Column('timestamp')
@Column({ type: 'timestamp', nullable: true })
From: Date;

@Column('timestamp')
Expand Down
4 changes: 0 additions & 4 deletions express-api/src/typeorm/Migrations/1707763864014-SeedData.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,6 @@ export class SeedData1707763864014 implements MigrationInterface {
const Users = SqlReader.readSqlFile(path.join(sqlFilePath, 'Users.sql'));
await queryRunner.query(Users.toString());

const Agencies = SqlReader.readSqlFile(path.join(sqlFilePath, 'Agencies.sql'));
await queryRunner.query(Agencies.toString());

const Provinces = SqlReader.readSqlFile(path.join(sqlFilePath, 'Provinces.sql'));
await queryRunner.query(Provinces.toString());

Expand Down Expand Up @@ -114,7 +111,6 @@ export class SeedData1707763864014 implements MigrationInterface {
await queryRunner.query('TRUNCATE TABLE property_classification cascade');
await queryRunner.query('TRUNCATE TABLE property_type cascade');
await queryRunner.query('TRUNCATE TABLE tier_level cascade');
await queryRunner.query('TRUNCATE TABLE agency cascade');
await queryRunner.query('TRUNCATE TABLE role cascade');
await queryRunner.query('TRUNCATE TABLE province cascade');
await queryRunner.query('TRUNCATE TABLE project_risk cascade');
Expand Down
159 changes: 159 additions & 0 deletions express-api/src/typeorm/Migrations/1708536844664-NullableFields.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
import { MigrationInterface, QueryRunner } from 'typeorm';

export class NullableFields1708536844664 implements MigrationInterface {
name = 'NullableFields1708536844664';

public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "project" DROP CONSTRAINT "FK_f033e95537c91a7787c7a07c857"`,
);
await queryRunner.query(`DROP INDEX "public"."IDX_7611151947b52a71323a46ea46"`);
await queryRunner.query(`DROP INDEX "public"."IDX_f033e95537c91a7787c7a07c85"`);
await queryRunner.query(
`ALTER TABLE "project" RENAME COLUMN "trier_level_id" TO "tier_level_id"`,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" DROP CONSTRAINT "FK_8988a5dafec2270a5191a1208cd"`,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" DROP CONSTRAINT "FK_c9230eab8c7df3197b3c96dcac6"`,
);
await queryRunner.query(`DROP INDEX "public"."IDX_e96934a5a81124580dd628602f"`);
await queryRunner.query(
`ALTER TABLE "notification_queue" ALTER COLUMN "project_id" DROP NOT NULL`,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" ALTER COLUMN "to_agency_id" DROP NOT NULL`,
);
await queryRunner.query(
`ALTER TABLE "project_agency_response" DROP CONSTRAINT "FK_67989daa98d90fafc2087a4871f"`,
);
await queryRunner.query(
`ALTER TABLE "project_agency_response" ALTER COLUMN "notification_id" DROP NOT NULL`,
);
await queryRunner.query(
`ALTER TABLE "project_property" DROP CONSTRAINT "FK_b2229cb8dfee8d1c2fe6b78bd91"`,
);
await queryRunner.query(
`ALTER TABLE "project_property" DROP CONSTRAINT "FK_d18ff54ded63e964ef42a66880e"`,
);
await queryRunner.query(`DROP INDEX "public"."IDX_ef992ea03a65f72e0b69644b35"`);
await queryRunner.query(
`ALTER TABLE "project_property" ALTER COLUMN "parcel_id" DROP NOT NULL`,
);
await queryRunner.query(
`ALTER TABLE "project_property" ALTER COLUMN "building_id" DROP NOT NULL`,
);
await queryRunner.query(`DROP INDEX "public"."IDX_e4998c66591ebea5e8c1c2d575"`);
await queryRunner.query(`ALTER TABLE "project_report" ALTER COLUMN "from" DROP NOT NULL`);
await queryRunner.query(
`CREATE INDEX "IDX_88df15c457daadc1d407c6d085" ON "project" ("tier_level_id") `,
);
await queryRunner.query(
`CREATE INDEX "IDX_5d29806a012b84ac1d014c563d" ON "project" ("status_id", "tier_level_id", "agency_id") `,
);
await queryRunner.query(
`CREATE INDEX "IDX_e96934a5a81124580dd628602f" ON "notification_queue" ("project_id", "template_id", "to_agency_id") `,
);
await queryRunner.query(
`CREATE INDEX "IDX_ef992ea03a65f72e0b69644b35" ON "project_property" ("project_id", "property_type_id", "parcel_id", "building_id") `,
);
await queryRunner.query(
`CREATE INDEX "IDX_e4998c66591ebea5e8c1c2d575" ON "project_report" ("id", "to", "from", "is_final") `,
);
await queryRunner.query(
`ALTER TABLE "project" ADD CONSTRAINT "FK_88df15c457daadc1d407c6d085f" FOREIGN KEY ("tier_level_id") REFERENCES "tier_level"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" ADD CONSTRAINT "FK_8988a5dafec2270a5191a1208cd" FOREIGN KEY ("project_id") REFERENCES "project"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" ADD CONSTRAINT "FK_c9230eab8c7df3197b3c96dcac6" FOREIGN KEY ("to_agency_id") REFERENCES "agency"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "project_agency_response" ADD CONSTRAINT "FK_67989daa98d90fafc2087a4871f" FOREIGN KEY ("notification_id") REFERENCES "notification_queue"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "project_property" ADD CONSTRAINT "FK_b2229cb8dfee8d1c2fe6b78bd91" FOREIGN KEY ("parcel_id") REFERENCES "parcel"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "project_property" ADD CONSTRAINT "FK_d18ff54ded63e964ef42a66880e" FOREIGN KEY ("building_id") REFERENCES "building"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
}

public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "project_property" DROP CONSTRAINT "FK_d18ff54ded63e964ef42a66880e"`,
);
await queryRunner.query(
`ALTER TABLE "project_property" DROP CONSTRAINT "FK_b2229cb8dfee8d1c2fe6b78bd91"`,
);
await queryRunner.query(
`ALTER TABLE "project_agency_response" DROP CONSTRAINT "FK_67989daa98d90fafc2087a4871f"`,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" DROP CONSTRAINT "FK_c9230eab8c7df3197b3c96dcac6"`,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" DROP CONSTRAINT "FK_8988a5dafec2270a5191a1208cd"`,
);
await queryRunner.query(
`ALTER TABLE "project" DROP CONSTRAINT "FK_88df15c457daadc1d407c6d085f"`,
);
await queryRunner.query(`DROP INDEX "public"."IDX_e4998c66591ebea5e8c1c2d575"`);
await queryRunner.query(`DROP INDEX "public"."IDX_ef992ea03a65f72e0b69644b35"`);
await queryRunner.query(`DROP INDEX "public"."IDX_e96934a5a81124580dd628602f"`);
await queryRunner.query(`DROP INDEX "public"."IDX_5d29806a012b84ac1d014c563d"`);
await queryRunner.query(`DROP INDEX "public"."IDX_88df15c457daadc1d407c6d085"`);
await queryRunner.query(`ALTER TABLE "project_report" ALTER COLUMN "from" SET NOT NULL`);
await queryRunner.query(
`CREATE INDEX "IDX_e4998c66591ebea5e8c1c2d575" ON "project_report" ("id", "is_final", "from", "to") `,
);
await queryRunner.query(
`ALTER TABLE "project_property" ALTER COLUMN "building_id" SET NOT NULL`,
);
await queryRunner.query(`ALTER TABLE "project_property" ALTER COLUMN "parcel_id" SET NOT NULL`);
await queryRunner.query(
`CREATE INDEX "IDX_ef992ea03a65f72e0b69644b35" ON "project_property" ("project_id", "property_type_id", "parcel_id", "building_id") `,
);
await queryRunner.query(
`ALTER TABLE "project_property" ADD CONSTRAINT "FK_d18ff54ded63e964ef42a66880e" FOREIGN KEY ("building_id") REFERENCES "building"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "project_property" ADD CONSTRAINT "FK_b2229cb8dfee8d1c2fe6b78bd91" FOREIGN KEY ("parcel_id") REFERENCES "parcel"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "project_agency_response" ALTER COLUMN "notification_id" SET NOT NULL`,
);
await queryRunner.query(
`ALTER TABLE "project_agency_response" ADD CONSTRAINT "FK_67989daa98d90fafc2087a4871f" FOREIGN KEY ("notification_id") REFERENCES "notification_queue"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" ALTER COLUMN "to_agency_id" SET NOT NULL`,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" ALTER COLUMN "project_id" SET NOT NULL`,
);
await queryRunner.query(
`CREATE INDEX "IDX_e96934a5a81124580dd628602f" ON "notification_queue" ("project_id", "to_agency_id", "template_id") `,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" ADD CONSTRAINT "FK_c9230eab8c7df3197b3c96dcac6" FOREIGN KEY ("to_agency_id") REFERENCES "agency"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "notification_queue" ADD CONSTRAINT "FK_8988a5dafec2270a5191a1208cd" FOREIGN KEY ("project_id") REFERENCES "project"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
await queryRunner.query(
`ALTER TABLE "project" RENAME COLUMN "tier_level_id" TO "trier_level_id"`,
);
await queryRunner.query(
`CREATE INDEX "IDX_f033e95537c91a7787c7a07c85" ON "project" ("trier_level_id") `,
);
await queryRunner.query(
`CREATE INDEX "IDX_7611151947b52a71323a46ea46" ON "project" ("agency_id", "trier_level_id", "status_id") `,
);
await queryRunner.query(
`ALTER TABLE "project" ADD CONSTRAINT "FK_f033e95537c91a7787c7a07c857" FOREIGN KEY ("trier_level_id") REFERENCES "tier_level"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`,
);
}
}
1 change: 1 addition & 0 deletions express-api/src/typeorm/Migrations/Seeds/EvaluationKeys.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

INSERT INTO evaluation_key (id,created_by_id,created_on,updated_by_id,updated_on,name,description) VALUES
(1, '00000000-0000-0000-0000-000000000000', '2023-01-17 17:58:34.7500000',NULL, NULL, N'Assessed', NULL),
(2, '00000000-0000-0000-0000-000000000000', '2023-01-17 17:58:34.7500000',NULL, NULL, N'Appraised', NULL),
Expand Down
Loading
Loading