Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: add async validation on uploading file #150

Merged
merged 11 commits into from
Jan 26, 2024
2 changes: 1 addition & 1 deletion .github/workflows/qe-dispatch.yml
Original file line number Diff line number Diff line change
@@ -13,7 +13,7 @@ on:
jobs:
quality-engineering:
name: QE
uses: vtex-apps/usqa/.github/workflows/quality-engineering.yml@v2
uses: vtex-apps/usqa/.github/workflows/quality-engineering.yml@v2.1.14
with:
cypress: true
cyRunnerBranch: ${{ inputs.cyRunnerBranch }}
2 changes: 1 addition & 1 deletion .github/workflows/qe-pull-request-target.yml
Original file line number Diff line number Diff line change
@@ -11,7 +11,7 @@ on:
jobs:
quality-engineering:
name: QE
uses: vtex-apps/usqa/.github/workflows/quality-engineering.yml@v2
uses: vtex-apps/usqa/.github/workflows/quality-engineering.yml@v2.1.14
with:
danger: true
dangerRequireChangelog: false
2 changes: 1 addition & 1 deletion .github/workflows/qe-pull-request.yml
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@ on:
jobs:
quality-engineering:
name: QE
uses: vtex-apps/usqa/.github/workflows/quality-engineering.yml@v2
uses: vtex-apps/usqa/.github/workflows/quality-engineering.yml@v2.1.14
with:
danger: true
dangerRequireChangelog: false
2 changes: 1 addition & 1 deletion .github/workflows/qe-push.yml
Original file line number Diff line number Diff line change
@@ -9,7 +9,7 @@ on:
jobs:
quality-engineering:
name: QE
uses: vtex-apps/usqa/.github/workflows/quality-engineering.yml@v2
uses: vtex-apps/usqa/.github/workflows/quality-engineering.yml@v2.1.14
with:
nodeLint: true
nodeTest: false
2 changes: 1 addition & 1 deletion .github/workflows/qe-schedule.yml
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@ on:
jobs:
quality-engineering:
name: QE
uses: vtex-apps/usqa/.github/workflows/quality-engineering.yml@v2
uses: vtex-apps/usqa/.github/workflows/quality-engineering.yml@v2.1.14
with:
cypress: true
cyRunnerTimeOut: 45
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -8,6 +8,10 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.

## [Unreleased]

### Added

- Refactor Bulk import Uploading Modal to Async Validation

## [1.29.2] - 2024-01-12

### Fixed
Original file line number Diff line number Diff line change
@@ -25,6 +25,7 @@ import type {
} from '../../types/BulkImport'
import useStartBulkImport from '../../hooks/useStartBulkImport'
import ReportDownloadLink from '../ReportDownloadLink/ReportDownloadLink'
import { ValidationScreen } from '../UploadingScreen'

const CreateOrganizationButton = () => {
const { formatMessage } = useTranslate()
@@ -70,6 +71,7 @@ const CreateOrganizationButton = () => {
onOpenChange={setUploadModalOpen}
uploadFile={uploadBulkImportFile}
onUploadFinish={handleUploadFinish}
uploadingScreen={props => <ValidationScreen {...props} />}
errorScreen={props => (
<ReportErrorScreen {...(props.data as BulkImportUploadError)} />
)}
2 changes: 1 addition & 1 deletion react/components/ImportReportModal/ImportReportModal.tsx
Original file line number Diff line number Diff line change
@@ -28,7 +28,7 @@ const ImportReportModal = ({
}: ImportReportModalProps) => {
const { translate: t, formatDate } = useTranslate()

const { data, error } = useBulkImportDetailsQuery(importId)
const { data, error } = useBulkImportDetailsQuery({ importId })

const reportDownloadLink = data?.importResult?.reportDownloadLink

58 changes: 58 additions & 0 deletions react/components/UploadingScreen/UploadingScreen.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import React, { useState } from 'react'
import { UploadingScreen as BulkImportUploadingScreen } from '@vtex/bulk-import-ui'

import type { UploadFileData } from '../../types/BulkImport'
import ValidatingScreen from './ValidatingScreen'
import useValidateBulkImport from '../../hooks/useValidateBulkImport'

export type UploadingScreenProps = {
name: string
size: number
uploadFile: () => Promise<UploadFileData>
onUploadFinished: (data: UploadFileData) => void
}

export type UploadingStep = 'UPLOADING' | 'VALIDATING'

const UploadingScreen = ({
uploadFile,
onUploadFinished: onUploadFinishedProp,
...otherProps
}: UploadingScreenProps) => {
const [step, setStep] = useState<UploadingStep>('UPLOADING')

const [importId, setImportId] = useState<string | undefined>(undefined)

const { startBulkImportValidation } = useValidateBulkImport({
onSuccess: () => {
setStep('VALIDATING')
},
})

const onUploadFinished = (data: UploadFileData) => {
if (data.status === 'error') {
onUploadFinishedProp(data)

return
}

startBulkImportValidation({ importId: data?.data?.fileData?.importId })
setImportId(data?.data?.fileData?.importId)
}

return step === 'UPLOADING' ? (
<BulkImportUploadingScreen
{...otherProps}
uploadFile={uploadFile}
onUploadFinished={onUploadFinished}
/>
) : (
<ValidatingScreen
{...otherProps}
importId={importId}
onUploadFinished={onUploadFinishedProp}
/>
)
}

export default UploadingScreen
80 changes: 80 additions & 0 deletions react/components/UploadingScreen/ValidatingScreen.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
import React from 'react'
import { Flex, Spinner, Text, csx } from '@vtex/admin-ui'

import { useTranslate } from '../../hooks'
import { bytesToSize } from '../utils/bytesToSize'
import type { UploadFileData } from '../../types/BulkImport'
import useBulkImportDetailsQuery from '../../hooks/useBulkImportDetailsQuery'

export type ValidatingScreenProps = {
importId?: string
name: string
size: number
onUploadFinished: (data: UploadFileData) => void
}

const ValidatingScreen = ({
name,
size,
importId,
onUploadFinished,
}: ValidatingScreenProps) => {
const { translate: t } = useTranslate()

useBulkImportDetailsQuery({
importId,
refreshInterval: 30 * 1000,
onSuccess: data => {
if (data.importState === 'ReadyToImport') {
onUploadFinished({
status: 'success',
data: {
fileData: {
...data,
percentage: data.percentage.toString(),
},
},
})

return
}

onUploadFinished({
status: 'error',
showReport: data?.importState === 'ValidationFailed',
data: {
error: 'FieldValidationError',
errorDownloadLink: data?.validationResult?.reportDownloadLink ?? '',
validationResult: data?.validationResult?.validationResult ?? [],
fileName: data.fileName,
},
})
},
})

return (
<Flex
className={csx({ backgroundColor: '$gray05', height: '100%' })}
align="center"
direction="column"
justify="center"
>
<Spinner className={csx({ color: '$blue40' })} size={120} />
<Text
className={csx({ marginBottom: '$space-3', marginTop: '$space-10' })}
variant="pageTitle"
>
{t('uploading')}
</Text>
<div>
<Text>{name}</Text>
<Text tone="secondary">
{' · '}
{bytesToSize(size)}
</Text>
</div>
</Flex>
)
}

export default ValidatingScreen
2 changes: 2 additions & 0 deletions react/components/UploadingScreen/index.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export { default as ValidationScreen } from './UploadingScreen'
export type { UploadingScreenProps } from './UploadingScreen'
10 changes: 10 additions & 0 deletions react/components/utils/bytesToSize.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
export const bytesToSize = (bytes: number) => {
const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB']

if (bytes === 0) return '0 Bytes'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be translated? In Portuguese I think we use the same term, but not sure if we can guarantee the same for other languages.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, good question! I will ask to translation team and confirm.

const i = Math.floor(Math.log(bytes) / Math.log(1024))

if (i === 0) return `${bytes} ${sizes[i]}`

return `${Math.round(bytes / 1024 ** i)}${sizes[i]}`
}
15 changes: 14 additions & 1 deletion react/hooks/useBulkImportDetailsQuery.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,26 @@
import useSWR from 'swr'

import { getBulkImportDetails } from '../services'
import type { BulkImportDetails } from '../services/getBulkImportDetails'

const useBulkImportDetailsQuery = (importId: string) => {
export type UseBulkImportDetailsQueryProps = {
importId?: string
onSuccess?: (data: BulkImportDetails) => void
refreshInterval?: number
}

const useBulkImportDetailsQuery = ({
importId,
onSuccess,
refreshInterval = 0,
}: UseBulkImportDetailsQueryProps) => {
return useSWR(
importId ? `/buyer-orgs/${importId}` : null,
() => getBulkImportDetails(importId),
{
refreshInterval,
revalidateOnFocus: false,
onSuccess,
}
)
}
2 changes: 1 addition & 1 deletion react/hooks/useBulkImportsQuery.ts
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@ const useBulkImportQuery = (
account ? '/buyer-orgs' : null,
() => getBulkImportList(account),
{
refreshInterval: shouldPoll ? 5 * 1000 : 0, // 30 seconds
refreshInterval: shouldPoll ? 30 * 1000 : 0, // 30 seconds
onError: errorData => {
const status = errorData?.response?.status ?? 0

18 changes: 18 additions & 0 deletions react/hooks/useValidateBulkImport.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import useSWRMutation from 'swr/mutation'

import { validateBulkImport } from '../services'

const useValidateBulkImport = ({ onSuccess }: { onSuccess?: () => void }) => {
const { trigger } = useSWRMutation(
'/buyer-orgs/start',
(_, { arg }: { arg: { importId: string } }) =>
validateBulkImport(arg.importId),
{
onSuccess,
}
)

return { startBulkImportValidation: trigger }
}

export default useValidateBulkImport
2 changes: 1 addition & 1 deletion react/package.json
Original file line number Diff line number Diff line change
@@ -36,7 +36,7 @@
},
"dependencies": {
"@vtex/admin-ui": "^0.136.1",
"@vtex/bulk-import-ui": "1.1.6",
"@vtex/bulk-import-ui": "1.1.8",
"@vtex/css-handles": "^1.0.0",
"apollo-client": "^2.6.10",
"axios": "1.4.0",
10 changes: 5 additions & 5 deletions react/services/getBulkImportDetails.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
import bulkImportClient from '.'
import type { ImportDetails, ImportReportData } from '../types/BulkImport'

type BulkImportList = Omit<ImportDetails, 'percentage'> & {
export type BulkImportDetails = Omit<ImportDetails, 'percentage'> & {
importReportList: ImportReportData[]
percentage: number
}

const getBulkImportList = async (importId: string): Promise<BulkImportList> => {
const getBulkImportDetails = async (
importId?: string
): Promise<BulkImportDetails> => {
const importListResponse = await bulkImportClient.get<ImportDetails>(
`/buyer-orgs/${importId}`
)
@@ -15,8 +17,6 @@ const getBulkImportList = async (importId: string): Promise<BulkImportList> => {

const { importResult } = data

if (!importResult?.imports) throw Error('Import result not provided')

const importList = importResult?.imports ?? []

const [totalSuccess, totalError] = importList.reduce(
@@ -49,4 +49,4 @@ const getBulkImportList = async (importId: string): Promise<BulkImportList> => {
}
}

export default getBulkImportList
export default getBulkImportDetails
7 changes: 4 additions & 3 deletions react/services/getBulkImportList.ts
Original file line number Diff line number Diff line change
@@ -15,9 +15,10 @@ const getBulkImportList = async (account: string) => {
const importListData = importListResponse.data as ImportDetails[]

return importListData
.filter(
item =>
!['ReadyToImport', 'Failed'].some(status => status === item.importState)
.filter(item =>
['InProgress', 'Completed', 'CompletedWithError'].some(
status => status === item.importState
)
)
.map(item => ({
importId: item.importId,
1 change: 1 addition & 0 deletions react/services/index.ts
Original file line number Diff line number Diff line change
@@ -3,3 +3,4 @@ export { default as getBulkImportList } from './getBulkImportList'
export { default as getBulkImportDetails } from './getBulkImportDetails'
export { default as uploadBulkImportFile } from './uploadBulkImportFile'
export { default as startBulkImport } from './startBulkImport'
export { default as validateBulkImport } from './validateBulkImport'
7 changes: 7 additions & 0 deletions react/services/validateBulkImport.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import bulkImportClient from '.'

const validateBulkImport = async (importId?: string): Promise<unknown> => {
return bulkImportClient.post(`/buyer-orgs/validate/${importId}`)
}

export default validateBulkImport
8 changes: 7 additions & 1 deletion react/types/BulkImport.d.ts
Original file line number Diff line number Diff line change
@@ -4,6 +4,8 @@ type ImportState =
| 'ReadyToImport'
| 'InProgress'
| 'Completed'
| 'InValidation'
| 'ValidationFailed'
| 'CompletedWithError'
| 'Failed'

@@ -56,6 +58,10 @@ export type ImportDetails = {
importedAt: string
importedUserEmail: string
importedUserName: string
validationResult?: {
reportDownloadLink: string
validationResult: ValidationResult[]
}
}

export type UploadFileResult = {
@@ -70,7 +76,7 @@ export type ValidationResult = {
}

export type FieldValidationError = {
description: string
description?: string
error: 'FieldValidationError'
errorDownloadLink: string
validationResult: ValidationResult[]
8 changes: 4 additions & 4 deletions react/yarn.lock
Original file line number Diff line number Diff line change
@@ -2192,10 +2192,10 @@
tiny-warning "^1.0.3"
use-debounce "^7.0.0"

"@vtex/bulk-import-ui@1.1.6":
version "1.1.6"
resolved "https://registry.yarnpkg.com/@vtex/bulk-import-ui/-/bulk-import-ui-1.1.6.tgz#aa4e812dfc9c9936ece5cee4158518f09eda6d73"
integrity sha512-1SrB3y4VOq2dD7NERYxeibnuLWAuu0um0lO/cet8b6F2z8ZivT1bE3bibDscoEI59eIr319796HTYXINjDCocg==
"@vtex/bulk-import-ui@1.1.8":
version "1.1.8"
resolved "https://registry.yarnpkg.com/@vtex/bulk-import-ui/-/bulk-import-ui-1.1.8.tgz#fcbdda205504b746ae87e1398944e3c14c5d9670"
integrity sha512-wZh6++7yOL8ZY29ga2lEGrpQJVUnDF/9ILIlC/m8grxCKCGu1FsVDL1dDBMtJ8l48sW7SrIquHJTMm5DhTr+QA==
dependencies:
"@vtex/admin-ui" "^0.136.1"


Unchanged files with check annotations Beta

Check warning on line 15 in react/MyOrganizationLink.tsx

GitHub Actions / QE / Lint Node.js

Unexpected any. Specify a different type

Check warning on line 17 in react/MyOrganizationLink.tsx

GitHub Actions / QE / Lint Node.js

Unexpected any. Specify a different type
const uid = setGUID(newAddressState)
const duplicated = data?.getCostCenterById?.addresses?.find(
(item: any) => item.addressId === uid

Check warning on line 305 in react/admin/CostCenterDetails.tsx

GitHub Actions / QE / Lint Node.js

Unexpected any. Specify a different type
)
let isDuplicatedError = false
</PageBlock>
<PageBlock title={formatMessage(messages.addresses)}>
<div className="flex">
{addresses.map((address: any, index) => {

Check warning on line 597 in react/admin/CostCenterDetails.tsx

GitHub Actions / QE / Lint Node.js

Unexpected any. Specify a different type
return (
<div key={index} className="w-25 ma3">
<Card>
interface CustomFieldsTableProps {
customFields: CustomField[]
handleDelete: (index: number) => void
handleUpdate: (index: number, customField: any) => void

Check warning on line 28 in react/admin/CustomFieldsTable.tsx

GitHub Actions / QE / Lint Node.js

Unexpected any. Specify a different type
}
const CustomFieldsTable: React.FC<CustomFieldsTableProps> = ({
const { data, getBodyCell, getHeadCell, getTable } = useTableState({
columns,
// item type is built using columns which makes it incompatible with customFields type
items: customFields as any,

Check warning on line 155 in react/admin/CustomFieldsTable.tsx

GitHub Actions / QE / Lint Node.js

Unexpected any. Specify a different type
})
return (
))}
</THead>
<TBody>
{data.map((item: any) => {

Check warning on line 167 in react/admin/CustomFieldsTable.tsx

GitHub Actions / QE / Lint Node.js

Unexpected any. Specify a different type
return (
<TBodyRow key={item.name}>
{columns.map(column => {
collectionsState,
setCollectionsState,
}: {
getSchema: (argument?: any) => any

Check warning on line 21 in react/admin/OrganizationDetails/OrganizationDetailsCollections.tsx

GitHub Actions / QE / Lint Node.js

Unexpected any. Specify a different type

Check warning on line 21 in react/admin/OrganizationDetails/OrganizationDetailsCollections.tsx

GitHub Actions / QE / Lint Node.js

Unexpected any. Specify a different type
collectionsState: Collection[]
setCollectionsState: (value: any) => void

Check warning on line 23 in react/admin/OrganizationDetails/OrganizationDetailsCollections.tsx

GitHub Actions / QE / Lint Node.js

Unexpected any. Specify a different type
}) => {
/**
* Hooks