Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relocate advanced materialization settings in workflows #1417

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 6 additions & 7 deletions src/components/collection/ResourceConfig.tsx
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import { Box, Typography } from '@mui/material';
import ResourceConfigForm from 'components/collection/ResourceConfigForm';
import AdvancedOptions from 'components/editor/Bindings/AdvancedOptions';
import Backfill from 'components/editor/Bindings/Backfill';
import FieldSelectionViewer from 'components/editor/Bindings/FieldSelection';
import TimeTravel from 'components/editor/Bindings/TimeTravel';
import { useEditorStore_queryResponse_draftedBindingIndex } from 'components/editor/Store/hooks';
import ErrorBoundryWrapper from 'components/shared/ErrorBoundryWrapper';
import { useEntityType } from 'context/EntityContext';
Expand Down Expand Up @@ -83,12 +83,11 @@ function ResourceConfig({
/>
) : null}

{entityType === 'materialization' ? (
<TimeTravel
bindingUUID={bindingUUID}
collectionName={collectionName}
/>
) : null}
<AdvancedOptions
bindingIndex={draftedBindingIndex}
bindingUUID={bindingUUID}
collectionName={collectionName}
/>
</>
);
}
Expand Down
65 changes: 65 additions & 0 deletions src/components/editor/Bindings/AdvancedOptions/index.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
import { Stack, Typography } from '@mui/material';
import WrapperWithHeader from 'components/shared/Entity/WrapperWithHeader';
import ErrorBoundryWrapper from 'components/shared/ErrorBoundryWrapper';
import { useEntityType } from 'context/EntityContext';
import { useIntl } from 'react-intl';
import { useBindingStore } from 'stores/Binding/Store';
import OnIncompatibleSchemaChange from '../OnIncompatibleSchemaChange';
import TimeTravel from '../TimeTravel';
import { AdvancedOptionsProps } from './types';

export default function AdvancedOptions({
bindingIndex,
bindingUUID,
collectionName,
}: AdvancedOptionsProps) {
const intl = useIntl();

const entityType = useEntityType();

const fullSourceErrorExists = useBindingStore((state) => {
const fullSourceErrors = state.fullSourceConfigs[bindingUUID]?.errors;

if (!fullSourceErrors) {
return false;
}

return fullSourceErrors.length > 0;
});

const onIncompatibleSchemaChangeErrorExists = useBindingStore(
(state) => state.onIncompatibleSchemaChangeErrorExists.binding
);
Comment on lines +20 to +32
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not need to change for this PR... but I think it would be safe to put these two into a single shallow call. I think that might help reduce how often these might change.


if (entityType !== 'materialization') {
return null;
}

return (
<WrapperWithHeader
forceOpen={
fullSourceErrorExists || onIncompatibleSchemaChangeErrorExists
}
header={
<Typography variant="formSectionHeader">
{intl.formatMessage({
id: 'workflows.advancedSettings.title',
})}
</Typography>
}
hideBorder
mountClosed
>
<Stack spacing={4}>
<ErrorBoundryWrapper>
<OnIncompatibleSchemaChange bindingIndex={bindingIndex} />
</ErrorBoundryWrapper>

<TimeTravel
bindingUUID={bindingUUID}
collectionName={collectionName}
/>
</Stack>
</WrapperWithHeader>
);
}
5 changes: 5 additions & 0 deletions src/components/editor/Bindings/AdvancedOptions/types.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
export interface AdvancedOptionsProps {
bindingIndex: number;
bindingUUID: string;
collectionName: string;
}
28 changes: 9 additions & 19 deletions src/components/editor/Bindings/Backfill/index.tsx
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
import ErrorBoundryWrapper from 'components/shared/ErrorBoundryWrapper';
import { useEntityType } from 'context/EntityContext';
import { useEntityWorkflow_Editing } from 'context/Workflow';
import { FormattedMessage } from 'react-intl';
import OnIncompatibleSchemaChange from '../OnIncompatibleSchemaChange';
import BackfillButton from './BackfillButton';
import SectionWrapper from './SectionWrapper';
import { BackfillProps } from './types';
Expand All @@ -16,24 +14,16 @@ export default function Backfill({

const showBackfillButton = isEdit && bindingIndex > -1 && collectionEnabled;

return showBackfillButton || entityType === 'materialization' ? (
return showBackfillButton ? (
<SectionWrapper>
{showBackfillButton ? (
<BackfillButton
bindingIndex={bindingIndex}
description={
<FormattedMessage
id={`workflows.collectionSelector.manualBackfill.message.${entityType}`}
/>
}
/>
) : null}

{entityType === 'materialization' ? (
<ErrorBoundryWrapper>
<OnIncompatibleSchemaChange bindingIndex={bindingIndex} />
</ErrorBoundryWrapper>
) : null}
<BackfillButton
bindingIndex={bindingIndex}
description={
<FormattedMessage
id={`workflows.collectionSelector.manualBackfill.message.${entityType}`}
/>
}
/>
</SectionWrapper>
) : null;
}
2 changes: 1 addition & 1 deletion src/components/editor/Bindings/FieldSelection/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ function FieldSelectionViewer({
const loading = formActive || formStatus === FormStatus.TESTING_BACKGROUND;

return (
<Box sx={{ mt: 3 }}>
<Box sx={{ my: 3 }}>
<Stack direction="row" spacing={1} sx={{ mb: 2 }}>
<Stack spacing={1}>
<Stack style={{ alignItems: 'center' }} direction="row">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ function Form({ bindingIndex = -1 }: OnIncompatibleSchemaChangeProps) {
return (
<IncompatibleSchemaChangeForm
currentSetting={currentSetting ? currentSetting : ''}
scope="binding"
updateDraftedSetting={updateServer}
/>
);
Expand Down
2 changes: 1 addition & 1 deletion src/components/editor/Bindings/TimeTravel/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ function TimeTravel({ bindingUUID, collectionName }: Props) {
<Stack>
<Stack spacing={1} sx={{ mb: 2 }}>
<Stack direction="row">
<Typography variant="formSectionHeader">
<Typography style={{ fontWeight: 500 }}>
<FormattedMessage id="notBeforeNotAfter.header" />
</Typography>
</Stack>
Expand Down
3 changes: 3 additions & 0 deletions src/components/editor/Bindings/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import { LocalZustandProvider } from 'context/LocalZustand';
import { alternativeReflexContainerBackground } from 'context/Theme';
import { useEntityWorkflow } from 'context/Workflow';

import AdvancedOptions from 'components/materialization/AdvancedOptions';
import { DraftSpecQuery } from 'hooks/useDraftSpecs';
import { useServerUpdateRequiredMonitor } from 'hooks/useServerUpdateRequiredMonitor';
import { ReactNode, useEffect, useMemo } from 'react';
Expand Down Expand Up @@ -109,6 +110,8 @@ function BindingsMultiEditor({
{entityType === 'materialization' ? <SourceCapture /> : null}

<Backfill />

<AdvancedOptions />
</Stack>

<ListAndDetails
Expand Down
10 changes: 10 additions & 0 deletions src/components/incompatibleSchemaChange/Form.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,25 @@ import useSupportedOptions from 'hooks/OnIncompatibleSchemaChange/useSupportedOp
import { useEffect, useMemo, useState } from 'react';
import { useIntl } from 'react-intl';
import { stringifyJSON } from 'services/stringify';
import { useBindingStore } from 'stores/Binding/Store';
import { useFormStateStore_isActive } from 'stores/FormState/hooks';
import { autoCompleteDefaultProps } from './shared';
import { BaseFormProps } from './types';

export default function IncompatibleSchemaChangeForm({
currentSetting,
scope,
updateDraftedSetting,
}: BaseFormProps) {
const intl = useIntl();

const [inputValue, setInputValue] = useState('');
const [invalidSetting, setInvalidSetting] = useState(false);

const setOnIncompatibleSchemaChangeErrorExists = useBindingStore(
(state) => state.setOnIncompatibleSchemaChangeErrorExists
);

const formActive = useFormStateStore_isActive();

const options = useSupportedOptions();
Expand Down Expand Up @@ -56,6 +62,10 @@ export default function IncompatibleSchemaChangeForm({
setInvalidSetting(false);
}, [currentSetting, selection]);

useEffect(() => {
setOnIncompatibleSchemaChangeErrorExists(invalidSetting, scope);
}, [invalidSetting, scope, setOnIncompatibleSchemaChangeErrorExists]);

return (
<Stack spacing={1}>
{invalidSetting ? (
Expand Down
1 change: 1 addition & 0 deletions src/components/incompatibleSchemaChange/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { choices } from './shared';

export interface BaseFormProps {
currentSetting: any;
scope: 'binding' | 'spec';
updateDraftedSetting: Function;
}

Expand Down
42 changes: 42 additions & 0 deletions src/components/materialization/AdvancedOptions.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import { Box, Typography } from '@mui/material';
import WrapperWithHeader from 'components/shared/Entity/WrapperWithHeader';
import ErrorBoundryWrapper from 'components/shared/ErrorBoundryWrapper';
import { useEntityType } from 'context/EntityContext';
import { useIntl } from 'react-intl';
import { useBindingStore } from 'stores/Binding/Store';
import OnIncompatibleSchemaChange from './OnIncompatibleSchemaChange';

export default function AdvancedOptions() {
const intl = useIntl();

const entityType = useEntityType();

const onIncompatibleSchemaChangeErrorExists = useBindingStore(
(state) => state.onIncompatibleSchemaChangeErrorExists.spec
);

if (entityType !== 'materialization') {
return null;
}

return (
<Box style={{ maxWidth: 850 }}>
<WrapperWithHeader
forceOpen={onIncompatibleSchemaChangeErrorExists}
header={
<Typography variant="formSectionHeader">
{intl.formatMessage({
id: 'workflows.advancedSettings.title',
})}
</Typography>
}
hideBorder
mountClosed
>
<ErrorBoundryWrapper>
<OnIncompatibleSchemaChange />
</ErrorBoundryWrapper>
</WrapperWithHeader>
</Box>
);
}
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ export default function Form() {
return (
<IncompatibleSchemaChangeForm
currentSetting={currentSetting}
scope="spec"
updateDraftedSetting={updateServer}
/>
);
Expand Down
28 changes: 11 additions & 17 deletions src/components/shared/Entity/Backfill.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,30 +3,24 @@ import SectionWrapper from 'components/editor/Bindings/Backfill/SectionWrapper';
import { useEntityType } from 'context/EntityContext';
import { useEntityWorkflow_Editing } from 'context/Workflow';
import { useIntl } from 'react-intl';
import OnIncompatibleSchemaChange from '../../materialization/OnIncompatibleSchemaChange';
import ErrorBoundryWrapper from '../ErrorBoundryWrapper';

export default function Backfill() {
const intl = useIntl();

const entityType = useEntityType();
const isEdit = useEntityWorkflow_Editing();

return isEdit || entityType === 'materialization' ? (
<SectionWrapper>
{isEdit ? (
<BackfillButton
description={intl.formatMessage({
id: `workflows.collectionSelector.manualBackfill.message.${entityType}.allBindings`,
})}
/>
) : null}
if (!isEdit) {
return null;
}

{entityType === 'materialization' ? (
<ErrorBoundryWrapper>
<OnIncompatibleSchemaChange />
</ErrorBoundryWrapper>
) : null}
return (
<SectionWrapper>
<BackfillButton
description={intl.formatMessage({
id: `workflows.collectionSelector.manualBackfill.message.${entityType}.allBindings`,
})}
/>
</SectionWrapper>
) : null;
);
}
2 changes: 1 addition & 1 deletion src/lang/en-US/Materializations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ export const Materializations: Record<string, string> = {
'materializationEdit.test.inProgress': `Please wait while we try to connect to the destination.`,

'materializationEdit.collectionSelector.heading': `Collection Selector`,
'materializationEdit.collectionSelector.instructions': `The collections bound to your existing materialization. To update the configuration, please update the fields under the Config tab. To update the schema, click Edit under the Collection tab.`,
'materializationEdit.collectionSelector.instructions': `The collections bound to your materialization. Update configuration under the Endpoint Config tab.`,

'materializationEdit.resourceConfig.heading': `Resource Configuration`,
'materializationEdit.save.failedErrorTitle': `Materialization Save Failed`,
Expand Down
15 changes: 15 additions & 0 deletions src/stores/Binding/Store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ const getInitialMiscData = (): Pick<
| 'discoveredCollections'
| 'evolvedCollections'
| 'onIncompatibleSchemaChange'
| 'onIncompatibleSchemaChangeErrorExists'
| 'rediscoveryRequired'
| 'resourceConfigErrorsExist'
| 'resourceConfigErrors'
Expand All @@ -185,6 +186,10 @@ const getInitialMiscData = (): Pick<
discoveredCollections: [],
evolvedCollections: [],
onIncompatibleSchemaChange: undefined,
onIncompatibleSchemaChangeErrorExists: {
binding: false,
spec: false,
},
rediscoveryRequired: false,
resourceConfigErrorsExist: false,
resourceConfigErrors: [],
Expand Down Expand Up @@ -926,6 +931,16 @@ const getInitialState = (
);
},

setOnIncompatibleSchemaChangeErrorExists: (value, key) => {
set(
produce((state: BindingState) => {
state.onIncompatibleSchemaChangeErrorExists[key] = value;
}),
false,
'On Incompatible Schema Change Error Exists Set'
);
},
Comment on lines +934 to +942
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

100% do not want to handle this in the PR. However, I think we should not have scope be involved with setting this and we should eventually add this to the meta data of the binding.

That way the new fields can also show an error icon in the selector.
image


setResourceSchema: async (val) => {
const resolved = await getDereffedSchema(val);

Expand Down
8 changes: 8 additions & 0 deletions src/stores/Binding/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,14 @@ export interface BindingState
value: string | undefined,
bindingUUID: string | null
) => void;
onIncompatibleSchemaChangeErrorExists: {
binding: boolean;
spec: boolean;
};
setOnIncompatibleSchemaChangeErrorExists: (
value: boolean,
key: 'binding' | 'spec'
) => void;

// Resource Config
resourceConfigs: ResourceConfigDictionary;
Expand Down
Loading