Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataFlow backfill - disabled but most of the functionality is there #1261

Merged
merged 70 commits into from
Sep 25, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
a4cf14b
Turning everything back on
travjenkins Sep 16, 2024
48a53e8
Moving steps stuff into a manual machine state in a store
travjenkins Sep 16, 2024
89736ca
Getting actions buttons wired up
travjenkins Sep 16, 2024
9a03b21
Moving more stuff into content
travjenkins Sep 16, 2024
46775ca
checking first step
travjenkins Sep 16, 2024
c0491d3
Updating typing a lot
travjenkins Sep 17, 2024
f820b36
typing tweaks
travjenkins Sep 17, 2024
38ee682
Storing off step definition
travjenkins Sep 17, 2024
6ffd672
Updating key to keep consistent
travjenkins Sep 17, 2024
afebcf9
Adding support for a context to store data... but this feels wrong
travjenkins Sep 17, 2024
d5b94e1
adding quick support to store off things in a keyed / machine way
travjenkins Sep 17, 2024
1cf48d3
cleaning up handled as we are not using this
travjenkins Sep 17, 2024
1cf6db5
Forcing to pass an index when updating a step
travjenkins Sep 17, 2024
2ce4b59
quick hack and typing update
travjenkins Sep 17, 2024
cf48496
Checking locked to skip showing the load bar
travjenkins Sep 18, 2024
0e329e9
Adding handled to get no response when an insert is done to reduce ne…
travjenkins Sep 18, 2024
8cacc59
Storing stuff to re-enable the capture
travjenkins Sep 18, 2024
232e8b9
marking valid so the next step works
travjenkins Sep 18, 2024
9503ce3
Starting to wire up the last step
travjenkins Sep 18, 2024
cdb05a9
Updating ChipList to:
travjenkins Sep 19, 2024
e94ff64
Making code a bit safer
travjenkins Sep 19, 2024
fd5ad1f
Typing for chiplist changes
travjenkins Sep 19, 2024
c46a656
styling the options better so the collections don't overlap the option
travjenkins Sep 19, 2024
653a701
Reset the state when leaving the flow
travjenkins Sep 19, 2024
67a86e6
Probably best to just store the publication status as a whole
travjenkins Sep 19, 2024
39e905f
Adding a "review selections" step as an idea
travjenkins Sep 19, 2024
3e43464
Refactoring to reduce need of functions
travjenkins Sep 19, 2024
6238b84
Resetting when closing as doing it in store is not working
travjenkins Sep 19, 2024
bdedc89
Moving hydrator within modal so it is always reset when opening and c…
travjenkins Sep 19, 2024
3a9d9fb
Breaking out the review table
travjenkins Sep 19, 2024
447d2b1
Adding more plural support
travjenkins Sep 19, 2024
84c8f67
Updating content as requested
travjenkins Sep 19, 2024
88e71f8
styling tweaks
travjenkins Sep 19, 2024
147c999
Putting routes nested
travjenkins Sep 19, 2024
6b0c903
Sharing the link to details
travjenkins Sep 19, 2024
0670fe8
A bit better error handling and messaging
travjenkins Sep 19, 2024
8ef6fed
clearing our errors when success
travjenkins Sep 19, 2024
b4eea48
Using related collections list
travjenkins Sep 19, 2024
a136944
Making sure we only count the enabled bindings in the list
travjenkins Sep 24, 2024
3c4a4dd
cleaning up hook that is not used
travjenkins Sep 24, 2024
6c95eab
Fixing the warning to display correctly
travjenkins Sep 24, 2024
1f4a57d
Making the save and publish logging and warning shared
travjenkins Sep 24, 2024
f72ddf4
Moved to a component fold
travjenkins Sep 24, 2024
29b55ce
Updating handling
travjenkins Sep 24, 2024
aa61b36
Ended up not needing the hook
travjenkins Sep 24, 2024
aa31af2
Updating messaging
travjenkins Sep 24, 2024
97682e3
Not show logs right away
travjenkins Sep 24, 2024
bb95c6a
Merge remote-tracking branch 'origin/main' into travjenkins/feature/f…
travjenkins Sep 24, 2024
ab92377
Merge remote-tracking branch 'origin/main' into travjenkins/feature/f…
travjenkins Sep 24, 2024
b38dc4d
Merge branch 'main' into travjenkins/feature/full-entity-refresh/serv…
travjenkins Sep 24, 2024
dc81e37
Adding a uuid so make things a bit more traceable
travjenkins Sep 25, 2024
f1abfca
Adding a LR event to try to make these sessions easier to find
travjenkins Sep 25, 2024
01b7099
Cleaning up comments
travjenkins Sep 25, 2024
9c4c128
Cleaning up comments
travjenkins Sep 25, 2024
e64c7d3
Adding some handling for draft errors
travjenkins Sep 25, 2024
e10ca70
Wiring up to exit the flow when complete
travjenkins Sep 25, 2024
6b3396b
Updating details message
travjenkins Sep 25, 2024
35ee6d8
Disabling all the new stuff so this can be merged
travjenkins Sep 25, 2024
a4a6de2
Removing `detail` from the forms.
travjenkins Sep 25, 2024
bb93b11
Removing code that is not needed
travjenkins Sep 25, 2024
44efb6b
PR: comments
travjenkins Sep 25, 2024
ff37f81
PR: commenting out function that isn't used
travjenkins Sep 25, 2024
7e7e6fe
PR: commenting out to disable this
travjenkins Sep 25, 2024
e2efcb6
Making code a bit simpler
travjenkins Sep 25, 2024
a89a4fe
Getting the chips aligned
travjenkins Sep 25, 2024
e901568
PR: putting original count back in
travjenkins Sep 25, 2024
c4e48f8
Updating content to be more clear
travjenkins Sep 25, 2024
6750662
PR: typo and small content change
travjenkins Sep 25, 2024
8396d89
PR: content typo
travjenkins Sep 25, 2024
2e80049
Merge branch 'main' into travjenkins/feature/full-entity-refresh/serv…
travjenkins Sep 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/components/editor/Bindings/Backfill/BackfillCount.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { useMemo } from 'react';
import { useIntl } from 'react-intl';
import {
useBinding_backfilledBindings_count,
useBinding_enabledCollections_count,
useBinding_collections_count,
} from 'stores/Binding/hooks';
import { BackfillCountProps } from './types';

Expand All @@ -13,7 +13,7 @@ function BackfillCount({ disabled }: BackfillCountProps) {
const entityType = useEntityType();

const backfillCount = useBinding_backfilledBindings_count();
const bindingsTotal = useBinding_enabledCollections_count();
const bindingsTotal = useBinding_collections_count();

// Only reason noBackfill is in here is because we are already running the memo on backfillCount change
const [noBackfill, itemType_backfill, itemType_bindings] = useMemo(() => {
Expand Down
8 changes: 3 additions & 5 deletions src/components/editor/Bindings/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -110,11 +110,9 @@ function BindingsMultiEditor({
{workflow === 'capture_edit' ||
workflow === 'materialization_edit' ? (
<Backfill
description={
<FormattedMessage
id={`workflows.collectionSelector.manualBackfill.message.${entityType}.allBindings`}
/>
}
description={intl.formatMessage({
id: `workflows.collectionSelector.manualBackfill.message.${entityType}.allBindings`,
})}
/>
) : null}
</Stack>
Expand Down
9 changes: 6 additions & 3 deletions src/components/shared/ChipList/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ function ChipList({
}: ChipListProps) {
const intl = useIntl();

const listScroller = useRef<any>(null);
const listScroller = useRef<HTMLDivElement>(null);

// Format data coming in so we can still pass in a list of strings
const formattedValues = useMemo(() => {
Expand Down Expand Up @@ -48,9 +48,11 @@ function ChipList({
// When all chips are shown scroll down just a hair to try to make
// sure the user knows that the list is scrollable
useEffect(() => {
if (maxRender === valueLength) {
listScroller.current?.scrollTo(undefined, 10);
if (!listScroller.current || maxRender !== valueLength) {
return;
}

listScroller.current.scrollTop += 20;
}, [maxRender, valueLength]);

return (
Expand All @@ -60,6 +62,7 @@ function ChipList({
display: 'flex',
flexWrap: 'wrap',
listStyle: 'none',
alignItems: 'center',
p: 0,
m: 0,
minWidth: 100,
Expand Down
4 changes: 2 additions & 2 deletions src/lang/en-US/Workflows.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,9 @@ export const Workflows: Record<string, string> = {
'workflows.collectionSelector.manualBackfill.notSupported.title': `This {entityType} doesn’t support backfills.`,
'workflows.collectionSelector.manualBackfill.notSupported.message': `To backfill, disable each binding, save and then re-enable and save.`,
'workflows.collectionSelector.manualBackfill.message.capture': `Trigger a backfill of this collection from the source when published.`,
'workflows.collectionSelector.manualBackfill.message.capture.allBindings': `Trigger a backfill of all collections from the source when published.`,
'workflows.collectionSelector.manualBackfill.message.capture.allBindings': `Trigger a backfill of all collection from the source when published. The UI will mark all collections as backfilled but the server will filter out those that cannot be backfilled (e.g. disabled collections).`,
travjenkins marked this conversation as resolved.
Show resolved Hide resolved
'workflows.collectionSelector.manualBackfill.message.materialization': `Trigger a backfill from the source collection to its materialized resource when published.`,
'workflows.collectionSelector.manualBackfill.message.materialization.allBindings': `Trigger a backfill from all source collections to their materialized resource when published.`,
'workflows.collectionSelector.manualBackfill.message.materialization.allBindings': `Trigger a backfill from all source collections to their materialized resource when published. The UI will mark all collections as backfilled but the server will filter out those that cannot be backfilled (e.g. disabled collections).`,
'workflows.collectionSelector.manualBackfill.cta.backfill': `Backfill`,
'workflows.collectionSelector.manualBackfill.count': `{backfillCount} of {bindingsTotal} {itemType} marked for backfill`,
'workflows.collectionSelector.manualBackfill.count.empty': `no {itemType} marked for backfill`,
Expand Down
3 changes: 3 additions & 0 deletions src/stores/Binding/hooks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,9 @@ export const useBinding_collections = () => {
return useBindingStore(useShallow((state) => state.getCollections()));
};

export const useBinding_collections_count = () =>
useBindingStore(useShallow((state) => state.getCollections().length));

export const useBinding_enabledCollections_count = () =>
useBindingStore(
useShallow((state) => state.getEnabledCollections().length)
Expand Down