Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Implemented ColumnAnnotationCard component #59

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Conversation

rmanaem
Copy link
Contributor

@rmanaem rmanaem commented Feb 25, 2025

Checklist

This section is for the PR reviewer

  • PR has an interpretable title with a prefix ([ENH], [FIX], [REF], [TST], [CI], [MNT], [INF], [MODEL], [DOC]) (see our Contributing Guidelines for more info)
  • PR has a label for the release changelog or skip-release (to be applied by maintainers only)
  • PR links to GitHub issue with mention Closes #XXXX
  • Tests pass
  • Checks pass

For new features:

  • Tests have been added

For bug fixes:

  • There is at least one test that would fail under the original bug conditions.

Summary by Sourcery

Implement column annotation functionality, allowing users to view and edit column metadata. Introduce a new ColumnAnnotationCard component for individual column annotation and integrate it into a paginated ColumnAnnotation view. Enhance the data store to manage and persist column metadata changes.

New Features:

  • Implement a ColumnAnnotationCard component that allows users to view and edit column metadata, including description, data type, and standardized variable.
  • Add a ColumnAnnotation component to display a paginated list of ColumnAnnotationCard components.
  • Integrate the new components with the data store to persist changes to column metadata.
  • Add a configuration file to store default standardized variables.

Enhancements:

  • Refactor the UploadCard component to use Material UI's CardHeader and CardContent components for improved structure and styling.

Tests:

  • Add Cypress component tests for the ColumnAnnotationCard component.

Copy link

sourcery-ai bot commented Feb 25, 2025

Reviewer's Guide by Sourcery

This pull request introduces the ColumnAnnotationCard component, which allows users to annotate columns with descriptions, data types, and standardized variables. The UploadCard component was also updated to use Material UI's CardHeader and CardContent components. The data store was modified to include functionality for managing column metadata and standardized variables. Finally, the theme was updated to remove text transformations from buttons and toggle buttons.

Updated class diagram for the Columns type

classDiagram
    class Columns {
      <<interface>>
      [key: number]: Column
    }
    class Column {
      header: string
      description: string | null
      dataType: 'Categorical' | 'Continuous' | null
      standardizedVariable: StandardizedVarible | null
    }
    class StandardizedVarible {
      <<interface>>
      identifier: string
    }

    Columns -- Column : contains
    Column -- StandardizedVarible : has *
Loading

Class diagram for ColumnAnnotationCard component

classDiagram
    class ColumnAnnotationCard {
        id: number
        header: string
        description: string | null
        dataType: 'Categorical' | 'Continuous' | null
        standardizedVariable: StandardizedVarible | null
        standardizedVariableOptions: StandardizedVaribles
        onDescriptionChange: (columnId: number, newDescription: string | null) => void
        onDataTypeChange: (columnId: number, newDataType: 'Categorical' | 'Continuous' | null) => void
        onStandardizedVariableChange: (columnId: number, newStandardizedVariable: StandardizedVarible | null) => void
    }

    class StandardizedVarible {
        identifier: string
    }

    class StandardizedVaribles {
        [key: string]: StandardizedVarible
    }

    ColumnAnnotationCard -- StandardizedVarible : uses
    ColumnAnnotationCard -- StandardizedVaribles : uses
Loading

File-Level Changes

Change Details Files
Introduced a new ColumnAnnotationCard component for annotating columns.
  • Created a new component to display and edit column metadata, including description, data type, and standardized variable.
  • Implemented functionality to toggle between editing and viewing the column description.
  • Added a toggle button group for selecting the data type (Categorical or Continuous).
  • Integrated an Autocomplete component for selecting a standardized variable from a predefined list.
  • Added Cypress component tests for the new component.
src/components/ColumnAnnotationCard.tsx
cypress/component/ColumnAnnotationCard.cy.tsx
Modified the UploadCard component to use Material UI's CardHeader and CardContent components.
  • Replaced the Typography component used for the title with CardHeader.
  • Wrapped the file uploader and preview components with CardContent.
src/components/UploadCard.tsx
Implemented column annotation functionality in the ColumnAnnotation component.
  • Added state management for the current page and columns per page.
  • Implemented pagination for displaying columns in chunks.
  • Added handlers for updating column descriptions, data types, and standardized variables in the data store.
  • Rendered ColumnAnnotationCard components for each column on the current page.
src/components/ColumnAnnotation.tsx
Updated the data store to include functionality for managing column metadata and standardized variables.
  • Added properties for storing standardized variables and a method to update the description of a column.
  • Added methods to update the data type and standardized variable of a column.
  • Included a default configuration file (default.json) containing a list of standardized variables.
  • Added a test suite for the new store actions.
src/stores/data.ts
src/stores/data.test.ts
configs/default.json
Updated the theme to remove text transformations from buttons and toggle buttons.
  • Modified the MuiButton and MuiToggleButton components in the theme to prevent text transformations.
src/theme.ts
Added new types for standardized variables.
  • Added new interfaces StandardizedVarible and StandardizedVaribles.
src/utils/types.ts

Assessment against linked issues

Issue Objective Addressed Explanation
#51 The component should have the column header as the title
#51 The component should have a free-form text field for description which user can edit/update
#51 The component should have an element for designating categorical/continuous data type
#51 The component should have a dropdown for selecting the standardized variable
#51 component test
#53 Update the state with changes made on the columnAnnotationCard component.

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!
  • Generate a plan of action for an issue: Comment @sourcery-ai plan on
    an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

netlify bot commented Feb 25, 2025

Deploy Preview for staging-annotation ready!

Name Link
🔨 Latest commit ce0aeb2
🔍 Latest deploy log https://app.netlify.com/sites/staging-annotation/deploys/67be4382c1f3d10008b6b46d
😎 Deploy Preview https://deploy-preview-59--staging-annotation.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@rmanaem rmanaem added the pr-minor Non-breaking feature or enhancement, will increment minor version (0.+1.0) label Feb 25, 2025
Copy link

codecov bot commented Feb 25, 2025

Codecov Report

Attention: Patch coverage is 73.23944% with 19 lines in your changes missing coverage. Please review.

Project coverage is 79.79%. Comparing base (e7d0640) to head (ce0aeb2).

Files with missing lines Patch % Lines
src/components/ColumnAnnotation.tsx 58.62% 12 Missing ⚠️
src/components/ColumnAnnotationCard.tsx 83.33% 5 Missing ⚠️
src/components/UploadCard.tsx 75.00% 1 Missing ⚠️
src/stores/data.ts 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #59      +/-   ##
==========================================
- Coverage   81.77%   79.79%   -1.99%     
==========================================
  Files          20       21       +1     
  Lines         225      292      +67     
  Branches       46       56      +10     
==========================================
+ Hits          184      233      +49     
- Misses         41       59      +18     
Flag Coverage Δ
tests 79.79% <73.23%> (-1.99%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rmanaem rmanaem requested a review from surchs February 25, 2025 23:59
Copy link

@surchs surchs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @rmanaem, thanks for the PR - and sorry it took so long to review.

A little like last time: the UI looks amazing, really very nice and close to what we would want in the final product. My main concerns are with the readability and modularity of the components - and how that will affect our ability to maintain and add things later.

There is also a bug with the standardized variables that happens because of duplicated state. But ultimately I think this is also related to the length and relative complexity of the components that make it harder to see what's going on and to reason about.

So I would say we factor things out, make the components very focused on single tasks. I would also say: be very hesitant to introduce complexity just for better UX at this point. E.g. the description editing UI is pretty, but you had to do a lot of work over a simple and less pretty textbox. I strongly suggest we leave such things for later and focus on getting the key functionality going now.


const totalPages = Math.ceil(columnEntries.length / columnsPerPage);

const handlePageChange = (_: React.ChangeEvent<unknown>, page: number) => {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍒 handlePageChange is really concerned with handling the paginated views of the column table/card element, correct? I'd say we make that explicit to differentiate from navigation. Maybe handlePaginationChange here, and then also make sure we have a distinct name for "stages" or "views" that is different from pages.

Comment on lines +82 to 103
updateColumnDescription: (columnId: number, description: string | null) => {
set((state) => ({
columns: R.assocPath([columnId, 'description'], description, state.columns),
}));
},

updateColumnDataType: (columnId: number, dataType: 'Categorical' | 'Continuous' | null) => {
set((state) => ({
columns: R.assocPath([columnId, 'dataType'], dataType, state.columns),
}));
},

updateColumnStandardizedVariable: (
columnId: number,
standardizedVariable: StandardizedVarible | null
) => {
set((state) => ({
columns: R.assocPath([columnId, 'standardizedVariable'], standardizedVariable, state.columns),
}));
},

// Data dictionary
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may have missed this before, but why did you go with Ramda over Immer here? My sense on a first read of https://zustand.docs.pmnd.rs/guides/updating-state#deeply-nested-object and some other docs on this is that Immer is more straightforward to read and has more docs / tutorials out there.

) => {
updateColumnStandardizedVariable(columnId, newStandardizedVariable);
};

return (
<div className="flex flex-col items-center">
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are two custom hooks in here you could factor out and improve readability: one for handling the pagination (and that might be reused in other places), and one for handling updates to the columns themselves.

Comment on lines +82 to +133
{isEditingDescription ? (
<div className="flex flex-col items-center gap-4 md:flex-row">
<TextField
data-cy={`${id}-column-annotation-card-description-input`}
fullWidth
multiline
rows={3}
value={editedDescription || ''}
onChange={(e) => setEditedDescription(e.target.value)}
variant="outlined"
label="Description"
className="flex-1"
/>
<Fab
data-cy={`${id}-column-annotation-card-save-description-button`}
color="secondary"
aria-label="save"
onClick={handleSaveDescription}
size="small"
className="mt-4 md:mt-0"
>
<SaveIcon />
</Fab>
</div>
) : (
<div className="flex flex-col items-center gap-4 md:flex-row">
<div className="flex-1">
<Typography variant="subtitle1" className="mb-2 text-gray-700">
Description:
</Typography>
<Typography
data-cy={`${id}-column-annotation-card-description`}
variant="body1"
className="text-gray-700"
>
{description || 'No description provided.'}
</Typography>
</div>
<Fab
data-cy={`${id}-column-annotation-card-edit-description-button`}
color="primary"
aria-label="edit"
onClick={handleEditDescription}
size="small"
className="mt-4 md:mt-0"
>
<EditIcon />
</Fab>
</div>
)}

<div className="mt-4 flex flex-col items-center gap-4 md:flex-row">
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking pretty, but it's also a whole lot to wrap your head around. I think the whole "Description editor" section should probably be it's own component.

Comment on lines +48 to +51
const [selectedStandardizedVariableKey, setSelectedStandardizedVariableKey] = useState<
string | null
>(standardizedVariable ? standardizedVariable.identifier : null);

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That doesn't seem right. You are already keeping the "selectedStandardizedVariable" inside the store. Why do you need to also handle this in state?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact there seems to be a bug where on initial select, the variable name uses the key of the Standardized Variable (i.e. human readable name), and when you navigate away and back to the component, it instead shows the identifier. I'm pretty sure this bug is related to duplicated state here - not fully clear yet how though

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aha, yes, here you set the "value" to standardizedVariable.identifier when instead we would probably want to use the label instead. I left another comment on types.ts but I think this is caused in part by the fact that you (I believe) use the key in StandardizedVariable**S** to encode the label, and then only add in the identifier in each given StandardizedVariable instance. The reason we don't see that the dropdown value gets passed the identifier is because of the duplicated and conflicting local component state that is set to the "label" in parallel. Only when you navigate away and back does the store state override the component state and you see the identifier show up.

So bottom line: let's keep single source of truth and let's keep all relevant info (label + identifier) inside the StandardizedVariable object.

@@ -35,3 +37,11 @@ export interface DataDictionary {
};
};
}

export interface StandardizedVarible {
identifier: string;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think StandardizedVarible should also have a .name or .label attribute. Otherwise you entirely rely on the string key in the StandardizedVaribles object for the human readable label - and that's no good.

}

export interface StandardizedVaribles {
[key: string]: StandardizedVarible;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar to above: I would advise against using the name of the StandardizedVarible as the key. Either a numeric key or the identifier or whatever. But something that is 1) guaranteed to be unique, and 2) used to look up important information (like the label or identifier), but not to encode that info directly

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also minor point: Can we find a more distinct name for this object of several StandardizedVarible? That's going to confuse someone pretty certainly. How about StandardizedVaribleCollection?

_: React.ChangeEvent<unknown>,
newValue: string | null
) => {
setSelectedStandardizedVariableKey(newValue);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect. 1) because you're passing the string "label" but expect a key, 2) and more importantly because you are creating duplicated state here between the component state and the store state (set just 2 lines below)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

Comment on lines +114 to +133
it('Fires the onStandardizedVariableChange event handler with the appropriate payload when the standardized variable is changed', () => {
const spy = cy.spy().as('spy');
cy.mount(
<ColumnAnnotationCard
id={props.id}
header={props.header}
description={props.description}
dataType={props.dataType}
standardizedVariable={props.standardizedVariable}
standardizedVariableOptions={props.standardizedVariableOptions}
onDescriptionChange={props.onDescriptionChange}
onDataTypeChange={props.onDataTypeChange}
onStandardizedVariableChange={spy}
/>
);
cy.get('[data-cy="1-column-annotation-card-standardized-variable-dropdown"]').type(
'age{downarrow}{enter}'
);
cy.get('@spy').should('have.been.calledWith', 1, { identifier: 'nb:Age' });
});
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add test cases also for "if I set the StandardizedVariable to XYZ in the store, it'll show up with the label" - that would have caught the current bug

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If not feasible for the AnnotationCard because it doesn't use the store directly, then maybe as a test of the parent component where we also load the AnnotationCard child component to ensure that setting the (mocked) store leads to the expected UI rendering outcome of "I now see the label of the Variable selected, not the identifier"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-minor Non-breaking feature or enhancement, will increment minor version (0.+1.0)
Projects
None yet
2 participants