DataSources module

[artf]

The goal of this new module is to allow loading generic data (eg. from API, DB, etc.) and designing the layout with them.

Some ideas:

const dataColl: DataSource = editor.DataSources.add({
    id: 'collection-id',
    label: 'Products',
    category: 'My product category',
    uniqueColumn: 'id',
    schema: {
      id: { type: 'string' },
      name: { type: 'string', order: 1 },
      age: { type: 'number' },
      available: { type: 'boolean' },
      avatar: { type: 'url-image' },
      author: { type: 'ref', ref: 'collection-authors', by: 'id' },
      categories: { type: 'refs', ref: 'collection-categories', by: 'id' },
    },
    records: [
      { id: 1, name: 'test', age: 22, author: '123', categories: [1,2,3] }
    ]
  })

  const dataSource = editor.DataSources.get('collection-id') as DataSource;
  dataSource.setRecords([{ id: 'id123', .... }]);
  dataSource.removeRecord('id123');
  dataSource.addRecord({ id: 'id123', .... });
  const dataRecord = datsSource.getRecord('id123') as DataRecord;
  dataRecord.set({ age: '30' });

  dataSource.getRecordValue('collection-id.id123.age');
  dataSource.setRecordValue('collection-id.id123.age', 35);

Started already something here: https://github.com/GrapesJS/grapesjs/tree/feature/data-sources

[lexoyo]

Great feature
It would be convenient to use graphql or openapi types to create the schema automatically

[artf]

Yeah here is more about having already the data.

I haven't thought yet about providing the proper API for the DataSourceFetcher but I think here it would be nice to have something you're suggesting (probably as a plugin), where you provide something like an OpenAPI file and a plugin is able to generate all the collections and proper data fetchers for you.

[lexoyo]

Awesome 👍
Did you get a chance to look at how Silex datasource plugin works?
We do WordPress front ends with it and no-code devs took it in hand easily

[lexoyo]

It retrieves graphql types and let the user create expressions with it
https://docs.silex.me/en/user/cms

[lexoyo]

I volunteer to contribute to such a plugin or the feature itself
Let me know where I can get started or if it is not expected or too early

[artf]

I'll try to check your work soon, thank you Alex 🙇‍♂️
I'd probably need to spend a bit more time defining the public API but I think you can start checking the branch I started here with the first related component (ComponentDataVariable)

[raaaahman]

Hello, I've started trying out this module on the dev branch.

Something that bothers me is the fact you have to hardcode the records' id in your paths:

editor.addComponents([
 {
   type: 'data-variable',
   tagName: 'h4',
   path: 'comments.c251.name'
  }
])

This is good as long as I add such components programmatically, so I can parse the data source and create one component per variable per record:

editor.DataSources.get('comments').getRecords().forEach(
  (record) => {
    editor.addComponents([
      {
        type: 'data-variable',
        tagName: 'h4',
        path: `comments.${record.id}.name`
      }
    ])
  }
)

But how should I create some blocks for such components?

Also, I found an issue with the DataSourceManager.getContext() method:

grapesjs/packages/core/src/data_sources/index.ts

Lines 104 to 105 in 9a6fa0c

	accR[i] = dataRecord.attributes;
	accR[dataRecord.id || i] = dataRecord.attributes;

With the duplicate properties, records with id == 1 can be overriden by records that are second in the array. So if I try to output data from the following endpoint: https://jsonplaceholder.typicode.com/posts/1/comments, I will get a list with:

• { id: 2 }
• { id: 3 }
• { id: 4 }
• { id: 5 }
• { id: 5 }

Should I report it like any regular issue? Should I submit a PR to solve it directly?

Thank you for the amazing work so far!

[artf]

Something that bothers me is the fact you have to hardcode the records' id in your paths

How would you access records in an array? By index?

But how should I create some blocks for such components?

I'm not sure what you mean. The same format you used for the component you can use for blocks.

Also, I found an issue with the DataSourceManager.getContext() method

Yeah that's a good case, @danstarns would you suggest removing accR[i] = dataRecord.attributes;?

[danstarns]

I have removed the id reassign here:

• fix: remove id reassign in data sources #6132

PR contains coverage for your JSONPlaceholder usage @raaaahman with snapshots.

[raaaahman]

How would you access records in an array? By index?

More likely in a loop. If I use records, I want to display several of them. I have written an example in my previous comment.

I'm not sure what you mean. The same format you used for the component you can use for blocks.

If I'm writing a component definition, so I can then use such components in a block, I can't really afford to hardcode a data source path:

editor.DomComponents.addType('comment-card', {
  isComponent: el => el.tagName === 'DIV' && el.classList.contains('comment-card'),

  model: {
    defaults: {
      tagName: 'div',
      components: [
        {
          type: 'data-variable',
          tagName: 'h4',
          path: 'comments.c1d45.title',
        }
      ],
      traits: [
        'name',
        'type',
        {
          type: 'number',
          label: 'Order',
          name: 'custom-order',
          value: {
            type: 'data-variable',
            defaultValue: undefined,
            path: 'comments.c1d45.order',
          },
        },
      ],
    }
  }
});

This component here will always use comment c1d45, and I won't be able to reuse it for any other record in my data source. That doesn't sound like a good usecase for blocks...

[raaaahman]

@danstarns Thank you!

Regarding your comment, I found that using the tagName property already works the way you describe, I used it in my other examples.

[artf]

@raaaahman ok got it, indeed, the first phase is to use DataSource only as variables. What you're mentioning is more the loop case which is not yet handled but will be in the future.