Skip to content

Commit

Permalink
Merge branch 'PS-748_subdefinition-to-rendition-migration' of github.…
Browse files Browse the repository at this point in the history 
…com:alchemy-fr/phrasea into w2502
  • Loading branch information
@nmaillat
nmaillat committed Jan 9, 2025
2 parents 20ffcca + 121fdfd commit 82397fe
Show file tree
Hide file tree
Showing 23 changed files with 1,441 additions and 402 deletions.
37 changes: 37 additions & 0 deletions databox/indexer/config/config.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,43 @@
"attributeDefinition": "idmp_attributeDefinition",
"renditionDefinition": "idmp_renditionDefinition"
},
"renditions": {
"original": {
"from": "document",
"useAsOriginal": true,
"class": "document"
},
"preview": {
"useAsPreview": true,
"class": "public",
"builders": {
"image": {
"from": "image:preview"
},
"video": {
"from": "video:preview"
},
"document": {
"from": "document:preview"
}
}
},
"thumbnail": {
"useAsThumbnail": true,
"parent": "preview",
"builders": {
"image": {
"from": "image:thumbnail"
},
"video": {
"from": "video:thumbnail"
},
"document": {
"from": "document:thumbnail"
}
}
}
},
"databoxMapping": [
{
"databox": "%env(PHRASEANET_DATABOX)%",
Expand Down
286 changes: 208 additions & 78 deletions databox/indexer/doc/conf_phraseanet.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
"collections": "",
"workspaceSlug": "phnet",
"searchQuery": "",
"recordsCollectionPath": "/records",
"storiesCollectionPath": "/stories",
"recordsCollectionPath": "/Records/{{ collection.name | escapePath }}/{{ record.getMetadata('Country', '_').value | escapePath }}/{{ record.getMetadata('City', '_').value | escapePath }}",
"storiesCollectionPath": "/Stories/{{ collection.name | escapePath }}/{{ record.getMetadata('Country', '_').value | escapePath }}/{{ record.getMetadata('City', '_').value | escapePath }}",
"copyTo": [
"{% for s in record.getMetadata('Subject', 'no_subject').values %}/classification/{{ record.getMetadata('Creator', 'no_creator').value | escapePath }}/{{ s | escapePath }}\n{% endfor %}"
],
Expand Down Expand Up @@ -90,90 +90,111 @@ The Phraseanet query to search for records to import. If empty: import all.
The Databox workspace where to import (created if not exists).

## `recordsCollectionPath`
Collection-path where to import records as "main" assets.
Collection-path where to import records as "**main**" assets. The asset name will always be the same ase the "original_name" of the record.

- The `recordsCollectionPath` can be a **[Twig](#About-Twig)** expression (see [About Twig](#About-Twig)), allowing to dispatch the assets into a tree of collections.

e.g.: Dispatch by phraseanet collection name, then country and city:

```json lines
...
"recordsCollectionPath": "/Collections/{{ collection.name | escapePath }}/{{ record.getMetadata('Country', '_').value | escapePath }}/{{ record.getMetadata('City', '_').value | escapePath }}",
...
```
--> `/Collections/MyPhraseanetCollection/France/Paris/IMG_1234.jpg`


- /!\ For backward compatibility: If the `recordsCollectionPath` is a simple string (no twig tags), it will be used as a root path
**and completd with phraseanet collection name**
```json lines
...
"recordsCollectionPath": "/Collections",
...
```
--> `/Collections/MyPhraseanetCollection/IMG_1234.jpg`


- `recordsCollectionPath` **can** be empty string (or not set): Since this is a simple (empty) string,
the backward compatibility applies, so the phraseanet collection name will be used as first-level collection.

--> `/MyPhraseanetCollection/IMG_1234.jpg`


## `storiesCollectionPath`
Collection-path where to import stories. If unset: do not import stories.

Each story becomes a collection, and each contained record (= "main" asset) is **copied / aliase** to this collection.
Each story becomes a collection, and each contained record (= "main" asset) is **copied / aliased** to this collection.

The name of the collection will be the same as the name of the story.

- The `storiesCollectionPath` can be a **[Twig](#About-Twig)** expression, allowing to dispatch the stories into a tree of collections.


e.g. 1: Import all stories in the same collection:

```json lines
...
"storiesCollectionPath": "/Stories",
...
```
--> `/Stories/JO-2024` where "JO-2024" is the name of a phraseanet story.


e.g. 2: Dispatch by phraseanet collection name, then country and city:

```json lines
...
"storiesCollectionPath": "/Stories/{{ collection.name | escapePath }}/{{ record.getMetadata('Country', '_').value | escapePath }}/{{ record.getMetadata('City', '_').value | escapePath }}",
...
```
--> `/Stories/MyPhraseanetCollection/France/Paris/JO-2024` where "JO-2024" is the name of a phraseanet story.


todo: How to import story metadata ?

## `copyTo`
list (array) of paths where to copy / alias "main" assets.

Each path is a **twig** expressions that must generate databox path(s), depending on run-time values like record metadata.
Each path is a **[Twig](#About-Twig)** expressions that must generate databox path(s), depending on run-time values like record metadata.

If the asset is to be copied in many places (paths), the twig must generate **one line per path**.

- ### `copyTo` Twig context :

- `record`: record object
- `record.record_id` : string
- `record.resource_id` : string
- `record.databox_id` : string
- `record.base_id` : string
- `record.uuid` : string
- `record.title` : string
- `record.original_name` : string
- `record.mime_type` : string
- `record.created_on` : string
- `record.updated_on` : string
- `record.status` : status[] ***use `getStatus()` method***
- `record.getStatus(<bit> [, <valueIfTrue> [, <valueIfFalse>]])` : boolean ; Value of sb <bit> (4...63).
Boolean value can be replaced by string value(s) `valueIf...`
- `record.subdef` : subdef[] ***use `getSubdef()` method***
- `record.getSubdef(<name>)` : subdef object
- `record.getSubdef(...).height` : number
- `record.getSubdef(...).width` : number
- `record.getSubdef(...).filesize` : number
- `record.getSubdef(...).player_type` : string
- `record.getSubdef(...).mime_type` : number
- `record.getSubdef(...).created_on` : string
- `record.getSubdef(...).updated_on` : string
- `record.getSubdef(...).url` : string
- `record.getSubdef(...).permalink` : permalink object
- `record.getSubdef(...).permalink.url` : string
- `record.metadata` : metata[] ***use `getMetadata()` method***
- `record.getMetadata(<fieldName> [,<default>])` : metadata object, with default value(s) if the field is not set for this record.
- `record.getMetadata(...).value` : The mono-value (if the field is multi-value : concat values with " ; ").
- `record.getMetadata(...).values` : The multi-values as array (if the field is mono-value : array with a single value).

- e.g. 1: Two levels dispatch with unique destination (mono-value fields):

- e.g. 1: Two levels dispatch with unique destination (mono-value fields):

```json lines
...
"copyTo": [
"/classification/{{record.getMetadata('Category', 'unknown_category').value | escapePath}}/{{record.getMetadata('SubCategory', 'unknown_subcategory').value | escapePath}}"
]
...
```

- e.g. 2: Multiple destinations (multi-values field):

```json lines
...
"copyTo": [
"{% for s in record.getMetadata('Keywords', 'no_keyword').values %}/classification/{{ s | escapePath }}\n{% endfor %}"
]
...
```
note: The `\n` is used to output one line (= one path) per keyword.

note: The default value "no_keyword" is a must-have, because if the record had no keyword, it would not be copied anywhere.

- e.g. 3: multiple destinations :

To dispatch the records in many "classification" places, one can set multiple `copyTo` settings.
```json lines
...
"copyTo": [
"/classification/author/{{record.getMetadata('Author', 'unknown_author').value | escapePath}},
"/classification/category/{{record.getMetadata('Category', 'unknown_category').value | escapePath}},
"/classification/year/{{record.getMetadata('Date', '').value is empty ? 'unknown_date' : {{record.getMetadata('Date').value | date('Y')}}
]
...
```
```json lines
...
"copyTo": [
"/classification/{{record.getMetadata('Category', 'unknown_category').value | escapePath}}/{{record.getMetadata('SubCategory', 'unknown_subcategory').value | escapePath}}"
]
...
```

- e.g. 2: Multiples destinations (multi-values field):

```json lines
...
"copyTo": [
"{% for s in record.getMetadata('Keywords', 'no_keyword').values %}/classification/{{ s | escapePath }}\n{% endfor %}"
]
...
```
note: The `\n` is used to output one line (= one path) per keyword.

note: The default value "no_keyword" is a must-have, because if the record had no keyword, it would not be copied anywhere.


- e.g. 3: multiples destinations :

To dispatch the records in many "classification" places, one can set multiple `copyTo` settings.
```json lines
...
"copyTo": [
"/classification/author/{{record.getMetadata('Author', 'unknown_author').value | escapePath}},
"/classification/category/{{record.getMetadata('Category', 'unknown_category').value | escapePath}},
"/classification/year/{{record.getMetadata('Date', '').value is empty ? 'unknown_date' : {{record.getMetadata('Date').value | date('Y')}}
]
...
```

## `fieldMap`
Map (key=AttributeDefinition name) of attributes to create / import.
Expand All @@ -191,14 +212,14 @@ Map (key=AttributeDefinition name) of attributes to create / import.
- `type`: how to evaluate the `value` ("metadata", "template", "text" ; default: "text")
- `value`: value expression, as metadata (=phraseanet field name), template (twig) code or simple text

"text" type: The value is the immediate value for the attribute.
**text** type: The value is the immediate value for the attribute.

"metadata" type: The value is the name of a Phraseanet field, like "Title".
**metadata** type: The value is the name of a Phraseanet field, like "Title".

"template" type: The value is a Twig code, to compose complex value(s)
**template** type: The value is a Twig code, to compose complex value(s)


For the "template" type, the Attribute value(s) is the result of the **twig** expression,
For the **template** type, the Attribute value(s) is the result of the **[Twig](#About-Twig)** expression,
which must generate **one item per line** for multi-values.
The Twig context is the same as `copyTo`

Expand All @@ -225,7 +246,116 @@ with `translatable=true`.
All the different AttributeDefinition locales are copied to the "Enabled locales" of the workspace.


_twig context technical note_:
## `renditions`

Allows to map Phraseanet subdef / (structures) to Phrasea renditions / (definitions).

A Phraseanet subdef is identified by it **type** (image, video, audio, document, unknown) and its **name**. e.g. `image:thumbnail`.

A Phrasea rendition-definition is declared by its **name** and **build settings** (sections image, video, ...).

### `from`
The `from` setting maps the phrasea rendition-definition to the phraseanet subdef.
The build settings will be generated from the phraseanet to match the subdef.

It is possible to declare a rendition with no `from`: not imported from Phraseanet, but created in Phrasea.

### `parent`
One can declare a `parent` relation between renditions, the parent rendition **must** be declared before the child.

/!\ `original` is **not** a rendition, but a place to declare settings specifics to the file of the asset.

The special `original` has no build settings, so the `from` setting is top-level.
Mostly it will be mapped to the Phraseanet special `document` subdef, but one could map it to any phraseanet subdef.

If a rendition is to be created from the original file, do **not** set it a parent.


### `useAsOriginal`, `useAsPreview`, `useAsThumbnail`, `class`, ...

Common settings for all renditions. If not set, the value will be "guessed" from the subdef name / class.

e.g.

```json lines
...
"renditions": {
"original": {
"from": "document",
"useAsOriginal": true,
"class": "document"
},
"preview": {
"useAsPreview": true,
"class": "public_preview",
"builders": {
"image": {
"from": "image:preview"
},
"video": {
"from": "video:preview"
}
}
},
"thumbnail": {
"useAsThumbnail": true,
"parent": "preview",
"builders": {
"image": {
"from": "image:thumbnail"
},
"video": {
"from": "video:thumbnail"
}
}
}
}
...
```

-------------------

# About Twig

When using twig expressions in the configuration, the context is the following:

- `record`: record object
- `record.record_id` : string
- `record.resource_id` : string
- `record.databox_id` : string
- `record.base_id` : string
- `record.uuid` : string
- `record.title` : string
- `record.original_name` : string
- `record.mime_type` : string
- `record.created_on` : string
- `record.updated_on` : string
- `record.status` : status[] ***use `getStatus()` method***
- `record.getStatus(<bit> [, <valueIfTrue> [, <valueIfFalse>]])` : boolean ; Value of sb <bit> (4...63).
Boolean value can be replaced by string value(s) `valueIf...`
- `record.subdef` : subdef[] ***use `getSubdef()` method***
- `record.getSubdef(<name>)` : subdef object
- `record.getSubdef(...).height` : number
- `record.getSubdef(...).width` : number
- `record.getSubdef(...).filesize` : number
- `record.getSubdef(...).player_type` : string
- `record.getSubdef(...).mime_type` : number
- `record.getSubdef(...).created_on` : string
- `record.getSubdef(...).updated_on` : string
- `record.getSubdef(...).url` : string
- `record.getSubdef(...).permalink` : permalink object
- `record.getSubdef(...).permalink.url` : string
- `record.metadata` : metata[] ***use `getMetadata()` method***
- `record.getMetadata(<fieldName> [,<default>])` : metadata object, with default value(s) if the field is not set for this record.
- `record.getMetadata(...).value` : The mono-value (if the field is multi-value : concat values with " ; ").
- `record.getMetadata(...).values` : The multi-values as array (if the field is mono-value : array with a single value).
- `collection`: collection object (of the record)
- `collection.databox_id`: string (same as `record.databox_id`)
- `collection.base_id`: string (same as `record.base_id`)
- `collection.collection_id`: number
- `collection.name`: string

## Twig context technical note:

To prevent twig to crash if a field doest not exists in a record (when trying to access a property like `.value`),
`getMetadata(...)` will return a "fake" empty metadata object.
Expand Down
5 changes: 3 additions & 2 deletions databox/indexer/src/databox/client.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -238,8 +238,9 @@ export class DataboxClient {
return res.data['hydra:member'];
}

async createRenditionDefinition(data: object): Promise<void> {
await this.client.post(`/rendition-definitions`, data);
async createRenditionDefinition(data: object): Promise<string> {
const res = await this.client.post(`/rendition-definitions`, data);
return res.data.id;
}

async flushWorkspace(workspaceId: string): Promise<string> {
Expand Down
Loading

0 comments on commit 82397fe

Please sign in to comment.