Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PERF] [JMAP] Improve attachement download #5372

Closed
chibenwa opened this issue Dec 12, 2024 · 2 comments
Closed

[PERF] [JMAP] Improve attachement download #5372

chibenwa opened this issue Dec 12, 2024 · 2 comments
Assignees
Labels
perf Contributes some performance enhencements

Comments

@chibenwa
Copy link
Member

Why?

Today attachment are handled as mime parts, so if I download 5 attachments on the same mail I would need to read its full content 5 times, parse it then decode the attachment.

This is innefficient because we have a projection for attachments : it could be a simple download of that projection...

This mismatch leads to:

  • Needless reads on the objectStore (billed by cloud provider)
  • Unecessary parsing on James side, very likely a good proportion of the JMAP workload

What?

Allign the blob id on the JMAP layer with the attachment id on the mailbox layer

That way it's a direct download.

How?

Long term known issue.

I tend to look at it by the "JMAP layer needs to use mailbox attachmentId" but it was very complex and involved a lot of coupling.

After discussing with @HoussemNasri I believe that the other way around might be easier: "mailbox layer can be using JMAP blobId as an attachement Id"

To do this we could:

  • Put MessageStorer behind an interface (and rename it! AttachmentParser? )
  • Provide an EmailBodyPart backed version of it.
  • Add the attachmentId back into the ParsedAttachement and make message parser responsible of the allocation
  • Upon part downloads lookup the attachment first, then fallback to blob parts.
  • And realign the tests

Should be doable...

@chibenwa chibenwa added the perf Contributes some performance enhencements label Dec 12, 2024
@chibenwa chibenwa self-assigned this Dec 19, 2024
@chibenwa
Copy link
Member Author

I started POCing around this.

A bit hacky but I got linagora/tmail-backend#1466 and apache#2602 that kind a demonstrate the idea.

Retrocompatible (forward and reverse) with current code!

@chibenwa
Copy link
Member Author

linagora/james-gatling#183 I can be helped by developing the benchmarks for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
perf Contributes some performance enhencements
Projects
None yet
Development

No branches or pull requests

1 participant