format: ipynb but keep yaml header

[MichelNivard]

Is there a way to render (not convert) to ipynb (format: ipynb) but keep the yaml header?

I want to potentially "round trip" a file qmd -> ipynb -> qmd where I have outputs in the ipynb for computational reasons.

[mcanouil]

You likely need a Lua filter or post-processing to do that.

+yaml_metadata_block variant won't give the wanted result for ipynb.

I'm not sure to follow what's your exact use case without any example though.

[cderv]

Is there a way to render (not convert) to ipynb (format: ipynb) but keep the yaml header?

Format .ipynb does not really have YAML header support. For quarto convert we do a specific processing so that the special cell is created.

If you do quarto convert, then execute you .qmd won't you get the computational output ?

Also when

"round trip" a file qmd -> ipynb -> qmd

What do you expect will happen with those outputs ?

I would sum up with the following

• quarto convert is a tool to pass between .qmd and .ipynb source for quarto render. This is exposed to user as a way to convert, but know that quarto render index.qmd will pass by an intermediate ipynb representation when the .qmd requires jupyter engine.

• quarto render will convert to an output. when doing --to ipynb there will be a conversion to ipynb output by Quarto and Pandoc. This means AST representation and then ipynb writer.

Hope it helps understand.

[MichelNivard]

So let me try and be clearer. at a scientific retreat a few weeks back (end of Nov) we had the forever discussion of wouldn't it be great to collaborate on these qmd docs with non technical ppl". Our science is a very collaborate one so you'd almost have o be able to send a google docs link to 8-80 authors, or something like it, at some point in any process. Anyway, I build the tool I. the vid below (this is a developer alpha running on localhost, currently in the process of planning a migration to vercel and server less infra) that OAuths with GitHub, then exposes all a user's repo's and ipynb files within them and lets the user edit in a (tiptap powered) WYSIWYG text editor with comments and track changes, and save to GitHub. the file stay within the ipynb spec, my goals is to 1. keep it within the spec where it 1. could be back converted to Qmd for further technical writing/publishing 2. exposure the unprocessed pandoc references such that the co-authors can add, remove, edit those with a WYSIWYG ref editor (still a work in progress). Anyway so I think I am looking to define a custom variation on the ipynb format that executes the compute (and any images et stored in the ipynb as base64), but passes over pandoc refs and keeps the header like quarto convert, such that it an be validly back converted to qmd?

notebook-writer.mov

[MichelNivard]

I think many ppl in science (myself included) work with quarto for the fine control over the presentation of the final output. I don't know I'd ever get that from ipynb. Though the tool I have developed will happily work with any ipynb.

So the goal is to make this possible:

Analysis [.r/.qmd] -> draft with output [ipynb] -> github [ipynb with metadata for texteditor] -> coloborate iterating prose/analysis [all in ipynb w metadata] -> export [clean ipynb then qmd] -> publish [from qmd]

All the things in the middle sound convoluted but the object really always remains a valid ipynb and is just acted on trough a google doc like shell that directly edits the file (and .bib) on github.

Looking at the quarto documentation I could just be looking for 'quarto convert --execute' and that would give me unrendered but executed ipynb? If I keep the raw yaml chunk intact it should go back to .qmd no problem right?

[MichelNivard]

O wait your saying I could just render the final ipynb with the raw header? Just never go back to .qmd unless that's somehow needed (Rstudio preference for example..)

[cderv]

O wait your saying I could just render the final ipynb with the raw header? Just never go back to .qmd unless that's somehow needed (Rstudio preference for example..)

Yes .ipynb is a valid input like .qmd. You can use quarto render on it, and you can also configure everything using a raw cell with yaml metadata content. So all the final output control is available.
.qmd as input is not mandatory, you can use .ipynb directly.

This is shown in Get Started with Jupyter part of our website
https://prerelease.quarto.org/docs/get-started/hello/jupyter.html

Regarding ipynb logic it is the following

By default, code cell in ipynb won't be executed by default when doing quarto render. This is because ipynb can store results and quarto leverages that. So you can use Quarto with ipynb only to get the final output you need. But this can be controlled with the --execute flag to recompute if needed.

Hope this clarifies

Just never go back to .qmd unless that's somehow needed (Rstudio preference for example..)

Yes RStudio does not support ipynb. Though nowadays, Positron will offer great support for .qmd and .ipynb https://positron.posit.co/

[MichelNivard]

Ah! So if rig it up where when my texteditor opens a ipynb it also sets up a a quarto github action (trying to develop something I could host at near zero expense) I could even render previews and output straight from the text-editor I am now building?

I love positron, still prefer the look and feel of a .qmd in editor, but more relevant I just send a manuscript of to 35 co-authors and colaborators that has to be word-like or it won't fly.

I am trying to build that "last mile" tool, where the compute is frozen, but prose, formulas and citation can be collaboratively edited in WYSIWYG.

[cderv]

I could even render previews and output straight from the text-editor I am now building?

quarto preview does run a preview server. So you could do the same in your tool I guess.

I love positron, still prefer the look and feel of a .qmd in editor, but more relevant I just send a manuscript of to 35 co-authors and colaborators that has to be word-like or it won't fly.

Totally understand that.

I am trying to build that "last mile" tool, where the compute is frozen, but prose, formulas and citation can be collaboratively edited in WYSIWYG.

Great challenge !