Can this be used commercially?

[juangea]

Can this be used commercially?
That's the question, thanks!

[bitplane]

This is likely a tough question. The LICENSE says you can, but it does not have patent waivers. So it'll ultimately depend on whether Microsoft filed any patents and whether they cover your product's use cases.

[juangea]

Maybe @sicxu , @YuDeng or @JeffreyXiang can confirm this can be used commercially.

But thanks for the input @bitplane :)

[YuDeng]

Hi @juangea, our pretrained models and most of our code are available under the MIT license, which permits commercial use with the only requirements being the preservation of copyright and license notices. However, the components related to rendering—such as rendering 3D Gaussians, Radiance fields, and extracting meshes and GLBs—are not available for commercial use. You will need to rewrite these parts if you intend to use them commercially. For more details, please refer to our license information at this link.

[juangea]

Thanks @YuDeng

It seems FlexiCubes is now part of Kaolin (The core functions of FlexiCubes are now in Kaolin starting from v0.15.0) so maybe that's an alternative, (it's under Apache 2.0)

I'm not sure what could be a good alternative to Differential Gaussian Rasterization.

In any case, since I'm not a low level programmer, this may be out of my capabilities, but I want to explore the possibilities, those libraries are used as a postproduction of the model output?

In what phases or what order are these libraries used?

I mean, if we use the model, we could investigate a different way of rendering the gaussian splatting or the glb extraction, but I'm not entirely sure what the output if the model is.

Cna you clarify this for me please?

No need for super long explanation, I don't want to steal too much time from you, just the basics to understand what are talking about here.

Thanks a lot!

[YuDeng]

The direct outputs of our models are: Gaussian primitives, local radiance fields within active voxels, or local signed distance values and parameters for Flexicubes to extract mesh surfaces. Up to these parts, everything follows the MIT license. After that, for rendering Gaussians, radiance fields, and extracting meshes, they are under different licenses and should be replaced.

[jclarkk]

@juangea

Even in kaolin it's stated that this part is non commercial:

[juangea]

LOL good to know.

A different approach will have to be taken then, it’s not going to be easy I presume.

[jclarkk]

@juangea
I do have some good news though, I managed to implement the renderer using gsplat which has a permissive license.

@YuDeng
Would you like a PR on this?

[YuDeng]

Hi @jclarkk, thanks for that. You can go ahead to make the PR or share the link of your fork here.

For now, I think it's better to keep this repo as is as a verified research project. We'll look into gsplat and evaluate its performance later (for both inference and training).

[DiamondGlassDrill]

@jcclark would love to get to know more about your fork and gsplat. Thanks in advance

[jclarkk]

I've opened a PR here:

#73

[tommi-pirttiniemi]

What about license of the output 3d models?

[YuDeng]

Hi @tommi-pirttiniemi, when exporting GLB files, the process involves using Flexicubes to extract meshes and a non-commercial Gaussian renderer for baking the albedos. It also involves using pymeshfix for mesh post-processing, which is under a GPL-3.0 license and allows for commercial use.

As suggested by @jclarkk in #73, gsplat can be an alternative to the Gaussian renderer; however, replacing the Flexicubes part requires further effort.

[jclarkk]

@juangea @YuDeng
I've also implemented Marching Cubes as a replacement for FlexiCubes. You can view the PR here:
#89

[juangea]

Awesome @jclarkk!!

I presume the PR won’t be accepted here being an official repo, is everything usable from your repo?

[jclarkk]

@juangea
Yes, you can simply use my main branch as it contains all the changes I've mentioned here.

[0lento]

As far as I know, we can now get as far as decoding gaussian and and raw mesh with permissive licenses, thanks to jclarkk.
Decimation is already MIT (regular pyvista) and also Trimesh (MIT) can do that as well.

I've been testing Trellis by fully removing the problematic libraries and found that there is still nvdiffrast dependency left for Utils3D and random direct nvdiffrast calls that get used at:

• MeshRenderer (when rendering mesh normals video preview for example, which could just be skipped)
• Fill Holes/invisible removal (I'd debate this isn't really mandatory either, as it seems to also make holes but it sure gets rid of extra polys, there's also potential for other, faster filtering options here?)
• Texture baking (both Fast and Optimized modes use utils3d/nvdiffrast atm)

Also if you need radiance fields, it seems to have diffoctreerast dependency which brings another limiting license, but afaik we don't need octrees or radiance fields to get the meshes or textures out?

In the big picture, I'd say that texturing phase is most crucial here to swap. There are a lot of MIT/BSD licensed options for differential renderers, but I don't really know which ones would be best suited for this. This would need some testing because speed and VRAM requirements play a notable role at this stage, at least with current baking strategy. Would SoftRas / DEODR / PyTorch3D etc work here?

There's also potential here for different texture projection strategies than what is used now, but just porting different renderer for the current texture bake would probably be simplest to implement. Did TRELLIS researchers experiment with other ways to accomplish the texture projection?

Also curious if it would be possible to just extract the raw mesh with colors applied to vertex colors for quick mesh visualizations?

[jclarkk]

@0lento
Thanks for mapping out all the tricky parts, it's really helpful.

Normally, projects utilize PyTorch3D for the texturing process, so I might try it there and attempt to replace the baking process.

[MooSpyker]

Is anyone working on a way to export using another MIT license software? Or are we screwed using 3D assets in video games using this tool all together? Its weird that Trellis itself is commercial but just not the exporter... how else could you use Trellis commercially then... without getting at the actual output models? Makes no sense to me.

[0lento]

Its weird that Trellis itself is commercial but just not the exporter...

Trellis is a research project. Commercial use isn't the aim of such projects. Trellis team has given us very generous MIT license for their part though (and it's great). Note that it's not just the exporter that originally uses non-commercial license. So does the originally used gaussian renderer, radiance fields, mesh generation and mesh rendering.

Now today, afaik you can get the actual mesh out using MIT compatible pipeline (for code side) if you use jclarkk's Marching Cubes PR. Even decimating it isn't an issue. Disclaimer: I haven't checked each involved license to that mentioned point manually, so that would still need some verification, we are still mapping this situation out with more broad strokes right now to see what all needs to change.

As mentioned on my previous post, current texture bake and some of the mesh cleanup are still using noncommercial licenses, but that part is definitely solvable one way or another. I'm fairly sure for example that you could even omit the whole gaussian step and sample the texture directly using structured latent data. Or just swap the current texture bake to use different renderer.

But the code licensing isn't really the only question mark here. To me the biggest question here is, how are these licenses actually going to affect the generated output or the weights license?

MS gives MIT license for their part, but hasn't the public weights been trained with code libraries that have noncommercial clause in them? Regular 3D-Future and HSSD-Models datasets don't allow commercial use, so this would also depend on the license MS got for them and what license MS can give for the trained models. Could @YuDeng clear the model's license bit, is it clearly MIT?

If the current models are not commercially usable, I'd assume you'd need to train the models using only datasets that allow commercial use, for example by only using Objaverse XL (which has Apache license). I'm not sure if I've seen anywhere how much compute resources did it take to train Trellis models from scratch but I'm just going to assume it's something that's out of reach for most here.

[MooSpyker]

So basically right now its impossible to use anything this tool outputs as a video game asset? Are they working towards that, or is this a tech demo thats basically out and done being worked on?

(I still have not been able to get this working in ComfyUI on my end after trying to the last 3 days... pretty frustrated at this point, we REALLY need an installation tutorial, none that are out right now get you even close...)

[juangea]

The model has been trained with a MIT license dataset that has been made public AFAIK so the model is MIT.

Also @jclarkk made some improvements to change the problematic components, however I’m not sure if the texture extraction is under that modification, but AFAIK for the mesh, it should be Ok

[0lento]

The model has been trained with a MIT license dataset that has been made public AFAIK so the model is MIT.

I can't find such information about MIT datasets on the project page anywhere, instead I find this:

"We provide TRELLIS-500K, a large-scale dataset containing 500K 3D assets curated from Objaverse(XL), ABO, 3D-FUTURE, HSSD, and Toys4k, filtered based on aesthetic scores."

If you follow the links, you'll find that Objaverse XL is Apache License, ABO is CC-BY (doesn't prevent commercial use but need to credit), 3D-Future has NC license, so does HSSD and Toys4K may have assets that have CC-BY-NC-SA license (their licenses vary per asset).

[colortelevision]

ask permission later

…

On Mon, Jan 6, 2025 at 11:36 AM 0lento ***@***.***> wrote: The model has been trained with a MIT license dataset that has been made public AFAIK so the model is MIT. I can't find such information about MIT datasets on the project page anywhere, instead I find this: "We provide TRELLIS-500K, a large-scale dataset containing 500K 3D assets curated from Objaverse(XL) <https://objaverse.allenai.org/>, ABO <https://amazon-berkeley-objects.s3.amazonaws.com/index.html>, 3D-FUTURE <https://tianchi.aliyun.com/specials/promotion/alibaba-3d-future>, HSSD <https://huggingface.co/datasets/hssd/hssd-models>, and Toys4k <https://github.com/rehg-lab/lowshot-shapebias/tree/main/toys4k>, filtered based on aesthetic scores." If you follow the links, you'll find that Objaverse XL is Apache License, ABO is CC-BY (doesn't prevent commercial use but need to credit), 3D-Future has NC license, so does HSSD and Toys4K may have assets that have CC-BY-NC-SA license (their licenses vary per asset). — Reply to this email directly, view it on GitHub <#41 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVPPWANERWH5TVSYK36YL3L2JKWKLAVCNFSM6AAAAABTJCL34WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNZTGQ3TOMZWGY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>