DiffusedTexture: AI-Powered Texture Generation for Blender

DiffusedTexture is a Blender add-on that uses Stable Diffusion to create textures directly on 3D meshes.

Table of Contents

• DiffusedTexture: AI-Powered Texture Generation for Blender
  • Table of Contents
  • Examples
  • Features
  • Installation (Windows)
  • Installation (Linux)
  • Setup
    • Blender Configuration
  • Usage
    • Additional Options
  • Troubleshooting
  • Roadmap
  • Acknowledgements

Examples

Features

• AI-Driven Texture Creation: Generate diffuse textures directly on 3D models
• Modes for Different Workflows:
  • Text2Image Parallel: Create textures from text prompts, ensuring global consistency.
  • Image2Image Parallel: Generates textures from input textures, applied parallel across all views.
  • Image2Image Sequential: Sequentially adjusts textures across views, great for refinement.
• LoRA Integration: Uses LoRA conditioning for specific styles.
• IPAdapter Integration: Fit specific styles or objects with images for enhanced flexibility and control.

Installation (Windows)

0. Download 7-Zip
1. Download all .tar files of the latest release
2. Untar the file diffused_texture_addon-0.0.2-windows_x64.7z.001 this will automatically untar the other .7z files

   WARNING: DO NOT unzip the resulting diffused_texture_addon-0.0.2-windows_x64.zip

3. If you did not already do so: You need to "Allow Online Access" under "System" in the Preferences.
4. Install the diffused_texture_addon-0.0.2-windows_x64.zip file in Blender as an Add-On.

   • Edit -> Preferences... -> Sidebar Add-ons -> Top right corner dropdown menu -> Install from Disk...

     

   • If neccessary, provide a custom HuggingFace Cache Path to install and/or load the checkpoints, else the default path is choosen.

     

   • Download necessary models (~10.6 GB total):

     • Tip: Open Blender's system console (Window > Toggle System Console) BEFORE starting the download to monitor download progress.

Installation (Linux)

• TODO: Test on Linux Instructions and test if it works

Setup

Blender Configuration

1. Enable CUDA or OPTIX in Blender if using an NVIDIA GPU.
   • Go to Edit > Preferences > System and configure GPU settings.
   • Note: Requires a modern NVIDIA GPU with at least 4GB(-ish) VRAM, 9 and 16 camera parallel runs will require more VRAM.

Usage

1. Load a 3D Model:
   • Import or create a .blend file containing the 3D model.
2. UV Unwrap the Model:
   • Apply a UV map (Smart UV Project works well).
3. Access the Add-On:
   • Open the DiffusedTexture panel in the N-panel (right-hand sidebar).
4. Set Up Texture Generation:
   • Prompt & Negative Prompt: Describe the desired texture/object and what to avoid.
   • Guidance Scale: Adjust creativity vs. fidelity.
   • Denoise Strength: Default to 1.0 for Text2Image.
5. Adjust Advanced Options:
   • Mesh Complexity:
     • Low: Depth ControlNet only.
     • Medium: Adds Canny ControlNet.
     • High: Adds Normalmap ControlNet for maximum detail.
   • Cameras: Use more viewpoints for better texture blending.
   • Texture & Render Resolution: Ensure render resolution is at least 2x texture resolution.
6. Generate Texture:
   • Click Start Texture Generation. Monitor progress in the system console.

Additional Options

• LoRA Models: Add one or multiple LoRA models to match specific results.
• IPAdapter: Supply the desired "look" as an image instead of a text prompt.

Troubleshooting

• Add-On Not Visible: Ensure it’s enabled in Edit > Preferences > Add-ons.
• Blender Freezes: Open the system console to track progress during long tasks.
• Permission Issues: Specify a valid output path.
• Out of GPU Memory:
  • Reduce camera count.
  • Close other GPU-intensive applications.
• Crashes: Restart Blender or your PC if crashes persist.

Roadmap

• Performance Enhancements:
  • Multi-threaded execution to prevent UI freezing.
• Checkpoint Flexibility:
  • Allow external Stable Diffusion checkpoints for varied outputs.
• Masking Support:
  • Apply textures to specific areas of the mesh.
• Multi-Mesh Workflow:
  • Simultaneously texture multiple objects.

Acknowledgements

• Inspired by Dream Textures by Carson Katri.
• Powered by:
  • Stable Diffusion
  • HuggingFace Diffusers
  • ControlNet
  • IPAdapter
• Influenced by research in TEXTure, Text2Tex, Paint3D, MatAtlas and EucliDreamer.