diff --git a/images/JonBarron.jpg b/images/JonBarron.jpg index d583ab98f7..bfaf851595 100644 Binary files a/images/JonBarron.jpg and b/images/JonBarron.jpg differ diff --git a/images/cat4d.jpg b/images/cat4d.jpg new file mode 100644 index 0000000000..b4a8463a10 Binary files /dev/null and b/images/cat4d.jpg differ diff --git a/images/cat4d.mp4 b/images/cat4d.mp4 new file mode 100644 index 0000000000..4a3ba543ae Binary files /dev/null and b/images/cat4d.mp4 differ diff --git a/images/r2r.jpg b/images/r2r.jpg new file mode 100644 index 0000000000..9463e8f23e Binary files /dev/null and b/images/r2r.jpg differ diff --git a/images/r2r.mp4 b/images/r2r.mp4 new file mode 100644 index 0000000000..5c68cb19e3 Binary files /dev/null and b/images/r2r.mp4 differ diff --git a/images/simvs.jpg b/images/simvs.jpg new file mode 100644 index 0000000000..88bcf9ff2e Binary files /dev/null and b/images/simvs.jpg differ diff --git a/images/simvs.mp4 b/images/simvs.mp4 new file mode 100644 index 0000000000..568be1564f Binary files /dev/null and b/images/simvs.mp4 differ diff --git a/index.html b/index.html index 8d32f54e7b..67c59301b5 100755 --- a/index.html +++ b/index.html @@ -33,17 +33,18 @@ Bio  /  Scholar  /  Twitter  /  + Bluesky  /  Github

- + profile photo -
+

Research

I'm interested in computer vision, deep learning, generative AI, and image processing. Most of my research is about inferring the physical world (shape, motion, color, light, etc) from images, usually with radiance fields. Some papers are highlighted. @@ -54,8 +55,154 @@

Research

+ + + + + + + + + + + + + + + + + + + + - - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+
+
+ +
+ +
+ + Generative Multiview Relighting for +3D Reconstruction under Extreme Illumination Variation + +
+ Hadi Alzayer, + Philipp Henzler, + Jonathan T. Barron, + Jia-Bin Huang, + Pratul P. Srinivasan, + Dor Verbin +
+ arXiv, 2024 +
+ project page + / + arXiv +

+

+ Images taken under extreme illumination variation can be made consistent with diffusion, and this enables high-quality 3D reconstruction. +

+
+
+
+ +
+ +
+ + SimVS: Simulating World Inconsistencies for Robust View Synthesis + +
+ Alex Trevithick, + Roni Paiss, + Philipp Henzler, + Dor Verbin, + Rundi Wu, + Hadi Alzayer, + Ruiqi Gao, + Ben Poole, + Jonathan T. Barron, + Aleksander Holynski, + Ravi Ramamoorthi, + Pratul P. Srinivasan +
+ arXiv, 2024 +
+ project page + / + arXiv +

+

+ Simulating the world with video models lets you make inconsistent captures consistent. +

+
+
+
+ +
+ +
+ + CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models + + +
+ Rundi Wu, + Ruiqi Gao, + Ben Poole, + Alex Trevithick, + Changxi Zheng, + Jonathan T. Barron, + Aleksander Holynski +
+ arXiv, 2024 +
+ project page + / + arXiv +

+

+ An approach for turning a video into a 4D radiance field that can be rendered in real-time. When combined with a text-to-video model, this enables text-to-4D. +

+
+
@@ -73,8 +220,8 @@

Research

ever_stop()
- + + EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis @@ -101,8 +248,9 @@

Research

+
+ CAT3D: Create Anything in 3D with Multi-View Diffusion Models @@ -151,7 +299,7 @@

Research

+
+ NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections @@ -198,7 +346,7 @@

Research

+
+ Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering @@ -245,7 +393,7 @@

Research

+
+ Nuvo: Neural UV Mapping for Unruly 3D Representations @@ -291,7 +439,7 @@

Research

+
+ Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis @@ -341,7 +489,7 @@

Research

+
+ SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration @@ -392,7 +540,7 @@

Research

+
+ Eclipse: Disambiguating Illumination and Materials using Unintended Shadows @@ -439,7 +587,7 @@

Research

+
+ ReconFusion: 3D Reconstruction with Diffusion Priors @@ -488,7 +636,7 @@

Research

+
+ SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-Wild @@ -541,7 +689,7 @@

Research

+
@@ -559,7 +707,7 @@

Research

internerf_stop()
+ InterNeRF: Scaling Radiance Fields via Parameter Interpolation @@ -582,7 +730,7 @@

Research

+
+ State of the Art on Diffusion Models for Visual Computing @@ -636,7 +784,7 @@

Research

+
+ CamP: Camera Preconditioning for Neural Radiance Fields @@ -680,7 +828,7 @@

Research

+
+ Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields @@ -726,7 +874,7 @@

Research

+
+ DreamBooth3D: Subject-Driven Text-to-3D Generation @@ -767,7 +915,7 @@

Research

+
+ BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis @@ -816,7 +964,7 @@

Research

+
+ MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes @@ -866,7 +1014,7 @@

Research

+
@@ -883,7 +1031,7 @@

Research

alignerf_stop()
+ AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training @@ -909,7 +1057,7 @@

Research

+
+ DreamFusion: Text-to-3D using 2D Diffusion @@ -953,7 +1101,7 @@

Research

+
@@ -970,7 +1118,7 @@

Research

guandao_stop()
+ Learning a Diffusion Prior for NeRFs @@ -990,7 +1138,7 @@

Research

+
@@ -1007,7 +1155,7 @@

Research

mira_stop()
+ MIRA: Mental Imagery for Robotic Affordances @@ -1030,7 +1178,7 @@

Research

+
@@ -1047,7 +1195,7 @@

Research

samurai_stop()
+ SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image Collections @@ -1074,7 +1222,7 @@

Research

+
@@ -1091,7 +1239,7 @@

Research

pnf_stop()
+ Polynomial Neural Fields for Subband Decomposition
@@ -1113,7 +1261,7 @@

Research

+
@@ -1130,7 +1278,7 @@

Research

malle_stop()
+ Fast and High-Quality Image Denoising via Malleable Convolutions @@ -1155,7 +1303,7 @@

Research

+
+ NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields @@ -1199,7 +1347,7 @@

Research

+
+ Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields @@ -1243,7 +1391,7 @@

Research

+
+ Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields @@ -1286,7 +1434,7 @@

Research

+
+ NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images @@ -1331,7 +1479,7 @@

Research

+
+ RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs @@ -1376,7 +1524,7 @@

Research

+
+ Block-NeRF: Scalable Large Scene Neural View Synthesis @@ -1422,7 +1570,7 @@

Research

+
+ HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video @@ -1465,7 +1613,7 @@

Research

+
+ Urban Radiance Fields @@ -1512,7 +1660,7 @@

Research

+
@@ -1529,7 +1677,7 @@

Research

ddp_stop()
+ Dense Depth Priors for Neural Radiance Fields from Sparse Input Views @@ -1553,7 +1701,7 @@

Research

+
+ Zero-Shot Text-Guided Object Generation with Dream Fields @@ -1596,7 +1744,7 @@

Research

+
@@ -1613,7 +1761,7 @@

Research

survey_stop()
+ Advances in Neural Rendering @@ -1646,7 +1794,7 @@

Research

+
@@ -1663,7 +1811,7 @@

Research

npil_stop()
+ Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition @@ -1690,7 +1838,7 @@

Research

+
+ HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields @@ -1735,7 +1883,7 @@

Research

+
@@ -1752,7 +1900,7 @@

Research

nerfactor_stop()
+ NeRFactor: Neural Factorization of Shape and Reflectance
Under an Unknown Illumination
@@ -1777,7 +1925,7 @@

Research

+
@@ -1793,7 +1941,7 @@

Research

dualfont_stop()
+ Scalable Font Reconstruction with Dual Latent Manifolds @@ -1811,7 +1959,7 @@

Research

+
+ Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields @@ -1857,7 +2005,7 @@

Research

+
+ Baking Neural Radiance Fields for Real-Time View Synthesis @@ -1903,7 +2051,7 @@

Research

+
+ Nerfies: Deformable Neural Radiance Fields @@ -1948,7 +2096,7 @@

Research

+
@@ -1965,7 +2113,7 @@

Research

c5_stop()
+ Cross-Camera Convolutional Color Constancy @@ -1987,7 +2135,7 @@

Research

+
@@ -2004,7 +2152,7 @@

Research

dualdefocus_stop()
+ Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image @@ -2032,7 +2180,7 @@

Research

+
+ NeRD: Neural Reflectance Decomposition from Image Collections @@ -2078,7 +2226,7 @@

Research

+
@@ -2095,7 +2243,7 @@

Research

flare_stop()
+ How to Train Neural Networks for Flare Removal @@ -2121,7 +2269,7 @@

Research

+
+ iNeRF: Inverting Neural Radiance Fields for Pose Estimation @@ -2163,7 +2311,7 @@

Research

+
+ IBRNet: Learning Multi-View Image-Based Rendering @@ -2209,7 +2357,7 @@

Research

+
+ NeRV: Neural Reflection and Visibility Fields for Relighting and View Synthesis @@ -2252,7 +2400,7 @@

Research

+
+ Learned Initializations for Optimizing Coordinate-Based Neural Representations @@ -2294,7 +2442,7 @@

Research

+
+ NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections @@ -2336,7 +2484,7 @@

Research

+
@@ -2353,7 +2501,7 @@

Research

dualrefl_stop()
+ Learned Dual-View Reflection Removal @@ -2379,7 +2527,7 @@

Research

+
+ Neural Light Transport for Relighting and View Synthesis @@ -2428,7 +2576,7 @@

Research

+
@@ -2445,7 +2593,7 @@

Research

lssr_stop()
+ Light Stage Super-Resolution: Continuous High-Frequency Relighting @@ -2473,7 +2621,7 @@

Research

+
@@ -2490,7 +2638,7 @@

Research

ff_stop()
+ Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains @@ -2518,7 +2666,7 @@

Research

+
@@ -2535,7 +2683,7 @@

Research

thresh_stop()
+ A Generalization of Otsu's Method and Minimum Error Thresholding @@ -2557,7 +2705,7 @@

Research

+
@@ -2574,7 +2722,7 @@

Research

uflow_stop()
+ What Matters in Unsupervised Optical Flow @@ -2598,7 +2746,7 @@

Research

+
+ NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis @@ -2649,7 +2797,7 @@

Research

+
@@ -2666,7 +2814,7 @@

Research

porshadmanip_stop()
+ Portrait Shadow Manipulation @@ -2689,7 +2837,7 @@

Research

+
@@ -2706,7 +2854,7 @@

Research

learnaf_stop()
+ Learning to Autofocus @@ -2731,7 +2879,7 @@

Research

+
+ Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination @@ -2777,7 +2925,7 @@

Research

+
@@ -2794,7 +2942,7 @@

Research

skyopt_stop()
+ Sky Optimization: Semantically Aware Image Processing of Skies in Low-Light Photography @@ -2817,7 +2965,7 @@

Research

+
@@ -2833,7 +2981,7 @@

Research

nightsight_stop()
+ Handheld Mobile Photography in Very Low Light @@ -2862,7 +3010,7 @@

Research

+
@@ -2878,7 +3026,7 @@

Research

font_stop()
+ A Deep Factorization of Style and Structure in Fonts @@ -2896,7 +3044,7 @@

Research

+
@@ -2912,7 +3060,7 @@

Research

dpzlearn_stop()
+ Learning Single Camera Depth Estimation using Dual-Pixels @@ -2932,7 +3080,7 @@

Research

+
@@ -2948,7 +3096,7 @@

Research

porlight_stop()
+ Single Image Portrait Relighting @@ -2975,7 +3123,7 @@

Research

+
@@ -2991,7 +3139,7 @@

Research

loss_stop()
+ A General and Adaptive Robust Loss Function @@ -3014,7 +3162,7 @@

Research

+
@@ -3030,7 +3178,7 @@

Research

mpi_stop()
+ Pushing the Boundaries of View Extrapolation with Multiplane Images @@ -3052,7 +3200,7 @@

Research

+
@@ -3068,7 +3216,7 @@

Research

unprocessing_stop()
+ Unprocessing Images for Learned Raw Denoising @@ -3092,7 +3240,7 @@

Research

+
@@ -3108,7 +3256,7 @@

Research

motionblur_stop()
+ Learning to Synthesize Motion Blur @@ -3130,7 +3278,7 @@

Research

+
@@ -3146,7 +3294,7 @@

Research

darkflash_stop()
+ Stereoscopic Dark Flash for Low-light Photography @@ -3166,7 +3314,7 @@

Research

+
@@ -3182,7 +3330,7 @@

Research

motionstereo_stop()
+ Depth from Motion for Smartphone AR @@ -3201,7 +3349,7 @@

Research

+
@@ -3217,7 +3365,7 @@

Research

portrait_stop()
+ Synthetic Depth-of-Field with a Single-Camera Mobile Phone @@ -3241,7 +3389,7 @@

Research

+
@@ -3257,7 +3405,7 @@

Research

aperture_stop()
+ Aperture Supervision for Monocular Depth Estimation @@ -3278,7 +3426,7 @@

Research

+
@@ -3294,7 +3442,7 @@

Research

deepburst_stop()
+ Burst Denoising with Kernel Prediction Networks @@ -3316,7 +3464,7 @@

Research

+
@@ -3332,7 +3480,7 @@

Research

friendly_stop()
+ A Hardware-Friendly Bilateral Solver for Real-Time Virtual Reality Video @@ -3348,7 +3496,7 @@

Research

+
@@ -3364,7 +3512,7 @@

Research

hdrnet_stop()
+ Deep Bilateral Learning for Real-Time Image Enhancement @@ -3383,7 +3531,7 @@

Research

+
@@ -3399,7 +3547,7 @@

Research

ffcc_stop()
+ Fast Fourier Color Constancy @@ -3422,7 +3570,7 @@

Research

+
@@ -3438,7 +3586,7 @@

Research

jump_stop()
+ Jump: Virtual Reality Video @@ -3458,7 +3606,7 @@

Research

+
@@ -3474,7 +3622,7 @@

Research

hdrp_stop()
+ Burst Photography for High Dynamic Range and Low-Light Imaging on Mobile Cameras @@ -3493,7 +3641,7 @@

Research

+
@@ -3509,7 +3657,7 @@

Research

bs_stop()
+ The Fast Bilateral Solver @@ -3532,7 +3680,7 @@

Research

+
@@ -3548,7 +3696,7 @@

Research

diverdi_stop()
+ Geometric Calibration for Mobile, Stereo, Autofocus Cameras @@ -3565,7 +3713,7 @@

Research

+
@@ -3581,7 +3729,7 @@

Research

dt_stop()
+ Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform @@ -3599,7 +3747,7 @@

Research

+
@@ -3615,7 +3763,7 @@

Research

ccc_stop()
+ Convolutional Color Constancy @@ -3631,10 +3779,10 @@

Research

+ + Scene Intrinsics and Depth from a Single Image @@ -3650,7 +3798,7 @@

Research

+
@@ -3668,7 +3816,7 @@

Research

defocus_stop()
+ Fast Bilateral-Space Stereo for Synthetic Defocus @@ -3689,10 +3837,10 @@

Research

+ PontTuset + Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation @@ -3711,7 +3859,7 @@

Research

+
@@ -3729,7 +3877,7 @@

Research

sirfs_stop()
+

Shape, Illumination, and Reflectance from Shading @@ -3751,10 +3899,10 @@

Research

+ ArbalaezCVPR2014 + Multiscale Combinatorial Grouping @@ -3770,7 +3918,7 @@

Research

+
@@ -3788,7 +3936,7 @@

Research

flyspin_stop()
+ Volumetric Semantic Segmentation using Pyramid Context Features @@ -3808,10 +3956,10 @@

Research

+ 3DSP + 3D Self-Portraits @@ -3826,7 +3974,7 @@

Research

+
+ Intrinsic Scene Properties from a Single RGB-D Image @@ -3860,10 +4008,10 @@

Research

+ Boundary_png + Boundary Cues for 3D Object Shape Recovery @@ -3882,7 +4030,7 @@

Research

+ @@ -3900,7 +4048,7 @@

Research

eccv12_stop()
+ Color Constancy, Intrinsic Images, and Shape Estimation @@ -3918,7 +4066,7 @@

Research

+
@@ -3936,7 +4084,7 @@

Research

cvpr12_stop()
+ Shape, Albedo, and Illumination from a Single Image of an Unknown Object @@ -3953,10 +4101,10 @@

Research

+ b3do + A Category-Level 3-D Object Dataset: Putting the Kinect to Work @@ -3978,10 +4126,10 @@

Research

+ safs_small + High-Frequency Shape and Albedo from Shading using Natural Image Statistics @@ -3996,10 +4144,10 @@

Research

+ fast-texture + Discovering Efficiency in Coarse-To-Fine Texture Classification @@ -4015,10 +4163,10 @@

Research

+ prl + Parallelizing Reinforcement Learning @@ -4033,10 +4181,10 @@

Research

+ blind-date + Blind Date: Using Proper Motions to Determine the Ages of Historical Images @@ -4049,10 +4197,10 @@

Research

+ clean-usnob + Cleaning the USNO-B Catalog Through Automatic Detection of Optical Artifacts @@ -4068,18 +4216,63 @@

Research

- +

Miscellanea

- +
+ + + + + + + - - + + + + + + + + + - - - - - - - - -
+
+

Micropapers

+
+
+ Squareplus: A Softplus-Like Algebraic Rectifier +
+ A Convenient Generalization of Schlick's Bias and Gain Functions +
+ Continuously Differentiable Exponential Linear Units +
+ Scholars & Big Models: How Can Academics Adapt? +
+ +
+

Recorded Talks

+
+
+ View Dependent Podcast, 2024 +
+ Bay Area Robotics Symposium, 2023 + +
+ EGSR Keynote, 2021 +
+ TUM AI Lecture Series, 2020 +
+ Vision & Graphics Seminar at MIT, 2020 +
+
+

Academic Service

+
+
+ Area Chair, CVPR 2025 +
Area Chair, CVPR 2024
Demo Chair, CVPR 2023 @@ -4093,11 +4286,16 @@

Miscellanea

Area Chair, CVPR 2018
- cs188 + +
+

Teaching

+
+ Graduate Student Instructor, CS188 Spring 2011
Graduate Student Instructor, CS188 Fall 2010 @@ -4106,23 +4304,6 @@

Miscellanea

-

Basically
Blog Posts

-
- Squareplus: A Softplus-Like Algebraic Rectifier -
- A Convenient Generalization of Schlick's Bias and Gain Functions -
- Continuously Differentiable Exponential Linear Units -
- Scholars & Big Models: How Can Academics Adapt? -
diff --git a/stylesheet.css b/stylesheet.css index d359156da5..36f08c4029 100644 --- a/stylesheet.css +++ b/stylesheet.css @@ -132,4 +132,11 @@ h2 { span.highlight { background-color: #ffffd0; +} + +.colored-box { + color: black; + padding: 20px; + display: inline-block; + border-radius: 10px; } \ No newline at end of file