Yunpeng Bai白云鹏

We revisit how DiTs organize visual content and discover that patch tokens exhibit a surprising degree of independence: even when PEs are perturbed, DiTs still produce globally coherent outputs, indicating that spatial coherence is primarily governed by PEs. Motivated by this finding, we introduce the Positional Encoding Field (PE-Field), which extends positional encodings from the 2D plane to a structured 3D field. PE-Field incorporates depth-aware encodings for volumetric reasoning and hierarchical encodings for fine-grained sub-patch control, enabling DiTs to model geometry directly in 3D space. Our PE-Field–augmented DiT achieves state-of-the-art performance on single-image novel view synthesis and generalizes to controllable spatial image editing.

We revisit the role of positional encoding in video diffusion transformers and show that it provides a useful spatial bias for geometry-aware control. If reference tokens are encoded according to their projected locations in the target view, the denoising model is encouraged to retrieve content from position-aligned regions of the input video. Building on this observation, we introduce a geometry-aware cross-attention mechanism that enables target video latent tokens to attend to structured context tokens derived from reference images or frames, with a projected positional encoding scheme that combines target-view 2D reprojection with depth-aware disambiguation.

We introduce geometric regularization losses into video generation by augmenting latent diffusion models with per-frame depth prediction. We adopt depth as the geometric representation because of the great progress in depth prediction and its compatibility with image-based latent encoders. To enforce structural consistency over time, we propose a multi-view geometric loss that aligns the predicted depth maps across frames within a shared 3D coordinate system.

We propose an efficient monocular depth estimation approach named FiffDepth, whose key feature is its use of diffusion priors. It transforms diffusion-based image generators into a feed-forward architecture for detailed depth estimation, preserving key generative features and integrating the strong generalization capabilities of models like DINOv2. FiffDepth achieves exceptional accuracy, stability, and fine-grained detail against state-of-the-art MDE approaches.

DreamDiffusion is the first method for generating high-quality images directly from brain electroencephalogram (EEG) signals, without the need to translate thoughts into text.

MesonGS is a codec for post-training compression of 3D Gaussians. We introduce a measurement criterion that considers both view-dependent and view-independent factors to assess the impact of each Gaussian point on the rendering output, enabling the removal of insignificant points, and decrease the entropy of attributes through two transformations that complement subsequent entropy coding to enhance the compression rate.

We propose a new method for NeRF-based facial avatar reconstruction that utilizes 3D-aware generative priors. Different from existing works that depend on a conditional deformation field for dynamic modeling, we propose to learn a personalized generative prior, formulated as a local and low-dimensional subspace in the latent space of a 3D-GAN.

We design an effective framework that allows the user to control the restoration process of degraded images with text descriptions, using CLIP's text-image feature compatibility to alleviate the difficulty of fusing text and image features. Our framework can be used for various image restoration tasks, including image inpainting, super-resolution, and colorization.

We propose a one-shot 3D facial avatar reconstruction framework, which only requires a single source image to reconstruct a high-fidelity 3D facial avatar, by leveraging the rich generative prior of 3D GANs and developing an efficient encoder-decoder network.

We propose to search the effective mixed-precision quantization policy by using a small proxy dataset for a model trained on a large-scale one, breaking the routine that requires a consistent dataset at model training and MPQ policy search time, which significantly improves MPQ searching efficiency.

We study how to represent a video with implicit neural representations (INRs) and propose a patch-wise solution, PS-NeRV, which represents videos as a function of patches and the corresponding patch coordinates. It naturally inherits the advantages of image-wise methods and achieves excellent reconstruction performance with fast decoding speed.

We propose the Semantic-Sparse Colorization Network (SSCN) to transfer both the global image style and detailed semantic-related colors to gray-scale images in a coarse-to-fine manner, perfectly balancing global and local colors while alleviating the ambiguous matching problem.

To tackle increasing ambiguity during forecasting, CMS-LSTM focuses on context correlations and multi-scale spatiotemporal flow with fine-grained local details, containing two elaborately designed blocks: Context Embedding (CE) for abundant context interactions and Spatiotemporal Expression (SE) for multi-scale spatiotemporal expression in hidden states.

We propose a simple deep network, TRNet, to estimate normal vectors based on a latent tangent space representation learned in the network. For each query point, the representation is a set of latent points spanning its tangent plane, generated using only the coordinates of its neighbors and regularized by a differentiable RANSAC-like component.