Documents Live, a web authoring and publishing system
If you see this, something is wrong
Table of contents
First published on Sunday, Oct 25, 2026 and last modified on Sunday, Oct 25, 2026 by François Chaplais.
Tongji University and Chongqing Afari Intelligent Drive Co., Ltd Email
Chongqing Afari Intelligent Drive Co., Ltd and Huazhong University of Science and Technology Email
Chongqing Afari Intelligent Drive Co., Ltd
Chongqing Afari Intelligent Drive Co., Ltd
Chongqing Afari Intelligent Drive Co., Ltd
Tongji University
Abstract
Controllable 3D part generation is pivotal for digital asset creation, yet existing methods lack the flexibility to accommodate diverse user workflows requiring varying levels of control. In this paper, we introduce FlexPart, a unified image-to-3D part generation framework that supports variable-granularity geometric conditions within a single model. By seamlessly integrating heterogeneous inputs, ranging from sparse points to dense bounding boxes and masks, via gated adaptive modulation, FlexPart unlocks a powerful cross-modal synergy. We also propose an asymmetric geometric guidance mechanism that leverages strong spatial priors to align representations across granularities, improving geometric consistency and facilitating inter-task synergy. Under mask conditioning on PartVerseXL, FlexPart reduces Part CD by 16.6% relative to TripoSG + SAM3D and by 10.5% relative to OmniPart, while maintaining competitive whole-mesh quality. The code can be found at https://github.com/Gucci233/FlexPart
1 Introduction
Perceiving objects as compositions of semantically coherent parts is essential for various tasks such as animation, simulation, and interactive editing. Unlike holistic 3D generation methods that generate whole entities [1, 2, 3, 4], image-to-3D part generation decomposes visual inputs into distinct, spatially assemblable, structurally detachable, and image-consistent 3D components.
However, this task presents significant challenges due to the inherent ambiguity of part decomposition. For example, the same object can be segmented at various granularities, and users may require different structural specifications depending on their goals. A straightforward solution is to project 2D segmentation masks [5] onto generated 3D meshes. However, such decoupled pipelines can introduce geometric inconsistencies during the lifting process. To address this limitation, recent approaches have integrated part control directly. Some methods rely on dense geometric prompts, such as pixel-wise 2D segmentation masks [6, 7, 8] or explicit bounding box layouts [9]. These inputs provide strong spatial constraints, enabling well-structured part generation. In contrast, other methods explore lighter forms of supervision, such as high-level prompts like part counts [10] or implicit volumetric partitioning [11]. These approaches reduce annotation effort and improve usability but often sacrifice precise spatial control.
While effective within their respective domains, existing methods are typically tailored to a single prompt granularity, as illustrated in Fig. 1(a). This specialization not only limits user flexibility by hindering seamless transitions between fine-grained control via dense masks and lightweight interaction via sparse points, but also overlooks the inherent structural relationships across different prompt modalities. Such rigidity is suboptimal, as prompts of varying granularities provide complementary information that could be jointly leveraged. This raises a fundamental question: can we develop a unified framework that accommodates variable-granularity inputs for flexible part generation, while exploiting their cross-modal synergy to enhance quality?
Driven by this motivation, we propose FlexPart, a unified framework for controllable 3D part generation. As shown in Fig. 1(a), FlexPart integrates sparse points, bounding boxes, and masks into a single Diffusion Transformer (DiT) via gated adaptive modulation. Joint training improves generation quality across prompt granularities. To further amplify this synergy, we introduce an Asymmetric Geometric Guidance mechanism. During training, it employs differentiable optimal transport to let strong spatial priors (masks/boxes) explicitly supervise the ambiguous representations of sparse points. The resulting model supports different prompt types at inference without retraining.
By consolidating diverse geometric constraints, FlexPart establishes a unified paradigm for versatile part creation with variable granularity. Under mask conditioning on PartVerseXL, FlexPart reduces Part CD by 16.6% relative to TripoSG + SAM3D and by 10.5% relative to OmniPart. Thanks to the learned cross-modal synergy, it remains highly competitive even with significantly sparser prompts. Ultimately, FlexPart delivers superior part-level generation quality (Fig. 1(b)) while preserving strong whole-mesh reconstruction fidelity.
Our contributions are summarized as follows: 1) We propose FlexPart, the first unified framework integrating point, box, and mask prompts into a single DiT to unlock cross-modal synergy for controllable 3D part generation; 2) We devise an Asymmetric Geometric Guidance mechanism that effectively transfers structural knowledge from dense priors to enhance the spatial reasoning of sparse inputs; 3) We demonstrate that our method achieves superior generation quality and geometric consistency compared to existing state-of-the-art approaches across diverse input modalities.
2 Related Work
General 3D Mesh Generation. Early 3D generation methods relied on optimization-based pipelines, using Score Distillation Sampling (SDS) to distill 2D priors into implicit 3D representations [12, 13, 14, 15]. While pioneering, these approaches often suffer from slow convergence and geometric artifacts. Feed-forward Large Reconstruction Models (LRMs) [16, 17] improve efficiency by directly predicting 3D representations from images. Native 3D generative models use compact latent representations [18, 1, 19] to support efficient diffusion training. Recent methods [4, 20, 21, 3] further improve geometric fidelity through structured sparse latents or flow matching. These object-level generators typically produce holistic meshes without explicit part decomposition, limiting part-level interaction and editing. Complementary work connects 3D analysis and generation. ROAD transfers discriminative semantic and structural priors to a generative model through reciprocal-objective alignment [22], while HERMES and HERMES++ jointly model scene understanding and future geometry generation in autonomous driving [23, 24]. Our work focuses on joint learning across geometric prompt granularities for controllable object-part generation.
3D Shape Analysis and Part Segmentation. Point-cloud analysis has advanced through efficient state-space backbones such as PointMamba [25] and model adaptation methods, including dynamic adapters with prompt tuning [26], spectral-domain fine-tuning [27], and input-aware parameter adaptation for 3D scenes [28]. Here, learned prompts used for model adaptation differ from the user-provided geometric prompts considered in our work. Language-driven approaches such as ShapeLLM [29] and Kestrel [30] further support object understanding and part-aware grounding. For part decomposition, supervised methods [31, 32] analyze existing shapes, while other approaches lift 2D foundation model priors, such as SAM [33] and GLIP [34], to 3D [5, 35, 36, 37, 38, 39]. Recent methods learn open-world 3D feature fields or native segmentation networks [40, 41, 42]. Conventional segmentation primarily identifies regions of existing surfaces. HoloPart [43] extends this setting by completing initially segmented regions into full 3D parts using a generative model. These pipelines rely on input geometry, whereas our task is to generate complete parts directly from an image and geometric prompts.
Part-level 3D Generation. Recent generative models synthesize objects as compositions of semantic components, with different ways of specifying part structure. Mask- and layout-conditioned methods [8, 7, 9] provide explicit spatial control, although precise masks or layouts can require substantial annotation effort. AutoPartGen [44] generates parts autoregressively and supports conditioning on images, part masks, or existing 3D objects. In a related programmatic reconstruction setting, MeshCoder [45] uses an LLM to translate point clouds into editable, part-structured mesh code. Methods such as PartCrafter [10] and PartPacker [11] internalize part decomposition within the generative process without requiring explicit geometric prompts. This simplifies interaction but offers less direct control over spatial part boundaries. A related challenge arises in count-controlled video generation, where NUMINA [46] uses countable latent layouts to improve numerical alignment, illustrating the value of spatial structure for abstract count-based control. Our framework unifies point, box, and mask prompts within one model, allowing users to choose the level of spatial control while benefiting from joint multi-granularity training.
3 Preliminaries
Optimal Transport. Optimal Transport (OT) provides a principled geometric framework for modeling soft correspondences between discrete sets. Given two sets of features with cardinalities \( N\) and \( M\) , OT aims to estimate a transport plan \( \mathbf{P} \in \mathbb{R}_{+}^{N \times M}\) , where each entry \( P_{ij}\) represents the mass transported from element \( i\) to element \( j\) .
Standard OT constrains \( \mathbf{P}\) to lie within the transport polytope \( \mathcal{U}(\mathbf{r}, \mathbf{q})\) , defined by marginal constraints:
(1)
where \( \mathbf{1}\) denotes a vector of ones, and \( \mathbf{r}\) and \( \mathbf{q}\) are the source and target mass vectors, respectively.
In high-dimensional matching problems, the entropically regularized formulation is widely adopted to enable efficient and differentiable optimization:
(2)
where \( \mathbf{C}\) and \( \varepsilon\) denote the cost matrix and regularization parameter, respectively. The entropy regularizer is defined as:
(3)
The optimal solution admits the factorized form \( \mathbf{P} = \mathrm{diag}(\mathbf{u}) \mathbf{K} \mathrm{diag}(\mathbf{v})\) , with \( \mathbf{K} = \exp(-\mathbf{C}/\varepsilon)\) , and can be efficiently computed using the Sinkhorn–Knopp algorithm.
Partial Optimal Transport. Standard OT enforces strict mass conservation, requiring all elements to be matched, which is ill-suited for scenarios with occlusions or noise. Partial OT addresses this by introducing a dustbin mechanism to absorb unmatched mass. Conceptually, the transport matrix is augmented to \( \bar{\mathbf{P}} \in \mathbb{R}^{(N+1) \times (M+1)}\) allowing elements with no confident correspondence to be assigned to the dustbin row or column. This relaxation handles partial and ambiguous assignments.
4 Methodology
Our method presents a unified multi-granularity framework for controllable image-to-3D part generation, accommodating diverse geometric user inputs, such as point prompts, 2D bounding boxes, and 2D masks, within a single model. Built upon the DiT architecture, FlexPart leverages a Gated Adaptive Layer Normalization (Gated AdaLN) mechanism to inject these geometric conditions, enabling unified modeling of heterogeneous inputs with varying precision. To further support this versatility, we incorporate a geometric guidance mechanism to maintain robustness under weak supervision. The overall architecture is illustrated in Fig. 2.
4.1 Gated Adaptive Modulation
Prompt Definition and Encoding. We consider multiple geometric prompt types for part-level structure, including point, box, and mask prompts. For each part \( i\) , we denote their embeddings as \( \mathbf{e}_i^{\mathrm{point}}, \mathbf{e}_i^{\mathrm{box}}, \mathbf{e}_i^{\mathrm{mask}} \in \mathbb{R}^{D}\) , all produced by lightweight modality-specific encoders. Point prompts are encoded via Gaussian Fourier features followed by an MLP to capture continuous 2D locations. Box prompts encode top-left and bottom-right corners separately with a learnable corner-type embedding inspired by SAM [33]. Mask prompts are processed using a small CNN with global average pooling to obtain compact representations. These encoders provide expressive, lightweight representations.
To train the model with different prompt combinations and improve robustness to missing inputs, we employ a probabilistic masking strategy. Although point prompts serve as the mandatory minimal input for users during inference, as shown in Fig. 2, we apply random masking to all modalities, including points, during training. Specifically, we simulate missing geometric prompts by replacing them with learnable null embeddings. For each part \( i\) and modality \( k \in \{\text{point}, \text{box}, \text{mask}\}\) , we use a binary drop mask \( m_i^k \in \{0,1\}\) with probability \( \Pr(m_i^k = 0) = p_\text{drop}\) , where \( p_\text{drop}\) is the manually set probability of masking a given modality. The effective prompt embedding is then defined as:
(4)
so that any dropped or absent prompt is replaced by its learnable null embedding \( \mathbf{e}_{\text{null}}^k \in \mathbb{R}^D\) .
The resulting embeddings for each part are aggregated and projected through a learnable fusion MLP \( \psi(\cdot)\) to yield a unified part-level conditioning vector:
(5)
where \( [\cdot \parallel \cdot]\) denotes feature concatenation. This mechanism produces a conditioning vector \( \mathbf{c}_i\) of fixed dimensionality for any input prompt combination.
Multi-Prompt Conditional Modulation. Given the fused part condition vector \( \mathbf{c}\) , we inject geometric control into the DiT via Gated AdaLN. This module dynamically modulates the transformer hidden states, concatenated with the timestep \( t\) and learnable part identity, by predicting scale and shift parameters based on \( \mathbf{c}\) .
A lightweight modulation network \( \Phi(\cdot)\) maps \( \mathbf{c}\) to adaptive scale, shift, and gating parameters \( (\boldsymbol{\gamma}, \boldsymbol{\beta}, \mathbf{g}) = \Phi(\mathbf{c})\) , which are applied to the hidden features \( \mathbf{H}\) as
(6)
where \( \mathrm{LN}(\cdot)\) denotes layer normalization and \( f(\cdot)\) is a small feed-forward projection. This gated formulation enables adaptive regulation of geometric conditions at each layer. Complementing the geometric prompts, image features from DINOv2 [47] are injected via cross-attention. Cross-attention provides global context, while Gated AdaLN controls part-level geometry.
Part-Aware Attention. Following the design of PartCrafter [10], we adopt a part-aware attention mechanism in the DiT to model both global inter-part interactions and local intra-part geometry. In our setup, we treat individual parts as distinct samples. A batch contains \( B\) parts, each represented by \( L\) latent tokens, collectively forming a hidden state tensor \( \mathbf{H} \in \mathbb{R}^{B \times L \times D}\) .
4.2 Asymmetric Geometric Guidance
To enrich sparse point prompts with spatial semantics, we propose asymmetric alignment: strong priors (boxes/masks) supervise weak point representations via differentiable optimal transport.
Asymmetric Visual Bridge. We project student point prompts and teacher box/mask prompts into a shared visual space using a Visual Bridge \( \mathcal{B}(\cdot)\) with cross-attention over image features \( \mathbf{I}_{\mathrm{enc}}\) . To prevent the alignment loss from updating the teacher prompt embeddings, we apply stop-gradient to the box/mask embeddings before the Visual Bridge:
Similarity-based Partial Optimal Transport. We align student and teacher features via differentiable partial optimal transport [48], which introduces global competition to resolve ambiguous matches. The affinity matrix is defined as:
(7)
where \( \tau\) is a learnable temperature and \( \alpha\) balances the two teacher sources.
To account for missing conditions (e.g., classifier-free guidance [49]), we introduce a learnable dustbin score \( \phi\) and construct an augmented matrix \( \bar{\mathbf{S}} \in \mathbb{R}^{(N+1)\times(M+1)}\) by padding \( \mathbf{S}\) with \( \phi\) . The optimal transport plan is computed via log-space Sinkhorn:
(8)
where \( \mathcal{U}\) denotes valid transport plans and \( H(\mathbf{P})\) is the entropy regularizer.
This mechanism transfers strong geometric priors from boxes and masks to point representations, improving spatial semantics without requiring dense prompts at inference.
Training Objective. We train FlexPart with a unified objective that combines conditional Rectified Flow and feature alignment supervision:
(9)
where the first term corresponds to the conditional Rectified Flow objective, encouraging accurate velocity prediction in the latent space, and the second term enforces index-wise alignment between student and teacher features via diagonal transport probabilities. Here, \( N\) denotes the number of student point representations in the alignment term, \( \delta\) is a small constant for numerical stability, and \( \lambda_{\mathrm{aux}}\) controls the strength of the alignment supervision.
Inference. During inference, we support flexible geometric conditions ranging from sparse points to dense masks. Point prompts are always required as the base condition, while higher-level cues are incorporated hierarchically for flexible control. Specifically, masks can be paired with bounding boxes, and boxes can be combined with point prompts. This design enables precise control of part generation under varying levels of geometric guidance.
5 Implementation Details and Metrics
5.1 Implementation Details
We use a pretrained 3D VAE from TripoSG [3] to obtain part latents. Each part is encoded into latent tokens following 3DShape2VecSet [18]. Following PartCrafter [10], we initialize the generative backbone with pretrained TripoSG [3] weights and fine-tune it using the following two-stage training strategy. We train FlexPart on a composite dataset derived from Objaverse [50] and PartVerseXL [9]. Stage 1 targets objects with 1–8 parts, each encoded with 512 tokens. To ensure robustness against suboptimal placement, we randomly sample points within masks and apply a 0.5 dropout rate to both boxes and masks. Stage 2 scales to 2–16 parts using 1024 tokens per part to handle varying granularities. In this stage, we set the dropout probability to 0.1 for points and 0.7 for masks, similar to [51, 52]. For bounding boxes, we implement a mixed dropping strategy with a probability of 0.25 for full dropout, and 0.25 for partial masking applied to half of the parts. Training is conducted on 8 NVIDIA H20 GPUs with a batch size of 256, using learning rates of \( 1\times10^{-4}\) and \( 5\times10^{-5}\) for the two stages, respectively.
5.2 Datasets and Metrics
To assess generalization, we conduct evaluations on a held-out test set from PartVerseXL [9] and PartObjaverse-Tiny [40], ensuring strict disjointness from the training set. For each dataset, we select the view that maximizes the product of visible pixel count and number of visible parts, ensuring sufficient part coverage.
Whole Mesh Evaluation Metrics. We first assemble the generated parts into a complete mesh and evaluate overall geometric quality using Chamfer Distance (CD) and F-Score, where F-Score is computed with a distance threshold of 0.1. Semantic consistency between the generated mesh and the input image is measured using ULIP [53].
Part-level Evaluation Metrics. A primary challenge in evaluating parts is that the number of generated components often differs from the ground truth. To ensure a fair comparison, we first filter out ground truth parts that are fully occluded in the input image. We apply CD-based Hungarian matching between the remaining ground-truth parts and generated parts to obtain one-to-one correspondences. We compute part-level metrics only over matched pairs; unmatched ground-truth and generated parts are excluded without additional penalties. Based on these matched pairs, we report Part CD to measure geometric deviation. Furthermore, we evaluate spatial alignment using part-level bounding-box IoU (IoU\( _B\) ) for coarse extent, and surface-based voxel IoU (IoU\( _V\) ) for fine-grained overlap. To handle non-watertight geometry, we define occupancy based on the distance field rather than interior filling. Specifically, we use a voxel grid with a pitch of \( h_{\mathrm{vox}} = 0.03\) , where a voxel is considered occupied if its distance to the nearest mesh surface is less than or equal to \( h_{\mathrm{vox}}\) . Unless otherwise stated, benchmark box inputs are derived from ground-truth masks, and points are sampled from the corresponding mask regions.
Detailed dataset construction and benchmark preparation are provided in App. A.
6 Experiments
| Method | Reference | Cond. | PartVerseXL | PartObjaverse-Tiny | ||||
| Part CD \( \downarrow\) | Part IoU\( _V\) \( \uparrow\) | Part IoU\( _B\) \( \uparrow\) | Part CD \( \downarrow\) | Part IoU\( _V\) \( \uparrow\) | Part IoU\( _B\) \( \uparrow\) | |||
| TRELLIS + SAM3D | CVPR 25 | MV Mask | 0.3471 | 0.1885 | 0.2173 | 0.3916 | 0.1226 | 0.2031 |
| TripoSG + SAM3D | TPAMI 25 | MV Mask | 0.3523 | 0.1727 | 0.1896 | 0.4235 | 0.1250 | 0.2078 |
| PartCrafter | NeurIPS 25 | Number | 0.5235 | 0.0907 | 0.1257 | 0.5193 | 0.0802 | 0.1455 |
| PartPacker | NeurIPS 25 | - | 0.3933 | 0.1717 | 0.2210 | 0.3932 | 0.1588 | 0.2327 |
| OmniPart | SIGGRAPH Asia 25 | Mask | 0.3283 | 0.1961 | 0.2831 | 0.3681 | 0.1568 | 0.2649 |
| FlexPart(ours) | - | Mask | 0.2938 | 0.2049 | 0.2840 | 0.3560 | 0.1571 | 0.2778 |
| FlexPart(ours) | - | Box | 0.3042 | 0.2078 | 0.2810 | 0.3675 | 0.1503 | 0.2693 |
| FlexPart(ours) | - | Point | 0.3334 | 0.1902 | 0.2506 | 0.3802 | 0.1411 | 0.2456 |
| Method | Reference | PartVerseXL | PartObjaverse-Tiny | ||||
| Mean CD \( \downarrow\) | F-Score \( \uparrow\) | ULIP \( \uparrow\) | Mean CD \( \downarrow\) | F-Score \( \uparrow\) | ULIP \( \uparrow\) | ||
| TRELLIS | CVPR 25 | 0.2125 | 0.7514 | 0.2370 | 0.2038 | 0.7478 | 0.2434 |
| TripoSG | TPAMI 25 | 0.1227 | 0.8176 | 0.2331 | 0.1504 | 0.7811 | 0.2410 |
| PartCrafter | NeurIPS 25 | 0.1666 | 0.7346 | 0.1950 | 0.1731 | 0.7280 | 0.2136 |
| PartPacker | NeurIPS 25 | 0.1489 | 0.7899 | 0.2266 | 0.1612 | 0.7682 | 0.2397 |
| OmniPart | SIGGRAPH Asia 25 | 0.1547 | 0.7636 | 0.2179 | 0.1585 | 0.7501 | 0.2335 |
| FullPart | ICLR 26 | - | - | - | 0.1625 | 0.7577 | 0.1775 |
| FlexPart(ours) | - | 0.1418 | 0.7877 | 0.2206 | 0.1587 | 0.7510 | 0.2350 |
6.1 Main Results
We compare FlexPart against six state-of-the-art methods, including two holistic mesh generation models [4, 3] and four native part-based generation approaches [10, 11, 7, 9].
Part-level Mesh Quality. We first evaluate the geometric fidelity of individual parts (Tab. 1) to assess controllable part generation. For comparison with whole-mesh generation methods, we employ SAM3D [5] to project multi-view masks onto their generated meshes for part segmentation. This post-hoc lifting process often introduces geometric inconsistencies, significantly degrading part-level accuracy. In contrast, FlexPart directly infers compact part geometry. Under the same mask condition, our method outperforms OmniPart by a large margin, reducing Part CD on PartVerseXL by 10.5%. This improvement is consistent with the benefits of direct part generation: OmniPart uses intermediate axis-aligned 3D boxes, which can enclose substantial empty space around non-axis-aligned shapes.
Crucially, FlexPart maintains robust performance across varying input granularities. Relaxing the condition from masks to boxes yields stable quality, with Part CD degrading by merely 3.5% on PartVerseXL. Even under these weaker priors, FlexPart achieves better Part CD and Part IoU\( _B\) than structurally unconstrained methods like PartPacker. FullPart is excluded here due to its reliance on 3D bounding boxes. A fairer evaluation under controlled settings is provided in Sec. 6.2.
Whole Mesh Quality. Complementing our superior part-level performance, we further assess the global quality of the assembled meshes. As shown in Tab. 2, FlexPart achieves strong and consistent mesh-level quality across both benchmarks. On PartVerseXL, we achieve a Mean CD of 0.1418, surpassing OmniPart by 8%, indicating that our improvements in local part precision translate into better global geometric accuracy. On PartObjaverse-Tiny, while some specialized methods achieve marginally better results, FlexPart remains highly competitive. These results show that FlexPart supports flexible part control while maintaining competitive whole-mesh quality.
Qualitative Results. Fig. 3 presents qualitative comparisons across diverse categories. FullPart provides rough layout control via boxes but often yields inaccurate shapes. PartPacker achieves decent global geometry but lacks explicit control to match user-specified structures. OmniPart tends to distort meshes, deviating noticeably from inputs, while PartCrafter, limited to part-count conditioning, struggles to align with intended divisions. In contrast, FlexPart generates distinct and semantically meaningful parts, ensuring visual realism and structural coherence suitable for downstream editing.
6.2 Analysis and Ablation Study
To investigate the architecture design, we conduct ablation studies on PartVerseXL unless otherwise specified. Default settings are marked in gray.
Effect of Multi-granularity Synergy. Tab. 3 demonstrates that our framework achieves deep cross-modal synergy instead of a naive combination of conditions. Specifically, stronger geometric priors effectively guide weaker ones during training. Our joint model improves Point only inference by 7.9% in IoU\( _V\) and 6.1% in CD because it internalizes precise boundary constraints from dense inputs. Furthermore, joint training consistently outperforms single-modality models across all evaluation settings. For example, under mask conditioning, the unified model reduces Part CD from 0.3224 for the mask-only training baseline to 0.2938. These results confirm that FlexPart learns integrated semantics in a shared space. This formulation provides a versatile solution that maintains high geometric fidelity across diverse interaction modes. Fig. 4 illustrates robust, semantically consistent results across prompt granularities.
| Eval Cond. | Train Setting | Whole Mesh | Part-level | ||||
| CD \( \downarrow\) | F-Score \( \uparrow\) | ULIP \( \uparrow\) | CD \( \downarrow\) | IoU\( _V\) \( \uparrow\) | IoU\( _B\) \( \uparrow\) | ||
| Point | Point only | 0.1623 | 0.7368 | 0.2014 | 0.3550 | 0.1763 | 0.2320 |
| Point + Mask | 0.1587 | 0.7505 | 0.2112 | 0.3421 | 0.1816 | 0.2396 | |
| Point + Box + Mask | 0.1525 | 0.7689 | 0.2171 | 0.3334 | 0.1902 | 0.2506 | |
| Box | Box only | 0.1489 | 0.7756 | 0.2171 | 0.3155 | 0.2011 | 0.2756 |
| Point + Box | 0.1458 | 0.7812 | 0.2201 | 0.3102 | 0.2023 | 0.2763 | |
| Box + Mask | 0.1434 | 0.7834 | 0.2209 | 0.3094 | 0.2089 | 0.2795 | |
| Point + Box + Mask | 0.1447 | 0.7828 | 0.2219 | 0.3042 | 0.2078 | 0.2810 | |
| Mask | Mask only | 0.1541 | 0.7564 | 0.2056 | 0.3224 | 0.1905 | 0.2688 |
| Point + Mask | 0.1501 | 0.7712 | 0.2169 | 0.3202 | 0.1921 | 0.2712 | |
| Box + Mask | 0.1450 | 0.7848 | 0.2195 | 0.3076 | 0.1998 | 0.2822 | |
| Point + Box + Mask | 0.1418 | 0.7877 | 0.2206 | 0.2938 | 0.2049 | 0.2840 | |
| Cond. | Guid. | Part-level | ||
| CD \( \downarrow\) | IoU\( _V\) \( \uparrow\) | IoU\( _B\) \( \uparrow\) | ||
| Points | - | 0.3406 | 0.1811 | 0.2323 |
| ✓ | 0.3334 | 0.1902 | 0.2506 | |
| Masks | - | 0.3011 | 0.2078 | 0.2813 |
| ✓ | 0.2938 | 0.2049 | 0.2840 | |
Effect of Asymmetric Geometric Guidance.
Building upon the observed multi-granularity synergy, we evaluate the proposed Asymmetric Geometric Guidance by ablating the auxiliary alignment loss. As shown in Tab. 4, it significantly improves performance under sparse point prompts, yielding gains of 7.9% in Part IoU\( _B\) and 5.0% in Part IoU\( _V\) over the baseline. This indicates that OT-based alignment effectively transfers structural priors from dense annotations to regularize learning from sparse inputs. Performance under mask conditions remains comparable, suggesting positive cross-granularity transfer without conflict. The guidance operates only during training, with no inference overhead.
| Method | Cond. | Whole Mesh | Part-level | ||||
| CD\( \downarrow\) | F-Score\( \uparrow\) | ULIP\( \uparrow\) | CD\( \downarrow\) | IoU\( _V\) \( \uparrow\) | IoU\( _B\) \( \uparrow\) | ||
| FullPart | 3D Box | 0.1625 | 0.7577 | 0.1775 | 0.2488 | 0.2064 | 0.4370 |
| Ours | 3D Box | 0.1385 | 0.8158 | 0.2209 | 0.2356 | 0.2653 | 0.4027 |
Analysis of Stronger 3D Conditions. Although our primary focus is on generating 3D parts from lightweight 2D inputs, we also evaluate FlexPart under explicit 3D constraints. For a fair comparison with FullPart, which uses 3D bounding boxes, we retrain a variant conditioned solely on 3D boxes and evaluate it on PartObjaverse-Tiny. As shown in Tab. 5, our method outperforms FullPart on most metrics under the same supervision. This result demonstrates that Gated AdaLN handles both sparse 2D inputs and explicit 3D constraints.
Analysis of SAM-based Alternatives. A seemingly straightforward alternative is to convert sparse prompts into dense masks using 2D segmentation models (e.g., SAM) before 3D generation. However, such cascaded pipelines are inherently limited: lacking 3D part-level semantics, SAM often produces localized fragments rather than complete functional parts, leading to accumulated errors and degraded geometry. In contrast, FlexPart directly models heterogeneous conditions end-to-end, avoiding intermediate information loss and enabling consistent part-level generation from sparse prompts. Detailed analyses are provided in App. H.
Robustness to Input Variations. Perturbing the position and size of 2D boxes by up to 20% increases Part CD from 0.3042 to 0.3171, while Part IoU\( _V\) changes from 0.2078 to 0.2032. Providing boxes for half of the parts and points for the remainder yields Part CD of 0.3287. Mixing masks for half of the parts with boxes for the rest achieves Part CD of 0.3190, despite this combination being unseen during training. These results support generation with noisy and heterogeneous prompts. A complementary study evaluates joint point-and-box inputs from three annotators on 50 PartObjaverse-Tiny objects. Full protocols and results, including count-only inputs, are provided in App. D and E.
Sensitivity of Point Prompts. We evaluate four point-placement strategies on PartVerseXL: Center (the part-mask centroid), Edge (near the part boundary), Offset (a perturbed centroid), and Cluster (the patch center closest to the object-level geometric center). Under point-only input, switching from Center to Edge increases Part CD from 0.3302 to 0.4190 and reduces Part IoU\( _V\) from 0.1905 to 0.1342. With additional box or mask prompts, Part CD varies across the four strategies within 0.3036–0.3178 and 0.2995–0.3087, respectively. Stronger geometric prompts therefore reduce sensitivity to point placement. Experimental details and quantitative and qualitative results appear in App. G.
7 Conclusion
We introduced FlexPart, a unified framework for controllable image-to-3D part generation with geometric prompts at varying granularities. Gated adaptive modulation integrates heterogeneous prompts within a single model, while Asymmetric Geometric Guidance uses strong spatial priors to improve sparse point representations during training. Experiments demonstrate improved part-level geometric quality and competitive whole-mesh fidelity across prompt conditions. These results support joint multi-granularity learning as a practical approach to flexible 3D part generation.
Limitations. Occlusion handling remains challenging. Single-view 2D conditions provide limited depth cues, and the model may place occluded parts incorrectly in visible regions. Improving spatial reasoning under occlusion is a direction for future work.
Acknowledgements
We thank Chongqing Afari Technology Co., Ltd. for providing computational resources.
This work was supported by The Explorers Program of Shanghai (Basic Research Funding) (26TS1404200); the Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China (JYB2025XDXM312); and Tongji University’s “Medicine+X” Cultivation Program (2026-0648-YB-05).
A Dataset Preparation and Benchmark Details
Data Preparation. We construct the training set from subsets of Objaverse [50] and PartVerseXL [9], totaling approximately 250K 3D models. Due to batch-wise part-packing during training, about 180K models are effectively used. The Objaverse subset provides heterogeneous annotations, where some models contain only complete object meshes while others include part-level meshes. PartVerseXL complements this with densely and carefully annotated part-level meshes. Since most Objaverse meshes are non-watertight, we convert all models into watertight representations, following prior work [19, 3, 20, 21], showing that watertight geometry leads to more stable training and improved generation quality. For each mesh, multiple viewpoints are rendered, and the view with the largest visible foreground area is selected as the training image.
For the selected view, we generate 2D part visibility masks via ID-based rendering. Each part is rendered with a unique color, and depth buffering naturally resolves inter-part occlusions. The rendered image is decoded into a part ID map, from which binary visibility masks are extracted. Bounding boxes and point prompts are then derived directly from the corresponding masks, enabling unified geometric supervision at multiple granularities. To simulate user interactions, we sample 10 points within each mask and select the one nearest its centroid.
Benchmarks. For evaluation, we construct benchmark subsets to test generalization. From PartVerseXL [9], which provides high-quality part annotations, we create a held-out test set with no overlap with our training data by randomly selecting 100 shapes with no more than 16 annotated parts, matching the maximum part count used during training. We also evaluate on PartObjaverse-Tiny [40], a publicly available part-annotated dataset, including all shapes with up to 16 parts. Although some annotations in PartObjaverse-Tiny are less precise, it provides a complementary setting for assessing part-level generalization. For each dataset, we select the view that maximizes the product of visible pixel count and the number of visible parts, ensuring an informative view with sufficient part coverage. Before computing geometric metrics, we align each generated mesh to its ground-truth counterpart using rigid Iterative Closest Point.
A.1 Evaluation Metric Definitions
Before ICP, each shape is normalized to fit within \( [-1,1]^3\) so that its longest bounding-box side is 2, with no additional scaling during rigid alignment.
Whole Mesh Evaluation Metrics. Whole CD is the symmetric unsquared L2 Chamfer distance over 10K surface samples. For sampled point sets \( X\) and \( Y\) ,
(10)
F-Score is the harmonic mean of precision and recall at distance threshold 0.1 (5% of the normalized longest side). ULIP [53] measures semantic consistency with the input image. The same CD definition is used for matched part pairs.
Part-level Evaluation Metrics. A primary challenge in evaluating parts is that the number of generated components often differs from the ground truth. To ensure a fair comparison, we first filter out ground truth parts that are fully occluded in the input image. We apply CD-based Hungarian matching between the remaining ground-truth parts and generated parts to obtain one-to-one correspondences. We compute part-level metrics only over matched pairs; unmatched ground-truth and generated parts are excluded without additional penalties. Based on these matched pairs, we report Part CD to measure geometric deviation. Furthermore, we evaluate spatial alignment using part-level bounding-box IoU (IoU\( _B\) ) for coarse extent, and surface-based voxel IoU (IoU\( _V\) ) for fine-grained overlap. To handle non-watertight geometry, we define occupancy based on the distance field rather than interior filling. Specifically, we use a voxel grid with a pitch of \( h_{\mathrm{vox}} = 0.03\) , where a voxel is considered occupied if its distance to the nearest mesh surface is less than or equal to \( h_{\mathrm{vox}}\) . The normalized longest axis spans approximately 67 voxels at this pitch, which is fixed across all methods. Unless otherwise stated, benchmark boxes are derived from masks, and points are sampled from the corresponding mask regions.
A.2 Baseline Protocol and Data Lineage
All baselines use official checkpoints, without retraining or fine-tuning. TRELLIS and TripoSG receive an image, with multi-view SAM3D lifting for part evaluation; PartCrafter receives an image and GT part count; PartPacker receives only an image; OmniPart receives an image and GT-derived 2D masks; FullPart receives an image and GT 3D boxes. We re-evaluate every method using the same view selection, shape normalization, ICP alignment, and metrics. The main table is a system-level comparison with different inputs and training sets. Controlled comparisons use mask inputs for OmniPart, 3D boxes for FullPart, and part count for PartCrafter.
TRELLIS uses TRELLIS-500K; TripoSG uses curated Objaverse(-XL) and ShapeNet data. PartCrafter uses part-annotated Objaverse, ShapeNet, and ABO data; PartPacker uses filtered/extracted parts from TRELLIS-500K/Objaverse-XL; OmniPart uses self-constructed part-labeled data with a TRELLIS-based backbone. FullPart trains on PartVerseXL and is omitted on that benchmark. Our PartVerseXL evaluation set is strictly held out, whereas PartObjaverse-Tiny shares Objaverse asset lineage with several pretrained methods, including our TripoSG initialization.
A.3 Alignment Implementation and Overhead
We use 50 log-domain Sinkhorn iterations with a learnable dustbin score and temperature. The box/mask fusion weight is \( \alpha=0.5\) , and the auxiliary alignment loss weight is \( \lambda_{\mathrm{aux}}=1.0\) . The alignment branch has 1.18M parameters (0.077% of the model), and transport matrices are at most \( 33\times33\) . An fp16 profile on one NVIDIA H20 GPU, with 8 parts and 1024 latent tokens per part, measures 8.9 ms for this branch versus 525 ms for the DiT forward pass (1.70%). This measures forward-pass overhead in the profiled configuration, not total training throughput.
B Conditioning Route Analysis
We compare Gated AdaLN with injecting geometric prompt tokens into the pretrained image cross-attention pathway. The token-injection variant uses the full training schedule. Under mask conditioning, it performs worse than Gated AdaLN and the count-conditioned PartCrafter reference (Table 6). A possible explanation is competition between prompt tokens and the 257 DINOv2 tokens within one attention softmax; the separate modulation route avoids this competition. This diagnostic supports our choice for this backbone, without establishing a general disadvantage of cross-attention conditioning.
| Conditioning route | Part CD \( \downarrow\) | IoU\( _V\) \( \uparrow\) | IoU\( _B\) \( \uparrow\) |
| Gated AdaLN (ours) | 0.2938 | 0.2049 | 0.2840 |
| Prompt tokens in cross-attention | 0.6147 | 0.0818 | 0.1089 |
| PartCrafter (reference) | 0.5235 | 0.0907 | 0.1257 |
C Analysis of Asymmetric Geometric Guidance
C.1 Effect of Stop-Gradient Design
| Condition | Setting | CD \( \downarrow\) | F-Score \( \uparrow\) | ULIP \( \uparrow\) |
| Point | w/ Grad | 0.1518 | 0.7702 | 0.2166 |
| wo/ Grad | 0.1525 | 0.7689 | 0.2171 | |
| Box | w/ Grad | 0.1531 | 0.7762 | 0.2198 |
| wo/ Grad | 0.1447 | 0.7828 | 0.2219 | |
| Mask | w/ Grad | 0.1512 | 0.7761 | 0.2212 |
| wo/ Grad | 0.1418 | 0.7877 | 0.2206 | |
We further analyze the effect of the stop-gradient design in geometric guidance. We find that it primarily affects whole-mesh reconstruction quality across different prompt granularities. As shown in Tab. 7, enabling gradient flow (w/ Grad) slightly improves performance under sparse point prompts, reducing CD from 0.1525 to 0.1518. However, it degrades performance under stronger geometric conditions, including both box and mask prompts, where CD increases from 0.1447 to 0.1531 and from 0.1418 to 0.1512, respectively.
This behavior suggests that allowing gradients leads to overly symmetric optimization, where strong and weak prompts are pulled toward each other in representation space. Our goal is to guide weak point representations using strong geometric priors while preventing the alignment loss from updating the teacher prompt embeddings. We stop gradients at box/mask embeddings before the Visual Bridge.
C.2 Partial OT versus Contrastive Alignment
We evaluate whether partial optimal transport (OT) improves generation beyond a simpler contrastive alignment objective. We train an InfoNCE variant from the same initialization and with the same training budget, changing only the alignment loss. Table 8 reports the resulting part-level generation metrics on PartVerseXL. Partial OT achieves lower Part CD and higher voxel and bounding-box IoU under all three prompt conditions. In particular, under point prompts, Part CD decreases from 0.3499 to 0.3334. This experiment evaluates the alignment objective in end-to-end generation.
| Condition | Alignment loss | Part CD \( \downarrow\) | IoU\( _V\) \( \uparrow\) | IoU\( _B\) \( \uparrow\) |
| Mask | Partial OT (ours) | 0.2938 | 0.2049 | 0.2840 |
| InfoNCE | 0.3053 | 0.1924 | 0.2673 | |
| Box | Partial OT (ours) | 0.3042 | 0.2078 | 0.2810 |
| InfoNCE | 0.3074 | 0.1874 | 0.2653 | |
| Point | Partial OT (ours) | 0.3334 | 0.1902 | 0.2506 |
| InfoNCE | 0.3499 | 0.1807 | 0.2346 | |
C.3 Global Assignment on Frozen Features
Known part correspondences provide diagonal supervision during training, but point features can still be ambiguous among similar strong priors. Paired L2 alignment treats each known pair independently, while InfoNCE normalizes similarities independently within each point row. Neither imposes a column marginal on the induced matching. Sinkhorn instead couples the assignments through both row and column marginals, introducing competition among points for the available priors.
To isolate this assignment effect, we freeze the trained model and use 199 PartObjaverse-Tiny objects containing 1,330 parts. Point, box, and mask prompts are derived from each ground-truth part as in training. We run only the prompt encoders, visual bridge, and DINOv2 feature extractor, without diffusion sampling or mesh decoding. We then apply row-wise similarity matching and Sinkhorn to the same similarity matrix. Top-1 matching measures correct point-to-prior matches, while many-to-one collapse measures duplicated prior selections by multiple points.
| Assignment operator | Top-1 matching \( \uparrow\) | Many-to-one collapse \( \downarrow\) |
| Row-wise similarity matching | 0.803 | 0.133 |
| Sinkhorn global assignment | 0.895 | 0.024 |
On normalized features, an InfoNCE similarity argmax and an L2 nearest-neighbor readout both reduce to row-wise similarity matching, although their training objectives differ. As shown in Table 9, Sinkhorn improves Top-1 matching from 0.803 to 0.895 and reduces many-to-one collapse by approximately \( 5.5\times\) . Ambiguity is particularly pronounced for symmetric parts: row-wise matching accuracy is 0.516 for symmetric pairs, compared with 0.931 for other pairs. These observations support global competition for resolving ambiguous feature correspondences.
C.4 Dustbin for Unmatched Priors
The dustbin addresses a separate issue: unmatched entries in batch-level alignment. In the same frozen-feature diagnostic, we append four box-prior columns from another object, none of which corresponds to a part of the current object. We compare strict OT with partial OT using the dustbin under this shared distractor setting. As shown in Table 10, partial OT improves Top-1 matching from 0.677 to 0.747. The dustbin supplies a channel for unmatched mass, relaxing the requirement to assign mass to irrelevant priors. Together with the preceding diagnostic, this separates the role of global competition among valid matches from the role of unmatched-mass handling.
| Transport formulation | Top-1 matching \( \uparrow\) |
| Strict OT | 0.677 |
| Partial OT with dustbin | 0.747 |
D Adaptability to Input Variations
| Method | Part-level | ||
| CD \( \downarrow\) | IoU\( _V\) \( \uparrow\) | IoU\( _B\) \( \uparrow\) | |
| 2D Box w/ Perturbation | 0.3171 | 0.2032 | 0.2759 |
| Half Box + Point | 0.3287 | 0.1998 | 0.2609 |
| Half Mask + Half Box | 0.3190 | 0.2053 | 0.2706 |
| Part Count Only | 0.4786 | 0.1255 | 0.1608 |
| Ours (2D Box) | 0.3042 | 0.2078 | 0.2810 |
We evaluate FlexPart under noisy, partial, and heterogeneous inputs through controlled experiments that approximate common user input variations. Drawn bounding boxes may be misaligned, loosely cover target parts, or specify only a subset of components.
Perturbed bounding boxes. We simulate noisy user annotations by randomly offsetting the position and size of each input 2D bounding box by up to 20%. This perturbation tests the model’s tolerance to spatial inaccuracy while preserving approximate part localization.
Partial box specification. To reflect scenarios where users focus on only a subset of parts, we provide bounding boxes for half of the parts and use point prompts for the remaining parts. This hybrid setting evaluates whether strong and weak geometric prompts can be jointly exploited within the unified framework.
Mixed-condition specification. To further assess the effectiveness of our unified framework, we consider a mixed condition setting that is never observed during training. Specifically, we provide 2D masks for half of the parts and 2D bounding boxes for the remaining parts. This setting evaluates whether FlexPart can seamlessly integrate heterogeneous strong priors and generalize to novel combinations of geometric conditions.
Minimal input with part count only. We also consider the extreme case where users specify only the desired number of parts. Since FlexPart requires at least point-level spatial prompts, we automatically derive point prompts from the input image. Specifically, we extract high-resolution patch features \( \mathbf{f}_{\mathrm{patch}}\) using DINOv2. To improve spatial consistency, we incorporate the 2D coordinates \( \mathbf{x}_{\mathrm{patch}}\) of each patch to form a joint representation \( \mathbf{v}_{\mathrm{patch}} = [\mathbf{f}_{\mathrm{patch}} \parallel \omega_{\mathrm{sp}} \mathbf{x}_{\mathrm{patch}}]\) , where the spatial weight is set to \( \omega_{\mathrm{sp}} = 5.0\) . We then perform spectral clustering on this combined representation using \( k=15\) nearest neighbors to obtain point prompts from the cluster centroids. In our preliminary experiments, we compared several clustering methods, including K-means and hierarchical clustering; however, spectral clustering consistently yielded the best performance in capturing semantically and spatially coherent part structures. This strategy enables effective part generation even from minimal input, and outperforms PartCrafter when only part counts are provided.
The quantitative results for these controlled input variations are summarized in Tab. 11. The results demonstrate a clear trade-off between the strength of geometric conditions and generation accuracy: stronger inputs provide more precise part localization, while weaker inputs offer greater flexibility with reduced supervision. Crucially, thanks to its unified multi-granularity design, FlexPart consistently maintains reasonable performance across all input settings and generalizes well to noisy, partial, and mixed conditions. This highlights the advantage of a unified framework that seamlessly integrates heterogeneous user inputs for robust, practical interactive 3D part generation.
Complementary evaluation with human-provided point and box prompts is reported in App. E.
E Evaluation with Human-Provided Prompts
The controlled perturbations in App. D approximate common input errors. To complement them with actual human-provided prompts, we invite three annotators to mark their intended parts on the input images of 50 PartObjaverse-Tiny objects using points and 2D bounding boxes. Every evaluated input contains both prompt types; this study evaluates joint point-and-box input. Table 12 reports the mean and standard deviation of the evaluation scores across the three annotators.
| Prompt source | Part CD \( \downarrow\) | IoU\( _V\) \( \uparrow\) | IoU\( _B\) \( \uparrow\) |
| Human points + 2D boxes | \( 0.384\pm0.022\) | \( 0.131\pm0.010\) | \( 0.231\pm0.013\) |
We compute these metrics against the dataset parts after Hungarian matching. Since an annotator’s intended decomposition may differ from the dataset annotation, the scores serve as reference measurements of agreement with the dataset, rather than strict measures of whether the user’s intended decomposition is satisfied. The observed cross-annotator variation suggests limited sensitivity to the annotator within this study. This small-scale evaluation provides evidence for generation with human-provided spatial prompts, but does not measure interaction time or downstream usability. Complementary qualitative results with manually provided point or box prompts are shown in Fig. 9.
F Part Generation Efficiency
Beyond accuracy, we evaluate efficiency in terms of inference latency and memory usage. Specifically, we measure the average time required to generate a single part, covering the entire process from condition encoding to final mesh extraction. Additionally, we analyze peak GPU memory (VRAM) consumption during object generation with varying numbers of parts. All evaluations are conducted on a single NVIDIA A800 GPU using random instances from the PartVerseXL validation set.
Regarding inference speed, Fig. 5(a) illustrates that FlexPart achieves competitive performance relative to existing baselines. Although our method shares a similar Diffusion Transformer backbone with PartCrafter [10], we concatenate part identity embeddings with the latent code rather than adding them directly. This design choice introduces a slight computational overhead during attention operations, resulting in a marginal increase in inference latency. Compared to two-stage approaches like OmniPart [7], which accelerates generation via latent downsampling, our method operates at a higher resolution and thus requires more computation. FlexPart strikes a balanced trade-off between generation quality and speed, leaving potential for further acceleration in future work.
In terms of memory consumption, as shown in Fig. 5(b), FlexPart maintains a modest peak VRAM footprint of \( \sim\) 6GB to 8GB, which is comparable to the lightweight baseline PartCrafter [10]. In contrast, dense generation methods such as OmniPart [7] and FullPart [9] demand significantly higher resources, consuming nearly twice the memory under identical settings. Furthermore, our approach exhibits robust scalability with respect to object complexity. When doubling the target part count from 8 to 16, the peak memory usage increases only marginally. This footprint suggests that consumer GPUs with sufficient VRAM can support complex multi-part generation.
G Sensitivity of Point Prompts
We investigate how point placement affects generation quality across input modalities, focusing on spatial ambiguity and the role of point prompts.
Point Sampling Strategies. We consider four representative strategies to simulate diverse user behaviors and annotation noise: 1). Center, the centroid of the part mask; 2). Edge, a point sampled near the part boundary to induce maximal spatial ambiguity; 3). Offset, a randomly perturbed centroid to mimic typical user imprecision; 4). Cluster, the patch center nearest the object’s geometric center.
Quantitative Analysis. As shown in Tab. 13, the sensitivity to point placement strongly depends on the availability of additional geometric priors. When using Point Only inputs, the model exhibits clear sensitivity to suboptimal point locations. In particular, sampling points near object boundaries (Edge) leads to a substantial degradation across all metrics, with CD increasing from 0.3302 to 0.4190 and IoU\( _V\) dropping from 0.1905 to 0.1342, indicating greater spatial ambiguity without supporting cues.
In contrast, when stronger geometric conditions such as bounding boxes or masks are provided, the influence of point placement becomes significantly less pronounced. For example, under the Point + Box setting, the variation across different strategies remains within a narrow range (e.g., CD varies only from 0.3036 to 0.3178), indicating stable performance. A similar trend is observed for Point + Mask, where IoU\( _V\) fluctuates marginally between 0.1962 and 0.2088, suggesting that geometric priors effectively dominate the spatial reasoning process.
Qualitative Analysis. Consistent with the quantitative findings, our qualitative results shown in Fig. 6 visualize the generation outcomes under the four point sampling strategies in the Point Only setting. The visual comparison clearly demonstrates how ambiguous point inputs, such as those near the edges, lead to a loss of geometric fidelity.
Overall, while FlexPart is sensitive to point locations under point-only conditions, its sensitivity to point placement is substantially reduced when stronger geometric prompts are also provided.
| Input Type | Point Strategy | CD \( \downarrow\) | IoU\( _V\) \( \uparrow\) | IoU\( _B\) \( \uparrow\) |
| Point Only | Center | 0.3302 | 0.1905 | 0.2513 |
| Edge | 0.4190 | 0.1342 | 0.1697 | |
| Offset | 0.3476 | 0.1898 | 0.2455 | |
| Cluster | 0.3803 | 0.1632 | 0.1905 | |
| Point + Box | Center | 0.3118 | 0.2074 | 0.2728 |
| Edge | 0.3178 | 0.1996 | 0.2703 | |
| Offset | 0.3064 | 0.2029 | 0.2805 | |
| Cluster | 0.3036 | 0.2062 | 0.2793 | |
| Point + Mask | Center | 0.2995 | 0.2030 | 0.2857 |
| Edge | 0.3087 | 0.1962 | 0.2766 | |
| Offset | 0.3011 | 0.2088 | 0.2809 | |
| Cluster | 0.3058 | 0.2042 | 0.2796 | |
| Setting | Condition | CD \( \downarrow\) | IoU\( _V\) \( \uparrow\) | IoU\( _B\) \( \uparrow\) |
| SAM (Random) | Mask | 0.3498 | 0.1796 | 0.2375 |
| SAM (Center) | Mask | 0.3387 | 0.1731 | 0.2244 |
| Direct point (ours) | Point | 0.3334 | 0.1902 | 0.2506 |
| Ground Truth (Random) | Mask | 0.2938 | 0.2049 | 0.2840 |
H Analysis of SAM-based Cascaded Pipelines
A seemingly plausible approach is to first convert sparse inputs (e.g., points) into dense masks using a 2D segmentation model such as Segment Anything (SAM) [33], followed by mask-conditioned 3D generation. We analyze this pipeline to understand whether such a decomposition can replace direct modeling of sparse conditions.
As shown in Tab. 14, generation conditioned on SAM-derived masks performs worse than generation conditioned on ground-truth masks under both evaluated point sampling strategies. Even with center-based point sampling, CD degrades from 0.2938 to 0.3387, and IoU\( _V\) drops from 0.2049 to 0.1731. Direct point conditioning achieves Part CD 0.3334, IoU\( _V\) 0.1902, and IoU\( _B\) 0.2506, outperforming both SAM-derived mask settings under the same evaluation protocol.
This performance gap may reflect a mismatch between image segmentation and the intended 3D part decomposition. As visualized in Fig. 7, given a sparse prompt, SAM typically segments a localized fragment rather than the complete functional part intended by the user. Incomplete intermediate masks can propagate segmentation errors into 3D generation.
In contrast, our end-to-end framework avoids this by directly modeling heterogeneous conditions. This joint modeling enables cross-modal synergy, allowing the network to learn robust semantics directly from sparse prompts. Furthermore, it naturally scales to structured 3D conditions like 3D bounding boxes, which cannot be elegantly reduced to 2D masks.
I Additional Qualitative Results
To further substantiate the versatility of FlexPart, we provide extended qualitative results in Fig. 8 across a spectrum of geometric conditions. In addition, we present a new set of visualizations in Fig. 9 focusing on objects with highly complex topological structures. By directly leveraging image conditioning alongside user-provided point or box prompts, our method successfully synthesizes high-quality, part-level 3D objects that preserve intricate geometric details.
These combined results highlight the core advantage of our unified framework, which empowers users to flexibly choose the most suitable prompt type for their specific needs. Regardless of the input modality, from lightweight points to dense masks, FlexPart consistently produces high-fidelity meshes with clean topology and precise alignment to the provided spatial layouts, demonstrating robust generalization across diverse interaction modes.
References
[1] Clay: A controllable large-scale generative model for creating high-quality 3d assets ACM Transactions ON Graphics 2024 43 4 1–20
[2] Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer Proc. of Advances in NeuIEEE Robotics and Automation Letters Information Processing Systems 2024 37 121859–121881
[3] Triposg: High-fidelity 3d shape synthesis using large-scale rectified flow models IEEE Transactions on Pattern Analysis and Machine Intelligence 2025 1-18
[4] Structured 3d latents for scalable and versatile 3d generation Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition 2025 21469–21480
[5] Sam3d: Segment anything in 3d scenes Proc. of IEEE Intl. Conf. on Computer Vision_W 2023
[6] Partgen: Part-level 3d generation and reconstruction with multi-view diffusion models Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition 2025 5881–5892
[7] Omnipart: Part-aware 3d generation with semantic decoupling and structural cohesion ACM ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques Conference on Computer Graphics and Interactive Techniques in Asia 2025 1–12
[8] From one to more: Contextual part latents for 3d generation Proc. of IEEE Intl. Conf. on Computer Vision 2025 8230–8240
[9] FullPart: Generating each 3D Part at Full Resolution Proc. of Intl. Conf. on Learning Representations 2026
[10] PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers Proc. of Advances in NeuIEEE Robotics and Automation Letters Information Processing Systems 2025
[11] Efficient Part-level 3D Object Generation via Dual Volume Packing Proc. of Advances in NeuIEEE Robotics and Automation Letters Information Processing Systems 2025
[12] Dreamfusion: Text-to-3d using 2d diffusion Proc. of Intl. Conf. on Learning Representations 2023
[13] Magic3d: High-resolution text-to-3d content creation Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition 2023 300–309
[14] Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation Proc. of Advances in NeuIEEE Robotics and Automation Letters Information Processing Systems 2023 36 8406–8441
[15] Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion Proc. of European Conference on Computer Vision 2024 439–457
[16] Lrm: Large reconstruction model for single image to 3d Proc. of Intl. Conf. on Learning Representations 2024
[17] Lgm: Large multi-view gaussian model for high-resolution 3d content creation Proc. of European Conference on Computer Vision 2024 1–18
[18] 3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models ACM Transactions ON Graphics 2023 42 4 1–16
[19] Dora: Sampling and benchmarking for 3d shape variational auto-encoders Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition 2025 16251–16261
[20] Hunyuan3d 1.0: A unified framework for text-to-3d and image-to-3d generation arXiv preprint arXiv:2411.02293 2024
[21] Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation arXiv preprint arXiv:2501.12202 2025
[22] ROAD arXiv preprint arXiv:2607.28581 2026
[23] HERMES Proceedings of the IEEE/CVF International Conference on Computer Vision 2025
[24] HERMES++ arXiv preprint arXiv:2604.28196 2026
[25] PointMamba Advances in Neural Information Processing Systems 2024 37
[26] Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024
[27] Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning IEEE Transactions on Pattern Analysis and Machine Intelligence 2025
[28] PointTPA Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2026
[29] Shapellm: Universal 3d object understanding for embodied interaction Proc. of European Conference on Computer Vision 2024 214–238
[30] Kestrel: 3D Multimodal LLM for Part-Aware Grounded Description Proc. of IEEE Intl. Conf. on Computer Vision 2025 8973–8983
[31] Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition 2019 909–918
[32] Pointnet: Deep learning on point sets for 3d classification and segmentation Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition 2017 652–660
[33] Segment anything Proc. of IEEE Intl. Conf. on Computer Vision 2023 4015–4026
[34] Grounded language-image pre-training Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition 2022 10965–10975
[35] Satr: Zero-shot semantic segmentation of 3d shapes Proc. of IEEE Intl. Conf. on Computer Vision 2023 15166–15179
[36] Partstad: 2d-to-3d part segmentation task adaptation Proc. of European Conference on Computer Vision 2024 422–439
[37] Part123: part-aware 3d reconstruction from a single-view image ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques 2024 1–12
[38] 3\( \times\) 2: 3d object part segmentation by 2d semantic correspondences Proc. of European Conference on Computer Vision 2024 149–166
[39] Meshsegmenter: Zero-shot mesh semantic segmentation via texture synthesis Proc. of European Conference on Computer Vision 2024 182–199
[40] Sampart3d: Segment any part in 3d objects arXiv preprint arXiv:2411.07184 2024
[41] Partfield: Learning 3d feature fields for part segmentation and beyond Proc. of IEEE Intl. Conf. on Computer Vision 2025 9704–9715
[42] P3-sam: Native 3d part segmentation arXiv preprint arXiv:2509.06784 2025
[43] Holopart: Generative 3d part amodal segmentation Proc. of Intl. Conf. on Learning Representations 2025
[44] Autopartgen: Autogressive 3d part generation and discovery Proc. of Advances in NeuIEEE Robotics and Automation Letters Information Processing Systems 2025
[45] Meshcoder: Llm-powered structured mesh code generation from point clouds Proc. of Advances in NeuIEEE Robotics and Automation Letters Information Processing Systems 2025
[46] When Numbers Speak: Aligning Textual Numerals and Visual Instances in Text-to-Video Diffusion Models Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2026
[47] DINOv2: Learning Robust Visual Features without Supervision arXiv preprint arXiv:2304.07193 2023
[48] Superglue: Learning feature matching with graph neural networks Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition 2020 4938–4947
[49] Classifier-free diffusion guidance arXiv preprint arXiv:2207.12598 2022
[50] Objaverse: A universe of annotated 3d objects Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition 2023 13142–13153
[51] Vincie: Unlocking in-context image editing from video Proc. of Intl. Conf. on Learning Representations 2026
[52] Composer: Creative and controllable image synthesis with composable conditions Proc. of Intl. Conf. on Machine Learning 2023
[53] Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition 2023 1179–1189