ED2IF2-Net: Learning Disentangled Deformed Implicit Fields and Enhanced Displacement Fields from Single Images Using Pyramid Vision Transformer

There has emerged substantial research in addressing single-view 3D reconstruction and the majority of state-of-the-art implicit methods employ CNNs as backbone network. On other hand, transformers have shown remarkable performance many vision tasks. However, it is still unknown whether are suitable for reconstruction. In this paper, we propose first end-to-end network based on Pyramid Vision Transformer (PVT), called ED2IF2-Net, which disentangles an field into topological structures recovery surface details to achieve high-fidelity shape ED2IF2-Net uses a encoder extract multi-scale hierarchical local features global vector input single image, fed three separate decoders. A coarse decoder reconstructs vector, deformation iteratively refines using pixel-aligned obtain deformed through multiple blocks (IFDBs), detail predicts enhanced displacement with hybrid attention modules (HAMs). The final output fusion field, four loss terms applied reconstruct structure novel loss, overall after fusion, via Laplacian loss. quantitative results obtained from ShapeNet dataset validate exceptional ED2IF2-Net. Notably, ED2IF2-Net-L stands out top-performing variant, exhibiting highest mean IoU, CD, EMD, ECD-3D, ECD-2D scores, reaching impressive values 61.1, 7.26, 2.51, 6.08, 1.84, respectively. extensive experimental evaluations consistently demonstrate capabilities reconstructing recovering details, all while maintaining competitive inference time.