Exploiting deep learning and augmented reality in fused deposition modeling: a focus on registration

Abstract The current study aimed to propose a Deep Learning (DL) based framework retrieve in real-time the position and rotation of an object need maintenance from live video frames only. For testing positioning performances, we focused on intervention generic Fused Deposition Modeling (FDM) 3D printer maintenance. Lastly, demonstrate possible Augmented Reality (AR) application that can be built top this, discussed specific case using Prusa i3 MKS FDM printer. This method was developed You Only Look Once (YOLOv3) network for detection locate subsequent Rotation Convolutional Neural Network (RotationCNN), trained dataset artificial images, predict rotations’ parameters attaching model. To train YOLOv3 used augmented 1653 real while RotationCNN utilized 99.220 synthetic showing Printer with different orientations, fine-tuned it 235 images tagged manually. obtained AP (Average Precision) 100% Intersection Over Unit parameter 0.5, showed mean Geodesic Distance 0.250 (σ = 0.210) accuracy detect correct r 0.619 0.130), considering as acceptable range [ − 10, + 10]. We then evaluate CAD system performances 10 non-expert users: average speed improved 9.61 1.53) 5.30 1.30) number actions complete task 12.60 2.15) 11.00 0.89). work is further step through adoption DL AR assistance domain. In future works, will overcome limitations this approach develop mobile could extended any presents counterpart