The implementation of modern ERP solutions has introduced tremendous opportunities as well as challenges into the realm of intensely competent businesses. The ERP implementation phase is a very costly and time-consuming process. The failure of the implementation may result in the entire business to fail or to become incompetent. This fact along with the complexity of data streams has led the researchers to develop a hierarchical multi-level predictive solution to automatically predict the implementation success of ERP solution. This study exploits the strength and precision of the Adaptive Neuro-Fuzzy Inference System (ANFIS) for predicting the implementation success of ERP solutions before the initiation of the implementation phase. This capability is obtained by training the ANFIS system with a data set containing a large number of ERP implementation efforts that have led to success, failure, or a mid-range implementation. In the initial section of the paper, a brief review of the recent ERP solutions as well as ANFIS architecture and validation procedure is provided. After that, the major factors of ERP implementation success are deeply studied and extracted from the previous literature. The major influential implementation factors in the businesses are titled as Change Orchestration (CO), Implementation Guide (IG), and Requirements Coverage (RC). The factors represent the major elements that guide the implementation project to a final success or to a possible failure if mismanaged. Accordingly, three initial ANFIS models are designed, trained, and validated for the factors. The models are developed by gathering data from 414 SMEs located in the Islamic Republic of Iran during a three-year period. Each model is capable of identifying the weaknesses and predicting the successful implementation of major factors. After this step, a final ANFIS model is developed that integrates the outputs of three initial ANFIS models into a final fuzzy inference system, which predicts the overall success of the ERP implementation project before the initiation phase. This model provides the opportunity of embedding the previous precious experiences into a unified system that can reduce the heavy burden of implementation failure.