Khandoba Temple holds special importance not only for its ancient origin, but also for being associated with Sai Baba. It is located on the main road in Shirdi. Baba came to Shirdi for the frst time with the marriage procession of Chand Patil. When the marriage party halted at the entrance gate of Khandoba Temple, the man priest Mhalsapati welcomed Baba saying "Aao Sai" (come in Sai). Since then, Baba, whose original name was unknown to all, came to be known as Sai Baba. Baba liked the peace and holiness of the temple so much that he decided to live there. But Mhalsapati assumed him to be a Muslim and asked him to leave the temple. In a very dignified manner, Baba left the temple and made an abandoned mosque (Dwarkamai) his home.

Years later, Sai Baba asked his prominent disciple and devotee, Shri Upasani Maharaj, to stay at this temple for four years for his spiritual upliftment. In the temple of Khandoba, there are icons of Khandoba and his divine consorts Banai and Mhalsai.