These remarkable and interesting rock-cut shrines and monasteries are situated in the Narmada valley among the southern slopes of the Vindhya hill in Kukshi Tahsil of Dhar District. These are 82 km by road from the nearest railway station Meghnagar. Buses ply on the road to reach the caves. From Dhar these are about 100 km to south-west and from Kukshi about 18 km to the north.
These caves belong to the Buddhist faith and it is not certain as to how and when these caves began to be called Bagh Gupha. In modern times these caves were first discovered in 1818. It seems that after extinction of Buddhism in Central India by 10th Century A.D. these caves remained effaced from human memory. During the intervening centuries the caves often became the abode of Tigers (Bagh) and this association of tigers with the caves gave them the present name.The caves were excavated in a small hill in front of which runs a small river called Bagheshwari. Of the nine caves except four (no. II to V) all have suffered severely at the hand of time. Of the four better preserved caves no. II locally called Gonsai Gupha or Pandavon ki Gupha is most elaborate in plan. This rock-temple has a large monastic hall at the centre. An imposing six pillared portico adorned its front. A large recessed cell enshrining a stupa stands at the back side in the centre which gave monks much needed privacy and atmosphere for sacred meditation. The central hall having support of massive pillars decorated with spiral fluting served the purpose of prayer hall. Here the Buddhist monks assembled for prayers and religious discourses. The cave was a combination of a chaitya and vihara.
The other caves were more or less identical in plan except certain minor modifications as visible in cave no. III which is purely a vihara or monastery locally called ‘Hathi Khana’. It was also very elaborately carved and designed.The fourth cave is the largest one of the series. The local people have given the name Rangmahal to it. In many respect it is the most remarkable cave. It has three entrances and two windows. The main door is most attractive and well-finished. The roof of the hall has the support of twenty-eight pillars which are square at the base and change to octagons and then polygons again change to octagons at the summit. Cave V was evidently neither a chaitya nor a vihara. It was by its shape and arrangement a lecture-hall locally called Pathashala. It has no aisles, cells, stupa or image and is completely plain devoid of any ornament.
The paintings at Bagh caves are almost contemporaneous with those at Ajanta and the two represent a school which exerted a far reaching influence of on the art not of India alone and on her colonies in the south-east Asia and China but also on every country into which Buddhism presented. It has been said that these paintings can very favourably compare with the best Europe could produce down to the time of Michael Angelo.
An inscription of Subandhu, the king of Mahishmati was picked up at Bagh caves. It is known from this epigraphic record that the name of this vihar was Kalyan, built by one Dattrakat. Maharaja Subandhu donated the villages Dasilak Patti for the upkeep of vihar and the maintenance of its inmates, the Buddhist monks. Order was issued by the same king to the officials of the village to co-operate with the monks without putting any obstacle to their work.