Gaya is one of the seven sacred cities of Hinduism and most people only pass through it on their way to Bodh Gaya. This is understandable because it is also one of the dirtiest cities in India. However the Buddha stayed in Gaya for some time and preached his third discourse here. A steep hill on the southern edge of the town, now called Brahmayoni, was known in ancient times as Gayasisa.
This is where the Buddha taught his Fire Sermon notable for its dramatic imagery and parts of which T. S Elliot included in his poem The Wasteland. A good translation of the Fire Sermon is to be found in Three Cardinal Discourses of the Buddha, Wheel No 17. If you take the old road to Bodh Gaya which follows the river you will see Gayasisa on the right. Climb the stairs and after passing through the gateway half way up, follow the rough path to the left. The large area of exposed rock is probably where the audience sat while the Buddha delivered his sermon. Another interesting place in Gaya is Tankitamancha, now a large tank called Sujikund, where the Buddha had his well known encounter and dialogue with the malevolent spirit Suchiloma; (see Sutta Nipata, discourse No 5). Sujikund is surrounded by a wall made of huge blocks of stone and you will notice a few stupas and Buddhist statues near the ghat at the northern end. It is a quiet, rather haunting place and worth a visit. Sujikund is near the Vishnupada Temple. After that consider walking down to the river just behind Vishnupada. You can watch Hindus doing there ablMaputions, burning their dead, performing puja, defecating, etc.
The only other place of interest to the Buddhist is the museum which has a modest collection of Buddhist and Hindu sculpture from in and around Gaya. The museum is next to the Dak Bungalow just near Gaya Court and is open from 10am to 5pm each day and closed on Monday. When you arrive in Gaya I recommend you take a bus or three wheeler straight to Bodh Gaya and come back to visit Gaya as a day trip from there. If you take the early bus from Bodh Gaya you can climb Gayasisa while it is still cool, see the other places and return to Bodh Gaya in the afternoon. If your train arrives late in Gaya and you have to stay over night there are several hotels just across from the railway station.