You will meet much Roman and old French history on this excursion, which also can be combined with a small hike around Pont-du-Gard, or a long walk in les Alpilles (at the expense of Avignon, for example). The classic walk in les Alpilles goes between St.-Remy-de-Provence and les Baux-de-Provence.
(This mountain range could also be closed for summer hikers between 1.7 and 2nd Saturday in September due to the risk of forest fires.)
Be also warned that les Baux is one of France's most visited tourist spots! Try very early in the morning. But the most interesting for many visitors is not the village of les Baux itself, but the Cathederal des Images just a few hundred metres further along. This is a gigantic artistic slide show with sound inside an old quarry. Different subjects each year.
It would also be a shame to miss the ruins of the Roman city of Glanum, just outside St.-Remy (southern side).