X and Y are +-10V signals from your cycloid generator of choice. AM, usually a ramp, is a +-10V control voltage signal that in this case goes to the AD633 four quadrant multipliers. Inverting the AM signal to either the X axis or the Y axis results in the "diamond" effect. I looked at your video and yes, that is obviously what you are already doing. Sweep is the the AM signal summed with one axis. In the circuit I posted, it is better to take the AM signal before it is summed with the DC offset voltage. The direction effect simply selects X or Y axis to apply the sweep to.
A caution: According to my notes, the circuits I posted are what I built, which works great. I think my original notes are correct, but I'm not guaranteeing it.