This is the first time I've seen a capacitor detach itself like that, with the leads still intact. The factory either forgot to solder it in place, or it was done very poorly. You're right, it does go in the area you indicated. The stripe on the capacitor (negative terminal) should line up with the shaded portion of the circuit board. If the other components don't show any signs of damage, you can solder the capacitor back and it might work, but consider that a temporary fix until you get a better PSU. The burning smell is probably just the transformer heating up. The rapid clicking and power cycling is the overload protection, better PSUs will shut off completely instead of attempt to turn back on again.