That´s why many 737 pilots find flying with the FD on tedious - you have to keep commanding "deselect SPD mode" to the pilot monitoring. Every time you change from FL CHG to ALT AQC, then again at ALT HOLD, when changing to V/S or FL/CHG or G/S, and so on...
I think some airlines allow, or even encourage flying manually with the autothrottle in SPD mode - which I think is bad practice, because it leads to uncontrolled pitching because you never know when the A/T adds or removes thrust...
Jan