There's a whole body of theory on Adaptive Control (what the control systems community has called what we would now called "learning based control").

This is one of the classic texts: https://www.amazon.com/Adaptive-Control-Karl-Johan-Astrom/dp...

In short, many techniques that look a lot like modern machine learning have been in use for decades in control theory for tasks exactly like you describe (e.g. automatic tuning of PIDs). If you want to dig into this a bit deeper you might look up the "MIT Rule", which is a gradient style approach for dynamically tuning (and re-tuning) adaptive control systems, and is fairly common for PID tuning.

This is a bit of a slog to read, but it's not bad: http://www.amazon.com/Adaptive-Control-Edition-Johan-Astrom/...