Simple question, but still - any help would be appreciated!

In a simple dynamic programming problem, is it wrong to take derivative of one control wrt another control?

That question is impossible to answer in generality. There may for example be outer derivatives where something like this occurs.