Matrix calculus is the extension of calculus to vector or matrix setting. I used to suffer a lot with matrix calculus in my early grad life. I’ve seen some other people to suffer as well. It primarily happens because the standard courses in linear algebra do not cover this topic very often. However, all the maths in Matrix calculus are basically trivial. Anyway, here are a few interesting facts that made my life much easier. I could use these to derive many expressions (e.g. calculating gradients in SGD based algorithms etc.) in my machine learning practice. Check if they can help you or not. …
