How do I modify the code of WordPress, a theme or an extension?

Your site needs a feature that is not offered by WordPress, your theme or a specific extension. The question then becomes how do you modify...