Why Does Closing Windows on a Mac not Quit the Application?

When switching from Windows to Mac many people wonder why Mac apps don't quit when you click the red close button at the top of the window. Here's how macOS and Windows differ when it comes to closing windows, plus lots of tips and tricks how to handle windows and apps.