Not only do we rely on global weather conditions being just right for growing food, but we also rely on the electric grid and oil supply for producing and distributing food. Most preppers start by stockpiling survival foods in their pantries and storm shelters. If you want to survive a complete SHTF situation, you must have a self-sustainable food system in place.
Gardening is far from the perfect off-grid survival food system. It is too dependent on factors like climate, water availability, and soil conditions. You need to have lots of space to produce enough food to be self-sustainable with a garden (a lot can be done with just a small plot of land but, again, there is a learning curve here). There is also the issue of time. While waiting for your crops to mature, they are vulnerable to theft. Even if your garden is just in buckets on your apartment balcony, do it!