none of those are correct. the answer would be [tex]3 x^{2} -24x+36=180[/tex].
you can draw a picture of the garden space, where x is the width of the space, 3x is the length. to account for the 3 inch border you need to subtract 3 from both ends, so the width of his garden is then x-6, and the length is 3x-6.
multiplying these together, which is the formula for area, gets you [tex]3 x^{2} -24x+36=180[/tex].