Pergola with wind protection side panels