Preheat oven to 350 degrees F (175 degrees C).

Fill a pot with lightly-salted water and bring to a boil. Stir the macaroni into the water and return to a boil. Cook and occasionally stir until the pasta has cooked through, but is still firm to the bite, about 8 minutes; drain.

Melt the butter in a large pot over medium heat. Add the flour to the melted butter and whisk continually for 1 to 2 minutes to make a roux. Whisk 2 cups of the milk into the roux until smooth and then add the remaining milk. Raise the heat to medium-high and continue cooking and stirring until thickened, but not boiling.

Stir the Gouda, Cheddar, Swiss, and 1 cup Parmesan cheese into the mixture; once the cheeses have melted completely, fold the drained macaroni into the mixture to coat. Pour the macaroni mixture into a 9x13-inch baking dish.

Stir 1/2 cup Parmesan cheese, bread crumbs, and cayenne pepper together in a small bowl; sprinkle over the macaroni. Drizzle the olive oil over the bread crumbs.