Base damage (aka unstyled damage) is not affected by swing speed, only the weapon speed itself matters, and hence increasing your swing speed via haste or quickness effectively increases your dps. The only thing that gets reduced / is affected by the actual swing speed is the style damage bonus.