Does my Steam profile need to be public to use the site?

No.

You'll need to switch to a public profile when registering

Right.

if you wish to enter giveaways, the site will ask you to switch to a public Steam profile once a week to bring your account up-to-date

Right.

Are not private members violating rules?

No.
As you said, they have to switch to a public profile at least once a week.

Of course you could and should ask us to check if they activated every game they won.
Indeed the winner didn't activate a game, so I've suspended him.

Reroll approved!