To use in-game chat in those few Wii U games which support the feature, you'll need a headset plugged into the Game Pad - even when you're not using the controller.
Nintendo told Kotaku that "select games" including Assassin's Creed 3, Call of Duty: Black Ops 2 and Mass Effect 3 will support in-game chat.
Rather than utilising the Game Pad's built-in mic, chat requires a licensed stereo headset from Turtle Beach or Mad Catz plugged into the Game Pad; there's no jack on the Pro Controller which is likely to be the input method of choice for serious multiplayer fans.
Nintendo has given no indication of a wireless headset option any time soon, and has not implemented universal support for in-game chat.
The Wii U launches worldwide in November and December; get full launch details here.