Maokai is a great initiator and has good tower diving/chasing capability thanks to Twisted Advance. His Sapling Toss is also able to chase retreating enemies past their towers and possibly kill them.
Maokai can farm using Sapling Toss, as mid-game the sapling can kill 3-5 minions easily. The Sapling deals damage twice: once upon impact and another after a short delay. It can also be used to damage enemies in brush safely.
Use Sapling Toss to harass enemies in the lane. Try to overshoot your target slightly, just enough to get the impact damage and allowing the enemy to dodge into the explosion.
Try to aim your saplings so there are no enemy minions nearby to trigger them, as they may choose to explode on them rather than the desired champion. You can also use it to zone out enemies by placing them behind the enemy minions, forcing the enemy to duck away from experience range or take damage.
Champions with Boots of Speed can outrun the sapling quite easily, making this form of lane-control mainly effective at the very start of a game.
Saplings won't detonate on stealthed enemies, but can hit them. This is very useful if the enemy tries to flee through both minions and stealth.
By using Sapling Toss multiple times in a bush, stacking them on top of each other, you can catch unsuspecting enemies off guard resulting in a huge burst of damage if they walk into your saplings.
Vengeful Maelstrom is not a channeled spell, so Maokai is free to use other skills while supporting his teammates. Remember to deactivate right before all enemies leave the AoE, or else they will suffer no damage.
Vengeful Maelstrom is very useful when the battle ground is stationary, such as during a tower push or defense. Using it in the open will usually cause enemies to run out of it, wasting its potential unless you want them to go away. During a push, however, the designated fighting grounds remains around the tower, which forces the enemy team to fight on your terms.
Small skirmishes in the jungle or even the occasional team fight are good times to use Vengeful Maelstrom, as escape routes are few and the fighting ground is often crowded and fast paced.
The damage output from Vengeful Maelstrom is very powerful with some ability power. Using it in team fights to hit as many enemies as possible can turn the tide greatly in your teams favor.
Vengeful Maelstrom is good for small skirmishes in laning as well. If you aren't sure you can kill your enemy with the normal E-W-Q combo, throw down Vengeful Maelstrom to add that extra bit of damage you may need.
Use Vengeful Maelstrom while tanking minion waves to quickly kill minions and shut down the enemy's push.
Vengeful Maelstrom can be very useful for a kamikaze operation. A fully built Maokai can easily store up the 300 bonus damage when 1 v 4 thanks to Sap Magic. In addition to free hits while they try to kill you, the resulting nuke from Vengeful Maelstrom can often get an unexpected kill.
Place it over enemies and yourself should you expect the enemy to focus on you, reducing your damage taken while possibly let it cause more to enemies. Place it over allies if you expect enemies to aggressively attack them instead.
Enemies running from Vengeful Maelstrom isn't necessarily a problem, as long as you are damaging them while they flee from it. With Arcane Smash, Twisted Advance and even other allied crowd-control effects, the enemy fleeing Vengeful Maelstrom can allow your team to do a lot of damage upon them without receiving reprisal before they escape the circle.
Maokai's typical initiation is using Sapling Toss, rooting them near the sapling with Twisted Advance to ensure the sapling explodes on them, and finishing off with Arcane Smash. Waiting until the root is just about to end before using his knock-up on Arcane Smash increases disable time on them as well. If a team battle is about to start, use Vengeful Maelstrom after you jump in with your Twisted Advance, positioning its location to net as many enemies and allies as possible.
Maokai's damage output early game is very high being able to bring a squishy enemy to about 1/3 of their health using a few well aimed Sapling Tosses. When they get low enough use the above combo, you can easily get a kill.
Maokai can have a support role similar to Galio, being able to deal decent damage and still protect allies, or Malphite, being able to disrupt enemy positions and set-up kills for his allies while still remaining relatively tanky.
a) Due to Maokai's reliance on mana in the jungle and for a safer jungle path, begin at Blue buff. Do Blue -> Red(Smite) -> Gank(optional, take Twisted Advance at level 3 if ganking) -> Wolves -> ... If you can get a big leash at blue you can save smite and go straight for Red, in order to perform a faster gank
b) You can also start red in order to perform a level 2 gank. Red -> Gank -> Recall -> Blue -> Wolves -> ... In this case, take Twisted Advance at level 2
Health items synergize well with Sap Magic and improve his sustainability.
Frozen Heart is an excellent item for Maokai, giving him armor for survivability, cooldown reduction to allow him to do more damage and control fights better with his abilities, mana to allow him to maintain Vengeful Maelstrom longer, and an excellent passive to reduce the damage of enemy autoattackers.
If using a Guardian Angel, it is possible to save your Sap Magic until after you have resurrected as the stacks stay with you through death. Doing this could save your life if a champion decides to wait around until you've revived.
Rylai's Crystal Scepter provides great synergy with Sapling Toss, as it applies the slow with its first hit and then again after it chases down and explodes near an opponent. The health will also benefit his tanking.
Ohmwrecker can be used to enable Maokai to tower dive more aggressively, and save himself significant damage in situations where he is dragged under tower by someone using an escape during Twisted Advance.
If your team is already very tanky and are in the need of damage, you can actually build ability power similar to an actual mage. Keep in mind however that you still have to get very close to an opponent, and as such you should try to remain hard to kill.
Part of Champion ability to tank comes from him being able to consistently heal through Sap Magic. If you can spare it, try using Executioner's Calling or another healing-reduction ability to minimize its effect.
When facing a laning Maokai, the most reliable way to avoid Sapling Toss is to stand near your minions, then run when he throws the sapling. This way, the minions draw aggro instead of you, allowing you to reliably get out of the explosion radius.
Avoid the bushes if possible. Not only is bush-hugging ineffective against Sapling TossMaokai, it can actually make landing the explosion damage easier.