Cameron,
Everyone has brought up good points in the discussion so far. If you like, here's my $0.02:
With the Mac, you get the best of all worlds. You'll get the OS X, which is not perfect but IMHO is still ahead of Vista and XP. I've owned an iBook G4 for two years now and I've never had an OS issue. It just works.
OS X is based on Linux, so pretty much anything that can be run under Unix or Linux will run on a Mac. I run Gempak and Grads on my iBook, and even though its G4 processor is a few generations behind, the programs perform beautifully.
The solutions from Boot Camp, Parallels, and VMWare are what really make the new Macs appealing, IMO. If you don't like OS X or Linux, or if you are more comfortable in Windows, or if you use programs that aren't available for the Mac (GR3, et al), the dual boot/virtual machine can give you the Windows OS as well.
In short, even if you don't use all of its capabilities, the Mac can easily give you essentially three OS's on one machine. Whether that's enough to offset the additional cost of a Mac vs. PC (plus the expense of getting a copy of Windows) is a decision that's ultimately up to you.
If you do opt for the Mac, you might want to wait until the new version of OS X (Leopard) is out and has the bugs worked out.
Hope this helps, and good luck!