What I use is a myriad of programs.
Heres my list
CPU-Z
RealTemp
CoreTemp
CrystalCPUID (Might not work for some)
HCI Memtest
Memtest86
Prime95 x32
Prime 95 x64 (Use if you got more memory than 4GB I suppose)
What I do is first get my best CPU overclock. I usually look around the net to see what speeds other people are achieving and at what voltage. Gives me a rough idea. If they reveal their VID of their CPU even more helpful. Although
Intel's Spec page tells you the max temps and max VCore the VID is different for each CPU. It's just the voltage required for the CPU to boot and run at stock. The lower the better because that means more room for overclocking within safe operating temperatures. You can check your VID by using CoreTemp or RealTemp.
So what I do is try a setting and then I also lower my ram speed to make sure it doesn't get overclocked. If you overclock two things it's way harder to find out what is causing the errors so 1 step at a time. So my ram is running under stock speeds so there's very little chance it'll be causing the errors. Now I've got my CPU overclocked I run Prime95 for around 30 minutes. If it passes I increase the frequency and run another short Prime95. If it fails but my VCore is low I'll try increasing that (within limits though) first to see if that was causing the problem. I keep on doing that until I reach my best overclock, using CoreTemp/RealTemp and CPU-Z to watch the temperatures and the frequency. Usually CPU-Z will report a lower VCore than what you set in bios. It's usually like that, but not too far off. If you have a setting called Load Line Calibration turn that on in your bios so it helps keep your voltage from being affected by something called VDroop. Once the 30 min test passes and I've got a good overclock I do a 6h test myself. Others do 24h, 8h, 12h etc. Once you've done either one of those then you know your CPU is going to be stable.
Once all this is done and you've got your nice CPU overclock you then set your ram back to a normal divider, unless it's going over it's rated speed then you'll have to test if it can run at those speeds. Usually if you're increasing the speeds on DDR2 you might have to loosen the timings (the 4-4-4-12 and 5-5-5-15/18 things. Remember the last number usually has to be the sum of the first 3 or more (ie 5+5+5=15, 4+4+4=12. 18 one is just there because it's a common timing too). You might be lucky if your ram can do lower. When overclocking the ram what I do is use the HCI memtest program to check the memory in Windows. You can open multiples and I think the max ram it can check is 1.5GB. So anyway I use those to check if my memory has errors first as it's a bit faster than memtest86. I let it check 300% over and then I proceed to boot memtest86 on my flash drive and do a thorough overnight test.
Now the CrystalCPUID is used on more advanced high overclocks. Sometimes when you go into high FSB's you've got to adjust things like NB voltage, FSB voltage, GTL's etc. When I overclocked my Q9550 to 4GHz I had to do things like that. What CrystalCPUID can do is basically lock your multiplier in Windows. It's a lazy tool for me I suppose seeing as I could easily do it in bios
But what I do is lock the multiplier to the lowest. This makes sure the overclock isn't unstable (the lowest multiplier is set but the vcore is for my high overclock). I then run the Prime95 for 1h or 2 to test if the FSB voltage is set okay. So I know if the overclock fails during the 6h test it must be the VCore that isn't high enough.
Long post I know but it might help.
Now onto your problem. Remember upping the multiplier is always better than upping the FSB because more frequency = more heat. Yes you still get heat from the overclock but low freq + high multi gives less heat than high freq + lower multi.
Have you set your voltages in bios? Ie for the CPU, Ram, NB, SB. If so what are they. Remember to never put anything on Auto and overclock. Fine tuning is much better than the mobo's tuning.