If your mistake is larger or happened several rows back, the quickest way to remedy the situation is to take the work off the needles entirely and unravel it to a point before the mistake occured.
Knitters love having cute names for things, and this method is called "frogging" because you just "rip it" out. (I don't make these things up.)
Steps for Frogging
Note where your mistake is and where you'll want to stop ripping. Remove the project from the needle and set it flat on a table with good light so you can see what you're doing.
Gently pull on the yarn and slowly allow the work to unravel. If you pull too hard or quickly, the fibers can get a little agitated, making it difficult to identify all the individual loops you need to put back on the needle.
Once you've ripped to a point past your mistake (I usually rip out one row beyond the messed up row for good measure), carefully thread the loops back on the needle in the right order, making sure you end up with the same number of stitches you started with.
Remember, your working yarn should be at the top of the needle when you start knitting again, so thread the stitches starting with the opposite end, regardless of which side was up when you started ripping.
Make sure you know where you are supposed to be in your pattern and start again from there.
Fixing mistakes can be a little nerve-wracking, but if you can notice them and fix them before you finish your project you will be a lot happier with the results.