Recycling yarn from old sweaters is a great way to get some new to you yarn without spending a lot of money. Whether you're starting with a sweater you knit yourself, someone else's handknit cast off or a sweater from the thrift shop, the steps for dismantling the sweater so you can reuse the yarn are basically the same.
The key, however, is that when choosing a sweater, you need one that's been hand knit with seams that are sewn by hand or crocheted together rather than machine serged seams.
Machine knit garments involve making a big piece of knit fabric and cutting it into pieces rather than shaping the knitting to form the pieces of the garment you need. If you try to rip up a sweater like this, you'll end up with a bunch of short lengths of yarn that you won't really want to reuse.
Once you've found your hand knit sweater, gather it and a pair of scissors. Turn the sweater inside out so it's easier to see the seams.

