Continue in this manner, sewing pieces together in a way that makes logical sense for you, until you have a couple of units sewn together that then need to be sewn to each other.
It's easier to do this if you're sewing a relatively straight edge to a more jagged edge, because you can use the straight edge to keep your sewing straight and be assured that you're always sewing through all the layers.
At this point if you've been sewing by machine, you'll have a lot of seams all over the back of your work. Because of this, you could be stitching through four or more layers of knitting in some places as you sew your units together. If you don't have a pretty heavy-duty sewing machine, this might not be possible, so you can also sew these seams by hand.
Either way, take your time and make sure you're stitching through all the layers and making your seams as straight as possible. This project is meant to have a scrappy look, however, so don't worry too much about perfection unless you can't help it.
Keep working in this way, adding units to units, until you're out of swatches or your blanket is the size you desire.
You can hide the seams on the back by sewing a piece of fabric to the back of the blanket, or leave it rustic. If you have big pieces of knitting hanging off the back where you squared off two pieces, if you machine sewed firmly you can actually cut away the excess and the seam will keep it from unraveling further.
