So I decided to break off a section of my 4GB thumb drive, make it an encrypted drive, and store the keys there as well. Following are the steps I used to do just that.
So I did a Google search and found this page, which got me through most of what I needed to do. But a few steps that didn't work included naming the partition itself.
On the page the author recommends doing:
/sbin/e2label /dev/mapper/cryptmap “Brad’s Files”
However, for me, this failed consistently (on my system the mapper is /dev/mapper/cryptmcp). What I had to do instead was refer to the partition by its UUID instead:
/sbin/e2label /dev/mapper/udisks-luks-uuid-ec00b89c-8a65-4fa5-8a9d-de0b7ecc5efa-uid500 "McPierceSecure"
Disconnecting and reconnecting the drive, entering my password and now the device is mounted with the correct label.
Accessing The Drive As A User
When that was all done, I found I wasn't able to write to either partition as my regular user. To fix that, just run the command:
sudo chown mcpierce:mcpierce /media/McPierceSecure
Verify by unmounting, removing then re-inserting the drive.