Don't sweat it, building my castle is one of those things I just gradually get the materials for while I'm out mining.
I figured up I need something like 200 stacks of cobblestone just to complete the outer wall so it's not like I'm in a hurry.
I went exploring and found some surface-accessible caves. One of them goes straight into a dungeon with a zombie spawner. Wicked. That's a prime candidate for turning into a trap, methinks. I just wish you could move spawners around. I'd love to get a bunch of them and put them in a trap room and have myself a nice little loot factory.