It may only be a rumor, but I thought the producers bumped Riley off because they had a Scotsman on board.
This is silly. Riley was a one-off character who only ended up making a second appearance because the studio owed Bruce Hyde another part. His character in "The Conscience of the King" wasn't even scripted to be Riley - that was a last minute name change that was done when he was cast.