In a nutshell, yes it is heat.
There is the light speed barrier too, as it takes time for a signal to travel across the chip. And when we're dealing with GHz, light speed is significant.
To put that into perspective, 3GHz means that with each clock tick, light can travel a maximum of 10cm distance. Electricity is slower than light. So given that your computer's ram is about 10cm from the processor, you know it can't load data from main memory in one clock tick. Ye canna change the laws o' physics.
So miniaturization is necessary to make progress. That means finer conductors, which generate more heat for their size.
Those fine silicon semiconductors will burn out if they're pulsed with current too frequently. This is why core voltages keep dropping to try and counteract heating, 5v --> 3.3v --> 1.3v. There's a limit how low that can go though.
Superconducting semiconductors will revitalise the clock speed race if we ever get them manageable on the domestic scale.