The preferred power source in the Star Trek era seems to be fusion, with M/AM reactors being reserved for really high powered applications, like warp drive.
So, while everything else on board can function just fine on the fusion reactors, you need that big sucker to operate the warp drive, at least at any level that might be considered practical. It is, essentially, the core
of the enitre warp drive system, hence "warp core".
As for the dilithium crystals, the high energy plasma that's produced by the matter/antimatter reaction is passed through the dilithium crystals, which focuses and intensifies the plasma stream, raising it up to the level necessary to generate a stable warp field. During TOS' time, the dilithium crystal assembly was separate from the reactor, whereas by TNG's time (maybe even as early as TUC), the crystals are now inside the reactor itself.