From someone who was a first-run viewer of late TNG, DS9, VOY, and ENT, here's what I remember on it.
Someone actually compiled all the ratings into a graph. Cool. A few years ago I saw it all in table form. From the actual numbers I saw, DS9 beat VOY every week except for at least part of VOY's Season 1 and maybe "Equinox" beat "Dogs of War" (DS9's lowest rated episode). DS9's ratings were sometimes harshly affected by holiday weekends (e.g. Memorial Day weekend). Some years they did well, some years they had crappy ratings. It was independent of the quality of the episode (some crappy eps got good ratings, some good eps got crappy ratings). Some eps even had better ratings in summer/reruns than in their debut.
Before people talk about DS9's complexity & serialized nature, that only got established later on. DS9's ratings declined before that. DS9 wasn't known for being serial in like 1993-95.
And people debating DS9's competition- Hercules, Xena, and Earth: Final Conflict
were competition. They were all syndicated shows competing for people's attention in a crowded syndicated market. Babylon 5 (PTEN was a glorified syndication package) too. Star Trek wasn't on cable til late 1998, Stargate was unknown then, being on Showtime Fri nights. You can't think solely in terms of sci-fi competition.
Trek fatigue is a concept being retroactively applied. It became popular during ENT's run and now people try to apply it onto the 1990s. It was not commonly talked about during DS9 & VOY's run.
Now onto what others mentioned.
* Season 1 was pretty bad, not Season 2 so much. All the annoying Bajoran stories are what drove people away, myself included (intermittantly saw Season 2, years later I saw all of it and I wished I was watching it because it was good, gradually watched Season 3 more and became a regular viewer again). "Move Along Home" was actually one of the more interesting episodes considering episodes like "Progress", "The Storyteller", "In the Hands of the Prophets", which drove people away. "Duet" reinforced this too (great episode, but at the time got lost in all the Bajoran navel-gazing).
Then there are the several subpar episodes, a rather empty Q episode, "The Passenger", "The Forsaken", "Dramatis Personae". And there was this whole unknown space on the other side of the wormhole basically left unexplored. Only Season 2 started to explore it. "Captive Pursuit" was an amazing episode at the time which enticed people with the Gamma Quadrant. Season 1 was a load of subpar episodes, bland space station drama (which also hurt Babylon 5 til they started their intrigue & action plots).
This was the most active element I would say that drove people away. Notice after Season 2, Bajoran episodes are fewer and tend to have more action ("Shakaar", "The Darkness and the Light", etc). Season 1 sucked badly. It had nothing you would want to draw you in and everything you didn't want to drive you away.
* The Dominion was cool, mysterious, but yes, not being introduced til the Season 2 finale/Season 3 hurt the show. Star Trek, like Batman, is partially defined by its villains. The Klingons & Romulans were a big draw (not quite villains in TNG but still), then the Borg were huge. All DS9 had early on was the Cardassians, who weren't quite good A-list villains (good B-list ones though).
Odo was one of DS9's most popular characters (him & Quark were the top 2 in 1994), and what I recall at the time, the Founders & Jem'Hadar were considered somewhat cool villains that people were curious about because you knew so little about them. Borg were hive cyborgs with advanced technology from far away. Klingons & Romulans were already generally known about, but the Dominion was mysterious.
* T&A. Seven was memorable, but she didn't join til Season 4. The only T&A before then would have to come from Torres or Kes... or Janeway. Thus, T&A is irrelevant before 1997.
* "That other show". I actually think this might be another big reason. When TNG & DS9 were on for 1.5 years, TNG had all the attention after people tuned out after giving "Emissary" the best syndicated rating ever. Despite people bashing Season 7, people were interested in seeing TNG thru to the end and they had a hyped countdown for the last 10 episodes (Season 7 for a sci-fi show was big back then. Most had only gone 4 or 5 seasons). TNG had a very big stature and its series finale even made the cover of TV Guide and got quite a bit of attention (like many stations showing it in a special time, primetime during the week).
DS9 was on in syndication and as independent stations became network affiliates (of WB, UPN), syndicated shows tended to get almost no advertising outside of the time blocks syndicated shows were on (i.e. you might see a DS9 commercial during Hercules, Xena, Earth: Final Conflict and vice-versa, but not a DS9 commercial during Buffy, Voyager, etc). Voyager, as part of a network, and the flagship show for that network, got a lot of advertisement. DS9's treatment vs. VOY was a reflection of new networks forcing syndication to the margins of the tv schedule rather than any special favoritism.
Has there been any study indicating whether these same downward slopes are more widespread among other shows of the same period, even shows outside science fiction?
Ratings for syndicated shows across the board were declining over the mid-late '90s. They were hit the hardest first and networks started to feel it a little then but not significantly until the '00s. As for any formal study, maybe ask the people who run the Tv by the Numbers website. They of all people would know.
* Part was cable becoming an increasing draw for people (more people getting cable. Cable was rather early in its history in 1993 but by 1999 was well established and becoming a hub of entertainment activity). With so many channels, it can draw a lot of people away, even though each channel's audience is tiny compared to broadcast tv.
* Part was the syndicated market being
oversaturated. There were hordes and hordes of shows. If I remember correctly, the 1996-7 season had the most shows in syndication of any season.
* And part was the rise of UPN & WB. This gobbled up
a lot of independent stations and their primetimes, forcing syndicated shows to all compete for a finite amount of space on Saturday & Sunday out of primetime as networks had exclusivity to primetime. Remember, Fox gobbled up a lot of stations, and in larger cities, Telemundo and/or Univision did too. There were loads of independent stations before the mid '90s.
* Combine an oversaturated market + ex-indy station's primetime being locked up= this forced many shows to late night Sat or Sun, like after 11PM. If you're on the air Sun or Mon morning at 12AM or 1AM, what kind of ratings do you think you're gonna get vs. something Sat or Sun from 2-6PM?
* The internet as a major medium for entertainment mainly became a factor in the '00s. It was almost non-existent early in DS9's run and by the mid-late '90s, when the internet truly "arrived" for many people, it was web 1.0 and remember, people are more prolific websurfers now than then. Projecting web 2.0 (youtube, facebook/myspace, hulu, blogs, etc) back onto the internet of the '90s will give you a completely false sense of what the '90s internet was.
Of course, DS9 was one of the most prestigious syndicated shows in the '90s, so usually got one of the better slots in Saturday or Sunday afternoons/early evening. Also note many independent stations also showed sports, particularly baseball and basketball (before regional sports channels arose on cable). On WGN, the Cubs frequently pre-empted or pushed around DS9, as well as Hercules & Xena (basketball didn't because they play games in primetime, not the afternoon, though others noted basketball pushing DS9 around in their market).
And DS9 had very wide coverage. It was one of the top syndicated shows. Any station getting synd. shows would want DS9 back in the early/mid '90s. After 1995 or 1996, Baywatch's ratings really started to tank and while Hercules & Xena were hot, those 2 shows + DS9 all shuffled among #s 1, 2, and 3 in syndicated programs (not counting game shows). By 1998 it was going lower, but it was still in the top picks of all synd. shows.
It's rather ironic, DS9 fans whining like the Bajorans that made people tune out in the early seasons and focusing obsessively on Seven's T&A when DS9 had better ratings than Voyager across its run. Too complex/serial, no T&A, it was the middle child, etc... all rather cliched explanations when the real reasons are mundane and tied to the changing tv landscape of the '90s and the timeslots DS9 occupied.