Maybe not exactly what the OP is looking for, but we'd be remiss if we went without mentioning

Flatland: A Romance of Many Dimensions

Published in 1884, Flatland depicts a 2-D world inhabited by polygons and lines, which then interacts with a 1-D world inhabited by points, a 3-D world inhabited by a sphere, and a 0-D world which is a point.

While none of these worlds could really be described as alternate or parallel dimensions to our own (or a fictional Earth-like dimension), they could be thought of as alternate dimensions to each other.

1928: "The Blue Dimension", a short story by Francis Flagg, published in Amazing Stories, June 1928, available at the Internet Archive.

The first-person narrator's friend, a retired optometrist, has discovered that there are coexistent worlds separated from ours by different rates of vibration:

"[. . .] Consider that we are living at a certain rate of vibration. Everything vibrating within range of our own rate would manifest itself to us as matter, that is, as concrete material, such as mountains, trees, cats, birds, snakes, etc. Anything below or above our range would to us be merely
space, non-existent. You follow me?"

"Not quite," I confessed.

"Well, let me put it differently. You know there are sounds so high in frequency or pitch that the human ear cannot hear them, and vice versa, so low as to be inaudible."

"I understand that."

"Good. Then please remember that everything we observe around us, the smoke of factories, the red of sunset, houses, trees, animals, men, are all things manifesting themselves to us at varying degrees of vibration. At a certain rate they impinge on the ear as sound, the eye as color, the tongue as taste, the flesh as feeling. If that be true, then there must be a wealth of things all around us we cannot taste, handle or see."

[. . . .]

"Robert," said the Doctor impressively, "the world, as we know it, the world of our five senses, has been pretty well explored. Lots of people think there remains nothing more to discover. But what if someone were to open the way into those hidden realms all around us, the countless planes above and below! Think of the strange races that might be found, the new lands that might be visited, the wealth of knowledge that might be garnered!"

The narrator tries on a pair of goggles the Doctor has devised to see into another dimension:

"I beg you not to be in the least alarmed, no matter what you see. Remember to keep quiet and not to endanger these lenses by any sudden move. Bear in mind the fact that you are in no bodily danger, that I am constantly by your side in this workshop, and tell
me, if you can, what you see."

With that the eye-glasses were brought down until the rest-piece fitted the nose, and the side-flaps drawn back, were made fast in the rear. For a moment I was dazzled. My blinking eyes were lost in a maze of contrasting crystals. Then, so suddenly as to galvanize me with the shock, the crystals merged into one harmonious whole, seemed to expand, clarify, and I was gazing — gazing through the incredible aisles of a blue forest. It was a blue world that I saw. The trees, the giant ferns, the sucker-like blooms, were all blue. Not one prevailing shade of blue, no. The flowers, in some cases, were almost purplish red, and in others, shaded away into the most delicious contrasts of creamy whites and yellows. But the predominating color was blue. What could be seen of
the sky was greenish blue. The very atmosphere had a bluish tinge, as if the winds were colored and could be seen. Whichever way I looked, the blue forest was before me. I turned my head. It was on either side of me — behind me. A shiver of fear ran down my back.

After demonstrating the goggles, the Doctor shows the narrator a device he has invented, but as yet tried out only on mice, for transporting bodies physically into the other dimension:

"You mean," I gasped, "that you have invented a way of getting there?"

"Exactly."

"But how?"

"Briefly, by altering the present rate of vibration and bringing it in harmony with that prevailing in the other dimension. Obviously, if my body can be made to vibrate in accord with the blue world, I shall manifest there and not here. At least, I think so."

He led the way to what looked not unlike a big wringing machine of the roller type. The rollers, however, were of fine wire coils, interlockingly arranged, and there were twelve of them supported above a large tub filled with a metallic fluid. Several powerful looking electric batteries lay at the tub's base, on the floor.

"This," said the Doctor, laying his hand affectionately on the complicated apparatus, "is the Re-vibrator. The person or thing to be re-vibrated is run through those rollers, at the same time an alternating current of electricity is maintained in the wire coils which affects the molecules of matter and brings about the vibratory change. Just how this is done, I cannot tell you, for I do not know; but take my word for it, it is done."

I stared at the inert piece of machinery with mixed emotions. That anyone or anything could be run through its rollers to another dimension seemed the height of absurdity. Yet, after my experience with the glasses, I was distrustful of my own doubt.

According to the Wikipedia article on Parallel Universes, In one of the stories-within-a-story of Thousand And One Nights, "The Adventures of Bulukiya",

the protagonist Bulukiya [learns] of alternative worlds/universes that are similar to but still distinct from his own.

I don't know if you consider 1001 Nights as a human belief system, but I think it might fit. Since 1001 Nights has stories dating back to the 10th century and maybe older, this seems like a likely candidate.

Wikipedia has this to say:

One of the first science fiction examples is Murray Leinster's
Sidewise in Time, in which portions of alternative universes replace
corresponding geographical regions in this universe. Sidewise in Time
describes it in the manner that similar to requiring both longitude
and latitude coordinates in order to mark your location on Earth, so
too does time: travelling along latitude is akin to time travel moving
through past, present and future, while travelling along longitude is
to travel perpendicular to time and to other realities, hence the name
of the short story. Thus, another common term for a parallel universe
is "another dimension", stemming from the idea that if the 4th
dimension is time, the 5th dimension - a direction at a right angle to
the fourth - are alternate realities.

According to the page on the story itself,

"Sidewise in Time" is a science fiction short story by American writer
Murray Leinster that was first published in the June 1934 issue of
Astounding Stories. "Sidewise in Time" served as the title story for
Leinster's second story collection in 1950.

So as far as "scientifically explained" alternate timelines, the date to beat is 1934.

I have been reminded that a related question, What was the earliest SF work that used the idea of the "Multiverse"?, was asked a couple of years ago, at which time I gave the following answer, which was accepted:

1915: A Drop in Infinity, a novel by Gerald Grogan, available at the Internet Archive. Reviewed by Everett F. Bleiler in Science-Fiction: The Early Years:

A robinsonade in the fourth dimension.

Jack Thorpe and Marjorie Matthews are walking along the shore in Cornwall when a seeming eccentric asks them directions. They humor him by showing the way, whereupon he produces a revolver and takes them captive. A scientist who has worked in dimensional research, he is brilliant, but unfortunately mad and irresponsible. He thinks of himself as a hubble-bubble, and so the characters call him.

The Hubble-Bubble reveals that he has obtained access, via the fourth dimension, to two other worlds which in modern terminology amount to parallel worlds. His technique involves electricity, vibrations, and a mental set. He now offers Jack and Marjorie the choice of death or entry to another world, which he claims is much like earth in fauna and flora, but without human or other intelligent life.

Jack and Marjorie have little choice, and in a short time find themselves in the world they later call Marjorie-land. Making the best of the situation, they work out a Crusoe-like primitive culture, building a house, cultivating certain plants, and domesticating animals. From time to time a few other humans are dropped in with them, a total of four batches in all. Most of them are congenial, but Michael Quelch, a lazy, vicious Cockney will eventually cause trouble.

On one occasion the Hubble-Bubble's apparatus seems to have "backfired," and Jack is temporarily returned to our world. But he makes terms of a sort with the mad scientist and goes back to Marjorie-land.

Time passes. Jack and Marjorie have two children, and the colonists thrive. Life seems reasonably secure and happy. But then Quelch causes trouble. Thinking that Jack is dead when he does not return on time from a journey of exploration, Quelch tries to seize control of the settlement, rape Marjorie, and murder the children. Fortunately, as in a novel by Edgar Rice Burroughs, Jack returns in the nick of time. After some complications Jack reluctantly sets out to hunt Quelch down and kill him, but Quelch is found dead of natural causes.

After a time it becomes apparent that the Hubble-Bubble is also dead and that the small human colony in Marjorie-land is permanently isolated.

The story is told by Thorpe, a generation or two later.

In the first part of the book, the treatment is flippant, but the story soon settles down to a rather dull development. Of historical interest as a very early parallel worlds story.

So I'm just going to be that guy and say, "The question is wrong."

You've explicitly disallowed religious texts from answers, but it means you'll never get the truth. In reality, the most well-documented, best-protected documents are those considered sacred by the people who wrote them. We will naturally see far more ancient documentation on religion and politics than anything else.

But just because something is part of a culture's mythology doesn't mean they believe it as fact. And things believed as fact aren't immune to being partially retold as fiction. The idea that modern sci-fi and fantasy writers didn't take many ideas from religion seems patently absurd to me.

Nat mentions in a comment,

... Paradise Lost was also a fictional work, but I've heard Christians cite it as actual religion. So where's the line? And if the line's original intention, then Dante's Inferno was clearly intended as fiction, yet it discusses alternate realms. Earlier works exploring the "underworld" were likely meant as fictions, too. Some argue that the Garden of Eden, from the start of the Bible, was also intended as allegory. I guess it just gets hard to tell what the original intention is once a work's old enough.

I would take it one step further and say that even if these original pieces were intended to be truth, there's no way other people haven't been creating deliberately fictional stories for just as long. Modern sci-fi is just the latest in a very long string of such stories stretching to before recorded history.

A note on Stack policies regarding religious beliefs and "sci-fi" or "fantasy".

To avoid turning the entire forum into a giant religious debate (and probably other reasons), there's a policy about not treating religious teachings or beliefs as fiction. But we're not treating these beliefs as either fact or fiction here. We're just noting that the concepts existed in some form.

If Odin came to Earth and gave the ancient Norse a physical tour of Valhalla, the concepts written about wouldn't be any more influential than if the entire story was invented around campfires over the centuries. This answer doesn't care one way or the other about the origin of the beliefs. Just that these beliefs were almost certainly the origin of many stories.

What is an alternate reality / parallel dimension, anyway?

Different people will, of course, differ in opinion on the exact definitions. But a simple definition of a parallel dimension is any physical location you can't access via normal travel through 3-space. Or 2-space if you're a Flatlander.

An alternate reality is any type of parallel universe that largely mirrors Earth (or the protagonist's homeworld if they're not from Earth) but has various shades of differences. Some of these are caused by branching timelines, such that normal Earth and alternate Earth were one and the same at some point in their shared past. Some are only superficially similar. Some invoke some type of shadow people, who not only mimic humans on Earth, but specific humans currently alive.

I doubt many ancient civilizations (if any) understood the concept of a separate 3-space reached by traveling ana or kata along a fourth spatial dimension. You could, of course, arbitrarily define parallel universes as requiring some such convention. But there are two problems with this.

First, not all dimension hopping is done via a fourth dimension. Narnia is reached through a wardrobe that makes no mention of dimensions. It's entirely possible for the wardrobe to connect directly between worlds without any external notion of direction. So you'd be arbitrarily defining many modern "parallel dimensions" as not being such.

Second, it completely misses the point of looking for a historical source of the trope if we only include highly pedantic definitions of the trope and act as if instances of the trope meeting these pedantic criteria were conceived in a vacuum.

What about the origin of "dimensions" themselves? (c. 1754)

The earliest example of the word "dimension" being used to refer to the modern mathematical concepts is from 1754⁴ when Jean Baptiste le Rond d’Alembert wrote⁵

"I said earlier that it is impossible to conceive of more than three dimensions (italics of d'Alembert). A clever acquaintance of mine [un homme d'esprit de ma connaissance] believes that one might nevertheless consider timespan as a fourth dimension, and that the product of time with volume would in a certain manner be a product of four dimensions; this idea may be contested, but it has, it would seem to me, some merit, if only because of its novelty" [D'Alembert 1751-, Vol. IV, lOlO]."

Clearly, the idea existed before this article was published, but we have no way of knowing how long before.

What I'm having no luck finding is a source for the first time the mathematical concept of traveling through a higher dimension is published. Clearly, d'Alembert is only talking about representing time as a dimension, not as a means of travel to another world.

But I would say this puts a reasonably firm cap on the oldest material that could use this trope in its most pedantic form as being early to mid 1700s.

The Field of Reeds (the Egyptian afterlife, c. 2700 BC), the earliest recorded alternate reality / parallel dimension story I could find.

We can go back to ancient Egypt (c. 2700-1800 BC) and find mythology relating to an afterlife.^{1 2} The deceased's soul leaves its body then travels to the Hall of Truth, (hopefully) passing various tests before approaching the Lake of Flowers, where a ferryman, Hraf-hef, would take the soul to the Field of Reeds, a paradise version of home where things were better and nobody died.

This is about as close to an "alternate universe" as you can get without using the label. Sure, you have to die to get there, but it's a physical location that largely mirrors Earth that can't be reached by conventional travel.

Arabian Nights -- The Adventures of Bulukiya (c. 750 AD), the oldest fictional alternate reality / parallel dimension story I could find (not verified).

This answer comes from another question asking about the first record of explicit portals used to travel to another world.⁷

According to the other scifi.se answer, and Wikipedia⁸, the story has elements of portals, other worlds, oh my! But I read through the story itself⁹ and couldn't find anything along those lines. I scanned through the days after the story heading but saw nothing there either, except mentions of the afterlife and Allah having created our world and the worlds of Hell and Heaven.

This would beat my next contender, but only if it actually contains these plot elements.

One thing to note here, is that the author of Arabian Nights is clearly influenced by religious mythology, and consistently writes from a pro-religion perspective. Whenever he mentions Allah or Muhammad, he interjects some type of reverent comment to ensure nobody believes those are just characters in a story.

‘O my mother, I have found, in one of my father’s treasuries, a book containing a description of Mohammed (whom Allah bless and keep!)

in that island he saw serpents as big as camels and palm trees, which repeated the names of Allah (be He extolled and exalted!) and blessed Mohammed (whom the Lord assain and save!)

Even if it's not dimension-hopping, it's clear evidence of old fiction directly including religious mythology into the fiction's mythology without treating the religion itself as fiction.

The Divine Comedy (c. 1321 AD), the first alternate reality / parallel dimension (probably) fictional story I can verify.

In The Divine Comedy³, the protagonist (and author), Dante Alighieri starts on Earth, travels through Hell, and ends up in Heaven.

The descent into Hell could be literal enough. The Earth is rather large and could potentially contain a vast chasm underneath.

But Heaven is reached by ascending from the Mount of Purgatory (an island in the ocean) after emerging from the bowels of Hell. And it's described as copies of our solar system, with Earth, Mars, the Moon, etc. being present along with several mythological constructs such as the Empyrean.

Again, this very snugly fits into the definition of not only a parallel dimension, but an alternate reality. It's a place that exists alongside ours, only accessible via some portal to Hell, and has a close resemblance to our Earth.

Note that there is some contention that The Divine Comedy was believed by the author to be some kind of spiritual journey, and might fall under the banner of "religious belief". However, most people would consider it a work of fiction, and it was labeled such officially by the Catholic Church of the era.⁶

¹Egyptian Book of the Dead from the Ancient History Encyclopedia
²Egyptian Afterlife - The Field of Reeds from the Ancient History Encyclopedia
³The Divine Comedy from Cummings Study Guides.
⁴Four-dimensional Space article on Wikipedia.
⁵d'Alembert and the Fourth Dimension by Rosine G. Van Oss via Science Direct.
⁶Quora Forum Post by Sonia Fanucchi.
⁷What is the first instance of a portal to another world?, answer by OrangeDog.
⁸One Thousand and One Nights article on Wikipedia.
⁹The Adventures of Bulukiya from the Adelaide ebooks collection. _{Note that it's a small part of a large page which the link should take you to.}

While it’s not fictional itself, it seems worth mentioning Hugh Everett III’s 1957 scientific paper (and PhD thesis) The Theory of the Universal Wave Function, which was the first thing to give an actual scientific basis to parallel universes, and turned such stories into science fiction instead of fantasy.

The concept of alternate dimensions/realities—also known as a multiverse—has strong roots in various religions and dates back thousands of years B.C.

Multiverses are not a modern or new concept. The concept of the multiverse has been explored for thousands of years to attempt to explain basic human existence, life and death… The ultimate “why” of this all.

And as with many things, Wikipedia sums things up nicely:

The concept of a multiverse is explored in various religious cosmologies that propose that the totality of existence comprises multiple or infinitely many universes, including our own. Usually, such beliefs include a creation myth, a history, a worldview and a prediction of the eventual fate or destiny of the world.

And even describes Hindu cosmology as well:

For example, Hindu cosmology includes the idea of an infinite cycle of births and deaths and an infinite number of universes with each cycle lasting 8.64 billion years.

As well as Kabbalah-based (Jewish mystical) theories:

There are five worlds between the Creator and our world. Each of them consists of five Partzufim and each Partzuf of five Sefirot. In total there are 125 levels between us and the Creator. Malchut, moving through all these levels, reaches the last one, and in this way, Behina Dalet, the only creation, merges with the four previous phases.

The concept of multiverses has existed since humans have imagined virtually anything. Heck, ever write? Where does what you write come from? And how is it that you wish to write about it and then share it with others to—hopefully—inspire them?

Sorry to sound hokey, but the world of creativity, fiction and science fiction is based upon someone creating another world that you—the reader—will then enter by consuming (reading, viewing, experiencing, etc…) their work. And pretty much all superhero stories no matter what are based on a god—or group of gods—coming to some place and saving people.

No matter how “Down to Earth” or relatable a superhero is, they are ultimately a “god” in the universe they inhabit. And all of these universes are unique and distinct from other universes created by others.

One very famous example of visiting the future, coming back, and creating a different timeline is Charles Dickens’ A Christmas Carol, from 1843. You’ve all read it: “Assure me that I yet may change these shadows you have shown me, by an altered life!”

If the question is, as you put it, "was there a definitive early example which defined and popularized the concept", one has to bring up H.G. Wells's 1923 utopian novel Men Like Gods as an early example.

Men Like Gods was written by a well-known author of the time, published in the US and UK, and received widespread critical attention. It even attracted a famous parody, Aldous Huxley's Brave New World. (It's not the concept of the multiverse being parodied; instead, it's Wells's utopian ideas).

The concept is introduced and explained in Chapter 4:

Serpentine had the manner of one who is taking great pains to be as simple as possible with a rather intricate question. He spoke, as it were, in propositions with a pause between each. "It had long been known," he began, "that the possible number of dimensions, like the possible number of anything else that could be enumerated, was unlimited!"

Yes, Mr. Barnstaple had got that, but it proved too much for Mr. Freddy Mush.

"Oh, Lord!" he said. "Dimensions!" and dropped his eye-glass and became despondently inattentive.

A little later:

Serpentine proceeded to explain that just as it would be possible for any number of practically two-dimensional universes to lie side by side, like sheets of paper, in a three-dimensional space, so in the many-dimensional space about which the ill-equipped human mind is still slowly and painfully acquiring knowledge, it is possible for an innumerable quantity of practically three-dimensional universes to lie, as it were, side by side and to undergo a roughly parallel movement through time. The speculative work of Lonestone and Cephalus had long since given the soundest basis for the belief that there actually were a very great number of such space-and-time universes, parallel to one another and resembling each other, nearly but not exactly, much as the leaves of a book might resemble one another. All of them would have duration, all of them would be gravitating systems—

(Mr. Burleigh shook his head to show that still he didn't see it.)

—And those lying closest together would most nearly resemble each other.

Source: Project Gutenberg (my emphasis)

Borges wrote a number of stories on the topic. The Garden of Forking Paths in 1941 and A New Refutation of Time in 1944. I think this is the one where he talks about a father putting a heavy iron sphere on the back of his son, crushing him so his double in another reality could fly.

Investigation of Borges' antecedents and footnotes would be a fertile ground for exploration of this concept.