Doing research isn’t an easy thing to do. There’s a reason that not everyone is an academic. Trying to bang one’s head up against the wall of science isn’t most people’s idea of a fun time.
That being said, when you do get an idea of a direction it can go, it’s exciting. You start gaining momentum, and before you know it, you’ve gotten a paper drafted up. Soon, you will be able to publish it, inserting yourself into the scientific literature. You will be able to stake your claim on the metaphorical mountain and declare, “I was able to come up with this!”
Of course, that’s until you see this.
Suddenly, all of your progress seems like a waste of time. While you were busy carving your own path up the mountain, someone had already beaten you to it.
This is frustrating, to say the least. Depending on what kind of work you did, it might all be useless now. Hopefully, you can still salvage the work, make it more general. However, that doesn’t change the fact that some of your results (perhaps even your key results) aren’t new.
One might honestly wonder how this can happen, particularly now, when we have such great search engines that can index more information than we could possibly ever consume. Are you just lazy for not checking the literature beforehand?
The truth is a bit more complicated. As historians know, preserving documents into the future isn’t easy. This is definitely true when considering the fact that we’ve gone from paper to digital, which means that a bunch of papers in the literature needed to be digitized. This isn’t too difficult, but getting these older papers catalogued is. Even with the power of search engines, it can be a hassle to find older papers, since they aren’t catalogued well. You need to dive deep down the rabbit hole of references from other papers to locate it. At that point, you better hope that you can find a PDF of the paper, because a lot of the time there’s a paywall that you can’t leapfrog.
Suffice to say, searching for older papers isn’t easy. That’s a big part of the reason why I ended up “discovering” an older paper that had part of my project. It’s not that the literature wasn’t searched, it’s that papers can be buried several layers deep and hidden from view. This is compounded by the fact that physics is a dense field with a lot of people.
Still, it makes me appreciate the scientists who did work before we had most papers digitized. The problems now are niche, but before digital finding any paper could be a challenge. I imagine you needed to have someone who was skilled at scouring the literature to make sure that you were doing new work.
The moral of the story is simple. The literature is vast, and so you need to be careful when starting a new project. If you do find that you’re repeating older work, try to see if you can build upon it in someway. Mistakes happen, so don’t get into a rage if you find yourself treading up an already-climbed mountain.