Yubetsu’s Post

🚀 Building the Knowledge Base of Tomorrow: Our Systems Started Crawling the Scientific Landscape! One of the most significant challenges of combining scientific research and artificial intelligence is the amount of data to be handled. To give you a sense of scale, around 3 million scientific articles are published each year—that is over 8,000 articles per day! At first sight, that might not seem like a lot. Analyzing 8,000 articles daily is undoubtedly an impossible task for a single researcher. For us, it totals to petabytes of raw data that need to be analyzed but, most importantly, discovered at first. After extensive research—and loads of optimization—we have put our experimental "discovery cluster" into operation. After this test run, we plan to launch the final version of our cluster, which is expected to index the majority of scientific literature in a matter of weeks 🎉 . 💭 Current estimates say there are around 250 million scientific articles in existence. We ourselves are very interested in how much larger (or smaller) the final number will turn out. Do you agree with the estimate or have a different number in mind? Feel free to comment. 🔔 Follow us for more updates

  • No alternative text description for this image

To view or add a comment, sign in

Explore topics