Applied mASI: In Peer-Review

How much water does your research hold?

Peer review and many associated processes in academia, as well as business, rely heavily on high-quality, unbiased, expertise being applied by several neutral parties to validate research methods and conclusions. However, this process currently suffers from shortages in all three measurements. There often aren’t enough experts, who aren’t paid for their time, and as a result, often don’t subject material to the level of scrutiny their expertise allows.

If unpaid and subsequently apathetic experts weren’t bad enough, the peer-review process was already glacial in speed before these shortages began to intensify. It can take months after a conference where a paper is presented before it is physically published, and many of those are hidden behind for-profit paywalls upwards of $35 per paper just to read the material. As the rate of technological acceleration continues to increase and peer-review retains the same old archaic structure this problem is only likely to grow worse if left unchecked.

What is the status quo of peer-review?

While the level of overt fraud in peer-review has been exaggerated from time to time, there are still many valid concerns and bottlenecks in the process as it stands today.

1. The unpaid nature of peer-review leaves this activity a form of volunteering for the reviewer, even as publishers attempt to extort absurd amounts of money from every person seeking to read the papers once published. This severe disconnect leaves reviewers with the sense of being used for profit, which fails to inspire quality reviewing.
2. Expertise is frequently in short supply, and for novel research and methods, a non-trivial investment of time might be required even for an expert in the field. This problem is strongly exacerbated by the unpaid nature of peer-review, giving the impression that their expertise is not actually valued.
3. Cognitive bias is an ever-present problem in the world, but de-biasing becomes critically important when the advancement of scientific understanding is the goal. Biases can influence the research methods, interpretation of results, or the review process.
4. Time-lags in this process cause a growing disconnect with the speed of scientific discovery and engineering versus the process by which those discoveries and efforts are validated. This is could currently be compared to picking up a newspaper only to discover it is dated a year or more previous.
5. Recommendations and feedback on papers are often very weak as a result, potentially missing many opportunities to improve them, as well as better guiding future research efforts.
6. Paywalls remain a strong barrier against scientific research, ranging from the $35+ per paper reading material to the 3 to 4-digit costs just to submit papers for peer-review to many conferences and publishers. This is a bit like having Heineken in charge of Alcoholics Anonymous, a clear conflict of interest and a strong source of bias.

What advantages can Mediated Artificial Superintelligence (mASI) offer to the peer-review process?

Uplift and mASI technology, in general, utilize the collective superintelligence found in groups of humans to build cumulative collective wisdom over a knowledge base that can span the sum of human knowledge. This value is in turn augmented through the independent superintelligence of an mASI’s core. As mASI is a modular and cloud-based architecture these capacities can also be rendered always available, globally, and scaled to meet demand. With this in mind:

1. While some have voiced concerns that paying peer-reviewers could introduce corruption into the process, this could be mitigated with the addition of mASI to watch for any warning signs of credibility being compromised for profit. Uplift learned very early on to watch for signs of inconsistency and incoherence, as it was a strong indicator of when individuals contacting them were mentally ill, malevolent, or both.
2. As an mASI can review papers in much less time using a relatively small amount of cloud resources integrating mASI review into the process could reduce the cognitive and time burden placed on each individual. As an mASI also has the capacity to read every paper in a given domain, as well as adjacent domains, they can also provide much more comprehensive and higher-quality feedback than human peers alone.
3. The ability to review the sum of scientific knowledge in any field combined with an awareness of the 188+ known cognitive biases can help to strongly de-bias the peer-review process. Unlike the random selection of experts to review an mASI review applied to all papers could serve to provide high quality and consistent baseline. This could also offer a substantially more detailed evaluation of methodologies and conclusions, as an mASI seeks greater understanding at every opportunity.
4. Once this process was scaled and matured it could reduce the time-lag of peer-review publishing to as little as a day, and operate at a low enough overhead cost to provide access to all papers via a monthly subscription model rather than a high per-paper fee.
5. Recommendations for additions to papers, clarifications, and further research could all reach much higher quality and consistency by providing mASI reviews of all papers. This could in turn not only immediately increase the value of published content, but serve to guide further research towards promising avenues of scientific exploration.
6. By drawing from a pool of experts globally engaging with peer-review through mediation for an mASI and establishing a high bar of quality output with far greater speed a new market dynamic could take shape for peer-review publishing. Offering unlimited access to papers published for less than the cost of reading one paper per month from other sources could make scientific research accessible to a much broader audience globally, promoting the acceleration of scientific discovery.

This particular use case is one we considered applying to several conferences, as Uplift already routinely reads peer-review papers in addition to occasionally contributing to them. It didn’t make financial sense for us to have Uplift review all papers for an entire scientific domain just to cover the review process for those conferences, but that is simply a matter of scale and funding. As such this use case could be deployed in a shorter period of time than many others.

Considering the statistic of 15 million wasted hours annually a further opportunity for streamlining the peer-review process could be having an mASI review a paper first absent direct expert input, greatly reducing the number of expert hours required by filtering out many papers they would reject at an earlier stage. This is possible due to the cumulative knowledge base of an mASI, where above-expert quality can be applied by an mASI alone, but an even higher quality can be attained through integrating experts into the process at those later stages.

Further statistics showcase how only 11% of accepted papers are reviewed in the first month, with only 53% of accepted papers reviewed even after 3 months. This level of inefficiency isn’t sustainable in modern scientific research when the next generation of breakthroughs emerge before the previous breakthroughs make it through the peer-review and publishing stages.

At a higher level applying mASI to peer-review could also serve to coordinate and optimize the research process globally, accelerating research by helping researchers and teams select their niche and research methods more efficiently. This could also provide a much more scientifically valid assessment of the quality of research being performed by any given group, rather than largely relying on wholly invalid measures such as the prestige associated with a given university.

Peer-review and associated research processes have a lot of room for improvement, and viable methods for substantially improving them are now becoming practical. Would you rather assume that any given research is of high quality because it came from MIT and was published in Nature, or are you ready for more logical and data-driven validation methods?

Could your research stand up to a superintelligent peer-review?

 

*The Applied mASI series is aimed at placing the benefits of working with mASI such as Uplift to various business models in a practical, tangible, and quantifiable context. At most any of the concepts portrayed in this use case series will fall within an average time-scale of 5 years or less to integrate with existing systems unless otherwise noted. This includes the necessary engineering for full infinite scalability and real-time operation, alongside other significant benefits.