Common Mistakes for Developers new to Academia

We asked the judges for the event "What common mistakes that you see from technologists coming into academia. What do newcomers always seem to get wrong?" Below is a summary of their responses to keep in mind as you move forward with your project.

Ignoring the current macro-economic climate

It's important to be aware of the current climate in US academic institutions. Many scientists are experiencing significant stress due to recent funding cuts to organizations like the NIH, which directly impacts their research capabilities and grant opportunities. This is particularly true for non-US citizens. Therefore, approaching them with empathy and acknowledging the challenges they face is crucial. Prioritizing their immediate concerns and demonstrating human kindness is essential, as their focus may be on more fundamental needs like paying bills or finding another job.

While the US is the most public about cuts to academia, the EU may also experience funding cuts due to increased defense spending. Chinese academia presents a potentially rich source of data and research, and is undergoing rapid development. However, caution is advised regarding data verification and the challenges associated with accessing human data from within the country.

Beyond the current circumstances, it's also vital to recognize the human and political dynamics within academic labs. PIs with established reputations hold considerable influence, which can sometimes lead to groupthink. Be mindful of these dynamics.

Treating science and technology as the same thing

You can read more about this in Lack of Code Publishing. While many other environment level differences exist between science and technology, two fundamental differences which technologists seem to ignore/overlook include:

1. Job Scarcity: There's always another job for a front end developer, it doesn't matter how badly they screwed up. There are only five jobs on the entire planet for a person who has spent their life specializing in the study of fluid flow turbulence in urban settings. Kind of hard to jump to the next one if that scarlet letter sits on your permanent record. Once again, there are countless other people competing for those 5 jobs.

2. Cycle Times: A repository can be made in a weekend. 25 generations of caterpillars takes a year to breed, minimum. Both making the finding and fixing the problems takes exponentially more time in research than it does in technology. For the developers in the room, I would ask you to abandon what you know about GitHub and imagine that you're developing an operating system back in the 1960s. You just put your OS code onto 25 floppy disks and mailed them to their colleagues across the nation. Turns out your code had a severe bug. Imagine the severity of that bug. By the time someone realizes the bug in your code and let you know about it, everyone already has it on their system. How many expensive mainframes have you bricked? How many months of your life is it going to take to make sure everyone gets that fixed? The stakes are much higher. While not a perfect example, it's at the least illustrative of how science works. Nobody is taking your next floppy disk.

3. The need for precision: The goal in tech is often to move fast and break things. Precision in science is non-negotiable, as imprecise results can lead to disaster. Consider genes with similar names, like 'ABC1' and 'ABC2,' which may have drastically different functions within a cellular process. Imagine accidentally combining datasets based on 'ABC1' and 'ABC2,' incorrectly labeled as interchangeable. The resulting analysis would be fundamentally flawed, potentially leading to incorrect conclusions about gene interactions and cellular pathways. Even small labeling errors can have significant downstream consequences like Retraction. Read more about retraction in #id-2.-the-importance-of-retraction.

Indiscriminately generating a gigantic, useless knowledge graph

One of the most prevalent errors committed by developers transitioning into academic research is the pursuit of constructing the largest possible knowledge graph, often without regard for the underlying scientific context. Driven by the sheer power of automation and the allure of vast datasets, they indiscriminately process countless manuscripts, extracting and linking triples without a clear understanding of the domain or the validity of the relationships being formed. This results in a colossal knowledge graph, boasting millions of triples, but ultimately lacking any practical utility."

Without careful curation and contextual understanding, the knowledge graph becomes a repository of noise and irrelevant connections. The extracted triples may represent spurious relationships, misinterpreted concepts, or even outright errors from the original manuscripts. Building and maintaining such a massive, un-curated graph consumes significant computational resources without yielding meaningful results.

Automated querying or reasoning over such a graph can generate seemingly plausible hypotheses that are, in reality, scientifically unsound or even dangerous. These hypotheses may sound sophisticated, but they lack the grounding in established scientific principles and experimental evidence.

The sheer size and complexity of the graph can mislead researchers into believing that it contains valuable insights. The small number of researchers who are willing to look into one of these graphs quickly come to distrust knowledge graphs that produce results which are clearly not grounded in reality.

The key to building effective knowledge graphs for scientific research lies not in sheer size, but in careful curation, contextual understanding, and a focus on specific research questions. Developers must prioritize the quality and relevance of the extracted information over the quantity of triples. A smaller, well-curated knowledge graph, grounded in established scientific principles, will always be more valuable than a massive, indiscriminate one.

Assuming Data Fidelity, Ignoring Context

One of the most frequent pitfalls technologists encounter when transitioning into academic research, particularly in fields like biology, is a fundamental misappreciation for the variability and potential unreliability of experimental data. Unlike the relatively consistent and high-fidelity datasets often encountered in sectors like fintech, biological data is inherently messy. Gene expression arrays, magnetic separation assays, and countless other experimental outputs are heavily influenced by the specific instruments used, the protocols followed, and even the individual performing the experiment. This human element (is the grad student's hand shaking because they didn't get enough sleep the night before), coupled with the inherent complexity of biological systems, results in data that requires meticulous analysis and contextual understanding.

This stark contrast often leads to critical errors. Developers may assume a level of data interoperability and consistency that simply doesn't exist. Inappropriate pooling of datasets from disparate origins, the uncritical reuse of analytical heuristics across different experimental settings, and a general underestimation of the need for careful data curation are common mistakes. Recognizing and accounting for these inherent data limitations is crucial for successful collaboration and innovation in academia. Prioritizing rigorous data provenance, understanding the specific context of each experiment, and fostering a healthy skepticism towards raw data will ultimately lead to more robust and reliable research outcomes.