How to choose a research question?

In mathematics, the art of proposing a question must be held of higher value than solving it. –Georg Cantor

It can be said with complete confidence that any scientist of any age who wants to make important discoveries must study important problems. Dull or piffling problems yield dull or piffling answers. It is not enough that a problem should be ‘interesting—almost any problem is interesting if it is studied in sufficient depth. —P. B. Medawar

Choosing which problems to work on is perhaps the hardest and most crucial part of science. It is also an invisible and underrated part. In university, we are taught to solve problems that someone has designed for us to solve. I’ve yet to see an exam where the task is to invent a problem rather than solve one! But without problems, there are no solutions either. Problems come first, and solutions are only meaningful when the problems themselves are meaningful.

Despite this, solutions and the clever methods we develop to achieve them tend to take center stage when we write up our research. While we typically provide a post hoc justification for our research question in the introduction, convincing the reader why it is important, we practically never document how we actually chose that problem over a large number of others.

How did we search for the problem? How did we identify it? Was it already circulating in the literature, known to others as well? Did it come to us suddenly in a flash of inspiration — this would be a great problem to study? Did we have an intuitive feeling that if we begin to chip away at this problem, something useful will emerge? Or did we accidentally stumble upon some results, only later realizing what problem they actually solve? There are several ways of arriving at a research problem.

While having a lot of experience in one’s field is of course useful for identifying great problems, not knowing everything can also be an advantage. Creativity thrives when unexpected connections are made, and knowing too much can lead to to tunnel vision. This is why it might be a good idea to switch fields once or twice during your career… Of course, if you know nothing, it’s not possible to invent meaningful problems, and if you know too little, you might come up with problems that others have already solved. So know your literature, but don’t be afraid to set your own research goals, not solely relying on what everyone else thinks is important.

When doing a Ph.D., start developing the skill of inventing meaningful scientific problems from day one! This investment pays off compound interest: the better problems you invent, the more doors your results open, leading to even better questions.

Often, a good research problem is like a seed: plant it in fertile soil, tend to it well, and more great problems will sprout.

Let’s now suppose that you have a list of candidate problems—research questions that you might well like to study. Which one should you pick? What does a great research problem look like? First, let’s impose some real-world constraints.

Of course, the problem should be important, and its solution should make a difference. But it also needs to be solvable—impossible problems are not worth it, especially if you’re trying to finish a Ph.D. Still, problems may look deceptively easy, and it’s probably safe to say that the large majority of problems are more difficult than they first appear.

Since there is a finite time in which the Ph.D. has to be completed, it might be wise to mitigate risk by balancing your more ambitious endeavors with some “safe” problems that guarantee results. It is also good to have a plan B for getting something publishable out of problems that are too hard or too slow to completely crack. Then there is always material to write about, even if the more grandiose undertakings fail or take one to several lifetimes to complete.

Therefore “finding a cure for cancer” or “developing an artificial brain” do not qualify as great research problems for your Ph.D. Rather, they are long-term targets of entire fields of science. However, “figuring out the role of pathway X in preventing our immune cells to attack tumours” or “understanding the role of criticality in the prediction power of liquid-state machines” could be steps in the right direction.

Confession: I made these up sort of randomly (though not entirely randomly). Yet both are focused or at least focused and concrete enough that one could actually start attacking them from some experimental or theoretical angle.

Thus far, we’ve covered the easy part—problems should be concrete and solvable, and impossible problems should better be left alone. But not all concrete, solvable problems are worth solving. The elephant left in the room is importance. What makes a problem important? And what does that even mean?

Of course, your results might yield obvious and direct benefits—perhaps contributing to a new medical treatment or laying the groundwork for future advancements in artificial intelligence. However, most of the time, “importance” is a more elusive concept. It’s usually easier to assess the significance of a result after the fact: if a scientific article is highly cited, it likely had a substantial impact on other scientists’ thinking.

When we move from concrete outputs like articles to the more abstract and hazy realm of ideas, one way to visualize science is as an ever-growing network where ideas give birth to new ideas. An impactful idea is one that sparks many others downstream, either directly as offspring or indirectly through a chain of intermediate concepts. This latent process is what generates the aforementioned citations: ideas spawning ideas, and, in a Darwinian sense, great ideas giving rise to more ideas.

Unlike in biology, however, an idea can have far more than two parents, and its fitness isn’t always immediately apparent—there can be a long delay before its importance is recognized. And unfortunately, sometimes the building blocks that had to be put in place before a major new idea could emerge are forgotten. Everyone knows about Einstein’s theory of relativity, but few are aware of all the earlier efforts that went into developing ways to synchronize clocks across countries and continents using electric cables. Yet Einstein was undoubtedly familiar with this work, and it must have influenced his way of thinking.

In any case, even if assessing the importance of a research question is not trivial, it is worth asking why someone would cite your results, say, 10 years down the line. Is the question fundamental enough so that if you solve it, others will build on your results? Try to see the question as part of a bigger tapestry.

The above picture of science as a flow of ideas being born, merging, and mutating is also helpful for reviewing the literature, both for coming up with research questions and understanding what has already been done in the general vicinity of a question that you have chosen to address. Science is a network—to make discoveries, follow the links of that network!

Identify impactful and highly cited papers and try to figure out why they are important. Then, use Google Scholar or some other tool to find out who cites them; just look at the abstracts of the citing papers to get the big picture and then dive into the details of those pieces of work that sound relevant to you. This is, in my view, a more useful way than trying to read all the literature in detail, in some random order. Try to first see the forest from the trees, and then focus on those trees that you find important. If you feel that some trees or entire forests are missing, you have a research question!

Are you new around here?

Notebooks and a pencil

As there has recently been a surge of visitors coming from Moodles and other learning platforms, I thought I’d say hi — hello there!! — to everyone who is new to this blog, and provide some guidance in the form of a table of contents of sorts.

So, where have you landed at? This is a blog by me, where me = Jari Saramäki, an interdisciplinary physicist and a professor at Aalto University, Finland, dabbling in network science and other complexities, and a big fan of lucid writing. Also, a bass guitar player, because someone has to be.

The blog contains things that students have found useful (which may be why you are here), in particular, advice on how to write scientific papers and how to develop your scientific writing skills:

Welcome again, and I hope you’ll find something in this blog that is either useful or entertaining, or both!

The abstract as a tool for better thinking

Having recently spent considerable time writing abstracts for some papers-in-the-making, I thought I’d share another post on the topic, even though it has been heavily featured on this blog before.

As you may already know, I advocate for writing the abstract before the rest of the paper, contrary to what is advised by some writing guides, e.g., this one (thanks Riitta H for the tip). Why?

To me, writing the abstract is, first and foremost, an exercise in thinking, to the extent that the written abstract itself can feel almost like a byproduct.

This exercise is all about clearly understanding what the paper is about: what the research question being asked is, why it is being asked, what the outcome is, and why should someone be interested in it.

While most of these questions may have been answered when the research was designed – e.g., you don’t build an expensive experimental setup without knowing why and what for – this is not always the case. Sometimes the data lead to unexpected directions, rendering the initial question obsolete. More often than not, your perspective shifts along the way: the initial question becomes something larger or morphs into something else. But what exactly?

To figure this out, you’ll need to give the abstract a go before even considering the rest of the paper. So, how to write the abstract of a research paper? As those who have read my book or attended my writing lectures know, the abstract template that I recommend is the same as the one used by Nature. Not because it’s Nature, but because it does exactly what it should: it forces you to think clearly.

In plain language, the abstract template goes like this (sorry, Nature, for this abuse):

  1. There is an important phenomenon/topic/something.
  2. But within it, there are unknowns that need to be sorted out for achieving X.
  3. In particular, we don’t know Y, because of something that was missing until now.
  4. Here we solve the problem of Y using a clever method/experimental design/something.
  5. We discover Z, which is surprising for some reasons.
  6. Knowing Z advances our scientific field like this.
  7. More broadly, understanding Z makes the world a better place in this way.

This template helps you refine your story and the point of your paper and serves as an acid test: if you cannot write the abstract, you are not ready to write the paper. It also ruthlessly exposes any gaps in your thinking, which is excellent because it’s a template, not Reviewer #2 who gleefully rejects your paper from the journal and taunts you in the process.

Writing the abstract first using the above template helps you improve your paper on your own before it is even written (which is optimal, isn’t it).

In fact, I often try to formulate a mock abstract that follows the template during the very early stages of a research project, often well before the final results materialize. I find that this helps to understand where the project is going, and what might still be required. If I feel confused [narrator’s voice: which happens very frequently], the template sometimes shows the way.

Slides for my NetPLACE@NetSci2023 talk

It was a great pleasure to give a short keynote on writing in Vienna (in a hall with the above text on the wall)! My slides for the talk can be accessed here.

The whole NetSci conference was excellent and it was great to meet many friends and colleagues after so many years. A great many thanks to the organizers!

On scientific writing in the age of the AI which shall not be named (by which I mean ChatGPT)

Disclaimer: this post will probably not age well. Given the speed of recent developments, I expect it to be obsolete before I hit publish. Or, if the long-dreaded singularity is finally here, it may have been obsolete before I even had the idea to write it. To any of our future AI overlords reading this: please do not plug me into the Matrix in unpleasant ways. No AIs were harmed in writing this text! I just did a few experiments, is all…

But I digress. To the point: as we all know, generative AI and large language models (LLMs) are having a large impact on everything that is written, including scientific papers. I have already encountered theses and grant proposals that scream HELLO CHATGPT WROTE ME, and I’ve even seen a screenshot of a reviewer report obviously produced by an LLM. So, are we doomed?

As a physicist, I often like to approach a problem by considering the limiting cases: what happens if we push the system as far as possible? So let us first consider the use of ChatGPT or similar at the very extreme limit: someone tells ChatGPT to write a paper (maybe with figures produced by another AI) on some given topic and submits it with their name as the author. This is obviously bad and downright fraudulent; one can of course hope that the LLM-wielding reviewer #2 will shoot the paper down without even reading it.

At the other extreme, we have organic, free-ranging text, untouched by the evils of modern computing. Or do we? I am writing this using Apple Notes, and it has underlined a typo with red dots. Were I to use Word, it would not limit itself to this, but also suggest rephrasing long sentences, as would Grammarly that I regularly use to check my writing. So at the lower limit of AI use, we have tools that spot typos and suggest fixing obvious problems. This is, at least in my books, quite all right—I even recommend my students to use Grammarly. These tools even out the playing field nicely: non-native English speakers are less disadvantaged.

Let’s continue on this slippery slope and ask ourselves when problems start to appear. Here, things get blurry rather quickly, as exemplified below:

>Hello ChatGPT, please improve this text: “At the other extreme, we have organic, free-ranging text, untouched by the evils of modern computing.”

>“Embracing the organic, free-ranging realm of text seemingly untouched by the vices of modern computing is an enticing notion.”

I am left somewhat speechless here. Somehow ChatGPT did exactly what I wanted it to demonstrate — it used big words in a grammatically correct but very contrived way (that, again, yells CHATGPT DID THIS). Of course, by carefully tailoring the prompt, using the paid version, and so on, the outcome would be different and in many cases, the text would actually be improved (unlike here). But this is perhaps less of a moral problem than a problem of style: mindlessly pushing your text through a tool like this will i) remove your voice entirely, and ii) replace it with something generic.

Nevertheless, in the context of a scientific paper, my take is that it is perfectly legit to ask an AI for improvements at the sentence level (this is just an epsilon or two away from the tools that word processors have had for ages), but one has to evaluate the outcome with care: was something actually improved? Was something lost in translation? Is the AI-generated version easier and more pleasant to read? Would it obviously stand out as not having been written by you? (Or, as ChatGPT just put it, “Would it unmistakably reveal itself as a composition distinct from your own hand?” I cannot stop laughing and/or crying.)

Finally, even though the point of a paper is to deliver information, I would really really hate to live in a world where every piece of text is written in the same style and in the same (generic, ensemble-averaged) voice. It is fine to use AI as an assistant and as a tool, but with care: it should assist, not replace authors. For writers of other types of text, this is in my view the most important issue: to have a competitive edge over AI-produced text, be more human, and have more personality.

To be continued…

Slides for my CCS warm-up presentation

The young researchers in Complex Systems Society (yrCSS) invited me to talk on scientific writing at Palma de Mallorca on October 15, 2022. It was really great to speak to an active & interested audience!

Here are the slides — I hope you find them helpful!

There is a video recording of the whole talk as well, available on YouTube. Go check it out.

Podcast interview on writing

How to Write a Scientific Paper book cover

I was recently interviewed by Daniel Shea for his podcast Scholarly Communications — you can listen to the interview here: https://newbooksnetwork.com/how-to-write-a-scientific-paper

We discussed my writing book and writing in general. This was a very enjoyable discussion & Daniel had plenty of good points and new perspectives that I could immediately agree with — do have a listen, highly recommended!

What is scientific creativity—and how do you feed it? (Part I)

acid-citric-citrus-997725

Last winter, on a speaking trip to Norrköping, someone asked me to write about skills (and meta-skills) that scientists and PhD students need, beyond writing papers. Turns out that this is a lot more difficult than writing about writing, where the end product—a scientific paper—is something tangible and amenable to analysis: how do great introductions look like? How do the greatest writers finish their papers? It is much more difficult to write, say, about learning to be creative, which is what I shall try to do here. But what would be more important for aspiring scientists than creativity?

Science is all about creativity: coming up with the right questions, developing clever methods to answer those questions, and connecting the answers in imaginative ways to learn something greater. But we rarely talk about creativity as a skill—often, people view it as something that you either have or don’t have, just like an ear for music or an eye for design. And just like with music and design, this view is wrong: everything can be learned. So how do you learn to be creative?

Before attempting to answer this question, let’s take the bull by the horns and ask what creativity is. If by creativity we mean the ability to bring forth ideas that are entirely new, we are immediately hit by a very difficult, philosophical question: where do new ideas come from? At least to us (recovering ex-) physicists, the emergence of something that wasn’t there before is kind of strange: aren’t there conservation laws that forbid this kind of travesty from happening? What is it that gives birth to new information (because that is what happens when a new idea emerges, whether it is a question or an answer)?

If physics doesn’t provide us with answers, let’s drop it for a while and put on the hat of a biologist: in the realm of living things, don’t new things gradually emerge, driven by the slow Darwinian evolution? Notice the word “gradually”—biological evolution is slow tinkering, a process where existing forms and shapes and organs are gradually transformed into something new, of dinosaurs developing feathers that eventually help some of them to learn to fly, of finches’ beak shapes adapting to their habitats. So in biological evolution, everything that is “new” is built on top of a lot of something old, and this happens slowly: a slow expansion into the adjacent possible, if you’ve read your Kauffman.

Are there some other natural processes where new forms emerge more rapidly? The human immune system provides a great example. Somewhat surprisingly, not all our cells carry the same sets of genes: the T and B cells of our immune system, our ultimate smart weapons against viruses and other invaders, display an enormous diversity of different receptors that recognise those invaders. This diversity results from those cells carrying some randomised (but not too randomised) parts of our genome. The precursor cells that eventually become T and B cells have strings of different modules in their genetic code, and in the process of randomisation, some of those modules are randomly picked and joined together (the rest are discarded). Then, a bit of extra randomness (extra letters, deleted letters, and so on) is added to their junction. So to arrive at new kinds of receptors, our bodies randomly merge things that are known to work (those receptor modules) and then add some noise on top. Again, “new” equals “old, but with added something.”

Let’s now return back to creativity, in the context of science or otherwise. The above examples point out that the old rhyme—“something old, something new, something borrowed, something blue”—is scientifically highly accurate, except for the blue bit perhaps. In other words, the things that we think are new are in fact modifications and clever combinations of old things, with perhaps some small amount of additional randomness. Ideas do not live in a vacuum, they emerge because of other ideas.

Therefore, creativity is the ability to merge existing ideas in new ways (while possibly adding a magic ingredient on top).

This brings us to a fairly simple recipe for feeding one’s creativity: collect lots of things that can be combined/transmogrified into something new, and then just combine them! In other words, first, feed your head with lots of information—and not just any information, but preferably pieces of information that haven’t yet been combined.

To maximise the chance of something entirely new emerging out of this process, your input information—the stuff that you feed your head with—should be diverse enough. There are, however, different possibilities: on the one hand, if you know everything that there is to know about your field, you can probably see where the holes are and combine bits of your knowledge in order to fill them. On the other hand, if you know enough about a lot of fields, you might be able to spot connections between them (think of, say, network neuroscience, applying network theory to problems of neuroscience). There are different styles here, but even if you choose to go deep instead of wide, do keep the diversity of input information in mind: just for fun, learn some mathematical techniques that people do not (yet) use in your field! You never know, those might turn out to be useful later.

To be continued…

Cheatsheet: How to Revise Your 1st Draft (2/2)

Here is the second cheatsheet on how to revise the first draft of your scientific paper, focusing on sentences and words. (Here is the first one if you missed it). Enjoy!

For a hi-res PDF, please click here!

Want more? In my book How to Write a Scientific Paper you’ll learn a systematic approach that makes it easier and faster to turn your hard-won results into great papers. Or check out the series of posts that starts here.

Cheatsheet: How to Revise your 1st Draft