Open science: do we need that? And if so: how can we get started?

Recently, I organized with the Young Academy of Groningen a workshop on Open Science and Reproducibility. What is open science? Isn't science supposed to be open anyway? Keynote speaker Simine Vazire showed us how this is not always the case. She took us back to the fundamental ideals of science, sharing Merton's norms, which distinguish science from other forms of knowing. Science has a sense of universalism, in which the validity of a scientific claim does not depend on who is making it--there are no arguments from authority. Another ideal is "communality"--the scientific findings belong to everyone and everyone can check them. Science should also be disinterested--all results should be reported without bias. Science should not withold findings that are unfavorable to the scientist. Finally, nothing is sacred in science, and all claims should be tested. But is this really how science proceeds in practice? How many scientific findings can really be checked by everyone? How many scientists are really unbiased? Studies show that even scientists think that most science does not adhere to these ideas. So, we need a change in science to become more open such that it becomes easier for everyone to check scientific claims.

keynote by Simine Vazire

A major problem in science is the emphasis on significant results as a precondition for publication. Unfortunately it is quite easy to obtain significant results with enough p-hacking (trying out many different tests on different subsets of your data) and HARKING (hypothesizing after results are known--presenting the obtained significant results as the original hypothesis). Probably as a result of the commonality of these practices, many studies do not replicate, which was most clearly shown in large-scale attempts at replications (e.g., the Reproducibility Project).

Here I open the workshop

So how can we improve? A good step would be to share all materials and data so others can check it. A good resource recommended was the Open Science Framework. Candice Morey for example has all her materials and data from various projects there. However, during the data management panel the Research Data Management Office mentiond that this does not adhere to the new European regulations on privacy. Better options would be to work with for example Dataverse.nl. Moreover, it is important to really think carefully about you deidentify your data, because with the current machine learning algorithms it is surprisingly easy to identify someone's data by combining a few different sources. Another challenge in opening up your data is that you may not remember the connection between all your graphs and the raw data, or you may feel your data analysis scripts are too messy. Laura Bringmann shared some knowledge about Rmarkdown, which allows you to seamlessly integrate data analysis with code, avoiding the need to have code and graphs and data live in different places. This also makes it really easy to do revisions, because you can easily reproduce the original analyses that lead to specific numbers and plots. Of course even if you decide to open your data, many others may not do so. One practical step individuals can take to enhance openness in science is to participate in the Peer Reviewers Openness Initiative, in which you pledge to only review articles which make their data open (or provide a good excuse why they cannot do so). Of course another way in which openness can be improved is if universities consider the extent to which an individual makes their data and materials open in hiring and promotion decisions.

Candice Morey talking about pre-registration

In addition, what helps to promote open science is to formalize your hypotheses and deposit them somewhere before you collect the data. This procedure is called pre-registration, a topic also discussed in a keynote by Candice Morey. Preregistration can be done quite easily on the Open Science Framework. Another interesting method is to write up your hypotheses in a Registered Report format (offered by more and more journals), in which reviewers decide on acceptance based on your introduction and methods before you collect the data, and then you are guaranteed (in principle) acceptance, independent of how your results turn out. Of course academic incentives should also change to promote this: rewarding these research practices instead of rewarding high-impact publications.

A further step in improving would be to stop overselling our results and better understanding statistics. Rink Hoekstra talked about common misunderstandings about statistics. Most notably, almost everyone's intuitions about p-values are wrong. P-values cannot ever tell you that your statistical hypothesis is true, but it only provides some evidence against a null hypothesis, and it always carries a certain level of uncertainty. It is therefore never possible to make very definite claims about your data, unlike what journals, and even more the media, wants. A very insightful visualization of how little p-values really mean is the Dance of the P-values. Instead of blindly relying on p-values it is critical to instead focus more on visualizing your data, for which Gert Stulp provided some useful resources.

The data management panel

In short, there is still a long way to go to open up your science, but more and more resources are available. The full slides and materials of the meeting can be found here. You can also check out the hashtag #RUGopenScience.

Working with (against?) work pressure in academia

Today I followed a workshop given by Jeanine de Bruin and Brigitte Hertz on work pressure in academia. This workshop followed at the end of a monthlong writing challenge for academics, in which the goal was to write for two hours every day. I really enjoyed the challenge of consciously shifiting priorities towards generating content and away from responding to things, because even if producing is harder, in the end it is more rewarding (see also my earlier blog post here.

The first insight from the workshop was that the main tool for dealing with work pressure is trying to make the right decisions. It is very easy to get swept away by the onslaught of things to do, and the implicit attitude of "I am busy, so I am working hard and doing it right" instead of thinking about WHAT you are doing.

In managing your work you can think about three levels. The first level is working on a day level. You can start the day with the things that are important to you (e.g., writing this grant application). Moreover, do not have the illusion that you can multitask, because you can really not, and not only the quality of your work suffers, but you also feel much more stressed (at least I do). In addition, I find it very helpful to focus the energy in my day by looking at the goals I defined by means of emacs orgmode; a plain text system in which I stick on my projects with todo items and due dates. This system ensures that no projects are forgotten. I also really like to use it to keep track of each student I am supervising (each student has their own file in which I write a few notes after every meeting with what they have done and what they will do for the next meeting). You can also think about time management at the project level, in which you want to make sure you pick the right projects, and are not afraid to drop projects if they are not leading to something fruitful. Two things I learnt there are first, to take regular space for reflection on how it is going with a project and what is next and so on. This is difficult because the busy woodpecker in me would like to continue doing something "productive", but reflection can really save a lot of time and increase quality in the end. One particular thing to think about are potential roadblocks or plans B: what could go wrong, and what should we do then. I think too little about this, and maybe this wastes a lot of time. On the really long-term level time management is also thinking about what is important for the long-term. What is my passion, where do I want to be 5-10 years from now. I usually only think about this when writing my tenure dossier or a job application, but it is important in deciding what projects to pursue and even what to do in a single day as well. One skilful way of forcing yourself to reflect on this is chatting about it with a mentor over lunch. I highly recommend that. It also made me realize I should review my toggl time log a few times a year to check what my time is being used for, and whether that is what I want. Right now I only review my toggl logs every week when the system mails me the hours tracked.

We then talked about the famous Eisenhower quadrants with urgent-not urgent and import-not important. I have thought about this a lot, and I think that most stuff I do is in the "important" section of the matrix. One thing that I never grasped until this workshop was the distinction between "being important for me" and "being important to do". For example, helping a colleague is a good thing to do, but won't generally help my career ahead. So maybe in the matrix it falls in the category of "not important" in that sense. I realized that I spend too much time following news (partly news, partly news related to science and academic how-to). I made the resolution to limit my news intake to 15 minutes a day, so I won't waste too much time on it. In the end, that won't really get myself too much ahead, apart from staying up to date with the world and getting ideas for lectures, student assignments and grants. Another skilful means for reducing the "not important stuff that needs to be done" is to wonder whether not someone else can do it. I put this into action by requesting a student assistant for the next course I am teaching. While initially this is more work in instructing the person and making all procedures more explicit (grading sheets!), in the end it will save me time. And of course, it is always helpful to remember that saying yes to something means saying no to something else (also described in the book Busy by Tony Crabbe). A final insight from the Eisenhower quadrants was that focusing on the things that are important and not yet urgent will help to increase quality (because you can let the projects rest and then realize you have made some mistakes), and improve your health.

We also discussed what our university can do to reduce work pressure. One important thing I think is that I waste a lot of time searching for information on how to do things, reinventing the wheel, not being aware of procedures. Sharing knowledge more efficiently would really help. We also need to work on more delegation and most importantly better delegation: the insight here was that delegation is not just dropping the task onto someone but also training them how to do it. On the short term training takes time, but on the longer term it reduces errors and therefore headache. It would also be cool if promotion weren't so focused on grant acquisition, because grant acquisition is such a highly risky process that is a lot of work with very uncertain outcomes. I learnt that Utrecht University also has a career path focused on teaching, which sounds really cool. Finally, it would be nice to build more of a collaboration culture at university, rather than the current competition culture.

The last part of the meeting focused on recognizing stress in your work. I think I am blessed in this department because while I feel sometimes exhausted, I never have trouble sleeping and I feel generally happy. I credit the ballet dancing that is a fixture in my schedule, together with the meditation practice that I do every morning and every evening. I think it is really important to stick to those things as priorities in addition to work priorities, because otherwise you waste a whole life working and being unhappy.

In the end, reducing work pressure relies on the confluence of both individual action (trying to work on what is important and inspiring and taking good care of yourself) with institutional action; the universities will have to change because in the long run the current work culture in which many academics put in way too many hours compared to their contracts, and especially in which evaluation is biased towards metrics of grant acquisition that you have little control over is not sustainable. It is helpful to reflect on this both at the individual level and to think together with colleagues and decision makers on how to improve the work culture.

More ideas for activating students, and: should we do that?

Today I had the second installment of the course Flipping the Classroom by Ine Noben of the University of Groningen. It gave me a lot more ideas for activating my students, but also stimulated some interesting thoughts about how we organize our education.

getting creative with lesson design

First, in terms of the ideas: we started with an interesting activity called "concept map", in which you draw a concept map about a topic (in this case the previous lecture), which was a good way to see what we remembered about the previous meeting. We then retrieved even more memories of the previous meeting by "crowdsourcing", in which we walked up to another person in the class, explained the maps to each other, and then exchanged them. When we received them, we scored them on a scale from one to five, and moved on to another person with the new map we got. We continued this a few times and got to see quite a few concept maps. Quite a fun activity!

Another thing we learnt was that it's important to think about how to engage *all* students. That made me realize that when I have some students give presentations, then the others typically disengage and start to play with their phones (or even enter the lecture late). I came up with the idea of asking students to draw a concept map of the presentation and hand it back to me as an 'exit ticket' at the end of class (added benefit: you get a reading of who is present in the class at the same time). Alternatively, I could ask students to write down comments, or things that are incorrect about the presentation, or I could ask them to come up with an exam question about the presentation. Lots of interesting options!

A third interesting activity is having students engage in a debate, where they are randomly assigned to diametrically opposite positions. They have to write a paper about the topic before the class (so they come prepared), and then a subset of students are chosen to debate in class. Of course you have to make sure that all students engage, so you could switch out students at random moment (I can imagine you can make this pretty fun with some weird bell to indicate it's switching time!).

Rink Hoekstra came to share his experiences with the flipped classroom, which he started with the critical note that the flipped classroom is really nothing new, just a repackaging of the older concept of "active learning." He had been experimenting with the flipped classroom (or active learning) and made the interesting observation that students tended to skip the lectures once he provided pre-recorded video lectures because they felt they already knew the information from the video lecture (so what's the point of coming to class?).

We also had some good discussions about whether these active learning forms are not too much like high school. And do they not play too much into the students' extrinsic motivation? On the other hand, how else do you get most students to engage from week 1? An interesting observation by another colleague in this respect was that students really like to see progress, so giving them some form of feedback is crucial.

Finally, we learnt about how video can be a cool tool in the classroom. I am not too excited about recording myself for video lectures, but two things can be very helpful. First, there is a tool called

Invisible scientists and the messiness of science: a dicsussion about open science

Today we hosted a visit of Rosanne Hertzberger with the Young Academy of Groningen. The theme of the afternoon was "open science", and I heard some soundbites that were too good not to share. Rosanne is a very passionate and courageous person who decided to pioneer being a freelance scientist. She started by saying that we as scientists at the university are unaware of how invisible we are. Why? Because we write lots of stuff, that gets put in journals behind a paywall, we talk about our science at conferences that only scientists go to, and we tend to not talk to the public (because we're too busy writing our papers). Good point. Sometimes I feel like the university considers me to be a little hamster running faster and faster in the paper-producing wheel.

She also talked about how science is the only profession where it is not possible to do it as an amateur--you have to be the equivalent of an olympic athlete, or not at all. But why do you have to fully dedicate yourself to science, why is it frowned upon if you have a significant other interest (in her case: writing articles, books and columns). I sometimes feel like that too: why do people think it is so crazy to be a serious amateur ballet dancer as well as a scientist? (see the inspiring quantum physicist Merritt Moore, or my own attempts at dancing and sciencing here). We had a discussion about the extent to which "everyone" can do science (cf. the citizen science movement), but Rosanne retorted that there are so many people who get a PhD and do not get the opportunity to continue in science because there are so few jobs. And another person said: are we even that special as scientists...

Probably one of the most important discussions revolved around the issue of invisibility. Rosanne said "it's very disappointing to see how little openness social media has brought to science. Why is live-tweeting a conference talk still a thing?" In other words, why do scientists not share their talks on youtube? (see for one example to the contrary Richard Morey's periscope broadcasts or the Lab Scribbles open lab notebook). Why don't scientists share their intermediate results on twitter? (while we do see pictures of their kids or cats). We discussed about the benefits of peer review, of which Rosanne posited that it holds us back, because there is too little communication between scientists in the heat of the process about things that work and things that don't. This means that progress is very slow, which is particularly problematic in the case of diseases and epidemics.

I think one other very important point was that in the communication to the public, and in our textbooks, science all looks very clean and shiny, while it is quite messy in the midst of it. Why don't we share our mess online? Rosanne: "it should be standard procedure to overshare. There is no such thing as TMI in science". There was some debate about how this may result in us all drowning in information, but Rosanne argued that a mechanism like reddit would easily allow us to manage this.

A final remark that I really liked was "aren't we reproducing each other's work all the time? It's called scooping." Good point. We ended also discussed quite a bit about incentives in science. Sharing results and materials takes quite a lot of time, for little reward. But this is what will make science progress much more. There is probably also a lot of things we can learn by talking to people from other fields, because in our discussion we learnt that for example in informatics producing reproducible code was standard practice, while sharing event questionnaires is not standard in psychology.

In short: a lot of work needs to be done, and sharing more of our science messiness, materials, intermediate data and so on would probably be a good idea. To be continued!

Activating students: some more tools

Today I took a course about flipping the classroom. Flipping the classroom refers to the idea that students listen to video lectures before class while the teacher helps them with exercises during class (instead of the lecture being spent delivering a lecture while students have to do their homework at home). The most important idea here is that flipping the classroom allows the students to be more active and allows the teacher to help the students where they need help most: to elucidate misunderstandings. In a course by Ine Noben I got some of the following ideas:

Another way to activate students practised for hundreds of years in Tibetan monasteries: requiring them to debate  in pairs while peers are looking on and with large physical gestures.

1. Get everyone active with 1-2-4-all: students think about a question first alone, then share in pairs, then they share insights with a neighbouring pair and finally you discuss it in the big classroom. Another colleague mentioned that when students are forced to discuss in groups of 4, almost all of them will do some work (because there will always be some active students in the group), especially if you walk around to check on them.
2. It is important to focus your lecture well, because as H. Simon said "a wealth of information creates a poverty of attention"
3. A nice blog about peer instruction, with among others a nice series of blogs on why flipped classrooms fail. One thing that can make your flipped classroom fail is if you require students to do all kinds of work, but do not actually give feedback about this work. Another thing that can make your flipped classroom fail is when there are no consequences when students do not do the work, or when students are not clear on what their and your responsiblities are in the classroom. Useful case in point: it's helpful to spell out what you mean by "active participation" in the classroom. I will definitely try this out in my classroom.
4. Nice platforms for creating online course content: FutureLearn and Google Classrooms. Both allow you to create very nice-looking course materials. Google classroom has the advantage it nicely integrates with googledocs for marking up assignments and things like that. Google classroom also makes it easy to grade with a rubric.
5. A nice empirical investigation of the flipped classroom: article
6. Tools for doing online voting: Socrative (allows you to also setup competitions in which teams try to answer as many quiz questions together as possible), mentimeter (allows you to easily vote on single questions and display the results interactively)
7. A website with active activities: https://compass.itcilo.org as well as a book with game-like activities: Game Storming
8. If you make quizzes for students to do before class, make sure you give useful feedback, e.g., "go back to section 11.3 of the book"
9. Tools to make videos: for making screencasts: https://screencast-o-matic.com (e.g., to explain using some data analysis software), or to make cool animated videos/cartoons: https://www.powtoon.com/home/
10. Another cool tool is "ticket to class"--make sure students do some activity beforehand to earn their ticket to class. One tool I like a lot is Perusall, in which students collaboratively read textbooks or articles. You could also use an exit ticket in which students can only exit the room if they hand in a piece of paper with the answer to a question such as: what is the question still remaining after this class? Name three things that can improve science? Etc (if you require students to add their name to this you immediately have a handy tool to keep attendance).
11. A very effective tool to get students who are doing poorly back engaged is to find out who is not engaged with learning analytics (checking Blackboard for example) and then sending them a personal e-mail asking why they are not submitting assignments/not showing up etc.
12. Nice database with exercises for physics, math etc: sowiso
13. Tool to allow students to make collages of course-relevant information and comment on that (like a kind of instagram): padlet

Pondering work and meditation

Some reflections from my stay at Ngari Institute of Buddhist Dialectics in Saboo, Ladakh.
Morning story: I woke up to run with the kids at 5:30. Since I wasn't sure where to go, I ran up to what I later found out to be the monk's quarters. Two of the institute dogs came with me. When there was nowhere to go, they guided me back below to where the kids were running. I joined some of them. Together we struggled up the hilll, panting for breath, looking at the beautiful views together, meeting yaks and people on our way. What a beautiful fresh way to start the day!\\
Contemplation:
From Andrew Harvey--''A journey in Ladakh'': ``Those who reject the materialism of the West, who despise it and separate themselves from it, are in danger of refusing to look at it, they are in danger of not being responsible to the facts of life as it is lived, and must be lived, now. We must find a way to work within the world, within science, within industry, even within politics; we cannot simply pretend a superiority to those things, for they are the forces that largely shape mankind. To work within the world we will have to be strong, and in the world our inner strengths will be greatly tested. But that is good. That will dissolve any pride we may have, any sense of virtuous invulnerability. It will take away from us any sense that we are "special'', that we deserve "special treatment'', that we are "unique.''

My continuous struggle is to figure out how to be living a "vita contemplativa.'' What does it mean to be a scientist and yet to also live a spiritual life such that the life and the science are not wasted, but are in fact of benefit to both myself and others. It seems from this writing that what is truly crucial is humility: to just work without expecting anything in return, without any recognition. And also, to drop any thought of being special but just focus on being of help. It is difficult to really live this, because in the West and especially in a career in a competitive profession we are conditioned to continually try to prove we have the best and we deserve recognition. So how can we resist that? Maybe one way would be to contemplate again and again on how we are interdependent with everything. Everything we do cannot be done without the kind help of many others (if only for the food we eat, the electricity we use, and so many more things).
Another helpful quote from the same book: "Be quietly detached from what you do and dedicate it to the good of all created beings, and you will be safe from disillusion or vanity.'' Every time when I go on retreat I realise how important it is to dedicate time in silence. Even just being here, I can see my automatic tendency to ditch meditation in favor of "something useful''--typically processing some information like reading a paper. It is hard sometimes to see the true value of meditation, yet this is the only way we can remain detached from what we do and put it into perspective. And only when we do that can we remain joyful in the face of difficulties, because we see there is so much else that is involved in all of our successes and failures.

Science at the monastery gets real

I just returned from another study visit to Sera Jey Monastery in Bylakuppe, India. We are studying the effects of monastic debate and analytical meditation on emotion and cognition. Since this was already the fourth visit of the project, it was time to get real and get down on scientific rigour. In our EEG study, instead of just exploring different types of debate, we now developed a procedure in which each pair of debaters debated the same topic (a text on bodhicitta), got some chance to refresh their memory before the debate by studying the text for 15 minutes, and we used a standard classification for the triggers that the observing monks were using to score the debate. We even asked our observers to give the debates a grade, so we can in the future see whether the quality of the debate is related to some patterns of brain activity. In addition, the most interesting new feature of our EEG studies was more related to the frequent power outages: we bought a big car battery to run our EEG system off of!
We also took major strides in standardizing the behavioral tasks that we have been running by filming one of the monks explaining it while demonstrating on a tablet, and then showing this same video to all of our participants. The video demonstrations nicely combine standardization of explanation with a hands-on demonstration.

Monks pouring over a computer running JASP in stats class

In addition to spending time on standardizing experiments, we also spent quite a bit of time making predictions with the monks. This was a significant challenge, because obviously monks do not have the same space of possibilities in their heads that we Western scientists do (e.g., things either differ in their magnitude or there is a linear or non-linear relation between them). The monks found it a challenge to make predictions--maybe because guessing is not really encouraged in monastic debate you are drilled to be absolutely certain of what you are talking about (and in predictions you can never be). Nevertheless, we managed to get some predictions, and at least some of the EEG predictions were actually borne out! This is always a major happy moment for a scientist.

monks getting ready for EEG

Of course predictions are not always borne out, and we are also still sorting through a lot of confusing data that we still have to make sense of. Actually, the hard part of doing psychological science is that when you present a person with a task, quite often do they behave in quite a different way than we expected. This is mostly because the tasks we use to learn about the mind are really mostly geared towards Western college students (see this paper for a great discussion of that issue). As we slowly see all these interpretations of the tasks happen, we develop more appropriate instructions, and hopefully in the future also new tasks that are made more for our target population. As a first step in preparing for this, we held a methods and statistics class, which was actually met with tremendous enthusiasm: almost the whole monastic core team showed up voluntarily for this class on their free day! We had a lot of fun talking about t-tests, anovas and different kinds of variables, and we even played around with Bayes Factors in JASP!

This trip also was one of lots and lots of discussions. Actually one of the things that became obvious to me this time around is that the most insightful discussions actually happen when you are working one-on-one, not when having a large group discussion. For example, I spent a good amount of time sitting with my monk-colleagues to translate the debates (well, they translated, I wrote down the translations). In the course of this process I learnt a lot of tricks that happen while debating, such as interpreting a question differently when you do not know the answer to this question.

behavioural testing in action

This time we also met for the first time some Western monks studying at Sera Jey. Both with them, and with some members of our monastic collaborator team who studied in the West, I had some really interesting conversations about how monastic university training differs from Western university training. One thing that differs is the extent to which it actually informs how one practises and lives one's life. In addition, the debate training is a tool that is very confrontational: if you do not know something in great detail then there is no place to hide. This has as a result that you learn to really understand things in depth, but probably also that you build up quite an emotional resilience to being continually challenged. In short, it was a very illuminating visit, and such a precious chance to get to hang out in the peaceful monastery environment and the amazing group of monastic collaborators!
Last but not least: our study was featured in the Times of India!

Where meditation meets reasoning: analytical meditation

Over the course of the last two years or so I have been studying, with several American colleagues, the practices of monastic debate and analytical meditation. As I am writing this I am once again in India for that study. It is a tremendous gift to be able to spend time in India working with a fantastic group of monks in the context of my work. During this visit, we had a short panel at the monastery on analytical meditation in which I shared some words about how we measure the brain with EEG, but most interestingly, two very bright monastic scholars shared textual and practical information about analytical meditation and debate. To ensure that I do not forget it, I decided to share it here on my blog (doing my best not to misrepresent what I have heard).

While analytical meditation (Tib. che gom) can be traced back to the historical Buddha (described for example in the King of Concentration sutra), and even Hindu saints before that time, it really became popular with the Buddhist saint Tsongkhapa. Tsongkhapa was the founder of the Geluk school of Tibetan Buddhism which has a strong focus on study and reasoning. Apparently there is also the criticism from other schools in Tibetan Buddhism that say that analytical meditation is just fake meditation invented by Tsongkhapa. Analytical meditation is complemented by stabilizing meditation, in which the mind is placed on an object and tries to stay there without moving. This stabilizing meditation is more well-known in the West, where meditating on the breath is a quite popular component of many mindfulness courses and interventions. However, it is said that such calm abiding does not really help to transform suffering in the long term--it can only give calm in the short term. Analytical meditation seeks to investigate the true causes of phenomena and thereby can lead to wisdom and new insight into the nature of phenomena.

However, neither type of meditation can exist without the other ones. Without stabilizing meditation, the analytical meditation cannot really thoroughly accomplished, because the mind is just too wild. Without analytical meditation, stabilizing meditation is just a brief respite from our crazy monkey mind (if we manage to get it quiet). Also interesting: in this tradition, meditation is referred to as familiarization (see also Dreyfus (2015) for an excellent discussion)--meditation is becoming thoroughly familiar with its object (such as the breath) by bringing your attention to it again and again.

Analytical and stabilizing meditation differ on various dimensions. As I mentioned above, the results of analytical meditation are more stable than the results of stabilizing meditation. While stabilizing meditation is only practised individually, analytical meditation can also be practised in smaller or larger groups, such as in the debates that we have been recording in the lab. While in stabilizing meditation, the body should remain still, in analytical meditation, it can also move (as it does in monastic debate).

This movement dimension is also the feature of analytical meditation that is fascinating to me, given my experience in dance. It seems like a genius way to make meditation palatable to young men in a monastery that have lots of physical energy: meditation in action! Analytical meditation is characterized by a continuous questioning of the topic at hand, looking at it from all directions and asking "why" and "how"? These questions then allow the practitioner to become more familiar with the topic at hand (traditional topics include concepts such as impermanence and interdependence) by thoroughly investigating it. Debating in the physical way that is used in the monastery (see this clip for an example makes it more interesting than just sitting down. In addition, the monks say that standing makes your thinking quicker and more clear. It brings all the senses together. Through a repeated investigation of concepts like impermanence, but also negative emotions like anger, slowly your mental patterns start to change, such that eventually thoughts of impermanence or patience come up more automatically in daily life, and in situations where you are about to become angry.

It seems to me that analytical meditation is worthy of more attention by contemplative neuroscientists. We have begun to do the first EEG studies and behavioral experiments. This surely is a slow process, frought with dangers of misunderstanding, but I think it is well worth our while. It is important not only because of potential applications in education or in therapeutic interventions to manage maladaptive thought patterns, but surely also because there is so much interest in science in the monastic community as well. I hope to be able to share some preliminary results in this space in the near future.

What I learnt about research on human trust

Last week I went to the Air Force Academy in Colorado Springs, United States to learn about research that is being done on trust. Mostly for the benefit of my future self, I will make an attempt at summarizing what I learnt about this somewhat foreign (to me) area of research. Trust is actually measured in lots of different ways, ranging from tightly controlled lab conditions to the messiness of the real world and foreign cultures. Trust models come larger from Roger Mayer. Fundamental components of trust appear to be ability, benevolence and integrity. In other words, you trust someone when you believe they can do things (ability), when you think they are meaing well (benevolence) and when you believe they act with integrity. Trust is most crucial in situations of risk, and when you are together with someone you trust, you are more willing to take risks in this relationship. Mei-Hua Lin discussed that trust depends on the amount of interaction you have had with a person, the similarity, affect, status, as well as situational factors. Mayer mentioned an interesting experiment to measure trust in a person, is to ask people how likely they would give ths person a project that is important to them when you cannot monitor them. Across the world, the integrity dimension appears to be the most important predictor of trust. Although mostly trust is considered to be positive, Alan Wagner is studying situations in which people overtrust. Most frequently people use Mayer's questionnaire for measuring trust, but another possibility is Rotter's Interpersonal Trust Questionnaire.

Trust in the laboratory can be related to confidence. It has been known for some time that confident testimony has greater influence, especially when it comes from people that also calibrate their confidence to their probability of being correct (Tenney et al., 2007). As such, you can examine trust in the laboratory by looking at how people take the advice of advisor that vary in how confident their advice is (see some cool new work by Yeung and Shea). It is apparently even possible to create computer models of trust, which update trust in an opponent on the basis of previous experiences (Juvinaet al., 2015). One interesting context in which these models were used were in peer-assisted learning of paired associates, in which your partner can inform your answers to the paired associates. In a slightly less cognitive lab setting, trust can be assessed by looking at people's facial expressions as they perform a task collaboratively (Social BART task). Even more, humans can extract trust from body odors, although this effect is modulated by gender. Extraction of social information from smell is also disrupted in people suffering from autism spectrum disorder.

Another dimension of trust occurs in teams of humans and robots collaborating. Antonio Chella thinks about whether recovery of trust can occur when we let a robot say "sorry". You can also look at how humans trust automation (e.g., in a factory) and look how often they notice failures of this automation, such as in the AF-MATB task. Apparently errors by the automated system can even elicit an error-related negativity ("oERN"). As the machine/factory makes more errors, people evidently trust it less. So in fact the reliability of one artificial agent affects how reliable we think another agent is: trust calibration.

On the other hand, do humans consider machines in the same ways as other humans? Jonathan Gratch looks at what aspects of robot behavior make us treat the robots like humans vs machines. Appararently the relevant dimensions are a sense of agency and displays of emotion--together he calls that mind perception. Humans treat robots unfairly and exhibit different emotions when they feel they are just machines. When you add emotions to the robot, people start to treat it more human-like. Apparently you can even decode from human brain activity whether people think they are dealing with humans versus machines. Also gaze is an important cue that humans use to decide whether to trust a robot. Angelo Cangelosi uses investment games to study how much people trust robots, and observed that people invest more in nice than in nasty naos. Amazingly enough, even rats prefer helpful robots over non-helpful robots! Also team interactions can be modelled with ACT-R, as Chris Myers' work on synthetic team mates shows.

Slightly less related to trust, but more to influence was work from Matt Lieberman, who showed that activity in the mPFC could predict behavior change in many contexts such as smoking cessation, wearing suncreen and more. Now what happens between two people are they are succesfully influenced? In experiments at Mount Jordan, Matt Lieberman showed that people's brains are more synchronized when they are watching a video together and are engaged and share a common reality. Also synchrony in speech (speech entrainment) can create social connectedness, because it is associated with increased positive feelings. However, this is not a simple phenomenon, because apparently it's not just more entrainment is better; rather, more variation in entrainment is better. The amount of speech entrainment seems to even affect whether people take advice from an avatar, although that is again a messy process. Less biological ways to measure connectedness include a questionnaire of social presence, which Kerstin Daubenhahn found to be sensitive to whether robots synchronized to the interaction with humans or not.

Other very interesting work by Clara Pretus looked at what is different in the brains of people who are wlling to fight and die for sacred values compared to people who don't. The main difference seemed to be less reliance on the dorsolateral prefrontal cortex for making these kinds of decisions. On a more positive note, very interesting work by Daniel Fessler showed how watching brief videos of prosocial behavior promotes real-world prosocial behavior (donations). The emotion of elevation appeared to be driving this real-world behavior. An important determinant in video content appeared to be reciprocation between the actors. Other happy news is a study by Adam Cohen who showed that when you ask people what kind of fictitious characters they would friend on Facebook, they trust Muslims and Christians equally, and the people they find most trustworthy are those who engage in costly religious practices (such as adhering to a kosher diet).

On a larger scale influence can be measured on twitter. People such as Vlad Barash have been developing network methods to study social contagion on this social media platform. Tim Weninger showed that social rating systems have a huge influence on how much other people like images/posts: to the extent that people are very poor at predicting what image will be more popular on social media, and popularity ratings are driven primarily by other users' ratings. In short, trust and influence are highly complex topics, on which very multidisciplinary research is done from many angles and perspectives.

Some useful tools I learnt about:

• facial analysis in OpenFace
• voice analysis with PRAAT
• Leanne Hirshfield has code for classifying brain data
• Online Experiment multitasking between N-back task and monnitoring an autonomic system

advice from successful scientists

I recently attended an evening with Spinoza laureates organised by the Young Academy of the University of Groningen, who shared their career advice.
Here you can find pictures and a collection of tweets about it: https://storify.com/mvugt/evening-with-spinoza-laureates-organized-by-young-

Running around a retreat with cell phone in hand--an opportunity to find spaciousness in busyness

I attended a retreat with my teacher Sogyal Rinpoche over Ascension day weekend. As I am writing this, after four intense days, I cannot help but feel tremendously spacious and relaxed. That is actually pretty odd given that I went there with a healthy dose of aversion, feeling tired and somewhat sick, and then spending the full four days of the retreat running around with phone in hand from 7:15 am to 9:00 pm. How could this be?

Sogyal Rinpoche teaching in Amsterdam (picture from Facebook Rigpa Nederland)

People attending the teaching; the dots dress on the far left is me (picture from Facebook Rigpa Nederland)

Going on retreat with Sogyal Rinpoche you know one thing, and that is that everything is going to change all the time. The only thing that is certain is the time at which the first session starts, and from that moment, everything is in a constant flow. The reason I was running around with the phone in my hand this time around was that I was the so-called "teaching services coordinator", who has to make sure that changes in the program are communicated to all the relevant teams, that every team is doing its work properly and that everyone has the materials they need and know what to do. In other words, this roles requires you to constantly be in touch with everything that is happening in all the different places, and to constantly be "in the change." It felt a bit like the retreat was an organism, and I as a teaching services coordinator had to feel what it was doing.

One of the big themes of the retreat was the "twit-twit mode"--a mode of being very busy and not being aware of what is actually happening. This mode of being sounded very familiar to me, and therefore the retreat was a fantastic way to work with it. The idea is that if you are connected to the awareness that is always present in your mind, then you will be able to hold your ground, instead of being swept away by busyness. I noticed that I was able to get a flavour of this at the retreat, because I was able to listen to the teachings on meditation in the nature of mind, and then immediately go back to working to hold the retreat. In that way it was quite easy to experiment with that awareness. Another beautiful idea that Sogyal Rinpoche used to describe this state of mind was that our ordinary mind would be consumed by the awareness of our nature of mind, instead of our ordinary being consumed by confusion. And when we are confused, we are easily burnt out because we cannot prioritize.

An important part of being aware is also being able to listen. Sometimes working together is benefits from less focus on results, and more focus on just getting together and listening. In fact, when you are with people, it's not what you say, but how you are. What is incredibly helpful is just to be open, loving, and compassionate, and then things may resolve themselves. This is both the case for situations between people and for our own mind: when we leave our mind in its pure awareness, then we can also find our true home. This is a true friend that is always there, whereas all other friends will some day disappear.

A further theme was the "throwaway culture" in which we live nowadays, in which we always want quick fixes yesterday, instead of going for more durable solutions. In particular, we are not willing to spend time and effort on a spiritual path that does not give very quick improvements. Moreover, we always want new things, and quickly get bored. I found that this analysis of present-day society rang true to my own experience, especially our consumerist society.

In addition, we often look for happiness outside ourselves. But actually, the source of happiness is inside ourselves: in a sense of joy and appreciation for what is always there. This ability to transform everything that happens into something good is the essence of Vajrayana buddhism. It reminds me of a Dutch poem: "if you look carefully, then you see that everything is colorful." Everything is in fact a display of clarity, joy, and wisdom, but it's so easy to forget that in the nitty-gritty of our everyday experience. Similarly, we can find joy in our work--even when we are very busy--when we are inspired by bodhicitta and caring for others. But if we just do the work to get ahead and to gain recognition then we will quickly burn out.

In summary, I felt the retreat was very much about bringing together the absolute teachings about the nature of awareness with the nitty-gritty of everyday life. I was able to get a taste of what happens when you get let go of the "twit-twit mode" and instead expand your awareness to everything that happens in the retreat, being in touch and not so much worried about little things. An inspiring way to find space in my own mind!

What happens when you try to transplant cognitive science to a Tibetan monastery

After a short night on the plane from Frankfurt to Bangalore, and a long drive to the monastery, we arrived safe and sound from hustling and bustling in India to the quaintness of a Tibetan monastery. Our mission? Trying to learn about the effects of the debating that Tibetan monks engage in on their cognitive and emotional functioning. But (obviously) none of it was as easy as that may sound.

cow in front of Sera Jey's science centre, where we conducted our studies

a visit by NIMHANS scientists

Doing research in India reminds you of how we take so many things for granted: drinkable water flowing from the tap, warm water to take a shower, steady internet, and electricity. The first few days, internet was mostly absent and electricity was intermittent. This meant we were really coding like monkeys, because apart from talking to our monastic collaborators, there was no distraction! We had brought a few tasks that we routinely use in our laboratories to study logical reasoning, working memory, and decentering (this is the ability to realize that your thoughts are just thoughts, and then to step outside a train of thought). We thought that translating the tasks and their instructions would be quite trivial, since it mostly involved common words like 'shoebox' and 'baseball bat.' But of course, shoeboxes and baseball bats are actually not that common in a Tibetan monastery! So we had to had long conversations about what the best Tibetan equivalent for a particular word would be. This really makes you realize how western-centric the psychology tasks that we normally use are! Another example of that came up in a working memory task that we use, in which you have to intersperse memorizing a visual object and making decisions about words. As it turns out, monks are really not used to doing two things at the same time (or maybe we should say: Westerners are chronic multitaskers). So is this actually a good measurement of working memory?

a testing session in action (with a visiting dog)

But doing research in India is not only being a crazy coding monkey: relationships with people are also very important. We received a visit from Prof. Shantala Hegde and Shivarama Varambally from the prestigious National Institutes of Mental Health And Neuroscience in Bangalore. They gave talks to the monastic core team about their work on how music affects emotions, and what goes wrong in patients with schizophrenia. And imagine, the tasks had to be improvised without slides because we still had no power! Nevertheless, we had a wonderful time together and learned a lot. We also were invited by the abbott of the monastery. What was very exciting was that some of the monastic collaborators actually presented the data and graphs single-handedly to the abbott, who was very inspired, and gave his blessing to the project. Not unimportant for a project that takes place at a monastery...

After a few days, and lots of struggles with Tibetan fonts, we got the tasks to show up on the little tablets we brought and we were able to start testing (victory!). We started doing our usual thing: trying to rush through the tasks to get in as much data as possible. However, this just doesn't work for the monks. As we slowed down, we realized the process was much more pleasant for everyone (and much more informative!) if we took a break after every task and asked the monks about their experience in doing the tasks--what worked, what didn't, whether it was related to things do would do in their debating or analytical meditation practice, and so on. We also quickly discovered the meaning of "Indian time"--groups of monks could show up 15, 30, 45, or even 60 minutes late! On the other hand, it is also quite rewarding when your participants tell you that a boring task is "good mind training." Even more rewarding was the fact that our monastic collaborators really manifested and single-handedly took charge of the testing sessions.

Monks taking charge of the testing sessions!

Altogether, it was a wonderful gift to hang out with my monastic collaborators once again. They are some of the most kind, funny, and joyful people I know. Quite illustrative for this was Easter, which fell in the period that we were visiting to conduct some studies. My collaborator Amir and I hid some chocolate easter eggs, and the monks playfully went around searching for them, and even rehiding some of them after they had found them!

How to activate students? Ideas for juicing up your teaching

me teaching during the Artificial Intelligence teacher of the year award ceremony. Picture by Niels Taatgen.

I recently attended a workshop at my university about innovations in teaching. There were several ideas that I thought were useful, so I decided to blog about them. Firstly, I attended a workshop on MOOCs by Tom Spits. There are more and more MOOCs. You can actually use those MOOCs to complement your classroom teaching. For example, you can use them in the flipped classroom: students watch the MOOC at home and then you spend the time in the classroom doing assignments and discussing difficult points. I think this makes a lot of sense, because sometimes lectures on MOOCs are excellent, and why would you waste your own time and effort on that? For example, I found this MOOC on methods and statistics excellent. I may actually refer students to that for some components of my Research Methods class. This could be also helpful for students who need a bit more explanation or background information. The only thing you have to watch out for is MOOCs taking place asynchronously with your class, so that students are not able to sign up and watch the lectures.

MOOCs can also be a helpful source of assignments. Of course you should check this with the MOOC teacher, but reusing some of these assignments can save you a lot of time (given that creating assignments takes *a lot* of time...). Moreover, many of these assignments even give automatic feedback, which can help students to practise the material well. Another potential use of MOOCs could be to remedy deficiencies. For example, in a multidisciplinary field like Artificial Intelligence, some students are weak on programming, others on psychology or machine learning. There are plenty of MOOCs in all these areas that students can do to remedy these deficiencies. Advantage of MOOCs over regular university courses is that they can be done at your own pace, and in your own time.

A significant challenge with MOOCs is finding the right one. One place to start is a MOOC search engine: moocse. There are also websites with reviews of MOOCs, e.g., class central. And of course you can follow your favorite academics on twitter and find out when they are teaching a MOOC.

Another teaching innovation are learning communities. mentimeter in the lecture to keep students engaged. And finally, a very simple thing you can do to have students take charge of their learning is to provide lecture slides before the lecture so that students can click links and explore during the lecture (of course this is also tricky, since it may be very distracting for the students...).

In summary, I got quite a few new ideas for enhancing active learning in my classroom. I am planning to check some of these ideas out in the next few weeks, when I am starting to teach again.

A new way of dieting or other habit changes: quantifying your wants

There is a cool new research project called qwantify.org in which some of my friends from the Mind and Life Institute participate. The project seeks to investigate what people think about and want and how those things influence each other. They do this study by means of a smartphone app, which everyone with a smartphone can download and try out. I downloaded it and have found it very insightful.

I often find myself wanting things, ranging from food to consuming media to--in fact--consuming anything. The app prompts you to investigate these wants. A few times a day, it buzzes you and asks you whether you want anything, a question that is immediately followed by questions like: how do you want to feel? (e.g., accomplished, comfortable, connected...).This is fascinating because it draws the connection between what you want immediately and why you might want that. For example, often I may want to check the twitter or consume other media. But why? Do I want to feel comfortable? Connected? Or actually, is feeling accomplished more important? If so, is social media the best thing to want right now? The app also asks you about your emotions and your social situation. For example, it asks you whether you feel content, angry, etc. It also asks you whether there are currently people around you, and whether you are interacting with them. Do you want more things when you are by yourself, or when with others?

After you have filled out a few prompts, the app starts to give you charts that index many different aspects of your behavior. For example, I found out that apparently, I am a pretty happy person, because my predominant emotions are content and happy. Also, want things more when I am alone than when I am with others. This makes sense to me, since when I am with others my mind is sufficiently occupied that I don't really need much more. Moreover, I want things more badly when I am in a bad mood compared to when I am in a good mood. Overall, I am pretty amazing that there are even such patterns in my own data. Quite interesting.

Most importantly, the app drew my attention to the kinds of choices I make in my life. Normally, when I want things, I unquestioningly follow those impulses (or resist them), but I don't investigate what is behind them, or whether they actually make me feel better. As a result of working with the app for a few days, I became more aware of what impulses make me really happy. For example, creating things and accomplishing things for others is what makes me happy, while my wants are usually quite the opposite: consuming things (mostly food and media). So, if I take a moment to reflect, I could actually become happier by following the longer-term goals of creating things for other people rather than simply following my mindless impulses... Do you want to try it out as well (and help science along the way)?

when you really need to dance... resources for doing ballet class/fitness at home

The holidays are coming up, which means that a lot of ballet classes are sadly not happening. Where can I still get my fix? For those occasions, I like to do ballet classes or workouts at home.

This is me doing a ballet class in the fitness centre of a hotel.

Since several people asked me about it, here is an overview of resources I really like for that:

• Lazy Dancer tips is a youtube channel with a lot of nice workouts, including this ballet barre for fitness and some 5-minute-beach-abs workouts. Good for dancers but also suitable for non-dancers
• For $15 you can buy a download of the barre and the centre this class by Kat Wildish, a ballet teacher from New York City specialising in adults. Her style is very easy to follow and focuses a lot on correct placement.
• Another youtube channel is Kathryn Morgan, who has some good workouts such as a pointe barre workout and a workout to strengthen your feet. She also holds online ballet classes, of which I really like the free stretch and strength class
• If you would like to improve your core and your form, floor barre is excellent. Yumiko sells a wonderful DVD
• On youtube, you can also easily dance along with Pacific northwest ballet school's Balanchine-style class
• Another DVD that I like a lot is one with Andrey Klemm, which is quite easy to follow, a good workout, and gorgeous dancers
• If you would like to improve your turns, then Finis Jhung has some good DVDs, although those are pretty pricy. I like the intermediate-advanced turning class, which unfortunately is no longer available

And of course, sometimes the most fun thing to do is just to put on some music that moves you and dance! 

does mindfulness mean we have no judgment whatsoever?

In a video on the bodhi Facebook page, Kimberly Poppe asks an important question here: can we suspend our judgment for a moment? What does this feel like? There has been a lot of controversy on mindfulness promoting non-judgmentalness and thereby condoning injustice, bigotry and violence in the world. I think what is shown here is that it is not about getting rid of judgments altogether, but rather taking a moment to first perceive what is, in a way that is as unbiased as possible, and then engaging in a way that is usually a lot more sane because you had a chance to think it over.

In this way, mindfulness can allow you to not become a zombie, but in fact allow you to see injustice more clearly and act in a more skilful way. By reducing the reliance on forming an opinion before you have had a chance to actually see, you can potentially go beyond your habits of thinking, especially those habits that are quite "sticky", i.e., those that involve your hopes and fears. Instead of running through patterns such as "I am a worthless..." or "s/he probably thinks I am a loser" or "these people are really stupid" again, and thereby making them stronger, we can also have a fresh look at what presents itself. Then we may be able to see different sides of a situation, and choose to do something different than the thing we usually do. So we can still have a strong opinion, and a strong commitment to act on an unjust situation, but now it is founded in a bit more of a grounded response.

Georges Dreyfus has a much more scholarly discussion of this topic: http://www.ahandfulofleaves.org/documents/Articles/Is%20mindfulness%20present-centred%20and%20non-judgmental%20A%20discussion%20of%20the%20cognitive%20dimensions%20of%20mindfulness_CB_Dreyfus_2011.pdf

review of the Society for Neuroscience conference

A few weeks ago I went to the big Society for Neuroscience conference in San Diego. This yearly conference usually draws around 30,000 neuroscientists. I was quite actively live-tweeting the conference, so for an impression of the current research on brain oscillations, decision making and mind-wandering, check out my storify: https://storify.com/mvugt/society-for-neuroscience-2016-review