Evaluating Grant Proposals for the European Commission

This past Sunday night, I hopped on the Eurostar from London St. Pancreas–and in just over two hours I disembarked at Midi station in Brussels. I love that Chunnel!

I’ve spent the week working alongside other experts from around Europe to evaluate projects proposed for funding. This is an activity I am doing to develop more skills with regard to grant writing and program design.

img_2058

Aongus took the underground over to St. Pancreas Sunday night, to see me off as I boarded the Eurostar.

This is a job that requires a great deal of concentration. We’ve each been working for weeks–studying 30-page proposals, 7-8 of them per expert,  and then creating very detailed individual reports, comparing and compiling these into group reports, and then meeting face-to-face on-site in Brussels to discuss each proposal in depth. The scores we assign will be used to determine which organizations will receive funds to support doctoral and post-doctoral researchers.

 

Through this process, the European Commission and its Research Executive Agency (REA) provide detailed, specific feedback to applicants as well as numeric scores.

Many applicants succeed and receive financial support, but I’ll admit that with the sums provided, competition is fierce.

I believe this funding is well spent. It builds the capacity of researchers to do great work and learn important new skills. It yields results that make life and systems better at the individual, organizational, national, regional, and international levels. It produces valuable research results in a vast array of fields and disciplines.

The evaluation process is extremely important. It has to be done with extreme care. It is a huge amount of work, and the experts involved take the job very seriously.

img_2069

The evaluation itself is confidential, but pictures of Brussels I can share. 🙂

Many dozen experts have been involved this week, as reviewers and quality control officers. Our purpose is to deliver accurate and reliable results.

 

As a scholar from the States, I particularly value the feedback given to applicants in this process. Great care is taken to keep the scoring open, transparent, and fair, and to yield consistency from year to year as well as between proposals.

It’s a tight-knit process with a demanding timetable. And we’ve done remarkably well at staying focused and on track.

Why do I see the results of this process as valuable? In the U.S., fellowship and grant applicants rarely get feedback. I suspect it’s a result of the litigious nature of “American” society that funding agencies don’t want to open themselves up to questioning, and they won’t let applicants know what was seen as weak about the proposal. They will provide only very general feedback if any at all. I’ve had this experience with at least three different funding agencies in the USA. It was exceedingly frustrating and turned me off from wanting to keep bashing my head against a rock (even though I had a relatively high level of success winning grants for educational/learning sciences!).

img_2132

The plaza next to the building where evaluations are conducted. 

Working here at REA, our primary focus is on achieving accurate scores that can hold water. There’s much less paranoia on the part of the funders, in my opinion. In fact, there doesn’t seem to be the same fear of redress–in the case of any mistake, the program managers actually do want to address it in a way that is fair to the applicant. Transparency and proper channels for redress/appeal are foundational principles of the programs that REA funds.

 

Because REA’s process provides reliable feedback, I myself was able to improve one past proposal that wasn’t successful on its first submission. I was able to learn and to re-submit. By addressing the points raised in the first evaluation, I was able to secure funding the second time around!

In the United States, I’d have been left in the dark, making the same mistakes over and over again. In my experience (having submitted 3 unsuccessful proposals, 2 successful proposals, and one pending proposal to various  MSCA programs evaluated via REA), the European evaluation system is FAR better than the US system. A knowledgable colleague told me yesterday that the overhead costs for evaluating and managing/overseeing the quality of these MSCA programs is lower than typical of other similar programs worldwide.

img_2111

Dinner at Lyon!

I can’t say this work is pleasurable, but I do enjoy being here, working hard, and feeling satisfaction by week’s end. It’s sometimes bittersweet, though, as it is Thanskgiving and, also, yesterday would have been Dad’s 74th birthday. He died five weeks ago, right after my assignments for this job arrived. Therefore, I didn’t get to talk with him yesterday. And, since this particular review always falls on Thanksgiving week, I’m spending my fourth Thanksgiving Day in Brussels, missing turkey in the States with family yet again.

 

In the evenings of this evaluation week, however, I do enjoy dinner out with other experts and my walks through the city to the Grand Place and the Royal Arcade. Hopefully tonight, the Christmas Market will be up and running! It’s 6:40PM so I need to get going and pack up my things for the night.

On Monday night I went out and I got to enjoy Moules et Frites at Lyon.

If you are capable and interested in serving as an expert evaluator, you can set up a personal profile in the Participant Portal (see instructions at https://ec.europa.eu/research/participants/docs/h2020-funding-guide/experts/experts_en.htm). When REA needs your expertise, they may well send you an invitation to serve.

 

Learning London: Fabulous February

img_0019

Visiting the Tower of London and the Crown Jewels.

I maintained a quick pace of work during month two of my Marie Curie Research Fellowship at University College London.

I have a grant-funded training fellowship and my activities are designed to build skills in specific areas, organized around the six “work packages” outlined below. This blog summarizes my academic achievements from February 2018.

Work Package 1: Qualitative Research

Analyzed data for a policy paper to improve women’s access to STEM education in Ireland. Located relevant policies from Poland to use as precedents and translated them into English with the help of Google Translate.

Prepared and submitted two draft papers to the Association for the Study of Engineering Education (ASEE) with:

  • Emerging Findings of a Longitudinal Study of Middle Eastern Women’s Experiences Learning Engineering Abroad
  • A model for spurring organizational change based on faculty experiences working together to implement Problem-Based Learning

Met with UCL’s Dr. Inês Direito to discuss how I can help with a future qualitative research study of women at UCL.

img_8655

Brushing up on research methods.

Collected follow-up interviews in Ireland (with 2 Middle Eastern and 1 Irish student) and connected with researchers in Portugal who will collect interview data to add to the Portuguese data I’ve collected with Dr. Bill Williams.

Reviewed literature relevant to my own research (Perry, 1999; Evans, Forney, & Guido-DiBrito, 1998; MIT case study on UCL’s Integrated Engineering Program)

Brushed up on methods for Qualitative Data Analysis by reading three chapters of Grbich, 2012

(Work Package 2: Mixed-Methods Research will build on findings of WP1, eventually.)

Work Package 3: Special Focus Journal Issue

img_8540

Visiting the Institute of the Arab World in Paris.

I pitched the idea for a special focus issue to the Editor of the IEEE Transactions on Education (ToE) on the topic of my current fellowship, got it accepted, assembled an all-star panel of guest editors for the issue, wrote and distributed the call for papers. It’s posted here, in case you or someone you know has interest in the subject of Using Design Projects to Spur Cognitive Development of Students in Science and Engineering.

I continued work on IEEE ToE’s upcoming special focus issue on social-cultural diversity. I saw one manuscript through to completion and worked closely with the Administrative Editor and Chief Editors to help our team of guest editors get the schedule moving forward, since work had stalled. I’m hoping for publication in August 2018, if we can keep our momentum going.

I only promised one special focus issue in my grant proposal–but why not aim to deliver two?

Work Package 4: Outreach (including Peer Reviews)

I drafted and submitted a 1000-word entry for The SAGE Encyclopedia of Higher Education on the topic of Problem-Based Learning and its use in engineering disciplines.

Reviewed ten proposed activities for a new children’s book by Usborne Publishing called “Scribble Engineering” and submitted an evaluation to the publisher.

Peer-reviewed a manuscript for the European Journal of Engineering Education and two others for IEEE ToE.

The Editor in Chief of the IEEE Transactions on Education (ToE) appointed me to the journal’s editorial board, so now I’m a full Associate Editor with a three-year term. In this job, I’m giving feedback to the Editor as to which manuscripts to forward though the peer review process and I’m managing the peer review process for one new manuscript each month.

Working with the Research in Engineering Education Network (REEN) in February involved a sub-committee meeting to edit guidelines and application forms for people interested in hosting a future Research in Engineering Education Symposium (REES, in 2021, 2023, or 2025). Our next symposium will be held in Cape Town, South Africa July 10-12, 2019. I also attend the monthly online meeting of REEN and followed up by contributing to the REEN Discussion Forum on LinkedIn, inviting colleagues to join the discussion.

img_7983-2

Meeting with Civil Service professionals in Dublin.

Work Package 5: Research Training

During this fellowship, I aim to develop skills in supervising PhD students and post-graduate level research teams. This month, I met face-to-face with four of the six Irish Civil Service professionals who I’m sponsoring in the training module they are taking related to policy and research.

Built new skills by attending:

img_8097

Hearing Louise Archer (left) and Angela Saini (right) speak at UCL’s Centre for Engineering Education.

I met with UCL’s Dr. Claire Ellul who teaches Building Information Modeling (BIM) and Geographic Information Systems (GIS) at UCL.

Joined the UK Society for Research in Higher Education (SRHE) and registered for future training sessions.

Met with Prof. Rao Bhamidimarri, VP of London South Bank University, about the engineering education center he runs, the STEM secondary schools he created, and PhD thesis projects I may be able to advise.

Work Package 6: Management

Met with my supervisor, Prof. Nick Tyler, for my one-month probationary review and to keep my Career Development Plan up to date.

img_8290

Studying urban design at London’s Building Centre.

Ongoing professional development:

Attended lectures at the Bartlett School of Architecture:

  • Fabio Gramazio of ETH Zurich and Gramazio Kohler Research
  • Jeremy Till, Head of College and Pro-Vice Chancellor at Central Saint Martins, University of the Arts London
img_8299

Transit maps at the Building Centre.

Visited and studied at:

 Other fronts

I had a bit of time left over for fun and adventure. I joined the UK’s Art Fund, which provides free or reduced entry prices at about 240 cultural sites in the UK. I also:

  • Using comp time, I took a three-day weekend in Paris to visit two lovely retired linguistic professors, Prof. Nancy Stenson from the University of Minsseota and Prof. Arthur Spears from CCNY. It was my first time through the Chunnel and my first time to meet Arthur, a friend of Nancy’s from grad school!
  • 3683f5c0-dea1-4161-8231-322fe514ca31

    Professors Arthur Spears, Nancy Stenson, and Shannon Chance in Paris

    Cheered on my partner, Aongus Coughlan in completing his coursework (in health, safety and legal aspects of building construction in the UK) and securing necessary certifications. He found a job after a grueling one-day search—he CVs emailed on Monday, interviewed on-site Tuesday, accepted a job offer on Wednesday!

  • Visited former colleagues and students in their bridge projects class at DIT during my Febraury research trip to Dublin.
  • Kept up my yoga and swimming, and at least 10,000 steps 6 of 7 days per week.
  • Celebrated my birthday with a massage, the play “Beginning” on the West End in London, pints out with my electrical engineering colleagues in Dublin, and a Turkish Bath at Ironmonger Row Baths in Islington.
  • Kept up with the achievements of my former architecture students via Facebook and LinkedIn. I’m thrilled with their achievements—books launched, exams passed, registrations earned, lives well-lived. For instance, I saw both The Shape of Water and Black Panther – the second being a movie to which my former students contributed.
  • We played in the snow on the last day of February, since the “Beast from the East” closed Dublin Airport and prevented a trip over to Ireland for research and speaking.