2  Methods

As noted above, we began by developing the research questions that the leadership team and community manager posed to better understand the Astropy community. In addition to work on tracking enegement and DEI, we constructed a survey that would address the research questions, going through several iterations of refining the survey questions as a team. We put the survey into Typeform (an online survey platform) and pretested the questions with two members of the Astropy community. These pretesters were asked them to take the survey with a member of the research team watching their screen. We asked them to “talk out loud” while filling out the survey, sharing any confusion, challenges or questions they had. Pretesting is a well known method of ensuring that the survey works for the community and it helps the research team troubleshoot problems with wording or question order before the survey goes to the wider community. From these pretests, we gained insights that changed the wording and order of a few survey questions as well as the suggestion of at least one additional question.

To recruit survey responses, we sent an email to the google group astropy-dev, the Astropy slack, the Facebook group (Python users in Astronomy) and engaged on X / Twitter as well. We did several follow up emails to give folks several opportunities to answer the survey, giving over a week and a half to respond. We ended with a total of n=98 of respondents to the survey with one duplicate.

Data quality is slightly impacted because the first 7 respondents had an erroneous logic loop in the survey software that made it impossible to continue unless answering one question in a particular way. We will note where this impacts the results, but we do not believe it impacts the findings significantly.

2.1 A Note About Qualitative Research

We qualitatively analyzed the open survey responses by annotating each one with codes (think of qualitative coding as tagging or annotating the data). We then grouped together like responses and found themes in the data. A few things to note about this type of analysis.

In this report we will share the themes that emerged in the data with a short description and a number of quotes that show examples of the theme. For the Astropy community, we elected to offer more quotes than we might in a non-academic white paper because we believed the raw data of the survey responses would be of interest to this community.

We omitted redundant or repetitive responses, but in qualitative research we do not omit data if it contradicts other data. This means that if there is variation in a theme - meaning that people in the community disagreed - we include both perspectives and note the contradiction.

Following the conventions of qualitative research, we do not edit what was written by respondent when we quote them, even if that means that there are typos, mistakes or misspellings in the responses. This done is to preserve the voice of the respondent.