Category: Uncategorized

Author: Caroline Appert

Thank you to everyone who participated in the CHI post-conference survey, which was open for approximately two weeks after the conference (from May 29 to June 10). Overall, 613 people participated in the survey, representing approximately 15% of the total number of CHI attendees (3,995 in total). This compares with 17% of the total number of attendees in 2023.

This article provides a concise overview of the survey findings. The interpretations are based on a thematic analysis of the open comments (the thematic analysis was done by Caroline Appert and Simone Barbosa).

Attendees’ experiences

The following chart shows the distribution of respondents based on their type of attendance at the conference. The green bar represents respondents who attended the conference in person, while the brown bar represents those who attended virtually. Among the respondents, approximately 15% attended the conference virtually. This percentage is close to the actual virtual attendance, which was approximately 17% for CHI ‘24.

In the survey, 78 respondents shared their reasons for choosing virtual attendance over on-site attendance. These reasons are reported in the table below (respondents were allowed to select multiple reasons).

What factor(s) did influence your decision to attend the conference virtually rather than in person?
————————————————————————-
| 56.41% | My decision to attend virtually was influenced by ecological reasons. |
| 14.10% | My decision to attend virtually was influenced by cultural reasons. |
| 03.85% | My decision to attend virtually was influenced by the Covid-19 pandemic.|
| 42.31% | My decision to attend virtually was influenced by personal reasons. |
| 38.46% | My decision to attend virtually was for other reasons. |

The distribution of responses to the question “How valuable was your CHI experience?” is presented below, using a breakdown by attendance type: virtual attendance on the left and on-site attendance on the right. The chart shows that on-site attendance was highly valued, mostly because it offers opportunities for networking (connecting with people and meeting new people) and for getting up-to-date with scientific content that is timely and of high quality. In contrast, feedback from virtual attendees was less positive. In their open comments, virtual participants highlighted the lack of networking and interaction opportunities (both as attendees and presenters), the fact that there was little to participate in and insufficient synchronous/hybrid interactions for the sessions they could participate in.

The survey included a series of questions where respondents rated various aspects of the conference, including sessions and activities (such as papers, demos, posters, panels, SIGs, student competitions, workshops, and keynotes) as well as conference events (such as receptions, exhibits, job fairs, and town hall meetings).

Overall, attendees gave mostly positive feedback about the sessions and activities, with all venues receiving a similar level of appreciation. However, some on-site attendees noted that activities in the exhibit hall could have been better. The most frequently mentioned issues included the lack of conviviality in the exhibit hall (quality of food and price of drinks during the reception), the perceived suboptimal poster layout for visibility and interactions, the limited number of demos during Interactivity, and the reported lifelessness of the Job Fair session. Virtual attendees, who did not experience the exhibit hall, gave consistently positive feedback across the sessions and activities they accessed. However, the low number of responses collected from virtual attendees makes it difficult to draw general conclusions.

Respondents also had the opportunity to evaluate how well different tools and aspects of the conference met their needs. This included the conference website, the Progressive Web App (PWA) used to navigate the program, and the different types of content offered by the conference, such as streamed content, recorded content, and the Sli.do application for handling questions and answers (QAs). The distribution of their responses is presented below.

Feedback regarding the PWA was mixed. Some attendees liked it, while others reported issues with getting the app to work on some mobile devices and found the navigation model confusing for such a large program. Attendees also suggested that the information on the conference website, especially instructions for presenters and remote attendees, should be better structured and clearer. Respondents mentioned that some information was included in specific blog posts that were not easy to find. To promote a better balance between remote and on-site participants, attendees were instructed to submit questions through the Sli.do app regardless of their mode of attendance. However, comments revealed that on-site attendees found this experience to be not fluid, and they quickly avoided using the app to ask questions during sessions. As a result, the appreciation of the Question and Answer experience was mixed. Finally, feedback regarding asynchronous presentations was notably negative, likely driven by the lack of interaction possibilities with pre-recorded videos.

To enable asynchronous presentations, presenters had to submit pre-recorded videos. For papers, case studies, and alt.chi presentations, the duration of these videos was capped at 15 minutes. For posters (including LBW, student competitions, and the Doctoral Consortium) and demos (Interactivity), the duration was set to 3 minutes. In the survey, we asked respondents to provide their feedback about whether these durations were suitable from both attendee and presenter viewpoints.

The charts below illustrate how well these durations aligned with what people expect either as attendees or as presenters. The large grey areas in the bars indicate that a significant proportion of attendees did not watch these videos. Regarding the duration of the videos, the trend for the 15-minute videos is that they are generally considered either appropriate or too long, while the trend for the 3-minute videos is that they are either appropriate or too short. Compared to last year, where long videos were 8–10 minutes long, a larger proportion of respondents this year mentioned that shorter videos would be preferable. In the comments, some attendees mentioned that the duration of videos should match the duration of on-site talks. As presenters, they also wish these videos to take less time to prepare and would like to receive instructions further in advance.

Accessibility

Among the respondents, approximately 2% requested an accessibility-related feature at the conference. The chart below indicates that these requests contributed to a positive experience. Very few accessibility-related issues were reported by CHI participants in the survey. Some of the issues reported included lack of food options, lack of air-conditioned spaces or difficulty using the PWA.

Transportation

The question about local travel in Hawai’i during the conference revealed that the majority of local travel was made on foot, and by car to a lesser extent.

How did you undertake local travel in Hawai’i while attending the conference?
———————————————————-
| 90.26% | On foot |
| 42.69% | By car |
| 03.48% | By metro (skyline) |
| 02.78% | By bike |
| 12.76% | Other |

Willingness to attend CHI ’25

In response to the final question on the likelihood of attending CHI ’25 in Yokohama, Japan, the majority of respondents will probably or definitely attend the conference physically, although some indicated that this would depend on the acceptance of papers or the availability of funds. The chart also illustrates that a very small number of respondents indicated that they would attend virtually, with a few stating that virtual attendance was not worth the effort.

As a final note, we would like to thank all the respondents who took the time to provide valuable feedback. We have carefully read and analyzed all the detailed comments, categorizing them into relevant themes and summarizing the key points. The input and insights shared by the attendees are very valuable to everyone involved in SIGCHI, the CHI Steering Committee, and the overall organization of CHI.

Author: Caroline Appert

The CHI post-conference survey ran for two weeks after the conference. We closed it on May 14. Overall, 832 persons took part to the survey, accounting for approximately 17% of the total number of CHI attendees (4,722 in total). This article gives a concise overview of the survey findings. Interpretations are based on a thematic analysis conducted to analyze the responses received for the open-ended questions.

Attendees’ experiences

The following chart depicts the distribution of respondents based on their attendance at the conference. The green bar represents respondents who attended the conference in person, while the brown bar represents those who attended virtually. Among the respondents, approximately 10% attended the conference virtually. This percentage is slightly lower than the actual percentage of virtual attendance, which was approximately 17% for this year’s conference.

The distribution of responses to the question “How valuable was your CHI experience?” is presented below, according to a breakdown by attendance type: on-site attendance on the left and virtual attendance on the right. The chart shows that on-site attendance was highly valued, primarily due to the opportunities for networking and the quality of scientific content presented at the conference. In contrast, feedback from virtual attendees was less positive. Analysis of respondents’ comments revealed two main reasons for their dissatisfaction: 1) remote presenters did not receive sufficient visibility during the on-site conference, and 2) virtual attendees expressed a sense of isolation as they were unable to engage and interact with other conference participants. These factors contributed to the less positive evaluation of the CHI experience among virtual attendees, as compared to those who attended in person.

The survey included a series of questions where respondents individually rated various aspects of the conference, including sessions and activities (such as papers, demos, posters, panels, SIGs, student competitions, workshops, and keynotes) as well as conference events (such as receptions, exhibits, job fair, and town hall).

Overall, attendees provided mostly positive feedback about the sessions and activities, with all venues receiving equal appreciation from on-site attendees. However, comments from some on-site attendees highlighted the need for improvements in the organization of the poster venue. Specifically, respondents mentioned that the spatial layout of the hall can be confusing, and the program schedule too packed, leaving insufficient time for interaction with poster presenters. This issue was further reflected in a subsequent question, where 32% of respondents stated that they were unable to see or interact with some posters or demos they had intended to.

In contrast, feedback from virtual attendees was consistently negative across all venues, although some comments mentioned that certain SIGs managed the hybrid experience well. Concerns were also expressed regarding the Job Fair event, as it lacked significant engagement. Additionally, comments revealed that some attendees expected the town hall sessions to allocate more time for discussing important topics, such as the environmental costs associated with the conference location.

Respondents had the opportunity to evaluate how well different tools and aspects of the conference met their needs. This included the conference website, the Progressive Web App (PWA) used to navigate the program, as well as the different types of content offered by the conference, such as streamed content, recorded content, and on-site presentations. The distribution of their responses is presented below.

Overall, people highly appreciated the on-site sessions. However, ratings for streamed and recorded content were lower, which is likely due to the limited availability of such content. Regarding the PWA, feedback was mixed, with some attendees expressing their satisfaction with its features, while others expressed certain concerns. Usability issues that were brought up by respondents have been compiled and will be taken into account by SIGCHI to the best of their abilities.

To accommodate the hybrid format, presenters were requested to create pre-recorded videos, which has become a standard practice. For papers, case studies, and alt.chi presentations, the duration of these videos was set to 7-to-10 minutes. For posters (including LBW, student competitions, and Doctoral Consortium) and demos (Interactivity), the duration was set to 3 minutes. In the survey, we asked respondents to provide their perspective on whether these durations were suitable, considering both the attendee and presenter viewpoints.

In the charts below, the large grey areas in the bars indicate that a significant proportion of attendees did not watch these videos. This observation is particularly pronounced for poster presentations. This is not very surprising given that a majority of survey respondents attended the conference on-site. However, many comments expressed satisfaction towards these videos, emphasizing their role in facilitating asynchronous consumption of content. It is worth noting that there were some comments that highlighted the amount of work involved in producing these videos for presenters.

Regarding the duration itself, which was the primary focus of these questions, a majority of respondents assessed the current durations as suitable (indicated by the blue bar in the charts). Very few respondents expressed a preference for shorter videos (represented by the light purple bar). We can conclude that the current video durations should not be reduced.

Evolution of the selection process

The survey included a series of questions about the selection process at CHI. In particular, we were interested in the comparison between the previous system based on rebuttals and the current system based on revise and resubmit (R&R).

The first question aimed to determine the percentage of respondents who had previously submitted a research paper and their experience with each system. Notably, over 40% of the respondents had submitted papers using both systems, enabling them to provide insightful comparisons as users of both approaches.

Have you already submitted a research paper to CHI?
————————————————-
| 43.06% | Yes, in both the previous system and the current system |
| 20.68% | Yes, in the current system (R&R) only |
| 10.73% | Yes, in the previous system (rebuttal) only |
| 26.44% | No |

The charts below provide a comparison between the two systems, based on different criteria. Green bars are when rebuttals are considered better, brown bars when R&R is considered better, and grey is “no difference”.

On the first line, we consider 1) the quality of papers, 2) the quality of the dialogue between authors and reviewers and 3) the quality of reviews. Overall, the quality of the review process is perceived as better with R&R than with rebuttals.

On the second line, we can observe the drawbacks associated with R&R. Specifically, the charts illustrate how respondents assessed the workload for authors, reviewers, and Program Committee (PC) members. The large green bars indicate that, in comparison to the rebuttal system, R&R increases the workload for all participants involved in the process.

The survey concluded with a final question that aimed to assess respondents’ overall preference between the two systems. The results indicate an inclination towards the revise and resubmit (R&R) system:

Overall, what system do you prefer?
—————————
| 40.72% | R&R |
| 19.94% | Rebuttals |
| 39.34% | No preference |

Accessibility

Among the respondents, approximately 2.79% requested an accessibility-related feature at the conference. The survey also explicitly inquired if attendees had encountered negative experiences in terms of accessibility. The chart below gives an overview of the responses, revealing that some respondents did report negative experiences.

Upon analyzing the comments provided in response to this question, the negative feedback is primarily attributed to the challenges faced by virtual attendees in interacting with the conference. Regarding experience on site, a few comments mentioned how overstimulating it can be for some individuals.

The survey included a couple of questions regarding positive experiences and suggestions for improving accessibility. Respondents provided valuable insights and mentioned several positive experiences that contributed to their overall satisfaction. Specifically, they highlighted the availability of sign language, captions for content, the presence of a quiet (desensitization) room in the conference center, and support for families (childcare).

Furthermore, respondents also provided suggestions for enhancing accessibility. In terms of virtual attendance, they want more support to interact with attendees and presenters on site and remote. As for the on-site experience, attendees expressed the desire for a wider range of food options with clear indications regarding their composition. Additionally, they would like more seating areas where they can rest and relax.

Transportation

The series of questions related to transportation revealed that approximately half of the attendees arrived by train (note though that a large proportion of respondents come from Europe). During the conference, participants either traveled on foot or took the train. This was greatly facilitated by the provision of complimentary passes for public transportation in Hamburg, which were included with the conference registration. The respondents expressed a high level of satisfaction regarding this initiative, with half of them stating that it had influenced their travel plans in some way.

How did you travel to Hamburg? (select all that apply)
—————————————-
| 58.08% | Plane |
| 53.48% | Train |
| 5.43% | Car |
| 4.04% | Other |

How did you travel around Hamburg while attending the conference?
——————————————-
| 82.61% | On foot |
| 73.71% | By train |
| 6.40% | By car |
| 3.62% | By bike |

Willingness to attend CHI ’24

In response to the final question about the location of the CHI ’24 conference, we received a diverse range of opinions. When asked about the likelihood of physically attending the conference in Hawaii, respondents were divided. Approximately half of them expressed a strong interest and stated that they would like to go or were likely to attend. On the other hand, the remaining half indicated that they would not attend or would prefer to participate virtually.

In the free comments section, respondents raised several concerns regarding the conference being held in Hawaii. These concerns primarily revolved around the carbon footprint associated with long-distance travel, the time and cost involved in reaching the destination, and the potential negative impact on the local communities in Hawaii.

As a final note, we would like to thank all the respondents who took the time to provide valuable feedback. We have carefully read and analyzed all the detailed comments, categorizing them into relevant themes and summarizing the key points. The input and insights shared by the attendees are very valuable to everyone involved in SIGCHI, the CHI Steering Committee, and the overall organization of CHI.