Research Projects

Meta-research in Human-Computer Interaction focuses on understanding how research itself is conducted, evaluated, and evolves within the HCI community.

Text-to-image generation with deep generative models has made giant leaps towards becoming a mainstream phenomenon since OpenAI's introduction of the multimodal CLIP model in 2021. This explosion of interest was facilitated by the availability of executable notebooks on Google's Colaboratory in which non-expert users can execute machine learning models for free and without deep knowledge of programming. Users on Google Colab interact with generative models via input prompts. Prompts are short texts that describe the expected end result, such as 'a beautiful painting of a landscape'. The online community that quickly emerged around this novel way of synthesizing artworks found that the quality of the generated artworks can be increased by using so-called prompt modifiers in the input prompt, such as 'trending on artstation' or 'rendered by Unreal Engine'. 'Prompt engineering' or 'prompting' refers to the practice and skill of finding effective input prompts for a text-to-image model. This project investigates 1) the skill and practice of prompt engineering, 2) the emerging AI art community, and 3) the specific input prompts and modifiers used in this community.

Prompt engineering is a valuable tool allowing to generate insights in application areas where traditional methods, including supervised deep learning, are too complex and difficult to use. I have applied prompt engineering in a number of applications, including questiona-answering over documents using retrieval-augmented generation (RAG), personalized safety assessment and accessibility recommendations using RAG and geo-location data, and application of prompt engineering for mining insights in research.

Feedback systems have become ubiquitous. You can find them in many public places, such as airports and restrooms. With these systems, people typically can give feedback using color-coded buttons with smiley faces on a scale from left (good) to right (bad). Feedback is provided in response to a prompt, such as 'How do you rate the cleanliness of this restroom?' or 'How do you rate your experience?'. During my PhD, I became interested in eliciting more complex feedback from situated and online crowds. For instance, I created a systems to elicit feedback for digital artworks on a public display and I created CrowdUI, a system in which users can manipulate a web page and send the altered page as feedback to the web designer.

Creativity is the process that leads to an artefact deemed both novel and useful in a given context and timeframe. Supporting creativity is considered as one of the grand challenges in HCI and inherently exploratory and transdisciplinary. Machines fall short in the fundamental characteristics that humans excel in — recombination, analogical transfer and divergent thinking. As machines cannot match the human intelligence when it comes to creativity, applying crowd-powered creativity support tools is an underexplored and promising area of research. As the World Wide Web has thoroughly permeated our work practices, supporting creative work with the crowd has become feasible. Microtasks are fast to complete, incur a low cognitive load and require little expertise. Crowdsourcing is especially effective in situations that require human cognition and judgement for optimal decision-making, such as creative work. Moreover, creativity is a social process. Studies from psychology show that groups of people with diverse backgrounds provide high quality ideas and can outperform skilled experts. Crowds offer this diverse set of skills because they are heterogeneous, providing different contexts and backgrounds leading to diverse ideas. Situated and mobile crowdsourcing further allows to tap into a local crowd with specialized knowledge.

Folk theories represent the users' beliefs of how an AI-driven system works. Prior research has investigated folk theories as a lens on users' reasoning about the algorithmically-curated content of news feeds on social media platforms, primarily in interview studies and surveys. In this project, we research and develop interactive interfaces for the structured capturing and crowdsourced collection of folk theories at scale.