Assistive robotic arms can support people with disabilities in various activities to offset the negative effects of the direct care worker crisis. In this paper, we demonstrate a semi-autonomous robot-assisted feeding system (RAF-FF), the first of it's kind to support the direct acquisition of finger foods.

RAF-FF delivers finger foods and a drink to the user by processing visual and auditory inputs with foundation models (GPT-4o, DINO-X). We evaluated this system with a laboratory study of seven able-bodied participants and one participant with a spinal cord injury (SCI). Participants selected three finger foods of a menu of seven different finger foods to be fed, along with a drink of their choice. During trials RAF-FF processed natural language commands to feed desired food items, or give the user the drink.

Overall, the robot successfully delivered a food item or drink on 94% of feeding cycles with cycles lasting from 28 to 96 seconds. Users generally rated the RAF-FF high on the System Usability Scale (score of 97.5 for SCI participant and mean 79.3 for able-bodied participants). Participants stated in post-trial interviews that they would use the robot at home but would like to have more control over the system's timing, and have more transparency into RAF-FF's decisions.