Vision-Based State Estimation for Underactuated Grippers

Marissa Jonker; Jan de Jong; Wesley Roozing; Wouter Hakvoort

doi:10.1016/j.ifacol.2025.10.200

Vision-Based State Estimation for Underactuated Grippers

Marissa Jonker^*
, Jan de Jong
, Wesley Roozing
, Wouter Hakvoort^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

4 Downloads (Pure)

Abstract

Grasping and handling of delicate objects is a challenge, especially in agri-food. The required force sensing to carefully handle these objects is ideally not done by tactile sensors, to upkeep cleanability and easy replacement of the grippers. To avoid the use of such sensors, we propose to estimate the contact forces from the actuator force and the configuration of the gripper using computer vision. We track integrated fiducials to obtain a pose estimate using an Extended Kalman Filter. By using a redundant set of fiducial markers, the method is robust to partial occlusions. We quantify its performance on an underactuated compliant gripper, which has a range of motion of 272 mrad for the joint angles, and 17.3 mm for the actuation phalanx movement. The results show we are able to estimate the configuration with an RMSE of approximately 29 and 17 mrad for the gripper angles, and 0.67 mm for the actuation phalanx displacement.

Original language	English
Title of host publication	14th IFAC Symposium on Robotics ROBOTICS 2025
Subtitle of host publication	Paris, France, July 15-18, 2025, Proceedings
Editors	Hyunjin Choi
Publisher	IFAC
Pages	79-84
Number of pages	6
DOIs	https://doi.org/10.1016/j.ifacol.2025.10.200
Publication status	Published - 2025
Event	14th IFAC Symposium on Robotics, ROBOTICS 2025, held jointly with the 10th IFAC Symposium on Mechatronic Systems, MECHATRONICS 2025 - Sorbonne University, Paris, France Duration: 15 Jul 2025 → 18 Jul 2025 Conference number: 14 https://ifac2025-msrob.com/

Publication series

Name	IFAC-papersonline
Publisher	IFAC
Number	18
Volume	59
ISSN (Print)	2405-8971

Conference

Conference	14th IFAC Symposium on Robotics, ROBOTICS 2025, held jointly with the 10th IFAC Symposium on Mechatronic Systems, MECHATRONICS 2025
Abbreviated title	ROBOTICS 2025
Country/Territory	France
City	Paris
Period	15/07/25 → 18/07/25
Internet address	https://ifac2025-msrob.com/

Keywords

gripper
underactuated
Vision-based state estimation

Access to Document

10.1016/j.ifacol.2025.10.200Licence: CC BY-NC-ND

Conference paperFinal published version, 1.48 MBLicence: CC BY-NC-ND

Cite this

@inproceedings{699a5acb82e94ee9b891638bbd22144e,

title = "Vision-Based State Estimation for Underactuated Grippers",

abstract = "Grasping and handling of delicate objects is a challenge, especially in agri-food. The required force sensing to carefully handle these objects is ideally not done by tactile sensors, to upkeep cleanability and easy replacement of the grippers. To avoid the use of such sensors, we propose to estimate the contact forces from the actuator force and the configuration of the gripper using computer vision. We track integrated fiducials to obtain a pose estimate using an Extended Kalman Filter. By using a redundant set of fiducial markers, the method is robust to partial occlusions. We quantify its performance on an underactuated compliant gripper, which has a range of motion of 272 mrad for the joint angles, and 17.3 mm for the actuation phalanx movement. The results show we are able to estimate the configuration with an RMSE of approximately 29 and 17 mrad for the gripper angles, and 0.67 mm for the actuation phalanx displacement.",

keywords = "gripper, underactuated, Vision-based state estimation",

author = "Marissa Jonker and \{de Jong\}, Jan and Wesley Roozing and Wouter Hakvoort",

note = "Publisher Copyright: Copyright {\textcopyright} 2025 The Authors.; 14th IFAC Symposium on Robotics, ROBOTICS 2025, held jointly with the 10th IFAC Symposium on Mechatronic Systems, MECHATRONICS 2025, ROBOTICS 2025 ; Conference date: 15-07-2025 Through 18-07-2025",

year = "2025",

doi = "10.1016/j.ifacol.2025.10.200",

language = "English",

series = "IFAC-papersonline",

publisher = "IFAC",

number = "18",

pages = "79--84",

editor = "Hyunjin Choi",

booktitle = "14th IFAC Symposium on Robotics ROBOTICS 2025",

url = "https://ifac2025-msrob.com/",

}

Jonker, M, de Jong, J , Roozing, W & Hakvoort, W 2025, Vision-Based State Estimation for Underactuated Grippers. in H Choi (ed.), 14th IFAC Symposium on Robotics ROBOTICS 2025: Paris, France, July 15-18, 2025, Proceedings. IFAC-papersonline, no. 18, vol. 59, IFAC, pp. 79-84, 14th IFAC Symposium on Robotics, ROBOTICS 2025, held jointly with the 10th IFAC Symposium on Mechatronic Systems, MECHATRONICS 2025, Paris, France, 15/07/25. https://doi.org/10.1016/j.ifacol.2025.10.200

Vision-Based State Estimation for Underactuated Grippers. / Jonker, Marissa; de Jong, Jan ; Roozing, Wesley et al.
14th IFAC Symposium on Robotics ROBOTICS 2025: Paris, France, July 15-18, 2025, Proceedings. ed. / Hyunjin Choi. IFAC, 2025. p. 79-84 (IFAC-papersonline; Vol. 59, No. 18).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Vision-Based State Estimation for Underactuated Grippers

AU - Jonker, Marissa

AU - de Jong, Jan

AU - Roozing, Wesley

AU - Hakvoort, Wouter

N1 - Conference code: 14

PY - 2025

Y1 - 2025

N2 - Grasping and handling of delicate objects is a challenge, especially in agri-food. The required force sensing to carefully handle these objects is ideally not done by tactile sensors, to upkeep cleanability and easy replacement of the grippers. To avoid the use of such sensors, we propose to estimate the contact forces from the actuator force and the configuration of the gripper using computer vision. We track integrated fiducials to obtain a pose estimate using an Extended Kalman Filter. By using a redundant set of fiducial markers, the method is robust to partial occlusions. We quantify its performance on an underactuated compliant gripper, which has a range of motion of 272 mrad for the joint angles, and 17.3 mm for the actuation phalanx movement. The results show we are able to estimate the configuration with an RMSE of approximately 29 and 17 mrad for the gripper angles, and 0.67 mm for the actuation phalanx displacement.

AB - Grasping and handling of delicate objects is a challenge, especially in agri-food. The required force sensing to carefully handle these objects is ideally not done by tactile sensors, to upkeep cleanability and easy replacement of the grippers. To avoid the use of such sensors, we propose to estimate the contact forces from the actuator force and the configuration of the gripper using computer vision. We track integrated fiducials to obtain a pose estimate using an Extended Kalman Filter. By using a redundant set of fiducial markers, the method is robust to partial occlusions. We quantify its performance on an underactuated compliant gripper, which has a range of motion of 272 mrad for the joint angles, and 17.3 mm for the actuation phalanx movement. The results show we are able to estimate the configuration with an RMSE of approximately 29 and 17 mrad for the gripper angles, and 0.67 mm for the actuation phalanx displacement.

KW - gripper

KW - underactuated

KW - Vision-based state estimation

UR - https://www.scopus.com/pages/publications/105023086834

U2 - 10.1016/j.ifacol.2025.10.200

DO - 10.1016/j.ifacol.2025.10.200

M3 - Conference contribution

AN - SCOPUS:105023086834

T3 - IFAC-papersonline

SP - 79

EP - 84

BT - 14th IFAC Symposium on Robotics ROBOTICS 2025

A2 - Choi, Hyunjin

PB - IFAC

T2 - 14th IFAC Symposium on Robotics, ROBOTICS 2025, held jointly with the 10th IFAC Symposium on Mechatronic Systems, MECHATRONICS 2025

Y2 - 15 July 2025 through 18 July 2025

ER -

Vision-Based State Estimation for Underactuated Grippers

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this