Haptic Eye Project

Understanding the Haptic Eye Project

Robotic devices that can perform teleoperation work in complex and unstructured environments (e.g. healthcare, education, automotive, telepresence, defense, etc.) rely mainly on visual and/or auditory feed- back to interact with a remote operator. There is a vital need for measuring the physical properties of ambient environments when performing precise manipulation tasks over a distance, to improve the quality of performance. In this project, we propose an approach to combine infrared thermography with machine learning for a fast, accurate way to classify objects of different material. A laser source stimulates the surface of the object while an infrared camera captures its thermal signature, extracts features about these signatures and feeds them into a machine learning-based algorithm for classification. 

The haptic Eye framework is shown in the following figure. It comprises the interaction module, the excitation module, the acquisition module and the material characterization module. The interaction module is responsible for launching the characterization process and using the gained insight to control the actuator component. The excitation module is responsible for performing the required excitation at the proper time and location, with proper waveform and duration. The acquisition module consists of a set of sensors that record the scene containing the object under examination, with a schedule that is synchronized with the excitation module. The data processing module handles the acquired data by applying a series of data processing steps and feeding the processed data forward to the material characterization module. Finally, the material characterization module takes the processed data and estimates the material properties of the examined object. Each of these modules can be further broken down into components. Some of these, such as the laser source, are essential in any realizations of the framework. Others, such as an RGB camera, are optional but have the potential to make the system more robust or enable additional functionalities. 

Screenshot 2021-09-22 171149.png

Publications

  • Tamás Aujeszky, George Korres, and Mohamad Eid, “Thermographic Material Recognition using Multi-Channel Neural Network”, IEEE Transactions on Instrumentation and Measurements (impact factor: 2.79), Vol. 69, No. 9, pp. 7061-7071, 2020.

    • Tamás Aujeszky, Georgios Korres, Mohamad Eid, Farshad Khorrami, “Estimating Weight of Unknown Objects Using Active Thermography”, Robotics, 8(4), 92, 2019.

    • Tamás Aujeszky, Georgios Korres, and Mohamad Eid, “An Approach to Estimate Emissivity For Thermography-based Material Recognition”, IEEE International Conference on Computational Intelligence & Virtual Environments for Measurement Systems and Applications (CIVEMSA 2019), Tianjin, China, June 14-16, 2019.

    • Tamás Aujeszky, Georgios Korres, and Mohamad Eid, “Texture Estimation Using Thermography and Machine Learning”, IEEE International Conference on Computational Intelligence & Virtual Environments for Measurement Systems and Applications (CIVEMSA 2019), Tianjin, China, June 14-16, 2019.

    • Tamás Aujeszky, George Korres, and Mohamad Eid, “Material Classification with Laser Thermography and Machine Learning”, Quantitative InfraRed Thermography Journal, DOI: 10.1080/17686733.2018.1539895, 2018.

    • Tamas Aujeszky, Georgios Korres, and Mohamad Eid, Measurement-Based Thermal Modeling Using Laser Thermography, IEEE Transactions on Instrumentation and Measurements, Volume: 67, Issue: 6, Pages: 1359-1369, 2018.

    • Tamás Aujeszky, George Korres, and Mohamad Eid, “Haptic Eye: A Contactless Material Classification System”, Asia Haptics, Songdo, Korea, 2018.

    • Tamas Aujeszky, Georgios Korres, and Mohamad Eid, “Thermography-based Material Classification using Machine Learning”, IEEE International Workshop on Haptic Audio Visual Environments and Gaming, Abu Dhabi, UAE, October 2017.

    • Georgios Korres, Tamas Aujeszky, and Mohamad Eid, “Characterizing Tactile Rendering Parameters For Ultrasound Based Stimulation”, IEEE World Haptics 2017, Munich, Germany, Jun 06-09, 2017.