Dual arm robot holding crisp. Image: Yijiong Lin

The new Bi-Touch system, designed by scientists at the University of Bristol and based at the Bristol Robotics Laboratory, allows robots to carry out manual tasks by sensing what to do from a digital helper.

The findings, published in IEEE Robotics and Automation Letters, show how an AI agent interprets its environment through tactile and proprioceptive feedback, and then control the robots’ behaviours, enabling precise sensing, gentle interaction, and effective object manipulation to accomplish robotic tasks.

This development could revolutionise industries such as fruit picking, domestic service, and eventually recreate touch in artificial limbs.

Lead author Yijiong Lin from the Faculty of Engineering, explained: “With our Bi-Touch system, we can easily train AI agents in a virtual world within a couple of hours to achieve bimanual tasks that are tailored towards the touch. And more importantly, we can directly apply these agents from the virtual world to the real world without further training.

“The tactile bimanual agent can solve tasks even under unexpected perturbations and manipulate delicate objects in a gentle way.”

Bimanual manipulation with tactile feedback will be key to human-level robot dexterity. However, this topic is less explored than single-arm settings, partly due to the availability of suitable hardware along with the complexity of designing effective controllers for tasks with relatively large state-action spaces. The team were able to develop a tactile dual-arm robotic system using recent advances in AI and robotic tactile sensing.

The researchers built up a virtual world (simulation) that contained two robot arms equipped with tactile sensors. They then design reward functions and a goal-update mechanism that could encourage the robot agents to learn to achieve the bimanual tasks and developed a real-world tactile dual-arm robot system to which they could directly apply the agent.

The robot learns bimanual skills through Deep Reinforcement Learning (Deep-RL), one of the most advanced techniques in the field of robot learning. It is designed to teach robots to do things by letting them learn from trial and error akin to training a dog with rewards and punishments.

For robotic manipulation, the robot learns to make decisions by attempting various behaviours to achieve designated tasks, for example, lifting up objects without dropping or breaking them. When it succeeds, it gets a reward, and when it fails, it learns what not to do. With time, it figures out the best ways to grab things using these rewards and punishments. The AI agent is visually blind relying only on proprioceptive feedback – a body’s ability to sense movement, action and location and tactile feedback.

They were able to successfully enable to the dual arm robot to successfully safely lift items as fragile as a single Pringle crisp.

Co-author Professor Nathan Lepora added: “Our Bi-Touch system showcases a promising approach with affordable software and hardware for learning bimanual behaviours with touch in simulation, which can be directly applied to the real world. Our developed tactile dual-arm robot simulation allows further research on more different tasks as the code will be open-source, which is ideal for developing other downstream tasks.”

Yijiong concluded: “Our Bi-Touch system allows a tactile dual-arm robot to learn sorely from simulation, and to achieve various manipulation tasks in a gentle way in the real world.

“And now we can easily train AI agents in a virtual world within a couple of hours to achieve bimanual tasks that are tailored towards the touch.”

• PAPER – Bi-Touch: Bimanual Tactile Manipulation With Sim-to-Real Deep Reinforcement Learning. Yijiong Lin, Alex Church, Max Yang, Haoran Li, John Lloyd, Dandan Zhang, and Nathan F. Lepora. IEEE Robotics and Automation Letters, vol. 8, no. 9, pp. 5472-5479, Sept. 2023, doi: 10.1109/LRA.2023.3295991.

Artificial Pancreas System with Reinforcement Learning. Image credit: Harry Emerson

Scientists at the University of Bristol have shown that reinforcement learning, a type of machine learning in which a computer program learns to make decisions by trying different actions, significantly outperforms commercial blood glucose controllers in terms of safety and effectiveness. By using offline reinforcement learning, where the algorithm learns from patient records, the researchers improve on prior work, showing that good blood glucose control can be achieved by learning from the decisions of the patient rather than by trial and error.

Type 1 diabetes is one of the most prevalent auto-immune conditions in the UK and is characterised by an insufficiency of the hormone insulin, which is responsible for blood glucose regulation.

Many factors affect a person’s blood glucose and therefore it can be a challenging and burdensome task to select the correct insulin dose for a given scenario. Current artificial pancreas devices provide automated insulin dosing but are limited by their simplistic decision-making algorithms.

However a new study, published in the Journal of Biomedical Informatics, shows offline reinforcement learning could represent an important milestone of care for people living with the condition. The largest improvement was in children, who experienced an additional one-and-a-half hours in the target glucose range per day.

Children represent a particularly important group as they are often unable to manage their diabetes without assistance and an improvement of this size would result in markedly better long-term health outcomes.

Lead author Harry Emerson from Bristol’s Department of Engineering Mathematics, explained: “My research explores whether reinforcement learning could be used to develop safer and more effective insulin dosing strategies.

“These machine learning driven algorithms have demonstrated superhuman performance in playing chess and piloting self-driving cars, and therefore could feasibly learn to perform highly personalised insulin dosing from pre-collected blood glucose data.

“This particular piece of work focuses specifically on offline reinforcement learning, in which the algorithm learns to act by observing examples of good and bad blood glucose control.

“Prior reinforcement learning methods in this area predominantly utilise a process of trial-and-error to identify good actions, which could expose a real-world patient to unsafe insulin doses.”

Due to the high risk associated with incorrect insulin dosing, experiments were performed using the FDA-approved UVA/Padova simulator, which creates a suite of virtual patients to test type 1 diabetes control algorithms. State-of-the-art offline reinforcement learning algorithms were evaluated against one of the most widely used artificial pancreas control algorithms. This comparison was conducted across 30 virtual patients (adults, adolescents and children) and considered 7,000 days of data, with performance being evaluated in accordance with current clinical guidelines. The simulator was also extended to consider realistic implementation challenges, such as measurement errors, incorrect patient information and limited quantities of available data.

This work provides a basis for continued reinforcement learning research in glucose control; demonstrating the potential of the approach to improve the health outcomes of people with type 1 diabetes, while highlighting the method’s shortcomings and areas of necessary future development.

The researchers’ ultimate goal is to deploy reinforcement learning in real-world artificial pancreas systems. These devices operate with limited patient oversight and consequently will require significant evidence of safety and effectiveness to achieve regulatory approval.

Harry added: ”This research demonstrates machine learning’s potential to learn effective insulin dosing strategies from the pre-collected type 1 diabetes data. The explored method outperforms one of the most widely used commercial artificial pancreas algorithms and demonstrates an ability to leverage a person’s habits and schedule to respond more quickly to dangerous events.”

• PAPER – Offline reinforcement learning for safer blood glucose control in people with type 1 diabetes. Emerson, Harry, Matthew Guy, and Ryan McConville. Journal of Biomedical Informatics, 142, 104376.

Robot sponge. Image credit: Tianqi Yue

This easy-to-make sponge-jamming device can help stiff robots handle delicate items carefully by mimicking the nuanced touch, or variable stiffness, of a human.

Robots can skip, jump and do somersaults, but they’re too rigid to hold an egg easily. Variable-stiffness devices are potential solutions for contact compliance on hard robots to reduce damage, or for improving the load capacity of soft robots.

This study, published at the IEEE International Conference on Robotics and Automation (ICRA) 2023, shows that variable stiffness can be achieved by a silicone sponge.

Lead author Tianqi Yue from Bristol’s Department of Engineering Mathematics explained: “Stiffness, also known as softness, is important in contact scenarios.

“Robotic arms are too rigid so they cannot make such a soft human-like grasp on delicate objects, for example, an egg.

“What makes humans different from robotic arms is that we have soft tissues enclosing rigid bones, which act as a natural mitigating mechanism.

“In this paper, we managed to develop a soft device with variable stiffness, to be mounted on the end robotic arm for making the robot-object contact safe.”

Robot sponge in action. Video Credit: Tianqi Yue.

Silicone sponge is a cheap and easy-to-fabricate material. It is a porous elastomer just like the cleaning sponge used in everyday tasks.

By squeezing the sponge, the sponge stiffens which is why it can be transformed into a variable-stiffness device.

This device could be used in industrial robots in scenarios including gripping jellies, eggs and other fragile substances. It can also be used in service robots to make human-robot interaction safer.

Mr Yue added: “We managed to use a sponge to make a cheap and nimble but effective device that can help robots achieve soft contact with objects. The great potential comes from its low cost and light weight.

“We believe this silicone-sponge based variable-stiffness device will provide a novel solution in industry and healthcare, for example, tunable-stiffness requirement on robotic polishing and ultrasound imaging.”

The team will now look at making the device achieve variable stiffness in multiple directions, including rotation.

Paper: “A Silicone-sponge-based Variable-stiffness Device” by Tianqi Yue at the IEEE International Conference on Robotics and Automation (ICRA) 2023.

Robot fish. Image credit: Tsam Lung You

The robot fish was fitted with a twisted and coiled polymer (TCP) to drive it forward, a light-weight low cost device that relies on temperature change to generate movement, which also limits its speed.

A TCP works by contracting like muscles when heated, converting the energy into mechanical motion. The TCP used in this work is warmed by Joule heating – the pass of current through an electrical conductor produces thermal energy and heats up the conductor. By minimising the distance between the TCP on one side of the robot fish and the spring on the other, this activates the fin at the rear, enabling the robot fish to reach new speeds. The undulating flapping of its rear fin was measured at a frequency of 2Hz, two waves per second. The frequency of the electric current is the same as the frequency of tail flap.

The findings, published at the 6th IEEE-RAS International Conference on Soft Robotics (RoboSoft 2023), provide a new route to raising the actuation – the action of causing a machine or device to operate – frequency of TCPs through thermomechanical design and shows the possibility of using TCPs at high frequency in aqueous environments.

Lead author Tsam Lung You from Bristol’s Department of Engineering Mathematics said: “Twisted and coiled polymer (TCP) actuator is a promising novel actuator, exhibiting attractive properties of light weight, low-cost high energy density and simple fabrication process.

“They can be made from very easily assessable materials such as a fishing line and they contract and provide linear actuation when heated up. However, because of the time needed for heat dissipation during the relaxation phase, this makes them slow.”

By optimising the structural design of the TCP-spring antagonistic muscle pair and bringing their anchor points closer together, it allowed the posterior fin to swing at a larger angle for the same amount of TCP actuation.

Antagonistic muscles. Image credit: Tsam Lung You

Although this requires greater force, TCP is a strong actuator with high work energy density, and is still able to drive the fin.

Until now, TCPs have been mostly used for applications such as wearable devices and robotic hands. This work opens up more areas of application where TCP can be used, such as marine robots for underwater exploration and monitoring.

Tsam Lung You added: “Our robotic fish swam at the fastest actuation frequency found in a real TCP application and also the highest locomotion speed of a TCP application so far.

“This is really exciting as it opens up more opportunities of TCP application in different areas.”

The team now plan to expand the scale and develop a knifefish-inspired TCP-driven ribbon fin robot that can swim agilely in water.

• PAPER – Robotic Fish driven by Twisted and Coiled Polymer Actuators at High Frequencies.. You, T. L., Rossiter, J. M., and Philamore, H. In 6th IEEE-RAS International Conference on Soft Robotics (RoboSoft).

RoboSalps in action. Credits: Valentina Lo Gatto

These robotic units called RoboSalps, after their animal namesakes, have been engineered to operate in unknown and extreme environments such as extra-terrestrial oceans.

Although salps resemble jellyfish with their semi-transparent barrel-shaped bodies, they belong to the family of Tunicata and have a complex life cycle, changing between solitary and aggregate generations where they connect to form colonies.

RoboSalps have similarly light, tubular bodies and can link to each other to form ‘colonies’ which gives them new capabilities that can only be achieved because they work together.

Researcher Valentina Lo Gatto of Bristol’s Department of Aerospace Engineering is leading the study. She is also a student at the EPSRC Centre of Doctoral Training in Future Autonomous and Robotic Systems (FARSCOPE CDT).

She said: “RoboSalp is the first modular salp-inspired robot. Each module is made of a very light-weight soft tubular structure and a drone propeller which enables them to swim. These simple modules can be combined into ‘colonies’ that are much more robust and have the potential to carry out complex tasks.