www.technologyreview.com
A healthy heart beats at a steady rate, between 60 and 100 times a minute. Thats not the case for all of us, Im reminded, as I look inside a cardboard box containing around 20 plastic heartseach a replica of a real human one. The hearts, which previously sat on a shelf in a lab in West London, were generated from MRI and CT scans of people being treated for heart conditions at Hammersmith Hospital next door. Steven Niederer, a biomedical engineer at the Alan Turing Institute and Imperial College London, created them on a 3D printer in his office. One of the hearts, printed in red recycled plastic, looks as I imagine a heart to look. It just about fits in my hand, and the chambers have the same dimensions as the ones you might see in a textbook. Perhaps it helps that its red. The others look enormous to me. One in particular, printed in black plastic, seems more than twice the size of the red one. As I find out later, the person who had the heart it was modeled on suffered from heart failure. The plastic organs are just for educational purposes. Niederer is more interested in creating detailed replicas of peoples hearts using computers. These digital twins are the same size and shape as the real thing. They work in the same way. But they exist only virtually. Scientists can do virtual surgery on these virtual hearts, figuring out the best course of action for a patients condition. After decades of research, models like these are now entering clinical trials and starting to be used for patient care. Virtual replicas of many other organs are also being developed. Engineers are working on digital twins of peoples brains, guts, livers, nervous systems, and more. Theyre creating virtual replicas of peoples faces, which could be used to try out surgeries or analyze facial features, and testing drugs on digital cancers. The eventual goal is to create digital versions of our bodiescomputer copies that could help researchers and doctors figure out our risk of developing various diseases and determine which treatments might work best. Theyd be our own personal guinea pigs for testing out medicines before we subject our real bodies to them. To engineers like Niederer, its a tantalizing prospect very much within reach. Several pilot studies have been completed, and larger trials are underway. Those in the field expect digital twins based on organs to become a part of clinical care within the next five to 10 years, aiding diagnosis and surgical decision-making. Further down the line, well even be able to run clinical trials on synthetic patientsvirtual bodies created using real data. But the budding technology will need to be developed carefully. Some worry about who will own this highly personalized data and how it could be used. Others fear for patient autonomywith an uncomplicated virtual record to consult, will doctors eventually bypass the patients themselves? And some simply feel a visceral repulsion at the idea of attempts to re-create humans in silico. People will say I dont want you copying me, says Wahbi El-Bouri, who is working on digital-twin technologies. They feel its a part of them that youve taken. Getting digital Digital twins are well established in other realms of engineering; for example, they have long been used to model machinery and infrastructure. The term may have become a marketing buzzword lately, but for those working on health applications, it means something very specific. We can think of a digital twin as having three separate components, says El-Bouri, a biomedical engineer at the University of Liverpool in the UK. The first is the thing being modeled. That might be a jet engine or a bridge, or it could be a persons heart. Essentially, its what we want to test or study. The second component is the digital replica of that object, which can be created by taking lots of measurements from the real thing and entering them into a computer. For a heart, that might mean blood pressure recordings as well as MRI and CT scans. The third is new data thats fed into the model. A true digital twin should be updated in real timefor example, with information collected from wearable sensors, if its a model of someones heart. Taking measurements of airplanes and bridges is one thing. Its much harder to get a continuous data feed from a person, especially when you need details about the inner functions of the heart or brain. And the information transfer should run both ways. Just as sensors can deliver data from a persons heart, the computer can model potential outcomes to make predictions and feed them back to a patient or health-care provider. A medical team might want to predict how a person will respond to a drug, for example, or test various surgical procedures on a digital model before operating in real life. By this definition, pretty much any smart device that tracks some aspect of your health could be considered a kind of rudimentary digital twin. You could say that an Apple Watch fulfills the definition of a digital twin in an unexciting way, says Niederer. It tells you if youre in atrial fibrillation or not. But the kind of digital twin that researchers like Niederer are working on is far more intricate and detailed. It could provide specific guidance on which disease risks a person faces, what medicines might be most effective, or how any surgeries should proceed. Were not quite there yet. Taking measurements of airplanes and bridges is one thing. Its much harder to get a continuous data feed from a person, especially when you need details about the inner functions of the heart or brain, says Niederer. As things stand, engineers are technically creating patient-specific models based on previously collected hospital and research data, which is not continually updated. The most advanced medical digital twins are those built to match human hearts. These were the first to be attempted, partly because the heart is essentially a pumpa device familiar to engineersand partly because heart disease is responsible for so much ill health and death, says El-Bouri. Now, advances in imaging technology and computer processing power are enabling researchers to mimic the organ with the level of fidelity that clinical applications require. Building a heart The first step to building a digital heart is to collect images of the real thing. Each team will have its own slightly different approach, but generally, they all start with MRI and CT scans of a persons heart. These can be entered into computer software to create a 3D movie. Some scans will also highlight any areas of damaged tissue, which might disrupt the way the electrical pulses that control heart muscle contraction travel through the organ. The next step is to break this 3D model down into tiny chunks. Engineers use the term computational mesh to describe the result; it can look like an image of the heart made up of thousands of 3D pieces. Each segment represents a small collection of cells and can be assigned properties based on how well they are expected to propagate an electrical impulse. Its all equations, says Natalia Trayanova, a biomedical engineering professor based at Johns Hopkins University in Baltimore, Maryland. This computer model of the human heart show how electrical signals pass through heart tissue. The model was created by Marina Strocchi, who works with Steven Niederer at Imperial College London.COURTESY OF MARINA STROCCHI As things stand, these properties involve some approximation. Engineers will guess how well each bit of heart works by extrapolating from previous studies of human hearts or past research on the disease the person has. The end result is a beating, pumping model of a real heart. When we have that model, you can poke it and prod it and see under what circumstances stuff will happen, says Trayanova. Her digital twins are already being trialed to help people with atrial fibrillation, a fairly common condition that can trigger an irregular heartbeattoo fast or all over the place. One treatment option is to burn off the bits of heart tissue responsible for the disrupted rhythm. Its usually left to a surgical team to figure out which bits to target. For Trayanova, the pokes and prods are designed to help surgeons with that decision. Scans might highlight a few regions of damaged or scarred tissue. Her team can then construct a digital twin to help locate the underlying source of the damage. In total, the tool will likely suggest two or three regions to destroythough in rare instances, it has shown many more, says Trayanova: They just have to trust us. So far, 59 people have been through the trial. More are planned. In cases like these, the models dont always need to be continually updated, Trayanova says. A heart surgeon might need to run simulations only to know where to implant a device, for example. Once that operation is over, no more data might be needed, she says. Quasi patients At his lab on the campus of Hammersmith Hospital in London, Niederer has also been building virtual hearts. He is exploring whether his models could be used to find the best place to implant pacemakers. His approach is similar to Trayanovas, but his models also incorporate ECG data from patients. These recordings give a sense of how electrical pulses pass through the heart tissue, he says. So far, Niederer and his colleagues have published a small trial in which models of 10 patients hearts were evaluated by doctors but not used to inform surgical decisions. Still, Niederer is already getting requests from device manufacturers to run virtual tests of their products. A couple have asked him to choose places where their battery-operated pacemaker devices can sit without bumping into heart tissue, he says. Not only can Niederer and his colleagues run this test virtually, but they can do it for hearts of various different sizes. The team can test the device in hundreds of potential locations, within hundreds of different virtual hearts. And we can do it in a week, he adds. This is an example of what scientists call in silico trialsclinical trials run on a computer. In some cases, its not just the trials that are digital. The volunteers are, too. El-Bouri and his colleagues are working on ways to create synthetic participants for their clinical trials. The team starts with data collected from real people and uses this to create all-new digital organs with a mishmash of characteristics from the real volunteers. These in silico trials could be especially useful for helping us figure out the best treatments for pregnant peoplea group that is notoriously excluded from many clinical trials. Specifically, one of El-Bouris interests is stroke, a medical emergency in which clots or bleeds prevent blood flow in parts of the brain. For their research, he and his colleagues model the brain, along with the blood vessels that feed it. You could create lots and lots of different shapes and sizes of these brains based on patient data, says El-Bouri. Once he and his team create a group of synthetic patient brains, they can test how these clots might change the flow of blood or oxygen, or how and where brain tissue is affected. They can test the impact of certain drugs, or see what might happen if a stent is used to remove the blockage. For another project, El-Bouri is creating synthetic retinas. From a starting point of 100 or so retinal scans from real people, his team can generate 200 or more synthetic eyes, just like that, he says. The trick is to figure out the math behind the distribution of blood vessels and re-create it through a set of algorithms. Now he is hoping to use those synthetic eyes in drug trialsamong other things, to find the best treatment doses for people with age-related macular degeneration, a common condition that can lead to blindness. These in silico trials could be especially useful for helping us figure out the best treatments for pregnant peoplea group that is notoriously excluded from many clinical trials. Thats for fear that an experimental treatment might harm a fetus, says Michelle Oyen, a professor of biomedical engineering at Wayne State University in Detroit. Oyen is creating digital twins of pregnancy. Its a challenge to get the information needed to feed the models; during pregnancy, people are generally advised to avoid scans or invasive investigations they dont need. Were much more limited in terms of the data that we can get, she says. Her team does make use of ultrasound images, including a form of ultrasound that allows the team to measure blood flow. From those images, they can see how blood flow in the uterus and the placenta, the organ that supports a fetus, might be linked to the fetuss growth and development, for example. For now, Oyen and her colleagues arent creating models of the fetuses themselvestheyre focusing on the fetal environment, which includes the placenta and uterus. A baby needs a healthy, functioning placenta in order to survive; if the organ starts to fail, stillbirth can be the tragic outcome. Oyen is working on ways to monitor the placenta in real time during pregnancy. These readings could be fed back to a digital twin. If she can find a way to tell when the placenta is failing, doctors might be able to intervene to save the baby, she says. I think this is a game changer for pregnancy research, she adds, because this basically gives us ways of doing research in pregnancy that [carries a minimal] risk of harm to the fetus or of harm to the mother. In another project, the team is looking at the impact of cesarean section scars on pregnancies. When a baby is delivered by C-section, surgeons cut through multiple layers of tissue in the abdomen, including the uterus. Scars that dont heal well become weak spots in the uterus, potentially causing problems for future pregnancies. By modeling these scars in digital twins, Oyen hopes to be able to simulate how future pregnancies might pan out, and determine if or when specialist care might be called for. Eventually, Oyen wants to create a full virtual replica of the pregnant uterus, fetus and all. But were not there yetwere decades behind the cardiovascular people, she says. Thats pregnancy research in a nutshell, she adds. Were always decades behind. Twinning Its all very well to generate virtual body parts, but the human body functions as a whole. Thats why the grand plan for digital twins involves replicas of entire people. Long term, the whole body would be fantastic, says El-Bouri. It may not be all that far off, either. Various research teams are already building models of the heart, brain, lungs, kidneys, liver, musculoskeletal system, blood vessels, immune system, eye, ear, and more. If we were to take every research group that works on digital twins across the world at the moment, I think you could put [a body] together, says El-Bouri. I think theres even someone working on the tongue, he adds. The challenge is bringing together all the various researchers, with the different approaches and different code involved in creating and using their models, says El-Bouri. Everything exists, he says. Its just putting it together thats going to be the issue. In theory, such whole-body twins could revolutionize health care. Trayanova envisions a future in which a digital twin is just another part of a persons medical recordone that a doctor can use to decide on a course of treatment. Technically, if someone tried really hard, they might be able to piece back who someone is through scans and twins of organs. Wahbi El-Bouri But El-Bouri says he receives mixed reactions to the idea. Some people think its really exciting and really cool, he says. But hes also met people who are strongly opposed to the idea of having a virtual copy of themselves exist on a computer somewhere: They dont want any part of that. Researchers need to make more of an effort to engage with the public to find out how people feel about the technology, he says. There are also concerns over patient autonomy. If a doctor has access to a patients digital twin and can use it to guide decisions about medical care, where does the patients own input come into the equation? Some of those working to create digital twins point out that the models could reveal whether patients have taken their daily meds or what theyve eaten that week. Will clinicians eventually come to see digital twins as a more reliable source of information than peoples self-reporting? Doctors should not be allowed to bypass patients and just ask the machine, says Matthias Braun, a social ethicist at the University of Bonn in Germany. There would be no informed consent, which would infringe on autonomy and maybe cause harm, he says. After all, we are not machines with broken parts. Two individuals with the same diagnosis can have very different experiences and lead very different lives. However, there are cases in which patients are not able to make decisions about their own treatmentfor example, if they are unconscious. In those cases, clinicians try to find a proxysomeone authorized to make decisions on the patients behalf. A digital psychological twin, trained on a persons medical data and digital footprint, could potentially act as a better surrogate than, for example, a relative who doesnt know the persons preferences, he says. If using digital twins in patient care is problematic, in silico trials can also raise issues. Jantina de Vries, an ethicist at the University of Cape Town, points out that the data used to create digital twins and synthetic quasi patients will come from people who can be scanned, measured, and monitored. This group is unlikely to include many of those living on the African continent, who wont have ready access to those technologies. The problem of data scarcity directly translates into technologies that are not geared to think about diverse bodies, she says. De Vries thinks the data should belong to the public in order to ensure that as many people benefit from digital-twin technologies as possible. Every record should be anonymized and kept within a public database that researchers around the world can access and make use of, she says. The people who participate in Trayanovas trials explicitly give me consent to know their data, and to know who they are [everything] about them, she says. The people taking part in Niederers research also provide consent for their data to be used by the medical and research teams. But while clinicians have access to all medical data, researchers access only anonymized or pseudonymized data, Niederer says. In some cases, researchers will also ask participants to consent to sharing their fully anonymized data in public repositories. This is the only data that companies are able to access, he adds: We do not share [our] data sets outside of the research or medical teams, and we do not share them with companies. El-Bouri thinks that patients should receive some form of compensation in exchange for sharing their health data. Perhaps they should get preferential access to medications and devices based on that data, he suggests. At any rate, [full] anonymization is tricky, particularly if youre taking patient scans to develop twins, he says. Technically, if someone tried really hard, they might be able to piece back who someone is through scans and twins of organs. When I looked at those anonymous plastic hearts, stored in a cardboard box tucked away on a shelf in the corner of an office, they felt completely divorced from the people whose real, beating hearts they were modeled on. But digital twins seem different somehow. Theyre animated replicas, digital copies that certainly appear to have some sort of life. People often think, Oh, this is just a simulation, says El-Bouri. But its a digital representation of an individual.
