How DeepMind’s AlphaZero Mastered Complex Games With No Human Input

It’s the end of an era in AI research. For decades complex board games like Go, chess, and shogi have been seen as the leading yardstick for machine intelligence. DeepMind’s latest program can master all three by simply playing itself, suggesting we need to set bigger challenges for AI.

Machines first started chipping away at humanity’s intellectual dominance back in 1997, when IBM’s Deep Blue beat chess grandmaster Garry Kasparov. In the first half of this decade, masters of Japan’s harder chess variant shogi started falling to computers. And in 2016 DeepMind shocked the world when its AlphaGo program defeated one of the world’s top-ranked masters of Go, widely considered the most complicated board game in the world.

As impressive as all these feats were, game-playing AI typically exploit the properties of a single game and often rely on hand-crafted knowledge coded into them by developers. But DeepMind’s latest creation, AlphaZero, detailed in a new paper in Science, was built from the bottom up to be game-agnostic.

All it was given was the rules of each game, and it then played itself thousands of times,  effectively using trial and error to work out the best tactics for each game. It was then pitted against the most powerful specialized AI for each game, including its predecessor AlphaGo, beating them comprehensively.

“This work has, in effect, closed a multi-decade chapter in AI research. AI researchers need to look to a new generation of games to provide the next set of challenges, “IBM computer scientist Murray Campbell, who has worked on chess-playing computers, wrote in an opinion for Science.

The generality of AlphaZero is important. While learning to master the most complex board games in the world is impressive, their tightly-controlled environments are a far stretch from the real world. The reinforcement learning approach that DeepMind has championed has achieved impressive results in Go, video games, and simulators, but practical applications have been harder to come by.

By making these programs more general, the company hopes they can start to break out of the confines of these more rigid environments and use the approach to tackle real-world challenges. Last week, there was evidence that transition may have begun—one of their algorithms won a competition for predicting how proteins fold, a complex problem at the heart of many biological processes and, perhaps more importantly, drug design.

One of the most impressive features of the new program is the efficiency with which it searches for the ideal move. The researchers used Monte Carlo tree search, which has long been the standard for Go-playing machines but is generally thought to be inappropriate for chess or shogi.

But the switch allowed the program to focus on the most promising potential positions rather than employing the brute force approach of its opponents. It assessed just 60,000 potential positions per second in both chess and shogi, compared with 60 million for its chess-playing opponent and 25 million in shogi.

Nonetheless, all three games are “perfect information games” where each player always has complete visibility of the game state, unlike card games like poker in which a player doesn’t know his opponents’ hands. Very few real-world situations have perfect information, so building the capacity to deal with uncertainty into these systems will be a crucial next step.

That’s leading to emerging interest among AI researchers in multiplayer video games, like StarCraft II and Dota 2. The games are incredibly open-ended, give players very limited visibility of their opponents’ actions, and require long-term strategic planning. And so far, AI programs have yet to beat their human counterparts.

One final aspect worth noting about this research is the enormous amount of computing power that went into it. AlphaZero took just 9 hours to master chess, 12 hours for shogi, and 13 days for Go. But this was achieved using 5,000 TPUs—Google’s specialized deep learning processors.

That’s a colossal amount of computing power, well out of the reach of pretty much anyone other than Silicon Valley’s behemoths. And as New York University computer scientist Julian Togelius noted to New Scientist, the system is only general in its capacity to learn. The program trained on shogi can’t play chess, so you need access to such enormous computing resources every time you repurpose it.

This is part of a trend noted by OpenAI researchers earlier this year; the computing resources used to train the programs behind some of the field’s most exciting advances are increasing exponentially. And that’s prompting both excitement and concern.

On the one hand, it raises the question of whether a substantial factor in the tractability of many problems is simply how many chips you can dedicate to them. With companies seemingly willing to dedicate extensive resources to AI, that would suggest a promising future for the field.

But at the same time, it also points to a future of AI haves and have nots. If computing resources become increasingly integral to making progress in the field, many bright minds outside the largest tech companies could be left behind.

Image Credit: Chill Chillz /

Give Your Plants Some Emotion With A.P.E.X.

I talk to my plants. Maybe I’m crazy, but I suspect that many of you do this too. In this project, you may be able to give your plants the ability to emote back just a little bit. A.P.E.X. is basically just a moisture sensor with some emojis as a […]

Read more on MAKE

The post Give Your Plants Some Emotion With A.P.E.X. appeared first on Make: DIY Projects and Ideas for Makers.

Are We Made of Memories? A Researcher’s Quest to Record His Life

How well do you remember what happened last week? Two weeks ago? Five weeks from last Tuesday? Unless you are meticulous in recording the events of your life, or something notable happened on a given date, your memories will likely be hazy at best. How many hours of your life, ignored, seep down into some dusty corner of your brain, to be retrieved only after great effort, or not at all? If we are nothing more than the sum of our memories and our experiences, how much of ourselves do we lose through forgetting?

Perhaps this natural amnesia is a good thing for keeping us all sane, but it’s also optional. This is what Neo Mohsenvand from MIT Media Lab aims to prove with his research project called Mnemo. The ambitious goal of the project is to record as much data about Mohsenvand as possible; throughout most of the day, he goes about his normal activities with a fisheye camera lens and a microphone  attached to his chest.

Mohsenvand’s project is reminiscent of previous studies. Morris Villarroel, a professor of animal physiology, used cameras and logbooks to keep track of the mundane day-to-day events of his life. Long before selfies and cell phone videos became popular, Sam Klemke filmed himself annually every year since 1977. But novel technology and big-data processing techniques—Mohsenvand’s kit includes an on-board GPU that he wears in a backpack—have enabled unprecedented data collection and analysis.

Using a newer generation of wearables, Mohsenvand is also able to monitor and measure other biometric signals; a device attached to his wrist keeps track of heart rate, skin temperature, and skin conductance, which are all associated with emotions or stress. Perhaps most intrusively, for the last five months he has worn a portable EEG headset that records his brain wave data. Mohsenvand aims to wear this apparatus for around nine hours a day—although not during the more, shall we say, private moments.

The Case for Documenting Our Lives

Collecting thousands of hours of footage of everything you do might seem like an act of obsessive preservation, a feeble attempt to preserve your life after you die. And the project raises ethical questions about privacy; strangers in Mohsenvand’s video might not have given their consent to be recorded. Is it worthwhile, then? What are the major benefits?

Mohsenvand notes that there is an obvious group for whom Mnemo could be a game-changer: Alzheimer’s patients. Both of his grandfathers suffered from the disease, and he described his frustration at the technology available, which mostly focuses on keeping sufferers safe—say, by locking medicine cabinets—rather than helping them to function. He dreams of an augmented memory that’s seamlessly integrated into a person, perhaps via more subtle wearable technology like augmented reality glasses, or even neural implants.

That way, short-term memory loss could be mitigated: if you can’t remember where you were an hour ago, you could just cue up that particular memory and work backwards. Combine it with image recognition software, and it’s more powerful still: you’d be able to remember where you put your keys, or when you last saw somebody.

In the extreme case, if our knowledge of how memory works and our ability to manipulate it becomes exponentially more precise, it may be possible to trigger or create long-term memories through analysis of the brain’s electrical signals (more research projects that would make Black Mirror writer Charlie Brooker shudder).

There may also be legal or safety reasons to encourage keeping a detailed record of our lives; after all, it’s a perfect alibi. The Lawfare Blog suggested that this might be one of the only ways to combat deepfake images or videos of celebrities; famous individuals could have an “immutable authentication trail” that could confirm what did or didn’t happen.

The idea of recorded experiences has already been portrayed in art and film; The Truman Show, for example, depicted a world transfixed by the daily life of an ordinary person and was partially inspired by real people doing life-streaming. In a world where YouTube and Instagram stars sell their lives and lifestyles to fans, offering a carefully-tailored but intimately personal image, there might be a market to watch them in real time.

The Fully-Examined Life

Socrates supposedly said that “the unexamined life is not worth living.” Technology allows us to generate reams and reams of data about ourselves to examine. Leaving aside futurist, transhumanist, and unproven medical applications, Mohsenvand views the Mnemo project as digitally-aided introspection. In the same way that keeping a diary is not only a record, but a way to make sense of one’s life, Mnemo provides insights into Mohsenvand’s psychology.

Rather than spending another eight hours watching the footage that was recorded during the day, Mohsenvand uses software to create an edited highlights package, speeding through the video and slowing down only at the most emotionally intense moments, as measured via the bio-signatures such as heart rate and skin conductance. Mohsenvand may gain new perspective on the most significant moments in a day after reviewing and reflecting on his video footage.

For example, when watching the film Whiplash, Mohsenvand’s heart rate was most responsive during interactions between the movie’s main character and his father. “I’m sensitive to father-son relationships, it turns out,” he told MIT Tech Review. By noticing which experiences he finds most enjoyable and which situations are the most stressful in each given day, he might be inspired to change his lifestyle accordingly. The ritual of reflection might seem self-obsessive, but it has something in common with meditation practices, philosophical thought, and therapy.

The Drawbacks of Remembering Every Minute

There are, of course, caveats associated with this research. Interpreting EEG data, which can be interfered with simply by blinking, remains notoriously difficult; unlike its portrayal in science fiction, it’s far from allowing us to read people’s thoughts.

Equally, people disagree about how to interpret things like heart rate, with some arguing that a person’s heart rate actually slows down momentarily when something significant happens. Intravenous monitoring of hormone concentration could provide a further means of calibrating emotional response, but would likely be too intrusive until technology improves.

Documenting our lives is also done inadvertently, every time we type anything into a computer, every time we appear in a public place, and with every link clicked. It creates strange forms of permanence, of record: forgotten social media accounts, endless digital snapshots of the day’s preoccupations that are strip-mined by advertisers for psychological gold. If we go out of our way to keep track of more of these actions, we may just be enabling advertisers or malicious actors to take advantage of us.

It’s yet to be determined which problems cannot be solved by the harvesting and analysis of endless data. Systems like Mnemo may not really hold the secret key to digital immortality, or a perfectly examined life. But if it can help those that need it most to remember who they are, it might just be worth the fashion faux-pas of wearing electrodes all day.

Image Credit: Arkela /

Switching to Electric Vehicles Could Save the US Billions, But Timing Is Everything

Today, less than two percent of the vehicles Americans buy are electric. But within the next three decades, some automotive industry experts expect electric vehicles could make up the majority of US and global car sales.

All told, American drivers log about 3 trillion miles per year, consuming more than 170 billion gallons of gasoline and diesel in the process. Converting all those road miles to electricity would place new demands on the nation’s system for producing and delivering electricity.

As part of a major energy infrastructure study, we are seeking to understand how an increase in electric vehicles (EVs) might change how energy is supplied and consumed. So far, we have figured out the impact of electric vehicles will depend on where you live and when they are charged.

How Much Electricity EVs Will Demand

Using a similar technique featured in our recent paper on hydrogen vehicles, we developed a state-by-state assessment of the amount of electricity that would be needed to charge an electrified fleet of personal cars, trucks and SUVs.


We started by estimating the amount of gasoline every county consumes today. We then converted vehicle miles traveled into electricity requirements based on the efficiency of today’s EVs.

Admittedly, these methods have limitations. The number of miles traveled could change significantly if autonomous vehicles become commonplace and more people rely on Uber, Lyft, and other vehicle sharing services, for example. However, we believe our approach provides a good starting point for estimating future electricity demand if EVs become the norm.

Regional Impacts

The US electric grid has continually evolved to accommodate new demands throughout the last century. But if the nation’s vehicles were to rapidly become electric, the grid would need to change faster. Depending on local driving habits and the grid infrastructure that’s already in place, our analysis shows that EVs will have different impacts in different regions.

Since Texas and California consume more electricity than any other states, they provide a good snapshot of what a future filled with electric vehicles might look like. In both cases, an increase in EVs would drive consumption higher, with the potential to strain local infrastructure.

If virtually all passenger cars in Texas were electrified today, the state would need approximately 110 more terawatt-hours of electricity per year—the average annual electricity consumption of 11 million homes. The added electricity demand would result in a 30 percent increase over current consumption in Texas.

By comparison, because of a more temperate climate, California might require nearly 50 percent more electricity than it currently consumes if passenger vehicles in the state were fully electrified. That means California would need to generate an additional 120 terawatt-hours of electricity per year.

A Tale of Two Grids

A look at the two states’ grids demonstrates how reliance on EVs for mobility could vary from place to place.

On hot summer afternoons, Texas uses about half of the electricity it generates to power air conditioning to keep buildings cool. The large seasonal variations in electricity demand due to air conditioning means the state has power plants that sit idle throughout many hours of the year. The spare capacity during off-peak hours could make it easier for Texas to meet future electricity demands of EVs.

California’s more temperate climate means the state needs less electricity on summer days, and less demand variability on the grid overall. As a result, California has less generation capacity available than Texas to meet future charging demands from electric vehicles.

In 2018, the Electric Reliability Council of Texas, the organization that manages most of Texas’s electric grid, hit a new peak demand of roughly 73 gigawatts on July 19. Looking at the off-peak hours for July 19, 2018, we found the ERCOT grid had spare capacity to provide more than 350 gigawatt-hours of additional electricity if idled power plants continued to operate throughout the day, not just during peak demand.

Based on our estimates, the charging requirements for a fully electrified fleet of personal cars in Texas would be about 290 gigawatt-hours per day, less than the available surplus of generation capacity. In other words, the Texas grid could theoretically charge a fully electrified vehicle fleet today if vehicles were charged during off-peak hours.

When we did the same analysis for California, however, we found that if EVs become the norm, it could push the total demand for electricity beyond the existing capacity of the Golden State’s grid.

Timing Is Everything

Perhaps even more important than how much electricity EVs would consume is the question of when it would be consumed.

We based the above estimates on optimal, off-peak charging patterns. If instead most EVs were to be charged in the afternoon, the electricity grid would need more generation capacity to avoid outages.

To meet that demand, California and Texas would need to build new power plants or buy more electricity from neighboring states than they already do. The states might also need additional transmission and distribution infrastructure to accommodate new automotive charging infrastructure.

All told, the transition to EVs from internal combustion engine vehicles could potentially cost tens of billions of dollars in Texas and even more in California to install new electricity infrastructure if many vehicles were to be charged during peak hours.

Incentives could reduce what it will cost to equip the grid for lots of electric vehicles. For example, utilities could charge different rates for electricity during different times of day and on different days of the week. Known as time-of-use pricing, this practice can encourage vehicle charging when electricity is more abundant during off-peak hours and therefore cheaper to supply.

California and other areas, including Austin, Texas, have already begun to use different strategies for implementing time-of-use rates. Other regions might want to watch closely, and adopt the lessons learned in those places as the number of electric vehicles on the road rises.

The Road Ahead

While EVs might increase the amount of electricity the US consumes, the investment required to accommodate them may be smaller than it appears. Many regions already have sufficient generation capacity if vehicles are charged during off-peak hours. The energy storage on board EVs could provide the flexibility needed to shift charging times and help grid operators better manage the supply and demand of electricity.

What’s more, based on our calculations, the money Americans would save in fuel costs alone could offset these investments.

For example, had most of California’s vehicles been electric by 2017, we estimate that its drivers would have saved around US$25 billion that year in fuel costs based on the average prices for electricity and gasoline.

In addition to fuel savings, some market analysts expect electric cars to be cheaper than conventional vehicles by 2026, another potential economic benefit.

While it’s challenging to predict the future prices for gasoline, electricity, and vehicles, we believe it is likely that the widespread use of EVs would reduce the overall costs of transportation in California and elsewhere. These savings are even greater if the environmental benefits, especially lower carbon emissions, are taken into account.The Conversation

The Union of Concerned Scientists has researched the carbon footprints of EVs versus gasoline-powered cars.

F. Todd Davidson, Research Associate, Energy Institute, University of Texas at Austin; Dave Tuttle, Research Fellow, The Energy Institute,, University of Texas at Austin; Joshua D. Rhodes, Research Fellow of Energy, University of Texas at Austin, and Kazunori Nagasawa, Postdoctoral Researcher, National Renewable Energy Laboratory

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Image Credit: Nadya Kubik /

Video Friday: Space Station’s New Robot Helper, and More

Your weekly selection of awesome robot videos

Corindus completes first-in-human telerobotic coronary intervention

Corindus Vascular Robotics

Corindus Vascular Robotics’ CorPath.

Corindus Vascular Robotics‘ CorPath robotic surgical platform was used in a first-in-human telerobotic intervention study in India. The Waltham, Mass.-based company said the study was the world’s first percutaneous coronary intervention conducted from a remote location outside of a catheterization lab.

In the trial, five patients at India’s Apex Heart Institute underwent an elective PCI procedure from a distance of approximately 20 miles away, Corindus said.

The procedures were performed by Apex Heart Institute chair and chief interventional cardiologist Dr. Tejas Patel from inside the Swaminarayan Akshardham temple in Gandhinagar, while his partner, Dr. Sanjay Shah, attended to the patient in person at the Apex Heart Institute.

“The first in human cases of remote robotic PCI represent a landmark event for interventional medicine. The application of telerobotics in India has the potential to impact a significant number of lives by providing access to care that may not otherwise have been possible. For the first time in cardiology’s history, India will shine for this ground-breaking innovation, and I am honored to be a part of this historic occasion,” Dr. Patel said in a prepared statement.

Cardiovascular disease, including stroke, is the number one cause of death worldwide, resulting in nearly 18 million deaths per year. Geographic barriers, socioeconomic status and a rapidly shrinking number of skilled specialists significantly hinder patient access to timely, specialized cardiovascular care. This is especially of concern during highly emergent medical events, such as heart attack and stroke, where ideally treatment is received in as little as 90 minutes or within 24 hours, respectively, to avoid death or permanent disability.

Corindus said that following the study, it plans to begin commercial product development to allow the CorPath system to be used in remote interventions.

“Cardiovascular disease, including stroke, is the world’s most significant and undertreated clinical problem due to limited access to specialized, timely medical care. As a result of existing barriers to care, including increased global poverty and a declining number of trained specialists, only a fraction of patients worldwide receives life-saving treatment, resulting in substantial death or disability,” said Corindus president & CEO Mark Toland. “We anticipate that our technology will revolutionize cardiovascular disease treatment by providing specialized and timely medical care to anyone, anywhere.”

Last month, Corindus saw shares rise over 10% after the robotic surgical platform maker beat loss-per-share expectations on Wall Street with its third quarter earnings. Earlier in 2018, Corindus inked a private placement deal worth $25 million through the offering of newly-designated Series A convertible preferred stock, with funds slated to support global commercialization of its CorPath GRX robotic surgical system.

The post Corindus completes first-in-human telerobotic coronary intervention appeared first on The Robot Report.

Tips of the Week: Cosplayer’s Project Notebooks, Silicon Molds from Caulk, Recip Saw Tips, Fire-Starting, and More

See what top tips Gareth Branwyn has collected over the past week.

Read more on MAKE

The post Tips of the Week: Cosplayer’s Project Notebooks, Silicon Molds from Caulk, Recip Saw Tips, Fire-Starting, and More appeared first on Make: DIY Projects and Ideas for Makers.

Tips of the Week: Cosplayer’s Project Notebooks, Silicon Molds from Caulk, Recip Saw Tips, Fire-Starting, and More

See what top tips Gareth Branwyn has collected over the past week.

Read more on MAKE

The post Tips of the Week: Cosplayer’s Project Notebooks, Silicon Molds from Caulk, Recip Saw Tips, Fire-Starting, and More appeared first on Make: DIY Projects and Ideas for Makers.

New Class of Metamaterials Changes Physical Properties in Seconds

Mechanical metamaterials can have their rigidity tuned, offering a new approach to soft robotics

A Smartphone App and 3D Printed Attachment for HIV Detection

There are more than 1.1 million people in the US living with HIV, and 1 in 7 of them don’t even know they have it. Young people are the most likely to be unaware of their infections. Globally, there were approximately 36.9 million people living with HIV in 2017, an estimated 1.8 million of them children.

Early detection of HIV is critical in preventing disease progression and transmission. Modern medications can inhibit the virus, allowing people with HIV to live long and healthy lives, but first, they need accessible and reliable diagnostics.

According to a recently published paper in Nature Communications, investigators from Brigham and Women’s Hospital have come up with a mobile diagnostic tool for detecting HIV and monitoring its management.

How It Works

With a single drop of blood, a 3D printed phone attachment can help identify the RNA nucleic acids of the virus.

The system detects HIV by studying the motion of bioengineered beads in a phone attachment. It also notably uses an alternative method for amplifying nucleic acids. Many scientists are already familiar with polymerase chain reaction (PCR), a method for duplicating a specific DNA segment. The researchers behind this study used loop mediated isothermal amplification (LAMP) instead, which is a faster method. LAMP’s rapid incubation time, efficiency, and specificity makes it particularly appropriate for point-of-care diagnostics, such as this phone-based system.

The system’s detection capability hinges on the use of micromotors. Tiny polystyrene beads move by themselves in a catalyzer. The system then studies the way that these micromotors interact with amplified nucleic acids. If HIV is present in the provided sample, it forms a tail on the micromotors that affects the motion.

The platform is expected to be particularly helpful in hard-to-reach areas where bulky medical equipment can’t be transported. The same type of technology could also be used to diagnose other viruses and bacteria rapidly and at a low cost.

When lives are on the line, detection precision is extremely important. This new platform detects HIV with 99 percent specificity. The material cost is less than $5 per test.

Fast, Cheap, and Portable

What does this mean for the world?

Given that antiretroviral drugs can control the virus and greatly reduce the risk of transmission, the continued suffering of those with HIV is in large part due to a lack of access to prevention, care, and treatment. A portable, fast, and low-cost diagnostic tool, powered in part by bioengineered catalytic motors, could be strategically essential.

Dr. Hadi Shafiee, co-author of the paper, is a faculty member at the Division of Engineering in Medicine and Renal Division of Medicine at Brigham and Women’s Hospital, Harvard Medical School, with an interest in developing innovative tools for global health. When I asked him about the platform, he said the world needs appropriate, low-cost diagnostics that measure viral load. Currently, viral load testing is complex, time-consuming, and expensive, which creates a significant barrier to diagnostics and treatment.

Shafiee described a scenario where a person might come from the middle of a jungle to a central lab somewhere in an African country, provide a blood sample, then have to leave because it takes time for the test to be performed—with no guarantee of being able to return for their test results and treatment information.

“There is a missed connection and then you cannot prescribe the right regimen,” explained Shafiee. “So the idea is that a health worker takes [the mobile device] in their backpack and goes to the site and gets the test done in less than an hour for a patient, and they make a decision right there.” The test can also be performed at a central lab or local physician’s office.

“Although we have a great treatment that works, a lot of people don’t have access to it, or when they get access to it, they don’t have the right regimens. So we get drug failure and drug resistance that adds to the complexity of the problem,” said Shafiee.

Hope For the Future

One day, it might also be possible to have an iteration of the kit that allows for self-testing. Over-the-counter technology or mail-in testing could correctly diagnose HIV, infectious diseases, and sexually transmitted diseases, providing a flexibility that is comparable to glucose meters or pregnancy tests already on the market. “That is doable, but going to take time,” Shafiee said.

Such technologies could be helpful in protecting privacy and working against stigma. For example, Shafiee pointed out that men actually contribute to 50 percent of infertility cases. However, the pressure is mostly on women, and when men go through the process of fertility testing in clinics, they often feel embarrassment.

A more private solution involving at-home analysis could reduce this problem. Shafiee expects that many similar diagnostic devices will hit the market in the future, addressing a wide range of medical needs.

Image Credit: ImageFlow /