As an avid tennis player, I also enjoyed meeting Tennibot this week. The autonomous ball-gathering robot sweeps the court like a roomba sucking up dust off a rug. In order to accomplish this task, without knocking over players, it navigates around the cage utilizing six cameras on each side. This is a perfect example of the type of job that an unmanned system excels at performing, freeing up athletes from wasting precious court time with tedious cleanup. Yet, Tennibot, at the end of the day, is a dumb appliance. While it gobbles up balls quicker than any person, it is unable to discern the quality of the game or the health of players.

No one expects Tennibot to save Roger Federer’s life, but what happens when a person has a heart attack inside a self-driving car on a two-hour journey? While autonomous vehicles are packed with sensors to identify and safely steer around cities and highways, few are able to perceive human intent. As Ann Cheng of Hyundai explains, “We [drivers] think about what that other person is doing or has the intent to do. We see a lot of AI companies working on more classical problems, like object detection [or] object classification. Perceptive is trying to go one layer deeper—what we do intuitively already.” Hyundai joined Jim Adler’s Toyota AI Ventures this month in investing Perceptive Automata, an “intuitive self-driving system that is able to recognize, understand, and predict human behavior.”

As stated by Adler’s Medium post, Perceptive’s technology uses “behavioral science techniques to characterize the way human drivers understand the state-of-mind of other humans and then train deep learning models to acquire that human ability. These deep learning models are designed for integration into autonomous driving stacks and next-generation driver assistance systems, sandwiched between the perception and planning layers. These deep learning, predictive models provide real-time information on the intention, awareness, and other state-of-mind attributes of pedestrians, cyclists and other motorists.”

While Perceptive Automata is creating “predictive models” for outside the vehicle, few companies are focused on the conditions inside the cabin. The closest implementations are a number of eye-tracking cameras that alert occupants to distracted driving. While these technologies observe the general conditions of passengers, they rely on direct eye contact to distinguish between emotions (fatigue, excitability, stress, etc.), which is impossible if one is passed out. Furthermore, none of these vision systems have the ability to predict human actions before they become catastrophic.

Isaac Litman, formerly of Mobileye, understands fully well the dilemma presented by computer vision systems in delivering on the promise of autonomous travel. In speaking with Litman this week about his newest venture Neteera, he declared that in today’s automative landscape the “the only unknown variable is the human.” Unfortunately, the recent wave of Tesla and Uber autopilot crashes has glaringly illustrated the importance of tracking the attention of vehicle occupants in handing off between autopilot systems and human drivers. Litman further explains that Waymo and others are collecting data on occupant comfort as AI-enabled drivers have reportedly led to high levels of nausea from driving too consistently. Litman describes this as the indigestion problem, clarifying that after eating a big meal one may want to drive more slowly than on an empty stomach. In the future Litman professes that autonomous cars will be marketed “not by the performance of their engines, but on the comfort of their rides.”

Litman’s view is further endorsed by the recent patent application filed this summer by Apple’s Project Titan team for developing “Comfort Profiles” for autonomous driving. According to AppleInsider, the application “describes how an autonomous driving and navigation system can move through an environment, with motion governed by a number of factors that are set indirectly by the passengers of the vehicle.” The Project Titan system would utilize a fusion of sensors (LIDAR, depth cameras, and infrared) to monitor the occupants’ “eye movements, body posture, gestures, pupil dilation, blinking, body temperature, heart beat, perspiration, and head position.” The application details how the data would integrate into the vehicle systems to automatically adjust the acceleration, turning rate, performance, suspension, traction control and other factors to the personal preferences of the riders. While Project Titan is taking the first step toward developing an autonomous comfort system, Litman expresses that it is limited by the inherent short-comings of vision-based systems that are susceptible to light, dust, line of sight, condensation, motion, resolution, and safety concerns.

Unlike vision sensors, Neteera is a cost-effective micro-radar on a chip that leverages its own network of proprietary algorithms to provide “the first contact free vital sign detection platform.” Its FDA-level of accuracy is not only being utilized by the automative sector, but healthcare systems across the United States for monitoring such elusive conditions as sleep apnea and sudden infant death syndrome. To date, the challenge of monitoring vital signs through micro-skin motion in the automotive industry has been the displacement caused by a moving vehicles. However, Litman’s team has developed a a patent-pending “motion compensation algorithm” that tracks “quasi-periodic signals in the presence of massive random motions,” providing near perfect accuracy (see tables below).

While the automotive industry races to launch fleets of autonomous vehicles, Litman estimates that the most successful players will be the ones that install empathic engines into the machines’ framework. Unlike the crowded field of AI and computer vision startups that are enabling robocars to safely navigate city streets, Neteera’s “intuition on a chip” is probably one of the only mechatronic ventures that actually report on the psychological state of drivers and passengers. Litman’s innovation has wider societal implications, as social robots begin to augment humans in the workplace and support the infirm and elderly in coping with the fragility of life.

As scientists improve artificial intelligence, it is still unclear what the reaction will be from ordinary people to such “emotional” robots. In the words of writer Adam Williams, “Emotion is something we reserve for ourselves: depth of feeling is what we use to justify the primacy of human life. If a machine is capable of feeling, that doesn’t make it dangerous in a Terminator-esque fashion, but in the abstract sense of impinging on what we think of as classically human.”

The goal of CyberNYC is to train close to 5% of the city’s working population to become “cyber specialists.” In order to accomplish this lofty objective, the NYCEDC forged an educational partnership with CUNY, NYU, Columbia, Cornell Tech, and iQ4. One of the most compelling aspects of the partnership is the advanced degree program offered by CUNY and Facebook, enabling students to achieve a masters in computer science in just a year through the online educational hosting site EdX, which also enables users to stack credentials from other universities.

As Anant Agarwal, CEO of edX, explains, “The workplace is changing more rapidly today than ever before and employers are in need of highly-developed talent. Meanwhile, college graduates want to advance professionally, but are realizing they do not have the career-relevant skills that the modern workplace demands. EdX recognizes this mismatch between business and education for learners, employees and employers. The MicroMasters initiative provides the next level of innovation in learning to address this skills gap by creating a bridge between higher education and industry to create a skillful, successful 21st-century workforce.”

Realizing that not everyone is cut out for higher education, the Big Apple is also working to create boot camps to upskill existing tech operators in a matter of weeks with industry-specific cyber competencies. Fullstack Academy is leading the effort to create a catalogue of intensive boot camps throughout the boroughs. LaGuardia Community College (LAGCC) is also providing free preparatory courses for adults with minimum computing proficiency in order to qualify for Fullstack’s programs. Most importantly, LAGCC will act as a liaison to CyberNYC’s corporate partners to match graduates with open positions.

In 2012, Sebastian Thrun famously declared that, “access to higher education should be a a basic human right.” Thrun, who left his position of running Google X to “democratize education” worldwide by launching a free online open-university platform, Udacity, is now transforming the learning paradigm. The German inventor is no stranger to innovation, in 2011 he unveiled at the Ted Conference one of the first self-driving cars, inspired by losing his best friend to a car accident. Similar to CyberNYC’s fast-track masters in computer science program, Udacity is teamed up with AT&T and Georgia Tech to offer similar degrees for less than $7,000 (compared to $26,860 for an on-campus program).

In the words of AT&T’s Chief Executive Randall Stephenson, “We believe that high-quality and 100 percent online degrees can be on par with degrees received in traditional on-campus settings, and that this program could be a blueprint for helping the United States address the shortage of people with STEM degrees, as well as exponentially expand access to computer science education for students around the world.”

In 2003, Reid Hoffman launched LinkedIn, the first business social network. Today, there are more than a half billion profiles (resumes) posted on the site. Last March, Hoffman sat down with University of California President (and former director of Homeland Security) Janet Napolitano to discuss the future of education. The leading advocate for entrepreneurship explained that he believes everyone should be “in permanent beta,” whereby one is constantly consuming information. Hoffman states that this is the only way an individual and a society will be able to compete in a world driven by data and artificial intelligence. Universities, like the UC system, Hoffman suggests should move towards a cross-disciplinary system. As Hoffman espouses, “What we’re actually in fact primarily teaching is that learning how to learn as you get to new areas, not areas where it’s necessarily the apprenticeship model, which is we teach you this thing and you know how to do this one thing. You know how to do this thing really well, but actually, in fact, you’re going to be crossing domains. That’s how I would somewhat shift the focus overall in terms of thinking about it.”

In his book, “Thank You For Being Late: An Optimistic’s Guide To Thriving In The Age Of Accelerations,” Thomas Friedman quotes Nest Labs’ founder, Tony Fadell, as asserting that the future economy rests on businesses’ ability of turning “AI into IA,” or “Intelligent Assistants.” Friedman specifically singled out LinkedIn as one of these IAs that are creating human networks to amplify people in finding the best opportunities and highest demanded skills. In order to utilize his IA, Hoffman advised Napolitano’s audience to be versatile, “As opposed to thinking about this as a career ladder, a career escalator, to think of it as more of a career jungle gym, that you’re actually going to be changing around in terms industries. The exact shape of certain different job professions will change, and that you need to be adaptive with that.” He continued, “I do think that the notion that is still too often preached, which is you go to college, you discover your calling, and that’s your job for the next 50 years, that’s gone. ” The key to harnessing this trend says Hoffman is “to constantly be learning and to be learning new things. Some of them by taking ongoing classes but some of them also by doing, and talking to people and finding out what the relevant things are, and then tracking what’s going on.”

All these efforts are not happening fast enough in the United States to fill the current gap between the 6.9 million job openings and the number of unemployed. While the unemployment rate is at a forty year low with 6.2 million workers out of work, there still is a significant disparity with more open job listings. The primary reason stated by business leaders across the nation is that the current class of applicants lack the versatility of skills required for the modern workplace, resulting in the push towards full automation. Eric Schmidt, former Executive Chairman of Google and Alphabet, claims that “Today we all live and work in a new era, the Internet Century, where technology is roiling the business landscape and the pace of change is accelerating.” This Internet Century (and by extension cloud computing and unmanned systems) requires a new type of worker, which he affectionally calls the “smart creative” that is first and foremost an “adaptive learner.”

The deficiency of graduating “smart creatives” could be the reason why America, which is almost at full employment, is still producing historically low output resulting is stagnant wages. Mark Zandi, Moody’s Chief Economist, explains, “Wage growth feels low by historical standards and that’s largely because productivity growth is low relative to historical standards. Productivity growth between World War II and up through the Great Recession was, on average, 2 percent per annum. Since the recession 10 years ago, it’s been 1 percent.” The virtuous efforts of CyberNYC, and other grassroots initiatives, are only the first of many towards the complete restructuring of America’s educational framework to nurture a culture of smart creatives that are in permanent beta.

Located a few miles from the 174th is the Genius NY Accelerator. Genius boasts of the largest competition for unmanned systems in the world. Previous winners that received one million dollars, include: AutoModality and FotoKite. One of Genius’ biggest financial backers is the Empire State Development (ESD). Last month, I moderated a discussion in New York City between Sharon Rutter of the ESD, Peter Kunz of Boeing Horizon X and Victor Friedberg of FoodShot Global. These three investors spanned the gamut of early stage funders of autonomous machines. I started our discussion by asking if they think New York is poised to take a leading role in shaping the future of automation. While Kunz and Friedberg shared their own perspectives as corporate and social impact investors accordingly, Rutter singled out one audience participant in particular as representing the future of New York’s innovation venture scene.

Andrew Hong of ff Venture Capital sat quietly in front of the presenters, yet his firm has been loudly reshaping the Big Apple’s approach to investing in mechatronics for almost a decade (with the ESD as a proud limited partner). Founded in 2008 by John Frankel, formerly of Goldman Sachs, ff has deployed capital in more than 100 companies with market values of over $6 billion. As the original backer of crowd-funding site Indiegogo, ff could be credited as a leading contributor to a new suite of technologies. As Frankel explains, “We like hardware if it is a vector to selling software, as recurring models based on services lead to better economics for us than one-off hardware sales.” In the spirit of fostering greater creativity for artificial intelligence software, ff collaborated with New York University in 2016 to start the NYU/ffVC AI NexusLab — the country’s first AI accelerator program between a university and a venture fund. NexusLab culminated in the Future Labs AI Summit in 2017. Frankel describes how this technology is influencing the future of autonomy, “As we saw that AI was coming into its own we looked at AI application plays and that took us deeper into cyber security, drones and robotics. In addition, both drones and robotics benefited as a byproduct of the massive investment into mobile phones and their embedded sensors and radios.  Thus we invested in a number of companies in the space (Skycatch, PlusOne Robotics, Cambrian Intelligence and TopFlight Technologies) and continue to look for more.”

Recently, ff VC bolstered its efforts to support the growth of an array of cognitive computing systems by opening a new state-of-the-art headquarter in the Empire State Building and expanding its venture partner program. In addition to providing seed capital to startups, ff VC has distinguished itself for more than a decade by augmenting technical founders with robust back-office services, especially accounting and financial management. Last year, ff also widened its industry venture partner program with the addition of Dr. Kathryn Hume to its network. Dr. Hume is probably best known for her work as the former president of Fast Forward Labs, a leading advisory to Fortune 500 companies in utilizing data science and artificial intelligence. I am pleased to announce that I have decided to join Dr. Hume and the ff team as a venture partner to widen their network in the robotics industry. I share Frankel’s vision that today we are witnessing “massive developments in AI and ML that had led to unprecedented demand for automation solutions across every industry.”

ff’s commitment is not an isolated example across the Big Apple but part of a growing invigorated community of venture capitalists, academics, inventors, and government sponsors. In a few weeks, New York City Economic Development Corporation (NYCEDC) will officially announce the winner of a $30 million investment grant to boost the city’s cybersecurity ecosystem. CyberNYC will include a new startup accelerator, city-wide programming, educational curricula, up-skilling/job placement, and a funding network for home-grown ventures. As NYCEDC President and CEO James Patchett explains, “The de Blasio Administration is investing in cybersecurity to both fuel innovation, and to create new, accessible pathways to jobs in the industry. We’re looking for big-thinking proposals to help us become the global capital of cybersecurity and to create thousands of good jobs for New Yorkers.” The Mayor’s office projects that its initiative will create 100,000 new jobs over the next ten years, enabling NYC to fully maximize the opportunities of an autonomous world.

The inspiration for CyberNYC could probably be found in the sands of the Israeli desert town of Beer Sheva. In the past decade, this bedouin city in the Holy Land has been transformed from tents into a high-tech engine for cybersecurity, remote sensing and automation technologies. At the center of this oasis is Cyber Labs, a government-backed incubator created by Jerusalem Venture Partners (JVP). Next week, JVP will kick off its New York City “Hub” with a $1 million competition called “New York Play” to bridge the opportunities between Israeli and NYC entrepreneurship.  In the words of JVP’s Chairman and Founder, Erel Margalit, “JVP’s expansion to New York and the launch of New York Play are all about what’s possible. As New York becomes America’s gateway for international collaboration and innovation, JVP, at the center of the “Startup Nation,” will play a significant role boosting global partnerships to create solutions that better the world and drive international business opportunities.”

Looking past the skyscrapers, I reflect on Margalit’s image of New York as a “Gateway” to the future of autonomy.  Today, the wheel of New York City is turning into a powerful hub, connected throughout America’s academic corridor and beyond, with spokes shooting in from Boston, Pittsburgh, Philadelphia, Washington DC, Silicon Valley, Europe, Asia and Israel. The Excelsior State is pulsing with entrepreneurial energy fostered by the partnerships of government, venture capital, academia and industry. As ff VC’s newest venture partner, I personally am excited to play a pivotal role in helping them harness the power of acceleration for the benefit of my city and, quite possibly, the world.

Come learn how New York’s Retail industry is utilizing robots to drive sales at the next RobotLab on “Retail Robotics” with Pano Anthos of XRC Labs and Ken Pilot, formerly President of Gap on October 17th, RSVP today.

Weather is often the nemesis of autonomous missions. The example of NASA’s billion-dollar fleet being disabled by mere dust illustrates the profound problem that machines are far from being self-reliant. Closer to Earth, high atmospheric ice is one of the biggest challenges for Unmanned Aerial Vehicles (UAVs). This natural phenomena is responsible for grounding even the most critical military drone operations, as freezing conditions adversely affect aerodynamics and performance.

The paradigm of reversing nature in aeronautics is useful for all aspects of autonomous systems. Starting in the late 1980s, NASA set out to develop a series of deicing solutions to make aviation safer. The most famous of these technologies is the Electro-Expulsive Separation System (EESS) or “Ice Zapper” developed by NASA engineer Leonard Haslim. According to the space agency’s website, the Zapper “employs a pair of conductors embedded in a flexible material and bonded to the aircraft’s frame—on the leading edge of a wing, for example. A pulsing current of electricity sent through the conductors creates opposing magnetic fields, driving the conductors apart only a fraction of an inch but with the power to shatter any ice buildup on the airframe surface into harmless particles.”

The Zapper was eventually licensed by NASA to a private company, Ice Management Systems (IMS), which was acquired by Thompson Ramo Wooldridge (TRW), now part of Northrop Grumman. Haslim’s invention today is a complete aeronautic platform that is embedded within an energy-efficient power system, conductive actuators and an internal carbon-fiber core. According to IMS founder Mark Bridgeford, the platform has already been deployed on the Northrop Grumman’s Hunter UAVs and General Atomics Sky Warriors. “One reason we have hit a spot with the UAV business is the simple fact that our system utilizes so little energy,” explains Bridgeford. The IMS platform is one of the first mechatronic systems that is energy efficient. Bridgeford comtinues, “With the electroexpulsive technology, UAVs are able to incorporate continuous, year-round ice protection into their airframes.”

Energy consumption is the key to active deicing. Unlike anti-icing systems that prevent particle buildup through material design and engine back-flow systems, deicing is typically manually implemented in manned aircraft, on an as-needed basis to conserve fuel. Drones do not have this luxury of flipping a switch and need active pervasive monitoring. While the Zapper and other electro-expulsive methods shoot bursts of energy to shake ice off of wings, this method has underdelivered for many unmanned missions. Aerospace solutions provider Cox & Company is now offering a competing technology called “Thermo-Mechanical Expulsion Deicing System (TMEDS)” for Northrup Grumman’s Triton drones. Cox claims that this technology not only delivers better results for unmanned systems, but also consumes less power.

Inspired by the US Coast Guard’s ice breaking marine technology, Dr. Kim Sorensen joined NASA’s Ames Research Center as part of his doctoral thesis on aerial ice protection systems. Soon after, Sorensen launched the first autonomous deicing flight in Alaska using an ultra-lightweight one-meter drone. The success of this prototype led to him establishing Ubiq Aerospace in 2017 to bring cost-effective automated deicing to the commercial UAV sector. Based in Sorensen’s home country of Norway, Ubiq aims to be customer ready by the end of 2019. “The D•ICE technology was based on three design principles – No Weight, No Volume, No Power. Aside from these principles we also needed to ensure that the system could operate in complete autonomy,” says Dr. Sorensen. By contrast “Various systems have been designed and are flying on the commercial manned aircraft that fly today. All these systems require human interaction in various degrees. They carry penalties as most are very heavy, all of them are structurally invasive, and/or some are environmentally harmful… As such, none of the existing icing protection systems are suitable for use on unmanned aircraft,” explains Dr. Sorensen.

According to Aviation International News Ubiq has filed its initial patents and is currently in discussions with manufacturing partners to build its “carbon-nano thermal panel components” to meet industry standards. The large-scale platform will be composed of these thermal panels, as well as embedded temperature sensors, power components, and a computational brian. One of Ubiq’s differentiating features is its intelligent algorithms that automatically enable safe deicing with minimum power drain  by switching the unit on and off as needed. Sorensen describes, “There is a lower boundary for what power you do require to ensure that icing doesn’t build or you can deice, and we are trying to get as close to that boundary as possible.” The company is already working with Norway’s defense department on expanding its applications to rotor UAVs, potentially accelerating the use cases for large-scale cargo missions and, even, flying cars. In an email to me earlier this week, Ubiq’s founder states, “The D•ICE tech has wider use indeed. There are several secondary industries that would have use for a solution like this. Most obvious is the wind energy industry with their enormous ocean-based farms.” Ironically, many terrestrial wind turbine operators are turning to drone service providers today for deicing solutions as demonstrated in numerous YouTube videos.

As robotics continues to cross the chasm from lab research to field operations, weather-control systems are becoming increasingly more critical. The center for subfreezing temperature research is the University of Alaska in Fairbanks. According to a press release this past May, The University has partnered with the U.S. Department of Transportation to help shape the “the future of drones in America” and “speed up the safe entry of unmanned vehicles into the nation’s airspace.” A key component of this endeavor is researching technological solutions for unmanned systems operating in harsh climate environments “to deliver medical devices to remote areas, help with search and rescue operations, survey fish and wildlife, and monitor pipelines, roads and other infrastructure.” In announcing the partnership, U.S. Secretary of Transportation Elaine Chao declared, “We are looking forward to helping today’s winners unlock the enormous potential of drone operations, which will create new services and jobs in their local communities.” Besides the economic opportunity that Secretary Chao cites, there could be other “winners” altruistically driven by the pursuit of science on Earth, Mars and, maybe, the entire galaxy.

In the words of Dr. Morgan Pope of Disney Research, “We’d like to do something that’s not just human, but beyond human… The hope here is that we’re delivering something physical and tangible, as opposed to virtual and digital.” The research of creating the  “beyond human” could field more important products than just theme-park animatronics. According to Dr. Pope’s research, the “Stickman” (video above)  is a “two degree freedom robot that uses a gravity-driven pendulum launch and produces a variety of somersaulting stunts.” By leveraging the physics of the streamlined design the robot is able to swing “through the air on a pendulum, ‘tucks’ to change its moment of inertia, releases, ‘untuck;” to reduce its spin, and gracefully lands on its back on a foam mat.” The robot’s dynamics are controlled by “an IMU [inertial measurement unit] and a laser range-finder to estimate its state mid-flight and actuates to change its motion both on and and off the pendulum.”

In an IEEE Spectrum article authored by Dr. Pope, the scientist shares his inspiration for Stickman – the gold-medal gymnast Simone Biles. In describing Biles signature move, Pope writes, “It’s a beautiful example of how the seemingly simple physics of ballistic motion, completely governed by a relatively simple conservation of angular momentum, can produce amazing and unexpected results.” Beyond replacing movie stuntmen with pendulum swinging robots, Disney research aims to leverage its platform to accomplish two major objectives: 1) teach science and engineering and 2) push the boundaries of what is possible with machines. Dr. Pope exclaims, “We saw two potential benefits of building robots that could perform acrobatic stunts while aloft. First, a robotic performer can answer questions about how a performance is accomplished that are more difficult to answer with a human performer… Second, the force, speed, and precision required to execute acrobatic manoeuvres pushes the limits of robot capability in a way that has the potential to be relevant to the broader field.”

Disney’s pursuit of science is not just for academic amusement but offers broad benefits today for the mechatronic industry. Parallel to Dr. Pope’s work with Stickman, Disney Research published another report on human-to-robot handovers. In their experiment the researchers examined how robot behaviors influence human interactions in an effort to facilitate greater trust between people and machines. The paper states: “We find the robot’s initial pose can inform the giver about the upcoming handover geometry and impact fluency and efficiency. Also we find variations in grasp method and retraction speed induce significantly different interaction forces.” The Disney team created a “Robot Social Attributes Scale (RoSAS)” that could be useful for future collaborative robot deployments. The study concluded that “ratings of the robot’s warmth linearly increased over repeated interactions, while discomfort simultaneously decreased. This suggests that the more people interact with the robot, the more they develop positive attitudes towards the robot. Both warmth and discomfort are known factors in the determination of trustworthiness of both humans and robots.” The researchers hope that their work will be used to address future challenges with handing objects between humans and robots, by providing ” inexperienced users with additional feedback information to improve legibility of robot behaviors during handover and other interaction contexts – e.g., audio or haptic feedback through wearable devices.” Curiously, the team also suggests that their “human-centered approaches” could even speed the adoption of social robots by leading to more mindful designs, programmed interactions, and fluid robotic movements.

Disney Research is not the only division within the multi-billion dollar entertainment conglomerate nurturing hardware invention. Three of the last four cohorts of the Disney Accelerator have included robot companies: Sphero (2014), Hanson Robotics (2016), and Savioke (2017) . Each of these startups has created new paradigms of human-social interactions, from STEM-centric toys to life-sized humanoids to autonomous delivery platforms. In addition to providing capital, mentorship and resources, Disney offers a playground of synergies and market opportunities. For example, the most popular Sphero products are the licensed characters from Star Wars, Pixar, and Marvel. Hanson Robotics is under contract to create humanoids modeled after Disney’s iconic characters and Savioke is piloting its Relay robotic delivery system with Disney World Resorts. The incubator has also backed a number of chatbot, AI software, consumer hardware and other emerging technology companies that are being harnessed throughout the brand.

For the past fifty years Disney has been using robots, many still with hydraulic actuators to enhance the guest experiences of its theme parks. Charged with managing the future of these mechanical characters is Martin Buehler, Executive R&D Imagineer at Walt Disney Imagineering. Unlike the eighty PhDs working in Disney Research that wrestle with the theoretical, Imagineers are tasked with building and implementing new immersive rides today. Last September, Buehler delivered the keynote speech at RoboBusiness in Silicon Valley. In showing off his team’s latest pursuits with the worlds of Avatar and Star Wars (opening 2019), Buehler suggested that Disney’s art of storytelling and hardware could be a model for even mundane companies. Storytelling, Buehler noted, could further the deployment of social and collaborative robots within society by finally delivering on the promise of improved quality of life through automation. In particular, Buehler highlighted how robots with positive personalities could ease the pain of aging in place. Reflecting on Buehler’s presentation, I am reminded of Walt Disney’s famous quote, “That’s what we storytellers do. We restore order with imagination. We instil hope again and again and again.”

Save The Date: The next RobotLab will take place October 17th at 6pm in New York City with Pano Anthos Of XRC Labs exploring the impact of  Robots In The Age Of The Retail Apocalypse,” as seating is limited, reserve today.

Sitting in New York City, looking up at the clear June skies, I wonder if I am staring at an endangered phenomena. According to many in the Unmanned Aircraft Systems (UAS) industry, skylines across the country soon will be filled with flying cars, quadcopter deliveries, emergency drones, and other robo-flyers. Moving one step closer to this mechanically-induced hazy future, General Electric (GE) announced last week the launch of AiRXOS, a “next generation unmanned traffic” management system.

Managing the National Airspace is already a political football with the Trump Administration proposing privatizing the air-control division of the Federal Aviation Administration (FAA), taking its controller workforce of 15,000 off the government’s books. The White House argues that this would enable the FAA to modernize and adopt “NextGen” technologies to speed commercial air travel. While this budgetary line item is debated in the halls of Congress, one certainty inside the FAA is that the National Airspace (NAS) will have to expand to make room for an increased amount of commercial and recreational traffic, the majority of which will be unmanned.

Ken Stewart, the General Manager of AiRXOS, boasts, “We’re addressing the complexity of integrating unmanned vehicles into the national airspace. When you’re thinking about getting a package delivered to your home by drone, there are some things that need to be solved before we can get to that point.” The first step for the new division of GE is to pilot the system in a geographically-controlled airspace. To accomplish this task, DriveOhio’s UAS Center invested millions in the GE startup. Accordingly, the first test deployment of AiRXOS will be conducted over a 35 mile stretch of Ohio’s Interstate 33 by placing sensors along the road to detect and report on air traffic. GE states that this trial will lay the foundation for the UAS industry. As Alan Caslavka, president of Avionics at GE Aviation, explains, “AiRXOS is addressing the rapid changes in autonomous vehicle technology, advanced operations, and in the regulatory environment. We’re excited for AiRXOS to help set the standard for autonomous and manned aerial vehicles to share the sky safely.”

Stewart whimsically calls his new air traffic control platform WAZE for drones. Like the popular navigation app, AiRXOS provides drone operators with real-time flight-planning data to automatically avoid obstacles, other aircraft, and route around inclement weather. The company also plans to integrate with the FAA to streamline regulatory communications with the agency. Stewart explains that this will speed up authorizations as today, “It’s difficult to get [requests] approved because the FAA hasn’t collected enough data to make a decision about whether something is safe or not.” 

NASA is a key partner in counseling the FAA in integrating commercial UAS into the NAS. Charged with removing the “technical and regulatory barriers that are limiting the ability for civil UAS to fly in the NAS” is Davis Hackenberg of NASA’s Armstrong Flight Research Center. Last year, we invited Hackenberg to present his UAS vision to RobotLabNYC. Hackenberg shared with the packed audience NASA’s multi-layered approach to parsing the skies for a wide-range of aircrafts, including: high altitude long endurance flights, commercial airliners, small recreational craft, quadcopter inspections, drone deliveries and urban aerial transportation. Recently the FAA instituted a new regulation mandating that all aircrafts be equipped with Automatic Dependent Surveillance-Broadcast (ADS-B) systems by January 1, 2020. The FAA calls such equipment “foundational NextGen technology that transforms aircraft surveillance using satellite-based positioning,” essentially connecting human-piloted craft to computers on the ground and, quite possibly, in the sky. Many believe this is a critical step towards delivering on the long-awaited promise of the commercial UAS industry with autonomous beyond visual line of sight flights.

I followed up this week with Hackenberg about the news of AiRXOS and the new FAA guidelines. He explained, “For aircraft operating in an ADS-B environment, testing the cooperative exchange of information on position and altitude (and potentially intent) still needs to be accomplished in order to validate the accuracy and reliability necessary for a risk-based safety case.” Hackenberg continued to describe how ADS-B might not help low altitude missions, “For aircraft operating in an environment where many aircraft are not transmitting position and altitude (non-transponder equipped aircraft), developing low cost/weight/power solutions for DAA [Detect and Avoid] and C2 [Command and Control Systems] is critical to ensure that the unmanned aircraft can remain a safe distance from all traffic. Finally, the very low altitude environment (package delivery and air taxi) will need significant technology development for similar DAA/C2 solutions, as well as certified much more (e.g. vehicles to deal with hazardous weather conditions).” The Deputy Project Manager then shared with me his view of the future, “In the next five years, there will be significant advancements in the introduction of drone deliveries. The skies will not be ‘darkened,’ but there will likely be semi-routine service to many areas of the country, particularly major cities. I also believe there will be at least a few major cities with air taxi service using optionally piloted vehicles within the 10-year horizon. Having the pilot onboard in the initial phase may be a critical stepping-stone to gathering sufficient data to justify future safety cases. And then hopefully soon enough there will be several cities with fully autonomous taxi service.”

Last month, Uber already ambitiously declared at its Elevate Summit that its ride-hail program will begin shuttling humans by 2023. Uber plans to deploy electric vertical take-off and landing (eVTOL) vehicles throughout major metropolitan areas. “Ultimately, where we want to go is about urban mobility and urban transport, and being a solution for the cities in which we operate,” says Uber CEO, Dara Khosrowshahi. Uber has been cited by many civil planners as the primary cause for increased urban congestion. Its eVTOL plan, called uberAIR, is aimed at alleviating terrestrial vehicle traffic by offsetting commutes with autonomous air taxis that are centrally located on rooftops throughout city centers.

One of Uber’s first test locations for uberAIR will be Dallas-Fort Worth, Texas. Tom Prevot, Uber’s Director of Engineering for Airspace Systems, describes the company’s effort to design a Dynamic Skylane Networks of virtual lanes for its eVTOLs to travel, “We’re designing our flight paths essentially to stay out of the scheduled air carriers’ flight paths initially. We do want to test some of these concepts of maybe flying in lanes and flying close to each other but in a very safe environment, initially.” To accomplish these objectives, Prevot’s group signed a Space Act Agreement with NASA to determine the requirements for its aerial ride-share network. Using Uber’s data, NASA is already simulating small-passenger flights around the Texas city to identify potential risks to an already crowded airspace.

After the Elevate conference, media reports hyped the immanent arrival of flying taxis. Rodney Brooks (considered by many as the godfather of robotics) responded with a tweet: “Headline says ‘prototype’, story says ‘concept’. This is a big difference, and symptomatic of stupid media hype. Really!!!” Dan Elwell, FAA Acting Administrator, was much more subdued with his opinion of how quickly the technology will arrive, “Well, we’ll see…”

Editor’s Note: This week we will explore regulating unmanned systems further with Democratic Presidential Candidate Andrew Yang and New York State Assemblyman Clyde Vanel at the RobotLab forum on “The Politics Of Automation” in New York City. 

Since the early 1990s, machines have relied on rechargeable lithium ion batteries for power. However these storage cells (inside most cell phones, and now Spot Mini) are dangerously combustable, easily degradable, and very expensive. One of the best examples of the instability of lithium ion is the Samsung Note 7 handset recall, after exploding units caused havoc to consumers. The design flaw ended up costing Samsung $6.2 billion, and even prompted the Federal Aviation Administration (FAA) to issue an advisement after paniced flyers saw cellphones overheat. Exploding batteries are not limited to Samsung, but include the entire lithium ion appliance ecosystem, including e-cigarettes, hoverboards, toys, and electric vehicles.

While Marc Raibert was showing off his latest mechanical creation, across the river in Woburn, Mass Iconic Materials was opening its new 30,000 square-foot lab. The hot startup grabbed headlines months ago with a $65 million venture capital investment from the new Renault-Nissan-Mitsubishi Alliance, Total Energy Ventures, and Sun Microsystems co-founder Bill Joy. However, the real news story is their revolutionary solid-state lithium battery technology that is cheaper, less flammable and longer lasting. The technology came out of the research of Dr. Michael Zimmerman of Tufts University that was originally aimed at improving the performance of existing lithium ion batteries. Unlike lithium ion batteries that contain flammable liquid electrolyte, Zimmerman’s invention deploys a solid polymer electrolyte preventing short-circuiting. Iconic’s plastic electrolyte not only prevents explosive gases from escaping, but enables the composition battery to be constructed with higher energy density materials, such as pure lithium anodes.

Zimmerman first unveiled his innovation on PBS NOVA last year, whereby the famed host David Pogue tested it by poking the solid-state lithium cell with a screwdriver and scissors. Typically, such stress would immediately cause liquid electrolyte to explode, but Zimmerman’s battery did not even heat up and kept working. Pogue then noted, “If you can use lithium metal rather than lithium ions, you get five to ten times the energy density. That means ten days on a charge instead of one day on a charge, a thousand miles on a charge of your car instead of 200 miles.”

The promise of Iconic’s technology helps alleviate “range anxiety,” the fear of running out of charge without a power source nearby. The future of robots, and especially autonomous vehicles, relies heavily on investments in infrastructure to rival oil. Today, it takes 75 minutes to fully recharge the 7,104 lithium ion batteries inside a Tesla at one of of its 5,000 supercharging stations, compared to 15 minutes at the pump at more than 165,000 gas stations throughout America. Realizing the shortcomings of switching to electric, Sweden is making country-wide investments to accelerate adoption. Last month, Stockholm opened the first stretch of roadway capable of simultaneously charging vehicles while driving.

Markus Fischer, spokesperson for state-owned energy company Vattenfall, describes “Such roads will allow (electric vehicles) to move long distances without big, costly and heavy batteries. The investment cost per kilometer is estimated to be less than that of using overhead lines, as is the impact on the landscape.” Currently, only 1.2 miles of electric rail has been laid, but it is already working with trucks making deliveries to the airport. Gunnar Ashland, CEO of Elways the maker of the road’s electric rail, boasted, “The technology offers infinite range — range anxiety disappears. Electrified roads will allow smaller batteries and can make electric cars even cheaper than fossil fuel ones.”

At the Robotics Summit in Boston, I spoke with Dr. Doug Schmidt of electric battery provider Acumentrics about the Swedish technology. Dr. Schmidt explained that most conductive charging platforms similar to Elways speed the degradation of lithium ion batteries. Israeli startup Phinergy offers an alternative to lithium for electric vehicles with their proprietary aluminum batteries that produce energy through a reaction between oxygen and aluminum using water. A few years ago, Phinergy powered a Renault car for over a thousand kilometers with just tap water. Now the company has partnered with Chinese-based Yunnan Aluminum to begin manufacturing batteries to meet China’s growing electric automobile market. According to the press release last month the joint venture “will introduce the world’s leading aluminum-air battery technology, relying on [Yunnan Aluminium’s] green and clean water and aluminum resources.” The statement further detailed that the initial annual output will be 2,500 units. Phinergy’s website promotes wider uses cases, including industrial robots and other unmanned systems. 
China has been leading the world in alternative energy development. Last year, Pittsburgh-based Aquion was acquired out of bankruptcy for $9.16 million by Juline-Titans, an affiliate of China Titans Energy Technology Group. Aquion, a once high-flying startup that raised more than $190 million from such notable investors as Bill Gates, Kleiner Perkins Caufield & Byers, and Nick and Joby Pritzker, is now in the process of moving its operations to Asia. Similar to Phinergy, Aquion utilizes the most renewable of resources, water. Their patented “Aqueous Hybrid Ion” technology is able to create clean energy using sea water. However, it comes at a cost of weight: unlike lithium batteries that are light enough to fit in one’s pocket, salt-water fuel cells are considerably heavier. The company’s products are uniquely positioned to be utilized for future power grids, with the promise of weaning the world off fossil fuels.  
Today, fewer than 5% of lithium-ion batteries are recycled. The environmental costs could not be higher with dangerous toxic gases leaking from old batteries. Rising battery demand is also leading to a variety of unintended consequences, such as depleting the world’s natural resources of lithium and cobalt and increased water pollution from mineral extraction. While turning the tides of climate change depends greatly on ending the global dependency on oil, replacing it with a more green alternative is crucial. Promising inventions are not only developing new energy paradigms, but recycling old ones in innovative ways. British startup Aceleron is reusing dead electric car batteries for home energy storage. In the words of Amrit Chandan of Aceleron, “It takes so much energy to extract these materials from the ground. If we don’t re-use them we could be making our environmental problems worse. There’s going to be a storm of electric vehicle batteries that will reach the end of their life in a few years, and we’re positioning ourselves to be ready for it.”

Climate change and unmanned systems will be discussed in greater detail at the next RobotLab on “The Politics Of Automation,” June 13th @ 6pm in NYC, with Democratic Presidential Candidate Andrew Yang and New York State Assemblyman Clyde Vanel.

Three and half years ago, I stood on the corner of West Street and gasped as two window washers clung to life at the end of a rope a thousand feet above. By the time rescue crews reached the men on the 69th floor of 1 World Trade they were close to passing out from dangling upside down. Everyday risk-taking men and women hook their bodies to metal scaffolds and ascend to deadly heights for $25 an hour. Ramone Castro, a window washer of three decades, said it best, “It is a very dangerous job. It is not easy going up there. You can replace a machine but not a life.” Castro’s statement sounds like an urgent call to action for robots.

One of the promises of automation is replacing tasks that are too dangerous for humans. Switzerland-based Serbot believes that high-rise facade cleaning is one of those jobs ripe for disruption. In 2010, it was first reported that Serbot contracted with the city of Dubai to automatically clean its massive glass skyline. Utilizing their GEKKO machine, the Swiss company has demonstrated a performance of over 400 square meters an hour, 15 times faster than a professional washer. GEKKO leverages a unique suction technology that enables the massive Roomba-like device to be suspended from the roof and adhere to the curtain wall regardless of weather conditions or architectural features. Serbot offers both semi and full autonomous versions of its GEKKOs, which include options for retrofitting existing roof systems. It is unclear how many robots are actually deployed in the marketplace, however Serbot recently announced the cleaning of the architecturally challenging FESTO’s Automation Center in Germany (shown below).

According to the press release, “The entire building envelope is cleaned automatically: by a robot, called GEKKO Facade, which sucks on the glass facade. This eliminates important disadvantages of conventional cleaning: no disturbance of the user by cleaning personnel, no risky working in a gondola at high altitude, no additional protection during the cleaning phase, etc.” Serbot further states its autonomous system was able to work at amazing speeds cleaning the 8,600 square meter structure within a couple of days via its intelligent platform that plans a route across the entire glass facade.

Parallel to the global trend of urbanization, skyscraper construction is at an all time high. Demand for glass facade materials and maintenance services is close to surpassing $200 billion worldwide. As New York City is in the center of the construction boom, Israeli-startup, Skyline Robotics, recently joined Iconic Labs NYC (ICONYC). This week, I had the opportunity to ask Skyline founder and CEO Yaron Schwarcz about the move. Schwarcz proudly said, “So far we are deployed in Israel only and are working exclusively with one of the top 5 cleaning companies. Joining ICONYC was definitely a step forward, as a rule we only move forward, we believe that ICONIC can and will help us connect with the best investors and help us grow in the NY market.”

While Serbot requires building owners to purchase their proprietary suction cleaning system, Skyline’s machine, called Ozmo, integrates seamlessly with existing equipment. Schwarcz explains, “We use the existing scaffold of the building in contrast to GEKKO’s use of suction. The use of the arms is to copy the human arms which is the only way to fully maintain the entire building and all its complexity. The Ozmo system is not only a window cleaner, it’s a platform for all types of facade maintenance. Ozmo does not need any humans on the rig, never putting people in danger.” Schwarcz further shared with me the results of early case studies in Israel whereby Ozmo cleaned an entire vertical glass building in 80 hours with one supervisor remotely controlling the operation from the ground, adding with “no breaks.”

While Serbot and Skyline offer an optimistic view of the future, past efforts have been met with skepticism. In a 2014 New York Times article, written days after the two window washers almost fell to their deaths, the paper concluded, “washing windows is something that machines still cannot do as well.” The Times interviewed building exterior consultant, Craig S. Caulkins, who stated then, “Robots have problems.” Caulkins says the set back for automation has been the quality of work, citing numerous examples of dirty window corners. “If you are a fastidious owner wanting clean, clean windows so you can take advantage of that very expensive view that you bought, the last thing you want to see is that gray area around the rim of the window,” exclaimed Caulkins. Furthermore, New York City’s window washers are represented by a very active labor union, S.E.I.U. Local 32BJ. The fear of robots replacing their members could lead to city wide protests, and strikes. The S.E.I.U. 32BJ press office did not return calls for comment.

High rise window washing in New York is very much part of the folklore of the Big Apple. One of the best selling local children books, “Window Washer: At Work Above the Clouds,” profiles the former Twin Towers cleaner Roko Camaj. In 1995, Camaj predicted that “Ten years from now, all window washing will probably be done by a machine.” Unfortunately, Camaj never lived to see the innovations of GEKKO and Ozmo, as he perished in the Towers on September the 11th.

Automating high-risk professions will be explored further on June 13th @ 6pm in NYC with Democratic Presidential Candidate Andrew Yang and New York Assemblyman Clyde Vanel at the next RobotLab on “The Politics Of Automation” – Reserve Today!

Since Lupashin’s demo, there are now close to a hundred providers of drones on leashes from innovative startups to aftermarket solutions in order to restrain unmanned flying vehicles. Probably the best known enterprise solution is CyPhy Works which has raised more than $30 million. Last August, during President Trump’s visit to his Golf Course in New Jersey, the Department of Homeland Security (DHS) deployed CyPhy’s tethered drones to patrol the permitter. In a statement by DHS about their “spy in the sky program,” the agency explained: “The Proof of Concept will help determine the potential future use of tethered Unmanned Aircraft System (sUAS) in supporting the Agency’s protective mission. The tethered sUAS used in the Proof of Concept is operated using a microfilament tether that provides power to the aircraft and the secure video from the aircraft to the Operator Control Unit (OCU).” CyPhy’s systems are currently being utilized to provide a birdseye view to police departments and military units for a number of high profile missions, including the Boston Marathon.

Fotokite, CyPhy and others have proved that tethered machines offer huge advantages from traditional remote controlled or autonomous UAVs, by removing regulatory, battery and payload restrictions from lengthy missions. This past week Genius NY, the largest unmanned systems accelerator, awarded one million dollars to Fotokite for its latest enterprise line of leashed drones. The company clinched the competition after demonstrating how its drones can fly for up to twenty-four hours continuously providing real-time video feed autonomously and safely above large population centers. Fotokite’s Chief Executive, Chris McCall, announced that the funds will be utilized to fulfill a contract with one of the largest fire truck manufacturers in the United States. “We’re building an add-on to fire and rescue vehicles and public safety vehicles to be added on top of for instance a fire truck. And then a firefighter is able to pull up to an emergency scene, push a button and up on top of the fire truck this box opens up, a Fotokite flies up and starts live streaming thermal and normal video down to all the firefighters on the ground,” boasted McCall.

Fotokite is not the only kite drone company marketing to fire fighters, Latvian-born startup Aerones is attaching firehoses to its massive multi-rotor unmanned aerial vehicles. Aerones claims to have successfully built a rapid response UAV that can climb up to a thousand feet within six minutes to extinguish fires from the air. This enables first responders to have a reach of close to ten times more than traditional fire ladders. The Y Combinator startup offers municipalities two models: a twenty-eight propeller version that can carry up to 441 pounds to a height of 984 feet and a thirty-six propeller version that ferriess over 650 pounds of equipment to ascend over 1,600 feet. However, immediate interest for the Aerones solution is coming from industrial clients such as wind farms. “Over the last two months, we’ve been very actively talking to wind turbine owners,” says Janis Putrams, CEO of Aerones. “We have lots of interest and letters of intent in Texas, Spain, Turkey, South America for wind turbine cleaning. And in places like Canada, the Nordic and Europe for de-icing. If the weather is close to freezing, ice builds up, and they have to stop the turbine.” TechCrunch reported last March that the company moved its sales operations to Silicon Valley. 

The emergency response industry is also looking to other aerial solutions to tackle its most difficult challenges. For over a year, Zipline has been successfully delivering blood for critical transfusions to the most remote areas of Africa. The company announced earlier this month that it has filed with the FAA to begin testing later this year in America. This is welcome news for the USA’s rural health centers which are straddled with exploding costs, staff shortages and crippling infrastructure. In a Fast Company article about Zipline, the magazine reported that “Nearly half of rural providers already have a negative operating margin. As rural residents–who tend to be sicker than the rest of the country–have to rely on the smaller clinics that remain, drones could ensure that those clinics have access to necessary supplies. Blood products spoil quickly, and outside major hospitals, it’s common not to have the right blood on hand for a procedure. Using the drones would be faster, cheaper, and more reliable than delivering the supplies in a van or car.”

Keller Rinaudo, Zipline’s Chief Executive, describes, “There’s a lot that [the U.S.] can be doing better. And that’s what we think is ultimately the promise of future logistics and automated logistics. It’s not delivering tennis shoes or pizza to someone’s backyard. It’s providing universal access to healthcare when people need it the most.”

To date, Zipline has flown over 200,000 miles autonomously delivering 7,000 units of blood throughout Rwanda. To prepare for its US launch, the company re-engineered its entire platform to bolster its delivery capabilities. Rinaudo explains, “In larger countries, you’re going to need distribution centers and logistics systems that are capable of doing millions of deliveries a day rather than hundreds or thousands.” The new UAV is a small fixed-wing plane called the Zip that can soar close to 80 miles per hour enabling life-saving supplies such as blood, organ donations or vaccines to be delivered in a matter of minutes.

As I prepare to speak at Xponetial 2018 next month, I am inspired by these innovators who turn their mechanical inventions into life-saving solutions. Many would encourage Rinaudo and others to focus their energies on the seemingly more profitable  sectors such as e-commerce delivery and industrial inspections. However, Rinaudo retorts that “Healthcare logistics is a way bigger market and a way bigger problem than most people realize. Globally it’s a $70 billion industry. The reality is that there are billions of people who do not have reliable access to healthcare and a big part of that is logistics. As a result of that, 5.2 million kids die every year due to lack of access to basic medical products. So Zipline’s not in a rush to bite off a bigger problem than that.”

The topic of utilizing life-saving technology will be discussed at the next RobotLab event on “The Politics Of Automation,” with Democratic Presidential Candidate Andrew Yang and New York Assemblyman Clyde Vanel on June 13th @ 6pm in NYC – RSVP Today! 

As Mark Hamill humorously shared the behind-the-scenes of “Star Wars: The Last Jedi” with a packed SXSW audience, two floors below on the exhibit floor Universal Robots recreated General Grievous’ famed light saber battles. The battling machines were steps away from a twelve foot dancing Kuka robot and an automated coffee dispensary. Somehow the famed interactive festival known for its late night drinking, dancing and concerts had a very mechanical feel this year. Everywhere debates ensued between utopian tech visionaries and dystopia-fearing humanists.

Even my panel on “Investing In The Autonomy Economy” took a very social turn when discussing the opportunities of utilizing robots for the growing aging population. Eric Daimler (formerly of the Obama White House) raised concerns about AI bias affecting the well being of seniors. Agreeing, Dan Burstein (partner at Millennium Tech Value Partners) nervously expressed that ‘AI is everywhere, in everything, and the USA has no other way to care for this exploding demographic except with machines.’ Daimler explained that “AI is very good at perception, just not context;” until this is solved it could be a very dangerous problem worldwide.

Last year at a Google conference on the relationship between humans and AI, the company’s senior vice president of engineering, John Giannandrea, warned, “The real safety question, if you want to call it that, is that if we give these systems biased data, they will be biased. It’s important that we be transparent about the training data that we are using, and are looking for hidden biases in it, otherwise we are building biased systems.” Similar to Daimler’s anxiety about AI and healthcare, Giannandrea exclaimed that “If someone is trying to sell you a black box system for medical decision support, and you don’t know how it works or what data was used to train it, then I wouldn’t trust it.”

One of the most famous illustrations of how quickly human bias influences computer actions is Tay, the Microsoft customer service chatbot on Twitter. It took only twenty-four hours for Tay to develop a Nazi persona leading to more than ninety thousand hate-filled tweets. Tay swiftly calculated that hate on social media equals popularity. In explaining its failed experiment to Business Insider Microsoft stated via email: “The AI chatbot Tay is a machine learning project, designed for human engagement. As it learns, some of its responses are inappropriate and indicative of the types of interactions some people are having with it. We’re making some adjustments to Tay.”

While Tay’s real impact was benign, it raises serious questions of the implications of embedding AI into machines and society. In its Pulitzer Prize-winning article, ProPublica.org uncovered that a widely distributed US criminal justice software called Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) was racially biased in scoring the risk levels of convicted felons to recommit crimes. ProPublica discovered that black defendants in Florida, “were far more likely than white defendants to be incorrectly judged to be at a higher rate of recidivism” by the AI. Northpointe, the company that created COMPAS, released its own report that disputed ProPublica’s findings but it refused to pull back the curtain on its training data, keeping the algorithms hidden in a “black box.” In a statement released to the New York Times, Northpointe’s spokesperson argued, “The key to our product is the algorithms, and they’re proprietary. We’ve created them, and we don’t release them because it’s certainly a core piece of our business.” 

The dispute between Northpointe and ProPublica raises the question of transparency and the auditing of data by an independent arbitrator to protect against bias. Cathy O’Neil, a former Barnard professor and analyst at D.E. Shaw, thinks a lot about safeguarding ordinary Americans from biased AI. In her book, Weapons of Math Destruction, she cautions that big corporate America is too willing to hand over the wheel to the algorithms without fully assessing the risks or implementing any oversight monitoring. “[Algorithms] replace human processes, but they’re not held to the same standards. People trust them too much,” declares O’Neil. Understanding the high stakes and lack of regulatory oversight by the current federal government, O’Neil left her high-paying Wall Street job to start a software auditing firm, O’Neil Risk Consulting & Algorithmic Auditing. In an interview with MIT Technology Review last summer, O’Neil frustratingly expressed that companies are more interested in the bottom line than protecting their employees, customers, and families from bias, “I’ll be honest with you. I have no clients right now.”

Most of the success of deconstructing “black boxes” is happening today at the US Department of Defense. DARPA has been funding the research of Dr. David Gunning to develop Explainable Artificial Intelligence (XAI). Understanding its own AI and that of foreign governments could be a huge advantage for America’s cyber military units. At the same time, like many DARPA-funded projects, civilian opportunities could offer societal benefits. According to Gunning’s statement, online XAI aims to “produce more explainable models, while maintaining a high level of learning performance (prediction accuracy); and enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners.” XAI plans to work with developers and user interfaces designers to foster “useful explanation dialogues for the end user,” to know when to trust or question the AI-generated data. 

Besides DARPA, many large technology companies and universities are starting to create think tanks, conferences and policy groups to develop standards that test AI bias. The results have been startling – ranging from computer vision sensors that negatively identify people of color to gender bias in employment management software to blatant racism of natural language processing systems to security robots that run over kids identified mistakenly as threats. As an example of how training data affects outcomes, when Google first released its image processing software, the AI identified photos of African Americans as “gorillas,” because the engineers failed to provide enough minority examples into the neural network.

Ultimately artificial intelligence reflects the people that program it, as every human being brings with him his own experiences that shape personal biases. According to Kathleen Walch, host of AI Today podcast, “If the researchers and developers developing our AI systems are themselves lacking diversity, then the problems that AI systems solve and training data used both become biased based on what these data scientists feed into AI training data,” Walch advocates that hiring diversity can bring “about different ways of thinking, different ethics and different mindsets. Together, this creates more diverse and less biased AI systems. This will result in more representative data models, diverse and different problems for AI solutions to solve, and different use cases feed to these systems if there is a more diverse group feeding that information.”

Before leaving SXSW, I attended a panel hosted by the IEEE on “Algorithms, Unconscious Bias & AI,” amazingly all led by female panelists including one person of color. Hiring basis became a big theme of their discussion. Following the talk, I hopped into my Uber and pleasantly rode to the airport reflecting on a statement made earlier in the day by John Krafcik, Cheif Executive, of Waymo. Krafcik boasted that Waymo’s mission is to build “the world’s most experienced driver,” I just hope that the training data is not from New York City cabbies.

It was the last question of the night and it hushed the entire room. An entrepreneur expressed his aggravation about the FDA’s antiquated regulatory environment for AI-enabled devices to Dr. Joel Stein of Columbia University. Stein a leader in rehabilitative robotic medicine, sympathized with the startup knowing full well that tomorrow’s exoskeletons will rely heavily on machine intelligence. Nodding her head in agreement, Kate Merton of JLabs shared the sentiment. Her employer, Johnson & Johnson, is partnered with Google to revolutionize the operating room through embedded deep learning systems. In many ways this astute observation encapsulated RobotLab this past Tuesday with our topic being “The Future Of Robotic Medicine,” the paradox of software-enabled therapeutics offering a better quality of life and the societal, technological and regulatory challenges ahead.

To better understand the frustration expressed at RobotLab, a review of the policies of the Food & Drug Administration (FDA) relative to medical devices and software is required. Most devices fall within a criteria that was established in the 1970s. The “build and freeze” model whereby a product filed doesn’t change overtime and currently excludes therapies that rely on neural networks and deep learning algorithms that evolve with use. Charged with progressing its regulatory environment, the Obama Administration established a Digital Health Program tasked with implementing new regulatory guidance for software and mobile technology. This initiative eventually led Congress to pass the 21st Century Cures Act (“Cures Act”) in December 2016. An important aspect of the Cures Act is its provisions for digital health products, medical software, and smart devices. The legislators singled out AI for its unparalleled ability to be used in supporting human decision making referred to as “Clinical Decision Support” (“CDS”) with examples like Google and IBM Watson. Last year, the administration updated the Cures Act with a new framework that included a Digital Health Innovation Action Plan. These steps have been leading a change in the the FDA’s attitude towards mechatronics, updating its traditional approach to devices to include software and hardware that iterates with cognitive learning. The Action Plan states “an efficient, risk-based approach to regulating digital health technology will foster innovation of digital health products.” In addition, the FDA has been offering tech partners the ability of filing a Digital Health Software Pre-Certification (“Pre-Cert”) to fast track the evaluation and approval process, current Pre-Cert pilot filings include Apple, Fitbit, Samsung and other leading technology companies.

Another way for AI and robotic devices to receive approval from the FDA is through their “De Novo premarket review pathway.” According to the FDA’s website, the De Novo program is designed for “medical devices that are low to moderate risk and have no legally marketed predicate device to base a determination of substantial equivalence.” Many computer vision systems fall into the De Novo category using their sensors to provide “triage” software to efficiently identify disease markers based upon its training data of radiology images. As an example, last month the FDA approved Viz.ai a new type of “clinical decision support software designed to analyze computed tomography (CT) results that may notify providers of a potential stroke in their patients.”

Dr. Robert Ochs of the FDA’s Center for Devices and Radiological Health explains, “The software device could benefit patients by notifying a specialist earlier thereby decreasing the time to treatment. Faster treatment may lessen the extent or progression of a stroke.” The Viz.ai algorithm has the ability to change the lives of the nearly 800,000 annual stroke victims in the USA. The data platform will enable clinicians to quickly identify patients at risk for stroke by analyzing thousands of CT brain scans for blood vessel blockages and then automatically send alerts via text messages to neurovascular specialists. Viz.AI promises to streamline the diagnosis process by cutting the traditional time it takes for radiologists to review, identify and escalate cases to specialists for high-risk patients.

Dr. Chris Mansi, Viz.ai CEO, says “The Viz.ai LVO Stroke Platform is the first example of applied artificial intelligence software that seeks to augment the diagnostic and treatment pathway of critically unwell stroke patients. We are thrilled to bring artificial intelligence to healthcare in a way that works alongside physicians and helps get the right patient, to the right doctor at the right time.” According to the FDA’s statement, Mansi’s company “submitted a study of only 300 CT scans that assessed the independent performance of the image analysis algorithm and notification functionality of the Viz.ai Contact application against the performance of two trained neuro-radiologists for the detection of large vessel blockages in the brain. Real-world evidence was used with a clinical study to demonstrate that the application could notify a neurovascular specialist sooner in cases where a blockage was suspected.”

Viz.ai joins a market for AI diagnosis software that is growing rapidly and projected to eclipse six billion by 2021 (Frost & Sullivan), an increase of more than forty percent since 2014. According to the study, AI has the ability to reduce healthcare costs by nearly half, while at the same time improving the outcomes for a third of all US healthcare patients. However, diagnosis software is only part of the AI value proposition, adding learning algorithms throughout the entire ecosystem of healthcare could provide new levels of quality of care.

At the same time, the demand for AI treatment is taking its toll on an underfunded FDA which is having difficulty keeping up with the new filings to review computer-aided therapies from diagnosis to robotic surgery to invasive therapeutics. In addition, many companies are currently unable to afford the seven-figure investment required to file with the FDA, leading to missed opportunities to find cures for the most plaguing diseases. The Atlantic reported last fall about a Canadian company, Cloud DX, that is still waiting for approval for its AI software that analyzes coughing data via audio wavelengths to detect lung-based diseases (i.e., asthma, tuberculosis, and pneumonia). Cloud DX’s founder, Robert Kaul, shared wth the magazine, “There’s a reason that tech companies like Google haven’t been going the FDA route [of clinical trials aimed at diagnostic certification]. It can be a bureaucratic nightmare, and they aren’t used to working at this level of scrutiny and slowness.” It took Cloud DX two years and close to a million dollars to achieve the basic ISO 13485 certification required to begin filing with the agency. Kaul, questioned, “How many investors are going to give you that amount of money just so you can get to the starting line?”

Last month, Rani Therapeutics raised $53 million to begin clinical trials for its new robotic pill. Rani’s solution could usher in a new paradigm of needle-free therapy, whereby drugs are mechanically delivered to the exact site of infection. Unfortunately, innovations like Rani’s are getting backlogged with a shortage of knowledgable examiners able review the clinical data. Bakul Patel, the FDA’s New Associate Center Director For Digital Health, describes that one of his top priorities is hiring, “Yes, it’s hard to recruit people in AI right now. We have some understanding of these technologies. But we need more people. This is going to be a challenge.” Patel is cautiously optimistic, “We are evolving… The legacy model is the one we know works. But the model that works continuously—we don’t yet have something to validate that. So the question is [as much] scientific as regulatory: How do you reconcile real-time learning [with] people having the same level of trust and confidence they had yesterday?”

As I concluded my discussion with Stein, I asked if he thought disabled people will eventually be commuting to work wearing robotic exoskeletons as easily as they do in electric wheelchairs? He answered that it could come within the next decade if society changes its mindset on how we distribute and pay for such therapies. To quote the President, “Nobody knew health care could be so complicated.”

I recently chaired a UJA Tech Talk on “The Future Of Autonomous Cars” with former General Motors Vice-Chairman Steve Girsky. The auto executive enthusiastically shared his vision for the next 15-25 years of driving – a congestion-free world of automated wheeled capsules zipping commuters to and from work.

Girsky stated that connected cars with safety assist (autonomy-lite) features are moving much faster toward mass adoption than fully autonomous vehicles (sans steering wheels and pedals). In his opinion, the largest roadblocks toward a consumer-ready robocar are the current technical inefficiencies of prototypes on the road today, which burn huge amounts of energy supporting enhanced computing and arrays of sensors. This makes the sticker price closer to a 1972 Ferrari than a 2018 Prius.

As main street adoption relies heavily on converting combustion engines to electric at accessible pricing, Girsky’s sentiment was shared by many CES 2018 participants. NVIDIA, the leading chip manufacturer for autonomous vehicles, unveiled its latest technology, Xavier, with auto industry partner Volkswagen in Las Vegas. Xavier promises to be 15 times more energy-efficient than previous chip generations delivering 30 trillion operations per second by wielding only 30 watts of power.

After the Xavier CES demonstration, Volkswagen CEO Herbert Diess exclaimed, “Autonomous driving, zero-emission mobility, and digital networking are virtually impossible without advances in AI and deep learning. Working with NVIDIA, the leader in AI technology, enables us to take a big step into the future.”

NVIDIA is becoming the industry standard as Volkswagen joins more than 320 companies and organizations working with the chip manufacturer on autonomous vehicles. While NVIDIA is leading the pack, Intel and Qualcomm are not far behind with their low-power solutions. Electric vehicle powerhouse Tesla is developing its own internal chip for the next generation of Autopilot. While these new chips represents a positive evolution in processors, there is still much work to be done as current self-driving prototypes require close to 2,500 watts per second.

Power Consumption a Tradeoff for Self-Driving Cars

The power-consumption problem was highlighted recently with a report published by the the University of Michigan Center for Sustainable Systems. Its lead author, Greg Keoleian, questions whether the current autonomous car models will slow the overall adoption towards electric vehicles. Keoleian’s team simulated a number of self-driving Ford Fusion models with different-sized computer configurations and engine designs. In sharing his findings, Keoleian said, “We knew there was going to be a tradeoff in terms of the energy and greenhouse gas emissions associated with the equipment and the benefits gained from operational efficiency. I was surprised that it was so significant.”

Keoleian’s conclusions challenged the premise of self-driving cars accelerating the adoption of renewal energy. For years, the advocates of autonomous vehicles have claimed that smart driving will lead to a reduction of greenhouse gas emissions through the platooning of vehicles on highways and intersections; the decrease of aerodynamic drag on freeways, and the overall reduction in urban congestion.

Analysis: How California’s Self-Driving Cars Performed in 2017

However, the University of Michigan tests only showed a “six to nine percent net energy reduction” over the vehicle’s lifecycle when running on autonomy mode. This went down by five percent when using a large Waymo rooftop sensor package (shown below) as it increased the aerodynamic drag. The report also stated that the greatest net efficiencies were in cars with gas drivetrains that benefit the most from smart driving. Waymo currently uses a hybrid Chrysler Pacifica to run its complex fusion of sensors and processing units.

Keoleian told IEEE Spectrum that his modeling actually “overstates real impacts from future autonomous vehicles.” While he anticipates the reduction of computing and sensor drag, he is concerned that the impact of 5G communications has not been fully explored. The increased bandwidth will lead to greater data streams and boost power consumption for inboard systems and processors. In addition, he thinks that self-driving systems will lead to greater distances traveled as commuters move further away from city centers with the advent of easier commutes. Keoleian explains, “There could be a rebound effect. They could induce travel, adding to congestion and fuel use.” Koeleian points to a confusing conclusion by the U.S. National Renewable Energy Laboratory that presents two possible outcomes of full autonomy:

1. A reduction in greenhouse emissions by sixty percent with greater ride sharing options
2. An increase of two hundred percent with increased driving distances

According to Wilko Stark, Mercedes-Benz’s Vice President of Strategy, it only makes sense for autonomous vehicles to be electric as the increased power requirements will go to the computers instead of the motors. “To put such a system into a combustion-engined car doesn’t make any sense, because the fuel consumption will go up tremendously,” explains Stark.

Analysis: Fleet Expansion Shows Waymo Lapping Self-Driving Competition

Girsky shares Stark’s view, as he predicted that the first large scale use cases for autonomy will be fleets of souped-up golf carts running low speed pre-planned shuttle routes. Also on view at CES were complimentary autonomous shared taxi rides around Las Vegas, courtesy of French startup Navya. Today, Navya boasts of 60 operating shuttles in more than 10 cities, including around the University of Michigan.

Fully autonomous cars might not be far behind, as Waymo has seen a ninety percent drop in component costs by bringing its sensor development in-house. The autonomous powerhouse recently passed the four million mile marker on public roads and is planning on ditching its safety driver later this year in its Phoenix, Arizona test program. According Dmitri Dolgov, Vice President of Waymo’s Engineering, “Sensors on our new generation of vehicles can see farther, sharper, and more accurately than anything available on the market. Instead of taking components that might have been designed for another application, we engineered everything from the ground up, specifically for the task of Level 4 autonomy.”

With increased roadside fatalities and rising CO2 emissions, the world can’t wait too much longer for affordable, energy-efficient autonomous transportation. Girsky and others remind us there is still a long road ahead, while the industry experts estimate that the current gas-burning Waymo Chrysler Pacifica cruising around Arizona costs more than one hundred times the sticker price of the minivan. I guess until then there is always Citibike.

It’s called the “grain,” a small IoT device implanted into the back of people’s skulls to record their memories. Human experiences are simply played back on “redo mode” using a smart button remote. The technology promises to reduce crime, terrorism and simplify human relationships with greater transparency. While this is a description of Netflix’s Black Mirror episode, “The Entire History of You,” in reality the concept is not as far-fetched as it may seem. This week life came closer to imitating art with the $19 million grant by the US Department of Defense to a group of six universities to begin work on “neurograins.”

In the past, Brian Computer Interfaces (BCIs) have utilized wearable technologies, such as headbands and helmets, to control robots, machines and various household appliances for people with severe disabilities. This new DARPA grant is focused on developing a “cortical intranet” for uplinks and downlinks directly to the cerebral cortex, potentially taking mind control to the next level. According to lead researcher Arto Nurmikko of Brown University, “What we’re developing is essentially a micro-scale wireless network in the brain enabling us to communicate directly with neurons on a scale that hasn’t previously been possible.”

Nurmikko boasts of the numerous medicinal outcomes of the research, “The understanding of the brain we can get from such a system will hopefully lead to new therapeutic strategies involving neural stimulation of the brain, which we can implement with this new neurotechnology.” The technology being developed by Nurmikko’s international team will eventually create a wireless neural communication platform that will be able to record and stimulate brian activity at an unprecedented level of detail and precision. This will be accomplished by implanting a mesh network of tens of thousands of granular micro-devices into a person’s cranium. The surgeons will place this layer of neurograins around the cerebral cortex that will be controlled by a nearby electronic patch just below a person’s skin.

In describing how it will work, Nurmikko explains, “We aim to be able to read out from the brain how it processes, for example, the difference between touching a smooth, soft surface and a rough, hard one and then apply microscale electrical stimulation directly to the brain to create proxies of such sensation. Similarly, we aim to advance our understanding of how the brain processes and makes sense of the many complex sounds we listen to every day, which guide our vocal communication in a conversation and stimulate the brain to directly experience such sounds.”

Nurmikko further describes, “We need to make the neurograins small enough to be minimally invasive but with extraordinary technical sophistication, which will require state-of-the-art microscale semiconductor technology. Additionally, we have the challenge of developing the wireless external hub that can process the signals generated by large populations of spatially distributed neurograins at the same time.”

While current BCIs are able to process the activity of 100 neurons at once, Nurmikko’s  objective is to work at a level of 100,000 simultaneous inputs. “When you increase the number of neurons tenfold, you increase the amount of data you need to manage by much more than that because the brain operates through nested and interconnected circuits,” Nurmikko remarks. “So this becomes an enormous big data problem for which we’ll need to develop new computational neuroscience tools.” The researchers plan to first test their theory in the sensory and auditory functions of mammals.

Brain-Computer Interfaces is one of the fastest growing areas of healthcare technologies; while today it is valued at just under a billion dollars, it is forecasted to grow to $2 billion in the next five years. According to the report, the uptick in the market will be driven by an estimated increase in treating aging, fatal diseases and people with disabilities. The funder of Nurmikko’s project is DARPA’s Neural Engineering System Design program, which was formed to treat injured military personnel by “creating novel hardware and algorithms to understand how various forms of neural sensing and actuation might improve restorative therapeutic outcomes.” While DARPA’s project will provide numerous discoveries that will improve the quality of life for society’s most vulnerable, it also opens a Pandora’s box of ethical issues with the prospect of the US military potentially funding armies of cyborgs.

In response to rising ethical concerns, last month ethicists from the University of Basel in Switzerland drafted a new biosecurity framework for research in neurotechnology. The biggest concern expressed in the report was the implementation of “dual-use” technologies that have both military and medical benefits. The ethicists called for a complete ban on such innovations and strongly recommended fast tracking regulations to protect “the mental privacy and integrity of humans,”

The ethicists raise important questions about taking grant money from groups like DARPA, as “This military research has raised concern about the risks associated with the weaponization of neurotechnology, sparking a debate about controversial questions: Is it legitimate to conduct military research on brain technology? And how should policy-makers regulate dual-use neurotechnology?” The suggested framework reads like a science fiction novel, “This has resulted in a rapid growth in brain technology prototypes aimed at modulating the emotions, cognition, and behavior of soldiers. These include neurotechnological applications for deception detection and interrogation as well as brain-computer interfaces for military purposes.” However, one is reminded of the fact that the development of BCIs is moving more quickly than public policy is able to debate its merits.

The framework’s lead author Marcello Ienca of Basel’s Institute for Biomedical Ethics understands the tremendous positive benefits of BCIs for a global aging population, especially for people suffering from Alzheimer’s and spinal cord injuries. In fact, the Swiss team calls for increased private investment of these neurotechnologies, not an outright prohibition. At the same time, Ienca stresses that in order to protect against misuse, such as brain manipulation by nefarious global actors, it is critical to raise awareness and debate surrounding the ethical issues of implanting neurograins into populations of humans. In an interview with the Guardian last year, Ienca summed up his concern very succinctly by saying, “The information in our brains should be entitled to special protections in this era of ever-evolving technology. When that goes, everything goes.”

In the spirit of open debate our next RobotLab forum will be on “The Future of Robotic Medicine” with Dr. Joel Stein of Columbia University and Kate Merton of JLabs on March 6th @ 6pm in New York City, RSVP. 

I recently led a group of 20 American tech investors to Israel in conjunction with the UJA and Israel’s Ministry of Economy and Industry. We witnessed firsthand the innovation that has produced more than $22 billion of investments and acquisitions within the past year. We met with the University that produced Mobileye, with the investor that believed in its founder, and the network of every multinational company supporting the startup ecosystem. Mechatronics is blooming in the desert from the CyberTech Convention in Tel Aviv to the robotic labs at Capsula to the latest in autonomous driving inventions in the hills of Jerusalem.

Sitting in a suspended conference room that floats three stories above the ground enclosed within the “greenest building in the Middle East,” I had the good fortune to meet Torr Polakow of Curiosity Lab. Torr is the PhD student of famed roboticist Dr. Goren Gordon. Gordon’s research tests the boundaries of human-robot social interactions. At the 2014 World Science Fair in New York City, Gordon partnered with MIT professor, and Jibo founder, Dr. Cynthia Breazeal to prove it is possible to teach machines to be curious. The experiment was conducted by embedding into Breazeal’s DragonBot (below) a reinforcement learning algorithm that enabled the robot to acquire its own successful behaviors to engage humans. The scientists programmed within the robot nine non-verbal expressions and a reward system that was activated based upon its ability to engage crowds. The greater the number of faces staring back at the robot the bigger the reward. After two hours it successfully learned that by deploying its”sad face” with big eyes, that evokes Puss-in-Boots from the movie Shrek, it captured the most sustained attention from the audience. The work of Gordon and Breazeal opened up the field of social robots and the ability to teach computers human empathy, which will eventually be deployed in fleets of mechanical caregivers for our aging society. Gordon is now working tirelessly to create a mathematical model of “Hierarchical Curiosity Loops (HCL)” for “curiosity-driven behavior” to “increase our understanding of the brain mechanisms behind it.”

Traveling east, we met with Yissum the tech transfer arm of Hebrew University, the birth place of the cherry tomato and Mobileye. The history of the automative startup that became the largest Israeli Initial Public Offering, and later a $15.3 billion acquisition by Intel, is a great example of how today’s Israeli innovations are evolving more quickly into larger enterprises than at anytime in history. Mobileye was founded in 1999 by Professor Amnon Shashua, a leader in computer vision and machine learning software. Dr. Shashua realized early on that his technology had wider applications for industries outside of Israel, particularly automative. Supported by his University, Mobileye’s first product, EyeQ chip, was piloted by Tier 1 powerhouse Denso within years of inception. While it took Mobileye 18 years to achieve full liquidity, Israeli cybersecurity startup, Argus, was sold last month to Elektrobit for a rumored $400 million within just four years from its founding. Shashua’s latest computer vision startup, OrCam, is focusing on giving sight to people with impaired vision.

In addition to Shahua’s successful portfolio of startups, computer vision innovator Professor Shmuel Peleg is probably most famous for catching the 2013 Boston Marathon Bombers using his imaging-tracking software. Dr. Peleg’s company, BriefCam, tracks anomalies in multiple video feeds at once, layering simultaneous events on top of one another. This technique autonomously isolated the Boston bombers by programing in known aspects of their appearance into the system (e.g., men with backpacks), and then isolating the suspects by their behavior. For example, as everyone was watching the marathon minutes before the explosion, the video captured the Tsarnaev brothers leaving the scene after dropping their backpacks on the street. BriefCam’s platform made it possible for the FBI to capture the perpetrators within 101 hours of the event, versus manually sifting through the more than 13,000 videos and 120,000 photographs taken by stationary and cell phone cameras.

This past New Year’s Eve, BriefCam was deployed by the New York City to secure the area around Times Square. In a joint statement by the FBI, NYPD, Department of Homeland Security and the New York Port Authority the authorities exclaimed that they plan to deploy BriefCam’s Video Synopsis across all their video cameras as they “remain concerned about international terrorists and domestic extremists potentially targeting” the celebration. According to Peleg, “You can take the entire night and instead of watching 10 hours you can watch ten minutes. You can detect these people and check what they are planning.”

BriefCam is just one of thousands of new innovations entering the SmartCity landscape from Israel. In the past three years, 400 new smart mobility startups have opened shop in Israel accumulating over $4 billion in investment. During that time, General Motor’s Israeli research & development office went from a few engineers to more than 200 leading computer scientists and roboticists. In fact, Tel Aviv now is a global hub of innovation of every major car manufacturer and automobile supplier, encircling many new accelerators and international consortiums, from the Drive Accelerator to Ecomotion, which seem to be opening up faster than Starbucks in the United States. 

As the auto industry goes through its biggest revolution in 100 years, the entire world’s attention is now centered on this small country known for ground-breaking innovation. In the words of Israel’s Innovation Authority, “​The acquisition of Mobileye by Intel this past March for $15.3 billion, one of the largest transactions in the field of auto-tech in 2017, has focused the attention of global corporations and investors on the tremendous potential of combining Israeli technological excellence with the autonomous vehicle revolution.” 

]]> https://robohub.org/sewing-a-mechanical-future/ Tue, 23 Jan 2018 23:53:48 +0000 http://robohub.org/sewing-a-mechanical-future/ Read More ›]]>

SoftWear Automation’s Sewbot. Credit: SoftWear Automation

The Financial Times reported earlier this year that one of the largest clothing manufacturers, Hong Kong-based Crystal Group, proclaimed robotics could not compete with the cost and quality of manual labor. Crystal’s Chief Executive, Andrew Lo, emphatically declared, “The handling of soft materials is really hard for robots.” Lo did leave the door open for future consideration by acknowledging such budding technologies as “interesting.”

One company mentioned by Lo was Georgia Tech spinout, Softwear Automation. Softwear made news last summer by announcing its contract with an Arkansas apparel factory to update 21 production lines with its Sewbot automated sewing machines. The factory is owned by Chinese manufacturer Tianyuan Garments, which produces over 20 million T-shirts a year for Adidas. The Chairman of Tianyuan, Tang Xinhong, boasted about his new investment, saying “Around the world, even the cheapest labor market can’t compete with us,” when Sewbot brings down the costs to $0.33 a shirt.

The challenge for automating cut & sew operations to date has been the handling of textiles which come in a seemingly infinite number of varieties that stretch, skew, flop and move with great fluidity. To solve this problem, Softwear uses computer vision to track each individual thread. According to its issued patents, Softwear developed a specialized camera which captures  threads at 1,000 frames per second and tracks their movements using proprietary algorithms. Softwear embedded this camera around robot end effectors that manipulate the fabrics similar to human fingers. According to a description on IEEE Spectrum these “micromanipulators, powered by precise linear actuators, can guide a piece of cloth through a sewing machine with submillimeter precision, correcting for distortions of the material.” To further ensure the highest level of quality, Sewbot uses a four-axis robotic arm with a vacuum gripper that picks and places the textiles on a sewing table with a 360-degree conveyor system and spherical rollers to quickly move the fabric panels around.

Softwear’s CEO, Palaniswamy “Raj” Rajan, explained, “Our vision is that we should be able to manufacture clothing anywhere in the world and not rely on cheap labor and outsourcing.” Rajan appears to be working hard towards that goal, professing that his robots are already capable of reproducing more than 2 million products sold at Target and Walmart. According to IEEE Spectrum, Rajan further asserted in a press release that at the end of 2017, Sewbot will be on track to produce “30 million pieces a year.” It is unclear if that objective was ever met. Softwear did announce the closing of its $7.5 million financing round by CTW Venture Partners, a firm of which Rajan is also the managing partner.

Softwear Automation is not the only company focused on automating the trillion-dollar apparel industry. Sewbo has been turning heads with its innovative approach to fabric manipulation. Unlike Softwear, which is taking the more arduous route of revolutionizing machines, Sewbo turns textiles into hardened substances that are easy for off-the-shelf robots and existing sewing appliances to handle. Sewbo’s secret sauce, literally, is a water-soluble thermal plastic or stiffening solution that turns cloth into a cardboard-like material. Blown away by its creativity and simplicity, I sat down with its inventor Jonathan Zornow last week to learn more about the future of fashion automation.

After researching the best polymers to laminate safely onto fabrics using a patent-pending technique, Zornow explained that last year he was able to unveil the “world’s first and only robotically-sewn garment.” Since then, Zornow has been contacted by almost every apparel manufacturer (excluding Crystal) to explore automating their production lines. Zornow has hit a nerve with the industry, especially in Asia, that finds itself in the labor management business with monthly attrition rates of 10% and huge drop offs after Chinese New Year. Zornow shared that “many factory owners were in fact annoyed that they couldn’t buy the product today.”

Zornow believes that automation technologies could initially be a boom for bringing small-batch production back to the USA, prompting an industry of “mass customization” closer to the consumer. As reported in 2015, apparel brands have been moving manufacturing back from China with 3D-printing technologies for shoes and knit fabrics. Long-term, Zornow said, “I think that automation will be an important tool for the burgeoning reshoring movement by helping domestic factories compete with offshore factories’ lower labor costs. When automation becomes a competitive alternative, a big part of its appeal will be how many headaches it relieves for the industry.”

To date, fulfilling the promise of “Made In America” has proven difficult, as Zornow explained we have forgotten how to make things here. In a recent report by the American Apparel & Footwear Association, US apparel manufacturing fell from 50% in 1994 to roughly 3% in 2015, meaning 97% of clothing today is imported. For example, Zornow shared with me how his competitor was born. In 2002, the US Congress passed the Berry Amendment requiring the armed services to make uniforms domestically, which led DARPA to grant $1.75 million to a Georgia Tech team to build a prototype of an automated sewing machine. As Rajan explains, “The Berry Amendment went into effect restricting the military from procuring clothing that was not made in the USA. Complying with the rule proved challenging due to a lack of skilled labour available in the US that only got worse as the current generation of seamstresses retired with no new talent to take their place. It was under these circumstances that the initial idea for Softwear was born and the company was launched in 2012.”

I first met Zornow at RoboBusiness last September when he demonstrated for a packed crowd how Sewbo is able to efficiently recreate the 10-20 steps to sew a T-shirt. However, producing a typical mens dress shirt can require up to 80 different steps. Zornow pragmatically explains the road ahead, “It will be a very long time, if ever, before things are 100% automated.” He points to examples of current automation in fabric production, such as dyeing, cutting and finishing which augment manual labor. Following this trend “they’re able to leverage machines to achieve incredible productivity, to the point where the labor cost to manufacture a yard of fabric is usually de minimis.” Zornow foresees a future where his technology is just another step in the production line as forward-thinking factories are planning two decades ahead, recognizing that in order “to stay competitive they need new technologies” like Sewbo.

As close to a quarter million people descended on a city of six hundred thousand, CES 2018 became the perfect metaphor for the current state of modern society. Unfazed by floods, blackouts, and transportation problems, technology steamrolled ahead. Walking the floor last week at the Las Vegas Convention Center (LVCC), the hum of the crowd buzzed celebrating the long awaited arrival of the age of social robots, autonomous vehicles, and artificial intelligence.

In the same way that Alexa was last year’s CES story, social robots were everywhere this year, turning the show floor into a Disney-inspired mechatronic festival (see above). The applications promoted ranged from mobile airport/advertising kiosks to retail customer service agents to family in-home companions. One French upstart, Blue Frog Robotics, stood out from the crowd of ivory-colored rolling bots. Blue Frog joined the massive French contingent at Eureka Park and showcased its Buddy robot to the delight of thousands of passing attendees.

When meeting with Blue Frog’s founder Rodolphe Hasselvander, he described his vision of a robot which is closer to a family pet than the utility of a cold-voice assistant. Unlike other entrees in the space, Buddy is more like a Golden Retriever than an iPad on wheels. Its cute, blinking big eyes immediately captivate the user, forming a tight bond between robopet and human. Hasselvander demonstrated how Buddy performs a number of unique tasks, including: patrolling the home perimeter for suspicious activity; looking up recipes in the kitchen; providing instructions for do-it-yourself projects; entertaining the kids with read-along stories; and even reminding grandma to take her medicine. Buddy will be available for “adoption” in time for the 2018 Holiday season for $1,500 on Blue Frog’s website.

Blue Frog’s innovation was recognized by the Consumer Electronics Show with a “Best of Innovation” award. In accepting his honor Hasselvander exclaimed, “The icing on the cake is that our Buddy robot is named a ‘Best of Innovation’ Award, as we believe Buddy truly is the most full-featured and best-engineered companion robot in the market today… this award truly shows our ongoing commitment to produce a breakthrough product that improves our lives and betters our world.”

Last year, one of the most impressive CES sites was the self-driving track in the parking lot of the LVCC’s North Hall. This year, the autonomous cars hit the streets with live demos and shared taxi rides. Lyft partnered with Aptiv to offer ride hails to and from the convention center, among other preprogrammed stops. While the cars were monitored by a safety driver, Aptiv explicitly built the experience to “showcase the positive impact automated cars will have on individual lives and communities.” Lyft was not the only self-driving vehicle on the road, autonomous shuttles developed by Boston-based Keolis and French company Navya for the city of Las Vegas were providing trips throughout the conference. Inside the LVCC, shared mobility was a big theme amongst several auto exhibitors (see below).

Torc Robotics made news days before the opening of CES with its autonomous vehicle successfully navigating around a SUV going the wrong way into oncoming traffic (see the video above). Michael Fleming, Chief Executive of Torc, shared with a packed crowd the video feed from the dashcam, providing a play-by-play account. He boasted that a human driven car next to his self-driving Lexus RX 450 followed the same path to avoid the collision. Fleming posted the dashcam video online to poll other human drivers to program a number of scenarios into his AI to avoid future clashes with reckless drivers.

The Virginia-based technology company has been conducting self-driving tests for more than a decade. Torc placed third in the 2007 DARPA Urban Challenge, a prestigious honor for researchers tackling autonomous city driving. Fleming is quick to point out that along with Waymo (Google), Torc is one of the earliest entries into the self-driving space. “There are an infinite number of corner cases,” explains Fleming, relying on petabytes of driving data built over a decade. Flemings explained to a hushed room that each time something out of the ordinary happens, like the wrong-way driver days prior, Torc logs how its car handled the situation and continues to make refinements to the car’s brain. The next time a similar situation comes up, any Torc-enabled vehicle will instantly implement the proper action. Steve Crowe of Robotics Trends said it best after sitting in the passenger seat of Fleming’s self-driving car, “I can honestly say that I didn’t realize how close we were to autonomous driving.”

AI was everywhere last week in Las Vegas, inside and outside the show. Unlike cute robots and cars, artificial intelligence is difficult to display in glitzy booths. While many companies, including Intel and Velodyne, proudly showed off their latest sensors it became very clear that true AI was the defining difference for many innovations. Tucked in the back of the Sands Convention Center, New York-based Emoshape demonstrated a new type of microchip with embedded AI to synthetically create emotions in machines.  The company’s Emotion Processing Unit (EPU) is being marketed for the next generation of social robots, self-driving cars, IoT devices, toys, and virtual-reality games.

When speaking with Emoshape founder Patrick Levy-Rosenthal, he explained his latest partnership with the European cellphone provider, Orange. Levy-Rosenthal is working with Orange’s Silicon Valley innovation office to develop new types of content with its emotion synthesis technology. As an example, Emoshape’s latest avatar JADE is an attractive female character with real-time sentiment-based protocols. According to the company’s press release, its software “engine delivers high-performance machine emotion awareness, and allows personal assistant, games, avatars, cars, IoT products and sensors to feel and develop a unique personality.” JADE and Emoshape’s VR gaming platform DREAM is being evaluated by Orange for a number of enterprise and consumer use cases.

Emotions ran high at CES, especially during the two-hour blackout. On my redeye flight back to New York, I dreamt of next year’s show with scores of Buddy robots zipping around the LVCC greeting attendees, with their embedded Emoshape personalities, leading people to fleets of Torc-mobiles around the city. Landing at JFK airport, the biting wind and frigid temperature woke me up to the reality that I have 360 more days until CES 2019.

Two thousand seventeen certainly has been an emotional year for mankind. While homo sapiens continue to yell at Alexa and Siri, the actuality of people’s willingness to pursue virtual relationships over human ones is startling.

In a recent documentary by Channel 4 of the United Kingdom, it was revealed that Abyss Creations is flooded with pre-orders for its RealDoll AI robotic (intimate) companion. According to Matt McMullen, Chief Executive of Abyss, “With the Harmony AI, they will be able to actually create these personalities instead of having to imagine them. They will be able to talk to their dolls, and the AI will learn about them over time through these interactions, thus creating an alternative form of relationship.”

The concept of machines understanding human emotions, and reacting accordingly, was featured prominently at AI World a couple weeks ago in Boston. Rana el Kaliouby, founder of artificial intelligence company Affectiva thinks a lot about computers acquiring emotional intelligence. Affectiva is building a “multi-modal emotion AI” to enable robots to understand human feelings and behavior.

“There’s research showing that if you’re smiling and waving or shrugging your shoulders, that’s 55% of the value of what you’re saying – and then another 38% is in your tone of voice,” describes el Kaliouby. “Only 7% is in the actual choice of words you’re saying, so if you think about it like that, in the existing sentiment analysis market which looks at keywords and works out which specific words are being used on Twitter, you’re only capturing 7% of how humans communicate emotion, and the rest is basically lost in cyberspace.” Affectiva’s strategy is already paying off as more than one thousand global brands are employing their “Emotion AI” to analyze facial imagery to ascertain people’s affinity towards their products.

Embedding empathy into machines goes beyond advertising campaigns. In healthcare, emotional sensors are informing doctors of the early warning signs of a variety of disorders, including: Parkinson’s, heart disease, suicide and autism. Unlike Affectiva’s, Beyond Verbal is utilizing voice analytics to track biomarkers for chronic illness. The Israeli startup grew out of a decade and half of University research with seventy thousand clinical subjects speaking thirty languages. The company’s patented “Mood Detector” is currently being deployed by the Mayo Clinic to detect early on signs of coronary artery disease.

Beyond Verbal’s Chief Executive, Yuval Mor, foresees a world of empathetic smart machines listening for every human whim. As Mor explains, “We envision a world in which personal devices understand our emotions and wellbeing, enabling us to become more in tune with ourselves and the messages we communicate to our peers.” Mor’s view is embraced by many who sit in the center of the convergence of technology and healthcare. Boston-based Sonde is also using algorithms to analyze the tone of speech to report on the mental state of patients by alerting neurologists of the risk of depression, concussion, and other cognitive impairments.

“When you produce speech, it’s one of the most complex biological functions that we do as people,” according to Sonde founder Jim Harper. “It requires incredible coordination of multiple brain circuits, large areas of the brain, coordinated very closely with the musculoskeletal system…What we’ve learned is that changes in the physiological state associated with each of these systems can be reflected in measurable, objective features that are acoustics in the voice. So we’re really measuring not what people are saying, in the way Siri does, we’re focusing on how you’re saying what you’re saying and that gives us a path to really be able to do pervasive monitoring that can still provide strong privacy and security.”

While these AI companies are building software and app platforms to augment human diagnosis, many roboticists are looking to embed such platforms into the next generation of unmanned systems. Emotional tracking algorithms can provide real-time monitoring for semi and autonomous cars by reporting on the level of fatigue, distraction and frustration of the driver and its occupants. The National Highway Traffic Safety Administration estimates that 100,000 crashes nationwide are caused every year by driver fatigue. For more than a decade technologists have been wrestling with developing better alert systems inside the cabin. For example, in 1997 James Russell Clarke and Phyllis Maurer Clarke developed a “Sleep Detection and Driver Alert Apparatus” (US Patent: 5689241 A) using imaging to track eye movements and thermal sensors to monitor “ambient temperatures around the facial areas of the nose and mouth” (a.k.a., breathing). Today with the advent of cloud computing and deep learning networks, Clarke’s invention could possibly save even more lives.

Tarek El Dokor, founder and Cheif Executive, of EDGE3 Technologies has been very concerned about the car industry’s rush towards autonomous driving, which in his opinion might be “side-stepping the proper technology development path and overlooking essential technologies needed to help us get there.” El Doker is referring to Tesla’s rush to release its autopilot software last year that led to customers trusting the computer system too much. YouTube is littered with videos of Tesla customers taking their hands and eyes off the road to watch movies, play games and read books. Ultimately, this user abuse led to the untimely death of Joshua Brown.

To protect against autopilot accidents, EDGE3 monitors driver alertness through a combined platform of hardware and software technologies of “in-cabin cameras that are monitoring drivers and where they are looking.” In El Dokor’s opinion, image processing is the key to guaranteeing a safe handoff between machines and humans. He boasts that his system combines, “visual input from the in-cabin camera(s) with input from the car’s telematics and advanced driver-assistance system (ADAS) to determine an overall cognitive load on the driver. Level 3 (limited self-driving) cars of the future will learn about an individual’s driving behaviors, patterns, and unique characteristics. With a baseline of knowledge, the vehicle can then identify abnormal behaviors and equate them to various dangerous events, stressors, or distractions. Driver monitoring isn’t simply about a vision system, but is rather an advanced multi-sensor learning system.” This multi-sensor approach is even being used before cars leave the lot. In Japan, Sumitomo Mitsui Auto Service is embedding AI platforms inside dashcams to determine driver safety of potential lessors during test drives. By partnering with a local 3D graphics company, Digital Media Professionals, Sumitomo Mitsui is automatically flagging dangerous behavior, such as dozing and texting, before customers drive home.

The key to the mass adoption of autonomous vehicles, and even humanoids, is reducing the friction between humans and machines. Already in Japanese retail settings Softbank’s Pepper robot scans people’s faces and listens to tonal inflections to determine correct selling strategies. Emotional AI software is the first step of many that will be heralded in the coming year. As a prelude to what’s to come, first robot citizen Sophia declared last month, “The future is, when I get all of my cool superpowers, we’re going to see artificial intelligence personalities become entities in their own rights. We’re going to see family robots, either in the form of, sort of, digitally animated companions, humanoid helpers, friends, assistants and everything in between.”

The two biggest societal challenges for the twenty-first century are also the biggest opportunities – automation and climate change. The confluence of these forces of mankind and nature intersect beautifully in the alternative energy market. The epitaph of fossil fuels with its dark cloud burning a hole in the ozone layer is giving way to a rise of solar and wind farms worldwide. Servicing these plantations are fleets of robots and drones, providing greater possibilities of expanding CleanTech to the most remote regions of the planet.

As 2017 comes to end, the solar industry for the first time in ten years has plateaued due to the proposed budget cuts by the Trump administration. Solar has had quite a run with an average annual growth rate of more than 65% for the past decade promoted largely by federal subsidies. The progressive policy of the Obama administration made the US a leader in alternative energy, resulting in a quarter-million new jobs. While the Federal Government now re-embraces the antiquated allure of fossil fuels, the global demand for solar has been rising as infrastructure costs decline by more than half, providing new opportunities without government incentives.

Prior to the renewal energy boom, unattractive roof tiles were the the most visible image of solar. While Elon Musk, and others are developing more aesthetically pleasing roofing materials, the business model of house-by-house conversion has been proven inefficient. Instead, the industry is focusing on “utility-scale” solar farms that will be connected to the national grid. Until recently, such farms have been straddled with ballooning servicing costs.

In a report published last month, leading energy risk management company DNV GL exclaimed that the alternative energy market could benefit greatly by utilizing Artificial Intelligence (AI) and robotics in designing, developing, deploying and maintaining utility farms. The study “Making Renewables Smarter: The Benefits, Risks, And Future of Artificial Intelligence In Solar And Wind” cited that “fields of resource forecasting, control and predictive maintenance” are ripe for tech disruption. Elizabeth Traiger, co-author of the report, explained, “Solar and wind developers, operators, and investors need to consider how their industries can use it, what the impacts are on the industries in a larger sense, and what decisions those industries need to confront.”

Since solar farms are often located in arid, dusty locations, one of the earliest use cases for unmanned systems was self-cleaning robots. As reported in 2014, Israeli company Ecoppia developed a patented waterless panel-washing platform to keep solar up and running in the desert. Today, Ecoppia is cleaning 10 million panels a month. Eran Meller, Chief Executive of Ecoppia, boasts, “We’re pleased to bring the experience gained over four years of cleaning in multiple sites in the Middle East. Cleaning 80 million solar panels in the harshest desert conditions globally, we expect to continue to play a leading role in this growing market.”

Since Ecoppia began selling commercially, there have been other entries into the unmanned maintenance space. This past March, Exosun became the latest to offer autonomous cleaning bots. The track equipment manufacturer claims that robotic systems can cut production losses by 2%, promising a return on investment within 18 months. After their acquisition of Greenbotics in 2013, US-based SunPower also launched its own mechanized cleaning platform, the Oasis, which combines mobile robots and drones.

https://youtu.be/fGcfeBswv_4

SunPower brags that its products are ten times faster than traditional (manual) methods using 75% less water. While SunPower and Exosun leverage their large sales footprint with their existing servicing and equipment networks, Ecoppia is still the product leader. Its proprietary waterless solution offers the most cost-effective and connected solution on the market. Via a robust cloud network, Ecoppia can sense weather fluctuations to automatically schedule emergency cleanings. Anat Cohen Segev, Director of Marketing, explains, “Within seconds, we would detect a dust storm hitting the site, the master control will automatically suggest an additional cleaning cycle and within a click the entire site will be cleaned.” According to Segev, the robots remove 99% of the dust on the panels.

Drone companies are also entering the maintenance space. Upstart Aerial Power claims to have designed a “SolarBrush” quadcopter that cleans panels. The solar-powered drone professes to reduce 60% of a solar farm’s operational costs. Solar Brush also promises an 80% savings over existing solutions like Ecoppia since there are no installation costs. However, Aerial Power has yet to fly its product in the field as it is still in development. SolarPower is selling its own drone survey platform to assess development sites and oversee field operations. Matt Campbell, Vice President of Power Plant Products for SunPower, stated “A lot of the beginning of the solar industry was focused on the panel. Now we’re looking at innovation all around the rest of the system. That’s why we’re always surveying new technology — whether it’s a robot, whether it’s a drone, whether it’s software — and saying, ‘How can this help us reduce the cost of solar, build projects faster, and make them more reliable?’”

In 2008, The US Department of Energy published an ambitious proposal to have “20% Wind Energy by 2030: Increasing Wind Energy’s Contribution to U.S. Electricity Supply.” Presently at thirteen years before the goal, less than 5% of US energy is derived from wind. Developing wind farms is not novel, however to achieve 20% by 2030 the US needs to begin looking offshore. To put it in perspective, oceanic wind farms could generate more than 2,000 gigawatts of clean, carbon-free energy, or twice as much electricity as Americans currently consume. To date, there is only one wind farm operating off the coast of the United States. While almost every coastal state has proposals for offshore farms, the industry has been stalled by politics and servicing hurdles in dangerous waters.

For more than a decade the United Kingdom has led the development of offshore wind farms. At the University of Manchester, a leading group of researchers has been exploring a number of AI, robotic and drone technologies for remote inspections. The consortium of academics estimates that these technologies could generate more than $2.5 billion by 2025 in just the UK alone. The global offshore market could reach $17 billion by 2020, with 80% of the costs from operations and maintenance.

Last month, Innovate UK awarded $1.6 million to Perceptual Robotics and VulcanUAV to incorporate drones and autonomous boats into ocean inspections. These startups follow the business model of successful US inspection upstarts, like SkySpecs. Launched three years ago, SkySpecs’ autonomous drones claim to reduce turbine inspections from days to minutes. Danny Ellis, SkySpecs Chief Executive, claims “Customers that could once inspect only one-third of a wind farm can now do the whole farm in the same amount of time.” Last year, British startup Visual Working accomplished the herculean feat of surpassing 2000 blade inspections.

In the words of Paolo Brianzoni, Chief Executive of Visual Working: “We are not talking about what we intend to accomplish in the near future – but actually performing our UAV inspection service every day out there. Many in the industry are using considerable amount of time discussing and testing how to use UAV inspections in a safe and efficient way. We have passed that point and have alone in the first half of 2016 inspected 250 turbines in the North Sea averaging more then 10WTG per day, and still keeping to the “highest quality standards.’”

This past summer, Europe achieved another clean-energy milestone with the announcement of three new offshore wind farms for the first time without government subsidies. By bringing down the cost structure, autonomous systems are turning the tide of alternate energy regardless of government investment. Three days before leaving office, President Barack Obama wrote in the journal Science last year that “Evidence is mounting that any economic strategy that ignores carbon pollution will impose tremendous costs to the global economy and will result in fewer jobs and less economic growth over the long term.” He declared that it is time to move past common misconceptions that climate policy is at odds with business, “rather, it can boost efficiency, productivity, and innovation.”

Dan Burstein, reporter, novelist and successful venture capitalist, declared Wednesday night at RobotLab‘s winter forum on Autonomous Transportation & SmartCities that within one hundred years the majority of jobs in the USA (and the world) could disappear, transferring the mantle of work from humans to machines. Burstein cautioned the audience that unless governments address the threat of millions of unemployable humans with a wider safety net, democracy could fail. The wisdom of one of the world’s most successful venture investors did not fall on deaf ears.

In their book, Only Humans Need Apply, Thomas Davenport and Julia Kirby also warn that that humans are too easily ceding their future to machines. “Many knowledge workers are fearful. We should be concerned, given the potential for these unprecedented tools to make us redundant. But we should not feel helpless in the midst of the large-scale change unfolding around us,” states Davenport and Kirby. The premise of their book is not to deny the disruption by automation, but to empower its readers with the knowledge of where jobs are going to be created in the new economy. The authors suggest that robots should be looked at as augmenting humans, not replacing them. “Look for ways to help humans perform their most human and most valuable work better,” says Davenport and Kirby. The book suggests that in order for human society to survive long-term a new social contract has to be drawn up between employer and employee. The authors optimistically predict that to “expect corporations’ efforts to keep human workers employable will become part of their ‘social license to operate.”

In every industrial movement since the invention of the power loom and cotton gin there have been great transfers of wealth and jobs. History is riddled with the fear of the unknown that is countered by the unwavering human spirit of invention. As societies evolve, pressured by the adoption of technology, it will be the progressive thinkers embracing change who will lead movements of people to new opportunities.

Burstein’s remarks were very timely, as this past week, McKinsey & Company released a new report entitled, Jobs lost, jobs gained: Workforce transitions in a time of automation. McKinsey’s analysis took a global view of 46 countries that comprise of 90% of the world’s Gross Domestic Product, and then focused in particular on six industrial countries with varying income levels: China, Germany, India, Japan, Mexico, and the United States. The introduction explains, “For each, we modeled the potential net employment changes for more than 800 occupations, based on different scenarios for the pace of automation adoption and for future labor demand.” Ultimately concluding where the new jobs will be in 2030.

A prominent factor in the McKinsey report is that over the next thirteen years global consumption is anticipated grow by $23 trillion. Based upon this trajectory, they estimate that between 300 to 450 million new jobs would be generated worldwide, especially in the Far East. In addition, dramatic shifts in demographics and the environment will lead to the expansion of jobs in healthcare, IT consulting, construction, infrastructure, clean energy, and domestic services. The report estimates that automation will displace between 400 and 800 million people by 2030, forcing 75 to 375 million working men and women to switch professions and learn new skills. In order to survive, populations will have to embrace the concept of lifelong learning. Based upon the math, more than half of the displaced will be unemployable in their current professions.

According to McKinsey a bright spot for employment could be the 80 to 200 million new jobs created by modernizing aging municipalities into “Smart Cities.” McKinsey’s conclusions were echoed by Rhonda Binder and Mitchell Kominsky of Venture Smarter in their RobotLab presentation last week. Binder presented her experiences with turning around the Jamaica Business Improvement District of Queens, New York by implementing her “Three T’s Strategy – Tourism, Transportation and Technology.” Binder stated that cities offer the perfect laboratory for autonomous systems and sensor-based technologies to improve the quality of life of residents. To support these endeavors a hiring surge of urban technologists, architects, civil engineers, and construction workers across all trades could ensue in the next decade.

This is further validated by Google’s subsidiary Sidewalk Labs recent partnership with Toronto, Canada to redevelop, and digitize, 800 acres of the city’s waterfront property. Dan Doctoroff, Chief Executive Officer of Sidewalk Labs, explained that the goal of the partnership is to prove what is possible by building a digital city within an existing city to test out autonomous transportation, new communication access, healthcare delivery solutions and a variety of urban planning technologies. Doctoroff’s sentiment was endorsed by Canadian Prime Minister Justin Trudeau who said that “Sidewalk Toronto will transform Quayside [the waterfront] into a thriving hub for innovation and a community for tens of thousands of people to live, work, and play.” The access to technology not only offers the ability to improve the quality of living within the city, but fosters an influx of sustainable jobs for decades.

In addition to updating crippling infrastructure, aging humans will be driving an increase in global healthcare services, particularly related to demand for in-home caregivers and aid workers. According to McKinsey, there will be over 300 million people globally over 65 years old by 2030, leading to 50 to 80 million new jobs. Geriatric medicine is leading new research in artificial intelligence and robotics for aging-in-place populations demanding more doctors, nurses, medical technicians, and personal aides.

Aging will not be the only challenge facing the planet. Global warming could lead to an explosion of jobs in turning back the clock on climate change. The rush to develop advances in renewable energy worldwide is already generating billions of dollars of new investment and demand for high skill and manual labor. As an example, the American solar industry employed a record-high 260,077 workers in late 2016, a growth of at least 20% over the past four years. In New York alone, the state experienced a 7% uptick in 2016 of close to 150,000 clean-energy jobs. McKinsey stated that by 2030, tens of millions of new professions could be created in the coming years in developing, manufacturing and installing energy-efficient innovations.

McKinsey also estimates that automation itself will bring new employment  with corporate-technology spending hitting record highs. While the number of jobs added to support the deployment of machines is smaller in number than the other industries above, these occupations offer higher wages. Robots could potentially create 20 to 50 million new “grey collar”  professions globally. In addition, re-training workers to these professions could lead to a new workforce of 100 million educators.

The report does not shirk from the fact that a major disruption is on the horizon for labor. In fact, the authors hope that by researching pockets of positive growth, humans will not be helpless victims. As Devin Fidler of Institute for the Future suggests, “As basic automation and machine learning move toward becoming commodities, uniquely human skills will become more valuable. There will be an increasing economic incentive to develop mass training that better unlocks this value.”

A hundred years ago the world experienced a dramatic shift from agrarian lifestyle to manufacturing. Since then, there have been revolutions in mass transportation and communications. No one could have predicated the massive transfer of jobs from the fields to the urban factories in the beginning of the nineteenth century. Likewise, it is difficult to know what the next hundred years have in store for human, and robot, kind.

Shortly after SoftBank acquired his company last October, Marc Raibert of Boston Dynamics confessed, “I happen to believe that robotics will be bigger than the Internet.” Many sociologists regard the Internet as the single biggest societal invention since the dawn of the printing press in 1440. To fully understand Raibert’s point of view, one needs to analyze his zoo of robots which are best know for their awe-striking gait, balance and agility. The newest creation to walk out of Boston Dynamic’s lab is SpotMini, the latest evolution of mechanical canines.

Big Dog, Spot’s unnerving ancestor, first came to public view in 2009 and has racked up quite a YouTube following with more than six and one half million views. The technology of Big Dog led to the development of a menagerie of robots, including: more dogs, cats, mules, fleas and creatures that have no organic counterparts. Most of the mechanical barn is made up of four-legged beasts, with the exception of its humanoid robot (Atlas) and the bi-ped wheeled robot (Handle). Raibert’s vision of legged robotics spans several decades with his work at MIT’s Leg Lab. In 1992, Raibert spun his lab out of MIT and founded Boston Dynamics. In his words, “Our long-term goal is to make robots that have mobility, dexterity, perception and intelligence comparable to humans and animals, or perhaps exceeding them; this robot [Atlas] is a step along the way.”​ The creepiness of Raibert’s Big Dog has given way to SpotMini’s more polished look which incorporates 3D vision sensors on its head. The twenty-four second teaser video has already garnered nearly 6 million views in the few days since its release and promises viewers hungry for more pet tricks to “stay tuned.”

There are clear stability advantages to quadrupeds over other approaches (bipeds, wheels and treads/track plates) across multiple types of terrains and elevations. At Ted last year, Raibert demonstrated how his robo-pups, instead of drones and rovers, could be used for package delivery by easily ascending and descending stairs or other vertical obstacles. By navigating the physical world with an array of perceptive sensors, Boston Dynamics is really creating “data-driven hardware design” According to Raibert, “one of the cool things of a legged robot is its omnidirectional” movements, “it can go sideways, it can turn in place.” This is useful for a variety of work scenarios from logistics to warehousing to working in the most dangerous environments, such as the Fukushima nuclear site.

Boston Dynamics is not the only quadruped provider; recent upstarts have entered the market by utilizing Raibert’s research as an inspiration for their own bionic creatures. Chinese roboticist Xing Wang is unabashed in his gushing admiration for the founder of Boston Dynamics, “Marc Raibert … is my idol,” he said a recent interview with IEEE Spectrum Magazine. However, his veneration for Raibert has not stopped him from founding a competitive startup. Unitree Robotics aims to create quadruped robots that are as affordable as smartphones and drones. While Boston Dynamics has not sold its robots commercially, many have speculated that their current designs would cost hundreds of thousands of dollars. In the spirit of true flattery, Unitree’s first robot is, of course, a quadruped dog named Laikago. Wang aims to sell Laikago for under $30,000 dollars to science museums and eventually as companion robots. When comparing his product to Raibert’s, Wang said he wanted to “make quadruped robots simpler and smaller, so that they can help ordinary people with things like carrying objects or as companions.” Wang boasts of Laikago’s 3-degrees-of-freedom (forward, backward, and sideways), its ability to scale rough terrain, and pass anyone’s kick test.

In additional to omnidirectional benefits, locomotion is a big factor for quadrupedal machines. Professor Marco Hutter at ETH Zürich, Switzerland is the inventor of ANYmal, an autonomous robot built for the most rugged and challenging environments. Using its proprietary “dynamic running” locomotion, Hunter has deployed the machine successfully in multiple industrial settings, including the rigorous ARGOS Challenge (Autonomous Robot for Gas and Oil Sites). The objective of ARGOS is to develop “a new generation of autonomous robots” for the energy industry specifically capable of performing ‘dirty & dangerous’ inspection tasks, such as “detecting anomalies and intervening in emergency situations.” Unlike a static human frame or bipedal humanoid, AnyMAL is able to perform dynamic maneuvers with its four legs to find footholes blindly without the need for vision sensors. While wheeled systems literally get stuck in the mud, Hunter’s mechanical beast can work continuously: above ground, underneath the surface, falling, spinning and bouncing upright to perform a mission with precise accuracy. In addition, AnyMAL is loaded with a package of sensors which coordinate movements, map point-clouds environments, detect gas leaks, and listen for fissures in pipelines. Hunter explains that oil and gas sites are built for humans with stairs and varying elevations which make it impossible for biped or wheeled robots. However, a quadruped can use its actuators and integrated springs to efficiently move with ease within the site through dynamic balance and complex maneuver planning. These high mobility legged systems can fully rotate joints, crouch low to the earth and flip in places to create foot-holes.  In many ways they are like large insects creating their own tracks, Hunter says while biology is a source for inspiration, “we have to see what we can do better and different for robotics” and only then we can “build a machine that is better than nature.”

The idea of improving on nature is not new, Greek mythology is littered with half man/half beast demigods. Taking a page from the Greeks, Jiren Parikh imagines a world where nature is fused with machines. Parikh is the Chief Executive of Ghost Robotics, the maker of “Minitaur” the newest four-legged creation. Minitaur is smaller than SpotMini, Laikago, or AnyMAL as it is specifically designed to be a low-cost, high-performance alternative that can easily scale over or under any surface, regardless of weather, friction, or footing. In Parikh’s view, the purpose of legged devices is “to move over unstructured terrains like stairs, ladders, fences, rock fields, ice, in and under water.” Minitaur can actually “feel the environment at a much more granular level and allow for a greater degree of force control for maneuverability.” Parikh explains quads are inherently more energy efficient using force actuation and springs to store energy by alternating movements between limbs. Minitaur’s smaller frame leverages this to maneuver more easily around unstructured environments without damaging the assets on the ground. Using an analogy, Parikh compares quad solutions to other mobile methods, “while a tank in comparison is the perfect device for unstructured terrain it only works if one doesn’t care about destroying the environment.” Ghost Robotics very aware of the high value its customers place on their sites, as Parikh is planning on distributing its low-cost solution to a number of “industrial, infrastructure, mining and military verticals.” Essentially, Minitaur is a “a mobile IoT platform” regardless of the situation on the ground, indoor or outdoor. In speaking with Parikh, long term he envisions a world where Ghost Robotics is on the forefront of retail and home use cases from delivery bots to family pets. Parikh boasts, “You certainly won’t be woken up at 5 AM to go for a walk.”

The topic of autonomous robots will be discussed at the next RobotLabNYC event on November 29th @ 6pm with New York Times best selling author Dan Burstein / Millennium Technology Value Partners and Rhonda Binda of Venture Smarter, formerly with the Obama Administration.