Amazon cloud-service outage affected companies from Shipt to Roku

Grocery-delivery company Shipt along with several other tech companies experienced large-scale service issues Wednesday due to an outage of Amazon.com's widely used cloud service, Amazon Web Services (AWS).

The hourslong outage hobbled companies that rely on AWS, including the Amazon-owned Ring security-camera service, iRobot's Roomba vacuum cleaner app, newspaper sites such as the Washington Post, Flickr and Roku.

AWS is the world's largest provider of cloud-computing services, which allows customers to rent data-storage and processing capabilities over the web instead of running their own data centers. AWS, which competes in the business of providing computing infrastructure via the cloud against Microsoft and Google, has emerged as Amazon's most profitable business.

Some Twin Cities Shipt contract workers, who deliver groceries and other products to customers, said they were not able to accept orders and encountered additional problems on the busy shopping day before Thanksgiving.

At noon, Target-owned Shipt announced on Twitter it was experiencing "disrupted service." Later in the afternoon, Shipt said it was back to successfully processing orders nationwide. The company said it was prioritizing rescheduling orders that were not processed during the outage.

"We apologize for the inconvenience," Danielle Schumann, a Shipt spokeswoman, said in an e-mail.

Some Shipt workers expressed frustration on social media as they dealt with widespread issues on a day that was expected to be busy with orders.

Adding to the discontentment, many of the workers, who are independent contractors, were working Wednesday to earn cash bonuses from $50 to $300 that Shipt said it was giving to workers as an incentive based on the number of orders completed between Wednesday and Cyber Monday.

"Shipt's app had systemwide issues where no one could accept orders and many could not process orders," said Twin Cities Shipt worker Kandy Halley, in an e-mail. "Many also [were] unable to reach customers by phone due to issues. Many shoppers [were] sitting and waiting for issues to get fixed. Lost wages. Possibly lost meeting bonus quotas. Customers also lose out. All when it's super busy the day before Thanksgiving."

Shipt told its contract workers that any orders affected by the outage would not affect their shopper stats and would still count toward bonus counts.

Halley has been vocally critical of Shipt leadership saying Shipt's Thanksgiving "bonuses" were not adequate for the amount of work to fill orders over the next few days and that a new payment algorithm that Shipt instituted across the country at the end of September has severely cut workers' pay.

Shipt has said the new pay structure is supposed to more accurately pay workers based on effort such as deliveries made during high traffic times.

Earlier this week, Shipt announced it was hiring 50,000 more workers during the holiday season in addition to 100,000 extra workers it announced last month it wanted to hire as the company is seeing increased, early demand, Shipt said.

Shipt is seeking 6,000 more workers in the Twin Cities. The new hires would push the number of Shipt's shoppers to more than 350,000 after the company already doubled the number of shoppers to 200,000 in the spring.

As Amazon wrestled to resolve the outage, it offered little in the way of explanation.

"We are working toward resolution," Amazon spokeswoman Mary Camarata said.

A few hours after the outage began, Amazon wrote on the AWS status page that it had "identified the root cause" of the problem and was working to resolve it.

By midafternoon on the East Coast, Amazon had updated its status report, saying it "expect(s) full recovery to still take up to a few hours."

The AWS outage appeared to be limited to companies serviced by the collection of data centers Amazon runs that are scattered over a handful of northern Virginia cities, according to the AWS status webpage. That data center grouping, known as a "region" in cloud-computing lingo, is the oldest in Amazon's operations, having launched in 2006.

But the effect of the outage was felt broadly. New-account activation and the mobile app for streaming media service Roku were hampered, even as the streaming service continued to work on televisions, spokeswoman Tricia Mifsud said via e-mail.

Amazon's cloud-service business is thriving on higher demand from companies switching to virtual offices due to the COVID-19 pandemic. Sales from AWS rose about 29% in the third quarter.

Amazon held 45% of the global cloud-services market in 2019, according to the market-research firm Gartner. Amazon's next closest cloud-infrastructure competition, Microsoft, had 17.9% of the market last year.

Includes reporting from the Washington Post and Reuters.