Menu Close

Petals is creating a free, distributed network for running text-generating AI

BigScience, a community project backed by startup Hugging Face with the goal of making text-generating AI widely available, is developing a system called Petals that can run AI like ChatGPT by joining resources from people across the internet. With Petals, the code for which was released publicly last month, volunteers can donate their hardware power to tackle a portion of a text-generating workload and team up others to complete larger tasks, similar to Folding@home and other distributed compute setups.

“Petals is an ongoing collaborative project from researchers at Hugging Face, Yandex Research and the University of Washington,” Alexander Borzunov, the lead developer of Petals and a research engineer at Yandex, told TechCrunch in an email interview. “Unlike … APIs that are typically less flexible, Petals is entirely open source, so researchers may integrate latest text generation and system adaptation methods not yet available in APIs or access the system’s internal states to study its features.”

Open source, but not free

For all its faults, text-generating AI such as ChatGPT can be quite useful — at least if the viral demos on social media are anything to go by. ChatGPT and its kin promise to automate some of the mundane work that typically bogs down programmers, writers and even data scientists by generating human-like code, text and formulas at scale.

But they’re expensive to run. According to one estimate, ChatGPT is costing its developer, OpenAI, $100,000 per day, which works out to an eye-watering $3 million per month.

The costs involved with running cutting-edge text-generating AI have kept it relegated to startups and AI labs with substantial financial backing. It’s no coincidence that the companies offering some of the more capable text-generating systems tech, including AI21 Labs, Cohere and the aforementioned OpenAI, have raised hundreds of millions of dollars in capital from VCs.

But Petals democratizes things — in theory. Inspired by Borzunov’s earlier work focused on training AI systems over the internet, Petals aims to drastically bring down the costs of running text-generating AI.

“Petals is a first step towards enabling truly collaborative and continual improvement of machine learning models,” Colin Raffel, a faculty researcher at Hugging Face, told TechCrunch via email. “It … marks an ongoing shift from large models mostly confined to supercomputers to something more broadly accessible.”

Raffel made reference to the gold rush, of sorts, that’s occurred over the past year in the open source text generation community. Thanks to volunteer efforts and the generosity of tech giants’ research labs, the type of bleeding-edge text-generating AI that was once beyond reach of small-time developers suddenly became available, trained and ready to deploy.

BigScience debuted Bloom, a language model in many ways on par with OpenAI’s GPT-3 (the progenitor of ChatGPT), while Meta open sourced a comparably powerful AI system called OPT. Meanwhile, Microsoft and Nvidia partnered to make available one of the largest language systems ever developed, MT-NLG.

But all these systems require powerful hardware to use. For example, running Bloom on a local machine requires a GPU retailing in the hundreds to thousands of dollars. Enter the Petals network, which Borzunov claims will be powerful enough to run and fine-tune AI systems for chatbots and other “interactive” apps once it reaches sufficient capacity. To use Petals, users install an open source library and visit a website that provides instructions to connect to the Petals network. After they’re connected, they can generate text from Bloom running on Petals, or create a Petals server to contribute compute back to the network.

The more servers, the more robust the network. If one server goes down, Petals attempts to find a replacement automatically. While servers disconnect after around 1.5 seconds of inactivity to save on resources, Borzunov says that Petals is smart enough to quickly resume sessions, leading to only a slight delay for end-users.

Testing the Bloom text-generating AI system running on the Petals network. Image Credits: Kyle Wiggers / TechCrunch

In my tests, generating text using Petals took anywhere between a couple of seconds for basic prompts (e.g. “Translate the word ‘cat’ to Spanish”) to well over 20 seconds for more complex requests (e.g. “Write an essay in the style of Diderot about the nature of the universe”). One prompt (“Explain the meaning of life”) took close to three minutes, but to be fair, I instructed the system to respond with a wordier answer (around 75 words) than the previous few.

Image Credits: Kyle Wiggers / TechCrunch

That’s noticeably slower than ChatGPT — but also free. While ChatGPT doesn’t cost anything today, there’s no guarantee that that’ll be true in the future.

Borzunov wouldn’t reveal how large the Petals network is currently, save that “multiple” users with “GPUs of different capacity” have joined it since its launch in early December. The goal is to eventually introduce a rewards system to incentivize people to donate their compute; donators will receive “Bloom points” that they can spend on “higher priority or increased security guarantees” or potentially exchange for other rewards, Borzunov said.

Limitations of distributed compute

Petals promises to provide a low-cost, if not completely free, alternative to the paid text-generating services offered by vendors like OpenAI. But major technical kinks have yet to be ironed out.

Most concerning are the security flaws. The GitHub page for the Petals project notes that, because of the way Petals works, it’s possible for servers to recover input text — including text meant to be private — and record and modify it in a malicious way. That might entail sharing sensitive data with other users in the network, like names and phone numbers, or tweaking generated code so that it’s intentionally broken.

Petals also doesn’t address any of the flaws inherent in today’s leading text-generating systems, like their tendency to generate toxic and biased text (see the “Limitations” section in the Bloom entry on Hugging Face’s repository). In an email interview, Max Ryabinin, the senior research scientist at Yandex Research, made it clear that Petals is intended for research and academic use — at least at present.

“Petals sends intermediate … data though the public network, so we ask not to use it for sensitive data because other peers may (in theory) recover them from the intermediate representations,” Ryabinin said. “We suggest people who’d like to use Petals for sensitive data to set up their own private swarm hosted by orgs and people they trust who are authorized to process this data. For example, several small startups and labs may collaborate and set up a private swarm to protect their data from others while still getting benefits of using Petals.”

As with any distributed system, Petals could also be abused by end-users, either by bad actors looking to generate toxic text (e.g. hate speech) or developers with particularly resource-intensive apps. Raffel acknowledges that Petals will inevitably “face some issues” at the start. But he believes that the mission — lowering the barrier to running text-generating systems — will be well worth the initial bumps in the road.

“Given the recent success of many community-organized efforts in machine learning, we believe that it is important to continue developing these tools and hope that Petals will inspire other decentralized deep learning projects,” Raffel said.

Petals is creating a free, distributed network for running text-generating AI by Kyle Wiggers originally published on TechCrunch

BigScience, a community project backed by startup Hugging Face with the goal of making text-generating AI widely available, is developing a system called Petals that can run AI like ChatGPT by joining resources from people across the internet. With Petals, the code for which was released publicly last month, volunteers can donate their hardware power
Petals is creating a free, distributed network for running text-generating AI by Kyle Wiggers originally published on TechCrunch   TechCrunch 

Generated by Feedzy

Disclaimer

Innov8 is owned and operated by Rolling Rock Ventures. The information on this website is for general information purposes only. Any information obtained from this website should be reviewed with appropriate parties if there is any concern about the details reported herein. Innov8 is not responsible for its contents, accuracies, and any inaccuracies. Nothing on this site should be construed as professional advice for any individual or situation. This website includes information and content from external sites that is attributed accordingly and is not the intellectual property of Innov8. All feeds ("RSS Feed") and/or their contents contain material which is derived in whole or in part from material supplied by third parties and is protected by national and international copyright and trademark laws. The Site processes all information automatically using automated software without any human intervention or screening. Therefore, the Site is not responsible for any (part) of this content. The copyright of the feeds', including pictures and graphics, and its content belongs to its author or publisher.  Views and statements expressed in the content do not necessarily reflect those of Innov8 or its staff. Care and due diligence has been taken to maintain the accuracy of the information provided on this website. However, neither Innov8 nor the owners, attorneys, management, editorial team or any writers or employees are responsible for its content, errors or any consequences arising from use of the information provided on this website. The Site may modify, suspend, or discontinue any aspect of the RSS Feed at any time, including, without limitation, the availability of any Site content.  The User agrees that all RSS Feeds and news articles are for personal use only and that the User may not resell, lease, license, assign, redistribute or otherwise transfer any portion of the RSS Feed without attribution to the Site and to its originating author. The Site does not represent or warrant that every action taken with regard to your account and related activities in connection with the RSS Feed, including, without limitation, the Site Content, will be lawful in any particular jurisdiction. It is incumbent upon the user to know the laws that pertain to you in your jurisdiction and act lawfully at all times when using the RSS Feed, including, without limitation, the Site Content.  

Close Bitnami banner
Bitnami