Jannah Theme License is not validated, Go to the theme options page to validate the license, You need a single license for each domain name.
Tech

When it comes to big language models, should you build or buy? • TechCrunch


Last summer could can only be described as an “AI summer”, especially with large language models making an explosive entrance. We have seen huge neural networks trained on a massive body of data that can perform extremely impressive tasks, none more famous than OpenAI’s GPT-3 and its trendy new offspring, ChatGPT.

Companies of all shapes and sizes across all industries are racing to figure out how to integrate and extract value from this new technology. But OpenAI’s business model has been no less transformative than its contributions to natural language processing. Unlike almost every previous version of a flagship model, this one does not come with open source pre-trained weights, i.e. machine learning teams cannot simply download the models and adjust them for their own use cases.

Instead, they either have to pay to use them as is, or pay to refine the models and then pay four times the usage rate as is to use them. Of course, companies can always choose other peer open source models.

This gave rise to an age-old – but entirely new to ML – business question: would it be better to buy or develop this technology?

It is important to note that there is no single answer to this question; I’m not trying to provide a catch-all answer. I mean highlighting the pros and cons of both paths and offering a framework that could help companies assess what works for them while providing middle paths that attempt to include components of both worlds.

Purchase: Fast, but with clear pitfalls

While construction looks attractive over the long term, it requires leadership with a strong appetite for risk, as well as deep coffers to support that appetite.

Let’s start with the purchase. There are plenty of template-as-a-service providers that offer custom templates as APIs, charging on demand. This approach is fast, reliable, and requires little to no upfront capital outlay. Indeed, this approach reduces the risks of machine learning projects, especially for companies entering the field, and requires limited internal expertise beyond software engineers.

Projects can be started without requiring staff experienced in machine learning, and model results can be reasonably predictable, given that the ML component is purchased with a set of guarantees around the output.

Unfortunately, this approach has very clear pitfalls, the main one being the limited defense of the product. If you’re buying a model that anyone can buy and integrate into your systems, it’s no exaggeration to assume that your competitors can achieve product parity just as quickly and reliably. This will be true unless you can create an upstream gap through non-repeatable data collection techniques or a downstream gap through integrations.

Additionally, for high-throughput solutions, this approach can be extremely expensive at scale. For context, OpenAI’s DaVinci costs $0.02 per thousand tokens. Conservatively assuming 250 tokens per request and similar sized responses, you pay $0.01 per request. For a product with 100,000 requests per day, you would pay over $300,000 per year. Obviously, text-heavy apps (attempting to generate an article or engage in a chat) would incur even higher costs.

You also need to consider the limited flexibility with this approach: either you use the models as-is, or you pay a lot more to fine-tune them. It’s worth remembering that the latter approach would involve an unspoken ‘lock-in’ period with the vendor, as the fine-tuned models will be kept in their digital custody, not yours.

Building: Flexible and defensible, but costly and risky

On the other hand, building your own technology allows you to circumvent some of these challenges.


techcrunch Gt

Not all news on the site expresses the point of view of the site, but we transmit this news automatically and translate it through programmatic technology on the site and not from a human editor.
Back to top button