Generative AI Series 2, New York: Fellow Discussions on Gen AI Foundations and Applications

This is the second part of the series on Generative AI by Fellows Fund. The first part based on a similar discussion in Silicon Valley in Jan ‘22 is here‍

At a beautiful restaurant in New York City with a stunning view of the top of Federal Hall, hosted by Fellows Fund, over 35 leaders and entrepreneurs in the field of AI, including leaders from Microsoft, Celonis, Vi, Hyperscience, Dataminr, Meroxa, CPNET.io, Spectre Data, Absci, MosaicML, Verneek, Morgan Stanley and investors from Insight Partners, Tru Arrow Partners and others, gathered to discuss the foundations and applications of the rapidly evolving Generative AI (Gen AI) technology in the finance, enterprise, life sciences and other sectors.

Fellows Fund is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Subscribe

TL;DR

1. There will be continued innovation to both improve the quality and reduce the cost in all 3 layers of the AI stack - infrastructure to train and serve AI models (also called foundation models or large language models when applied to language and text data), a middle layer where foundation models are developed for specific domains and problems (for example, models for genomics, chemistry, law, etc.), as well as an application layer where Gen AI will improve the user experience and outcomes of a broad range of consumer and enterprise applications.
2. Application companies that own the end-user experience, and create a flywheel where the AI models are trained on domain-specific data, and data on outcomes and experiences are used to further improve the AI models in a closed-loop system, will create the most value using Gen AI.
3. As content generated by Gen AI becomes ubiquitous, it’s imperative to have scalable capabilities for detecting human-generated content versus machine-generated content and to verify that the generated content is safe and trustworthy.
4. Currently, most of the infrastructure, middleware, applications, data and application ecosystems are yet to be built to realize the potential of Gen AI, and we believe that this presents a timely opportunity for entrepreneurs and investors to innovate in all three layers of the stack to shape how we live and work in a world powered by Gen AI.

The evening saw 2 panels discussing the GAI foundations, applications in enterprises, finance, life sciences, and value creation.

Three layers of the Generative AI stack:

The first panel discussed the emerging stack for Gen AI.

1. The infrastructure layer for training and serving the AI foundation models at scale and efficiently has evolved a lot in the last 12 months. Some of the large foundation models like GPT3 are now hosted on serving farms in the cloud with serving SLAs, and accessible as APIs for developers to use in applications. The training of these large foundation models are predominantly done in GPU clusters on the cloud with high-speed networking for fast data access to keep pace with the computing speed. The training workloads will continue to rely on GPUs in the near to mid-term, while the model serving currently in the cloud infrastructure will become more diverse in the medium term, moving closer to the end applications. Even though the cost of AI infrastructure is high at present, with new hardware and software innovations the cost should continue to drop, and it will become cheaper once a sufficient amount of AI infrastructure capacity is built up to be available on demand.  

2. The middle layer involves training foundation models per domain (for example, enterprise functions like legal, finance, marketing, HR, etc., industry verticals like retail, pharma, manufacturing, etc.). These domain-specific models can either be trained by fine-tuning general-purpose foundation models like GPT3, BERT, etc. or building them from scratch, using both public and private domain-specific datasets. The domain-specific models will likely remain more accurate than general-purpose foundation models, for the next few years.

3. The application layer, where the foundation models (or fine-tuned per domain) are combined with domain-specific data to build applications and create novel end-user experiences. As the quality of the foundation models improves, the applications will become more effective. However, to further optimize these AI models for domain-specific outcomes and experiences, data that is specifically used to train or fine-tune the application-specific models, as well as data collected from using the applications, will be even more valuable. These optimizations will be achieved through techniques like reinforcement learning with both explicit (e.g. human generated labels) or implicit feedback (e.g. how the AI generated experiences are adopted by the end users). These datasets will create a defensible moat for Gen AI application companies. A good example of this type of application company is the new age drug discovery company Absci. The AI-powered drug discovery company designs new antibodies from scratch using AI generative models, tests promising candidates in the lab for efficacy and feeds the test results back to improve the model quality. The models are trained to produce not just promising candidates that will efficiently bind to the target of interest, but also have specific properties that make them easy to develop at scale and have low immunogenicity.

An emerging AI infrastructure and middle layer will enable a large number of AI-powered applications to run on a new operating system for AI. We expect there will be a handful of companies powering the AI infrastructure layer, more companies differentiating themselves by building domain-specific foundation models, and a large number of companies (including startups) creating domain-specific applications and human-centered experiences. The quality of these applications will steadily improve when the underlying AI models and the user experiences are optimized using feedback data on the quality and effectiveness of generated artifacts. Furthermore, we expect a few ecosystems to emerge that are centered around domain-specific data assets, software frameworks, and tooling (such as gaming, life sciences, manufacturing, finance, etc.).

For AI application companies using large foundation models, a current dilemma is their reliance on a few large vendors in the infrastructure and middle layers and the risk of these vendors moving up to the application stack. Our view is that despite the current situation, application companies that differentiate through the use of domain-specific data and user experiences will sustain their advantage. This is likely to be similar to the public clouds, where the cloud providers excel at the lower layers of the infrastructure stack while a number of successful applications and companies are built atop them without having to incur the large cost of one-time investments in infrastructure.

Potential applications of Gen AI in enterprises and financial services:

The second panel focused on the potential applications of Gen AI in enterprises and a few specific industries.

1. One of the biggest impacts of Gen AI in enterprises will be to increase the efficiency of knowledge workers. This increased efficiency will be achieved by reducing the time to find and use information from both structured and unstructured content within an enterprise (better search and answer questions on the enterprise content and data sources), reducing the barrier and improving the fidelity of communication (between employees, between the enterprise and customers, across languages, etc.), create engaging content for external communication, discover and design novel enterprise processes and workflows that improve the quality, speed up the process to desired outcomes, etc. This improved efficiency will free up some time for knowledge workers to focus on higher value-added activities and personal projects.

2. Many Gen AI applications in the near future will be aimed at use cases where quality is subjective and can be improved iteratively by human domain experts. As examples, this can be seen in creative workflows - such as games, fashion, media, etc.; services - such as customer support, digital marketing assets, etc.;  and even software development. The common themes in these use cases are that there could be multiple outputs, each having its own attributes such as style, taste, complexity, etc., but the outcomes are reviewed by a human expert before being acted upon, and an incorrect result is relatively inexpensive.

Important problems and startup opportunities:

The cost of infrastructure and middle layers continues to decrease, making it increasingly easier to develop cost-effective Gen AI applications. However, a few other factors will become more important for companies to build differentiated and defensible applications and experiences using Gen AI.

1. Acquire and access relevant data: This includes not just the data used to train or fine-tune the foundation models, but also other data like the input prompts used to generate the outputs from these models, labeled or annotated data specific to the domain/problem that can be used to optimize the models, data on the usage and feedback from using the outputs of the applications, data on the properties of the outputs themselves, etc. Building and operating closed-loop data systems that include creation, usage, feedback, and iteration quickly will distinguish companies and institutions from those without them.

2. Origin and safety of generated content: As Gen AI technologies produce content at scale, the time and cost to produce machine-generated content at scale diminish. However, human-generated content has a substantial and rising cost. As the content ecosystem gets flooded with machine-generated content, it will become increasingly important to be able to identify, tag, and track the provenance (human versus machine-generated) and lineage (copyright, derivative works, etc.) of content using technologies like NFT, create technologies and systems to monitor and moderate the safety and trustworthiness of generated content and develop guardrails around the use of machine-generated content. This presents opportunities for new AI companies (like SpectrumLabsai.com) to develop AI-powered content moderation products.

3. Human-centered AI experiences: In the near to medium term, there is greater potential for AI to augment human expertise and amplify productivity. To achieve this goal, the AI system must be designed with humans at the center, in control, and with AI in the loop providing routine assistance to the human experts.

Over the last 12 months, Gen AI technology has demonstrated enormous potential for amplifying human creativity and productivity. Despite a few large companies positioning to be ahead in building and leveraging Gen AI in their products, we believe that a lot of the infrastructure, middleware, applications, data, and application ecosystems are yet to be built to realize the potential of Gen AI. We also see a number of talented platform and product development professionals explore promising ideas in Gen AI. We are highly optimistic that this is a timely opportunity for entrepreneurs and investors in all three layers of the stack to shape how we live and work in a future where Gen AI is ubiquitous in our personal and professional lives!

Launch your startup project with Fellows Fund! Connect with a supportive community and secure funding for your project. Join us today at Fellows Collective, which is a startup fellow community built by Fellows Fund fellows.

‍