AI on Azure: Developing Private-Data Apps using Azure AI Services (2/n)

In the last article, we learned the foundations of Azure AI services. We learned how to get started building AI powered applications on Azure.

In this article, we will build a RAG (retrieval-augmented-generation) chatbot for spring-framework documentation. The final app should look something like this:

Spring Framework AI Bot for rescue!

If you are from a Java software developer, you most likely have worked on Spring Framework which is a framework for developing enterprise-ready applications, letting the developers work on developing the application rather than worrying about the integrations, configurations and what not.

While Spring Framework has become de-facto standard for developers, there’s a steep learning curve (I wrote my first book Just Spring on the very same framework — a straight-to-the-point book — please don’t buy it as it is outdated) and I do see not only juniors and mid-level, but also pros not having a solid understanding of the framework.

We will be building a AI bot that would help learn, develop, decipher Spring Framework using Azure AI services.

Creating the Project

Jump on to the portal and get your AI services in place — follow the part 1 if you haven’t set that up yet.

Once you are in AI studio, create the “spring-docs-bot” project, as shown below:

Creating Spring Docs Bot project

The AI hub is a collaboration workspace for all of our projects that would share the project settings, security, configurations and so on.

Once the project is created, you’d be navigated to the project’s Build page. The overview would be nothing but empty and so are your deployments and others.

Let’s quickly go over to the All Settings by clicking the bottom left link. This will navigate to the project settings:

Spring-docs-bot project settings

As you can see from the settings, a set of connections were already available for the project — connection to the Azure Open AI models and a connection to the blob storage containers. If you are working with any other resources such as Azure AI search (vector store), you can create a new connection here to your already existing Search service in this project space.

Deploy Models

The first step is to get required models deployed for your project. Azure OpenAI provides a set of models (LLMs) that we can deploy for our project’s inference needs.

When working with private data, you need two models — one a LLM for inferencing purposes and another one — an embedding model to vectorise the data and queries. Choose GPT-x model and text-embedding-ada-002 for these purposes respectively.

Navigate to the Deployments menu, click on the “Create” button, select the appropriate GPT model (I’ve chosen GPT-4) and get it deployed. Repeat the same process to get the text-embedding-ada-002 model deployed too.

Once these models are deployed, we are ready to play in the Playground — no pun intended :)

Adding Data

We will need to add spring-framework data so our AI bot get grounded. In the last article, we gone over how to upload a document via the Playground (uploading via add-you-data button).

In this article, we will go over Azure AI Search’s new function “Import and Vectorise Data” which is still preview.

Vector stores represent data in a floating point format — called vectors — in a multi-dimensional space. They use sophisticated search algorithms like k-nearest neighbour (k-NN)or hierarchical navigable small world (HNSW) to retrieve relevant results for a user’s query

With the advent of GenAI on our door steps, most search vendors embraced the change and quickly adapted vector capabilities into their existing product (Azure AI search, Elasticsearch etc) while new products have come up on to the stage (ChromaDB, Pinecone, Weviate and others) as newcomers. Either way, there are a ton of products out there, both opensource and closed source to choose from.

We will be using Azure AI search in this article, afterall, the series is “Azure on AI” :)

Azure has rebranded its Cognitive Search to Azure AI search with its brand new vector search capabilities.

Setting up Azure AI Search

Head over to portal and search for Azure AI search — the resource creation is straight forward.

Once the vector db is created, navigate to the dashboard, which should be seen something like this:

Azure AI Search service (vector db)

The Indexes are the ones which will hold your vectorised data for your document. The indexer on the other hand is an integration component that would connect to datasource and fetches on demand or periodically. Of course, the data sources are your pre-configured data locations, such as blog storage, Cosmos DB etc, where your data lies.

One wouldn’t start vectorising/indexing data via the UI in a real world other than testing/trying purposes. I chose this to onboard ourselves quickly and for simplicity. Later on (perhaps in an another article) in this series, we will code how to vectorise data using Python by connecting to your vector store.

Before you start thinking of vectorising the data, there’s one more piece of pre-requisite that needs sorting — the storage account

Create a Storage Blob

As of now, the “Import and vectorise data” feature works loading data from a blob storage account for now. The “import data” on the other hand would provide us a handful of options including uploading from databases, share point and others.

Azure AI search has added a “integrated vectorisation” feature that would carry out the required steps to chunk and vectorise data. The vectorisation is carried out by adding additional skills — skills are like data transformers.enhancers/enrichers on the data inbound pipeline. Azure AI search added a new skill called “Text Split skill” for chunking the data while “Open AI Embedding skill” for vectorising the data.

Provide the required information in the form for creating the storage account — make sure you provide anonymous access to the blobs for now (don’t do this in production, please!). This can be done in the security section under Advanced tab by checking the “Allow enabling anonymous access on individual containers” check box.

In the storage account, once it gets created, head down to Containers and create a new containers — I’ve added “spring-framework-docs” container with anonymous access:

Blob Storage container for hosting spring-framework-docs

Once the blob container is provisioned, the next step is to upload the document to this container. Click on the container and upload the file following the instructions mentioned on the next page.

Now let’s head back to Azure AI Search dashboard and start importing the data and vectorising it.

Vectorise Data

On the AI Search dashboard, click the “Import and vectorise data” link — this will lead to the next page where we will need to select the location on our blob container:

Connecting to our blob store

In the next page, there are a few configurational settings that would be required to set up the vectorisation process.

The following image demonstrates the setup.

Vectorise the data

Choose your open-ai service and the text embedding model (the tex-embedding-ada-002 is our choice) and acknowledge the costs.

Make sure you tick the check box that says “Enable semantic ranker” — this will help search not just by keyword but also using user’s intent (semantic meaning). Provide a prefix to the index (something like “spring-docs”) when asked in the next page (or accept it as shown).

The indexer gets created and so is the index once we complete this step. It also creates the data source and the skillsets required for vectorisation. After a couple of minutes, you should see documents (chunks) appearing in the index:

Searching through the vector data

We have vectorised the data via the Azure AI search portal. The “import data and vectorise” function performed quite a few actions behinds the scene:

• Created a new index with appropriate keyword as well as vector types
• Provisioned a blog storage container to store the docs uploaded from the local file system
• Created a indexer whose job is to crawl through the data sources to chunk and vectorise the data
• As part of vectorisation, the appropriate skillsets (Text Split and Embedding) were created and associated with the indexer
• Invoked the indexer to kick start the loading of the data from the blob storage to the pipeline for chunking and vectorisation to the index

The same can be done by using SDKs too — I’ll go over this code in the upcoming articles.

Now that we have our vector store primed, the next step is to glue our application with the vector store so the relevant results retrieved from the vector store will be fed into the LLM as the contextual data.

Adding Data in Azure AI Studio

We will need to jump into Azure AI studio to associate the vector store to the application that we has just provisioned.

In the Playground, click on the Add your data button, select “Azure AI Search” from the drop down (we can upload from local folders too — AI studio is expanding to provide few additional options straight from the studio) and go to next page.

Fill in the Azure AI Search details such as the vector store name, search index and the embedding model. In the Data management form page, select the search type as “Hybrid + semantic” and the semantic config that was created by the Azure AI Search during the indexing data:

Search type is hybrid

Review the settings in the final form page and confirm. The vector index will now be associated with the application as you can see in the Playground. This helps the application to be grounded. The image below shows this:

Application is grounded with private data

Testing the App

As all the required pieces are in place, we can now test the app manually in the AI Studio. But before we do this, let’s update the system prompt:

Chat with the app asking questions around Spring Framework

Once the system message is updated, test the application by asking questions around Spring Framework. We will look at evaluation of the model in the upcoming articles.

Once we are satisfied with the testing, we can then deploy the application to Azure App Service to the world can have access to it! We will look at the deployment of the application in the upcoming articles but for now, let’s wrap up!

That’s a wrap! We learned how to develop a RAG application from ground up using Azure AI Studio, Azure OpenAI service, Azure AI service and Azure AI Search!

Stay tune for more articles in the “Azure in AI” series!

Me @ Medium || LinkedIn || Twitter || GitHub ||

Of course, if you like my work, you can always catchup for a coffee BuyMeACoffee :)