Components

Prompting

from langchain.prompts import PromptTemplate

prompt_template = PromptTemplate.from_template("Tell me about a {genre} movie which {actor} is one of the actors.")
prompt_template.format(genre="romance", actor="Brad Pitt")

LLMs

Core component when building compound AI systems. They are the brain of the system for reasoning and generating the response.

from langchain_community.chat_models import ChatDatabricks

llm_dbrx = ChatDatabricks(endpoint="databricks-dbrx-instruct", max_tokens = 500)

for chunk in llm_dbrx.stream("Who is Brad Pitt"):
	print(chunk.content, end="\n", flush=True)

Chaining

from langchain_core.output_parsers import StrOutputParser

chain = prompt_template | llmdbrx | StrOutputParser()
print(chain.invoke({"genre": "romance", "actor": "Brad Pitt"))

Building the Multi-stage Chain

Create Vector Store

vs_endpoint_prefix = "vs_endpoint_"
vs_endpoint_name = vs_endpoint_prefix + str(get_fixed_integer(DA.unique_name("_")))
print(f"Assigned Vector Search endpoint name: {vs_endpoint_name}.")

source_table_fullname = f"{DA.catalog_name}.{DA.schema_name}.dais_text"
vs_index_table_fullname = f"{DA.catalog_name}.{DA.schema_name}.dais_embeddings"

create_vs_index(vs_endpoint_name, vs_index_table_fullname, source_table_fullname, "Title")

Load Data

image.png

Build the First Chain

from langchain_community.chat_models import ChatDatabricks
from langchain.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_community.tools import YouTubeSearchTool
from databricks.vector_search.client import VectorSearchClient
from langchain.schema.runnable import RunnablePassthrough

llm_dbrx = ChatDatabricks(endpoint="databricks-dbrx-instruct", max_tokens = 1000)
tool_yt = YouTubeSearchTool()

prompt_template_1 = PromptTemplate.from_template(
    """You are a Databricks expert. You will get questions about Databricks. Try to give simple answers and be professional.

Question: {question}

Answer:
"""
)

chain1 = ({"question": RunnablePassthrough()} | prompt_template_1 | llm_dbrx | StrOutputParser())
print(chain1.invoke({"question": "How machine learning models are stored in Unity Catalog?"}))

Build the Second Chain