sc-openai-c2-L5-vid6_1.srt

1
00:00:05,166 --> 00:00:07,666
In this video, we'll learn how to split
complex

2
00:00:07,666 --> 00:00:11,966
tasks into a series of simpler subtasks
by chaining multiple prompts together.

3
00:00:12,733 --> 00:00:16,033
You might be wondering why would you split
up a task into multiple prompts

4
00:00:16,200 --> 00:00:18,000
when you could achieve it with one prompt

5
00:00:18,000 --> 00:00:20,700
and chain of thought reasoning
like we learned in the previous video?

6
00:00:21,300 --> 00:00:24,300
We've already shown that language
models are very good at following complex

7
00:00:24,300 --> 00:00:27,600
instructions, especially the more advanced
models like GPT four.

8
00:00:28,466 --> 00:00:31,400
So let me explain why we would do this
with two analogies

9
00:00:31,400 --> 00:00:34,366
comparing chain of thought,
reasoning and chaining multiple prompts.

10
00:00:34,866 --> 00:00:37,433
The first analogy to compare
the two is the difference between

11
00:00:37,433 --> 00:00:41,066
cooking a complex meal in one go versus
cooking it in stages.

12
00:00:41,866 --> 00:00:45,133
Using one long, complicated instruction
can be like trying

13
00:00:45,133 --> 00:00:48,200
to cook a complex meal all at once
while you have to manage

14
00:00:48,200 --> 00:00:52,100
multiple ingredients, cooking techniques
and timings simultaneously.

15
00:00:52,533 --> 00:00:53,366
It can be challenging

16
00:00:53,366 --> 00:00:56,700
to keep track of everything and ensure
that each component is cooked perfectly.

17
00:00:57,400 --> 00:01:00,433
Training prompts, on the other hand,
is like cooking the meal in stages

18
00:01:00,733 --> 00:01:03,433
where you focus on one component
at a time, ensuring

19
00:01:03,433 --> 00:01:06,000
that each part is cooked
correctly before moving on to the next.

20
00:01:06,600 --> 00:01:09,133
This approach breaks down
the complexity of the task,

21
00:01:09,133 --> 00:01:12,033
making it easier to manage
and reducing the likelihood of errors.

22
00:01:12,466 --> 00:01:14,666
However,
this approach might be unnecessary

23
00:01:14,666 --> 00:01:16,800
and overcomplicated
for a very simple recipe.

24
00:01:17,700 --> 00:01:20,966
A slightly better analogy for the same
thing is the difference between reading

25
00:01:20,966 --> 00:01:25,133
spaghetti code with everything in one long
file and a simple modular program.

26
00:01:25,633 --> 00:01:28,433
The thing that can make spaghetti code bad
and difficult to debug

27
00:01:28,566 --> 00:01:32,200
is ambiguity and complex dependencies
between different parts of the logic.

28
00:01:32,600 --> 00:01:36,600
The same can be true of a complex single
step task submitted to a language model.

29
00:01:37,233 --> 00:01:39,466
Chaining problems is a powerful strategy.

30
00:01:39,466 --> 00:01:40,666
When you have a workflow

31
00:01:40,666 --> 00:01:43,800
where you can maintain
the state of the system at any given point

32
00:01:44,000 --> 00:01:46,366
and take different actions
depending on the current state.

33
00:01:46,800 --> 00:01:50,100
And so an example of the current state
would be after you've classified

34
00:01:50,100 --> 00:01:54,366
an incoming customer query,
the state would be the classification.

35
00:01:54,366 --> 00:01:58,200
So the it's an account question
or it's a product question.

36
00:01:58,200 --> 00:02:00,400
And then based on the state,
you might do something different.

37
00:02:01,733 --> 00:02:04,466
Each subtasks
contains only the instructions required

38
00:02:04,466 --> 00:02:08,000
for a single state of the task,
which makes the system easier to manage.

39
00:02:08,366 --> 00:02:09,333
Make sure the model has

40
00:02:09,333 --> 00:02:12,333
all the information
it needs to carry out a task and reduces

41
00:02:12,333 --> 00:02:13,800
the likelihood of errors.

42
00:02:13,800 --> 00:02:17,666
As I mentioned, this approach
can also reduce and lower costs

43
00:02:17,866 --> 00:02:20,633
since longer problems
with more tokens cost more to run.

44
00:02:20,900 --> 00:02:24,000
And outlining all steps
might be unnecessary in some cases.

45
00:02:24,600 --> 00:02:27,966
Another benefit of this approach
is that it is also easier to test

46
00:02:27,966 --> 00:02:31,533
which steps might be failing more often,
or to have a human in the loop

47
00:02:31,766 --> 00:02:33,166
at a specific step.

48
00:02:33,166 --> 00:02:35,633
So to summarize,
because this is a long explanation.

49
00:02:35,900 --> 00:02:39,000
Instead of describing a whole complex
workflow and dozens of bullet points

50
00:02:39,000 --> 00:02:42,000
or several paragraphs in one prompt,
like in the previous video,

51
00:02:42,233 --> 00:02:44,833
it might be better
to keep track of the state externally

52
00:02:45,000 --> 00:02:47,400
and then inject relevant
instructions as needed.

53
00:02:48,066 --> 00:02:50,133
And what makes a problem complex?

54
00:02:50,566 --> 00:02:52,400
I think in general a problem is complex.

55
00:02:52,400 --> 00:02:54,166
If there are many different instructions

56
00:02:54,166 --> 00:02:57,800
and potentially all of them could apply
to any given situation, as these

57
00:02:57,800 --> 00:03:01,200
are the cases where it could become hard
for the model to reason about what to do.

58
00:03:01,233 --> 00:03:03,500
And as you build with and interact
with these models more,

59
00:03:03,700 --> 00:03:06,800
you'll gain an intuition for when
to use this strategy versus the previous.

60
00:03:07,300 --> 00:03:09,500
And one additional benefit
that I didn't mention yet

61
00:03:09,833 --> 00:03:13,200
is that it also allows the model
to use external tools

62
00:03:13,400 --> 00:03:15,833
at certain points of the workflow
if necessary.

63
00:03:16,200 --> 00:03:18,900
For example,
it might decide to look something up

64
00:03:18,900 --> 00:03:22,566
in a product catalog
or call an API or search a knowledge base,

65
00:03:22,933 --> 00:03:25,033
something that could not be achieved
with a single prompt.

66
00:03:25,933 --> 00:03:27,900
So with that, let's dive into an example.

67
00:03:29,000 --> 00:03:31,866
So we're going to use the same example
as in the previous video

68
00:03:31,866 --> 00:03:34,933
when we want to answer a customer's
question about a specific product.

69
00:03:35,300 --> 00:03:38,100
But this time we'll use more products
and also break the steps

70
00:03:38,100 --> 00:03:40,566
down into a number of different prompts.

71
00:03:42,366 --> 00:03:43,966
So we'll use the same delimiter

72
00:03:43,966 --> 00:03:46,200
that we've been using
in the previous videos,

73
00:03:47,700 --> 00:03:51,433
and that's read through our system
message.

74
00:03:53,033 --> 00:03:55,200
You will be provided with customer service
queries.

75
00:03:55,466 --> 00:03:59,700
The customer service query will be
delimited with four hashtag characters.

76
00:04:00,366 --> 00:04:03,700
I'll put a python list of objects
where each object has the following format

77
00:04:05,100 --> 00:04:08,466
category,
which is one of these predefined fields

78
00:04:09,333 --> 00:04:12,400
or products and this is a list of products

79
00:04:12,400 --> 00:04:14,800
that must be found in the allowed products
below.

80
00:04:14,800 --> 00:04:18,000
While the categories and products must be
found in the customer service query.

81
00:04:19,600 --> 00:04:21,633
If a product
does mentioned, it must be associated

82
00:04:21,633 --> 00:04:24,100
with the correct category
in the allowed products list below.

83
00:04:24,566 --> 00:04:27,400
If no products or categories
are found, output an empty list.

84
00:04:27,866 --> 00:04:30,533
So now we have allowed list of products.

85
00:04:30,566 --> 00:04:35,833
So we have the categories and
then the products within those categories.

86
00:04:35,900 --> 00:04:38,066
Now, final instruction is only output.

87
00:04:38,066 --> 00:04:41,533
The list of objects with nothing else.

88
00:04:43,566 --> 00:04:45,966
So next we have our user message.

89
00:04:45,966 --> 00:04:47,766
And so this messages.

90
00:04:47,766 --> 00:04:52,800
Tell me about the small x profile on
the photo is not camera the DSL one.

91
00:04:53,166 --> 00:04:55,200
Also tell me about your TV's.

92
00:04:55,200 --> 00:04:57,966
So we're asking
about two specific products

93
00:04:58,733 --> 00:05:02,166
and also the general category
of televisions

94
00:05:03,500 --> 00:05:07,133
on both of these products are mentioned
in the allowed products list.

95
00:05:07,600 --> 00:05:13,033
Then we have a television section as well,

96
00:05:13,033 --> 00:05:17,333
and we format the system message
and user message into the messages array,

97
00:05:18,066 --> 00:05:24,900
and then we get the completion
from the model.

98
00:05:24,900 --> 00:05:30,366
So as you can see for our output,
we have a list of objects

99
00:05:30,833 --> 00:05:34,633
and each object has category and products.

100
00:05:35,033 --> 00:05:37,600
So we have the small X pro phone

101
00:05:38,000 --> 00:05:41,633
and the photo snapped DSLR camera.

102
00:05:41,966 --> 00:05:44,700
And then in the final object
we actually only have a category

103
00:05:44,700 --> 00:05:46,766
because we didn't
mention any specific TVs.

104
00:05:47,833 --> 00:05:48,366
And so

105
00:05:48,366 --> 00:05:52,433
the benefit of outputting
this structured response is that we can

106
00:05:52,433 --> 00:05:57,866
then read it into a list in Python,
which is very nice.

107
00:05:58,433 --> 00:06:01,200
And so let's try another example.

108
00:06:01,500 --> 00:06:04,266
So our second user message is

109
00:06:05,900 --> 00:06:07,800
my router isn't working.

110
00:06:07,800 --> 00:06:10,833
And if you notice in the list,
we don't actually have any routers

111
00:06:11,766 --> 00:06:13,033
and then

112
00:06:14,000 --> 00:06:20,400
format this correctly
and get the completion.

113
00:06:20,400 --> 00:06:23,866
And so as you can see in this case,
the output is an empty list.

114
00:06:25,300 --> 00:06:28,700
And so now that we have this step
to identify the category and products,

115
00:06:29,533 --> 00:06:32,700
if we find any products and categories,
we want to note some information

116
00:06:32,700 --> 00:06:35,666
about those requested products
and categories into the prompt

117
00:06:35,700 --> 00:06:38,233
so that we can better
answer the customer question.

118
00:06:38,233 --> 00:06:42,033
And so in our workflow,
the state now after this prompt is run

119
00:06:42,300 --> 00:06:45,533
as either products have been listed
or they haven't been listed.

120
00:06:45,800 --> 00:06:46,800
And in that case

121
00:06:46,800 --> 00:06:50,400
we wouldn't try to look anything up
because there's nothing to look up.

122
00:06:50,566 --> 00:06:52,900
Well, so if I were to actually build this
into a system,

123
00:06:52,900 --> 00:06:56,700
I might use category
names, maybe something like

124
00:06:57,700 --> 00:07:00,666
computers and laptops or something

125
00:07:01,633 --> 00:07:04,933
to avoid any weirdness
with spaces and special characters.

126
00:07:05,400 --> 00:07:07,333
But this should work for now.

127
00:07:07,333 --> 00:07:09,200
So now we want to look up some information

128
00:07:09,200 --> 00:07:10,966
about the products
that the user mentioned.

129
00:07:10,966 --> 00:07:14,233
So about this phone, this camera,
and about the TV's in general.

130
00:07:14,900 --> 00:07:15,833
And so we need to have

131
00:07:15,833 --> 00:07:18,300
some kind of product catalog
to look up this information from.

132
00:07:18,833 --> 00:07:20,833
So here we have our product information.

133
00:07:21,266 --> 00:07:24,466
I just paste it in.

134
00:07:24,466 --> 00:07:26,866
So as you can see,
we have a large number of products

135
00:07:28,733 --> 00:07:32,200
available at our store

136
00:07:32,500 --> 00:07:34,600
and all of these products are fake

137
00:07:34,600 --> 00:07:36,900
and were actually generated by GBG for.

138
00:07:37,933 --> 00:07:40,933
And so for each product
we have a couple of different fields.

139
00:07:40,933 --> 00:07:43,566
We have name category, brand

140
00:07:44,166 --> 00:07:46,500
warranty and so on.

141
00:07:46,800 --> 00:07:50,133
And so the products is just a dictionary
from product name

142
00:07:50,433 --> 00:07:53,700
to this object that contains
the information about the product.

143
00:07:54,933 --> 00:07:57,133
And notice
that each product has a category.

144
00:07:57,633 --> 00:07:59,533
So remember,
we want to look up information

145
00:07:59,533 --> 00:08:01,433
about the products
that the user asks about.

146
00:08:01,433 --> 00:08:05,300
So we need to define some helper functions
to allow us to look up

147
00:08:05,700 --> 00:08:08,666
product information by product name.

148
00:08:08,666 --> 00:08:10,866
So let's create a function

149
00:08:10,866 --> 00:08:13,366
gap, product by name,

150
00:08:14,533 --> 00:08:16,766
the input the name,

151
00:08:16,866 --> 00:08:19,366
and then we're going to return

152
00:08:19,366 --> 00:08:22,733
product dictionary
and we're going to get the value

153
00:08:22,966 --> 00:08:26,400
for the item with the name as the key.

154
00:08:26,400 --> 00:08:28,933
And then I'll pull
back is just going to be none.

155
00:08:29,266 --> 00:08:32,000
And so we also want to define
another helper function

156
00:08:32,000 --> 00:08:34,366
to get all of the products
for a certain category.

157
00:08:34,366 --> 00:08:35,333
For example,

158
00:08:35,333 --> 00:08:37,500
when the user is asking about the TVs
we have,

159
00:08:37,500 --> 00:08:40,966
we want to load all of the information
about all of the different TVs.

160
00:08:42,400 --> 00:08:45,266
So get products

161
00:08:45,266 --> 00:08:47,100
by category,

162
00:08:47,133 --> 00:08:51,000
the input, the category name string.

163
00:08:51,000 --> 00:08:55,300
And to do this, we want to look through
all of the products

164
00:08:55,300 --> 00:08:58,300
in the product's dictionary and check

165
00:08:59,100 --> 00:09:02,366
each one to see if the category
is equal to the input category.

166
00:09:02,366 --> 00:09:04,666
And if so, we want to return that.

167
00:09:04,666 --> 00:09:07,233
So we'll do this as follows.

168
00:09:07,766 --> 00:09:12,000
So first we want to loop through
each product and we have to get the values

169
00:09:12,000 --> 00:09:16,833
because we need to actually
access the category, which is the value.

170
00:09:18,033 --> 00:09:21,300
And then we'll return this product.

171
00:09:21,300 --> 00:09:23,633
If the product

172
00:09:25,166 --> 00:09:27,200
category

173
00:09:28,500 --> 00:09:30,800
is equal to

174
00:09:30,966 --> 00:09:36,900
our input category.

175
00:09:36,900 --> 00:09:38,533
So let's

176
00:09:39,066 --> 00:09:41,400
do an example
for each of these helper functions.

177
00:09:42,033 --> 00:09:43,800
So first

178
00:09:44,433 --> 00:09:46,866
we have a product called the Tech pro
Ultrabook,

179
00:09:47,166 --> 00:09:51,766
so let's get the product information
by name.

180
00:09:51,766 --> 00:09:54,600
So here you can see we've just fetched all
of the product information

181
00:09:57,033 --> 00:10:00,533
and let's do an example to
get all of the products for our category.

182
00:10:00,533 --> 00:10:07,000
So let's get all of the products
in the computers and laptops category.

183
00:10:07,000 --> 00:10:09,700
Hey, you see,
we fetched one of the products

184
00:10:10,233 --> 00:10:16,800
with this category.

185
00:10:16,800 --> 00:10:20,833
So let's continue our example.

186
00:10:20,833 --> 00:10:24,000
And just to remember where we are,
that's the user message.

187
00:10:24,300 --> 00:10:27,166
So the user message was
tell me about the smart X pro

188
00:10:27,200 --> 00:10:29,733
phone on the camera, on the TVs.

189
00:10:30,300 --> 00:10:32,900
And then the initial output from the model

190
00:10:32,900 --> 00:10:34,900
from the first step was this.

191
00:10:36,033 --> 00:10:37,966
And so what we also need to do

192
00:10:37,966 --> 00:10:40,800
is read this output from the model,
which is a string.

193
00:10:41,066 --> 00:10:45,100
We need to pass that into a list
so that we can use it as input

194
00:10:45,500 --> 00:10:49,066
to the help of functions
that we just wrote.

195
00:10:49,066 --> 00:10:54,733
So that's why
a helper function to do this.

196
00:10:54,733 --> 00:10:56,300
So we're going to use the Python

197
00:10:57,366 --> 00:11:00,366
JSON module and we're going to write
a function

198
00:11:00,366 --> 00:11:03,166
called read string to list and very

199
00:11:04,200 --> 00:11:05,500
descriptive title.

200
00:11:05,500 --> 00:11:09,700
The input string.

201
00:11:09,700 --> 00:11:12,900
And so first we'll just check
if the input string

202
00:11:12,900 --> 00:11:13,966
is non

203
00:11:15,900 --> 00:11:17,966
in case something in the previous step
failed.

204
00:11:17,966 --> 00:11:20,900
We're just going to return nothing

205
00:11:22,133 --> 00:11:22,466
and now

206
00:11:22,466 --> 00:11:24,766
we're going to have a try except block

207
00:11:25,833 --> 00:11:29,366
to make sure that we catch any errors.

208
00:11:29,366 --> 00:11:31,500
And so first

209
00:11:31,500 --> 00:11:36,400
we replace any single quotes
with double quotes in the input string

210
00:11:36,900 --> 00:11:42,033
to make sure we can pass the Jason
and then we're going to use the JSON nodes

211
00:11:42,033 --> 00:11:46,766
function to read the input string
into the array or the left.

212
00:11:47,266 --> 00:11:49,233
And then we're going to return this.

213
00:11:49,233 --> 00:11:54,100
And if there's a decode error, we're going
to print the error and then return non.

214
00:11:55,533 --> 00:11:57,700
So let's try this with our example.

215
00:11:58,366 --> 00:12:05,700
So we're going to get our category
and product list using the read string

216
00:12:05,700 --> 00:12:10,300
to list helper function and apply it
to this response from the model.

217
00:12:10,300 --> 00:12:14,400
And then we're going to print this list
so it should look the same.

218
00:12:15,200 --> 00:12:17,100
Let me run this first.

219
00:12:17,200 --> 00:12:21,300
And so as you can see, it's
just the same thing, except now

220
00:12:21,300 --> 00:12:25,233
the type of this variable
is actually a list instead of a string.

221
00:12:25,766 --> 00:12:29,166
So the whole point of what we're doing
is to get the product information into

222
00:12:29,366 --> 00:12:32,733
a less that we can add
to the next instruction to the model,

223
00:12:32,733 --> 00:12:36,333
which is going to be the instruction where
we ask it to answer the user question.

224
00:12:36,600 --> 00:12:39,166
And so to do this,
we need to put the product information

225
00:12:39,166 --> 00:12:42,766
into a nice string format
that we can add to the prompt.

226
00:12:42,766 --> 00:12:47,300
And so let's also create a helper function
to do this.

227
00:12:47,300 --> 00:12:49,366
So we're going to call it
generate output string,

228
00:12:50,300 --> 00:12:54,033
and it's going to take in the list of data
that we just created.

229
00:12:54,033 --> 00:12:54,600
So this

230
00:12:56,433 --> 00:12:57,566
and then I'm going to copy

231
00:12:57,566 --> 00:12:59,700
some code and then we'll walk through
what it's doing.

232
00:13:00,600 --> 00:13:03,366
So now I'm going to patient some code
and show you an example

233
00:13:03,366 --> 00:13:06,033
and then we'll talk about
what this function is doing.

234
00:13:06,033 --> 00:13:11,700
So we're going to get the product
information from our first user message.

235
00:13:12,000 --> 00:13:15,066
And so we're going to use this helper
function, generate output string

236
00:13:15,333 --> 00:13:23,400
on our category and product list,
which if we remember was this.

237
00:13:23,400 --> 00:13:27,766
And so here we have one of the product
information for the products

238
00:13:27,766 --> 00:13:31,233
that were in the mentioned
in the user message.

239
00:13:31,700 --> 00:13:35,866
So we have the phone that they mentioned,
we have the camera that they mentioned,

240
00:13:35,866 --> 00:13:42,000
and then we have all of the product
information for all of our TVs.

241
00:13:42,000 --> 00:13:45,733
And this is information
that will be helpful for the model

242
00:13:45,733 --> 00:13:48,000
to be able
to answer the user's initial question.

243
00:13:48,733 --> 00:13:51,066
And if you're interested in
how this function works,

244
00:13:51,900 --> 00:13:55,366
I'll give a brief overview,
but you can feel free to pause the video

245
00:13:55,366 --> 00:13:57,100
and read it more thoroughly.

246
00:13:57,100 --> 00:14:00,266
So it basically just loops through
all of the objects

247
00:14:00,266 --> 00:14:03,166
in this list
and first checks with that products.

248
00:14:03,500 --> 00:14:05,233
If so, it gets the

249
00:14:06,200 --> 00:14:09,200
information for each product
and then it checks off as a category.

250
00:14:09,200 --> 00:14:10,333
If they want any products.

251
00:14:10,333 --> 00:14:14,200
So that would be for this object,
for example.

252
00:14:14,400 --> 00:14:18,566
And then it gets all of the product
information for the products

253
00:14:18,566 --> 00:14:21,600
in that category,
and it just adds them to this string

254
00:14:22,300 --> 00:14:24,600
and then that's what it returns.

255
00:14:24,600 --> 00:14:27,733
So at this point,
we've found the relevant product

256
00:14:27,733 --> 00:14:29,833
information to answer the user question.

257
00:14:29,833 --> 00:14:32,366
Now it's time for the model
to actually answer the question.

258
00:14:32,766 --> 00:14:36,300
So let's have our system message.

259
00:14:36,300 --> 00:14:38,533
So this is the instruction here.

260
00:14:38,633 --> 00:14:39,433
Our customers service

261
00:14:39,433 --> 00:14:43,200
assistant for a large electronics store
respond in a friendly and helpful tone

262
00:14:43,500 --> 00:14:46,500
with, let's say,
with very concise answers.

263
00:14:46,800 --> 00:14:49,800
Make sure to ask the user relevant
follow up questions.

264
00:14:49,800 --> 00:14:51,966
So I want this to be
an interactive experience for the user.

265
00:14:52,600 --> 00:14:56,166
And so just as a reminder,
this was our initial user message.

266
00:14:56,600 --> 00:15:00,000
I'll just add it again.

267
00:15:00,000 --> 00:15:04,400
And so now we're going
to have our messages array,

268
00:15:04,400 --> 00:15:07,133
and this is the input to the model.
So let's go through this.

269
00:15:07,133 --> 00:15:09,600
We have our first message,
which is the system message.

270
00:15:09,600 --> 00:15:10,200
As usual,

271
00:15:11,766 --> 00:15:14,566
we have the user message

272
00:15:14,566 --> 00:15:17,866
and then we have this

273
00:15:18,166 --> 00:15:21,033
additional assistant message,
and this is the message

274
00:15:21,033 --> 00:15:24,000
that contains all of the product
information that we just looked up.

275
00:15:24,300 --> 00:15:27,900
And so we're saying relevant product
information, new line,

276
00:15:27,900 --> 00:15:31,366
and then this product information
that we just found.

277
00:15:32,700 --> 00:15:34,700
And so now the model has the relevant
context.

278
00:15:34,700 --> 00:15:36,866
It needs to be able
to answer this user's question.

279
00:15:38,166 --> 00:15:40,366
So let's get the final response

280
00:15:40,366 --> 00:15:44,866
and print it.

281
00:15:44,866 --> 00:15:48,700
And we're hoping that in the models
going to use relevant information from

282
00:15:49,866 --> 00:15:53,533
the product information
in order to answer the user

283
00:15:53,866 --> 00:15:58,000
in a helpful way.

284
00:15:58,000 --> 00:16:01,266
So first it tells the user about the smart
phone,

285
00:16:03,066 --> 00:16:04,733
tells the user about the photo

286
00:16:04,733 --> 00:16:08,766
stop camera and then talks
about the different televisions

287
00:16:09,566 --> 00:16:12,500
that we have in stock
and then ask the follow up question.

288
00:16:13,800 --> 00:16:14,433
So as you can

289
00:16:14,433 --> 00:16:17,233
see by breaking this up
into a series of steps,

290
00:16:17,500 --> 00:16:20,600
we were able to load information
relevant to the user query

291
00:16:20,966 --> 00:16:24,533
to give the model the relevant context
it needed to answer the question

292
00:16:24,533 --> 00:16:25,466
effectively.

293
00:16:25,466 --> 00:16:26,300
So you might be wondering

294
00:16:26,300 --> 00:16:29,100
why are we selectively loading product
descriptions into the prompt

295
00:16:29,300 --> 00:16:32,400
instead of including all of them
and letting the model use the information

296
00:16:32,400 --> 00:16:33,466
it needs?

297
00:16:33,466 --> 00:16:37,533
And so what I mean by this is why didn't
we just include all of this product

298
00:16:37,533 --> 00:16:41,000
information in the prompt and we wouldn't
have to bother with all of this

299
00:16:41,000 --> 00:16:45,066
intermediate steps to actually look up
the product information.

300
00:16:45,066 --> 00:16:47,633
There's a couple of reasons for this.

301
00:16:47,633 --> 00:16:50,000
Firstly,
including all of the product descriptions,

302
00:16:50,233 --> 00:16:52,300
might make the context
more confusing for the model,

303
00:16:52,300 --> 00:16:52,966
just as it would

304
00:16:52,966 --> 00:16:56,100
for a person trying to process
a large amount of information at once.

305
00:16:56,600 --> 00:17:01,233
I will say this is a lot less relevant
for more advanced models like JPG for

306
00:17:01,466 --> 00:17:03,333
especially when the context was
well structured

307
00:17:03,333 --> 00:17:06,133
like it is in this example on
the model is small enough

308
00:17:06,133 --> 00:17:08,500
just to ignore the information
that clearly isn't relevant.

309
00:17:10,133 --> 00:17:11,933
The next reasons are more compelling.

310
00:17:11,933 --> 00:17:15,466
So the second reason is that language
models have context limitations.

311
00:17:15,733 --> 00:17:19,533
A fixed number of tokens
allowed as input and output.

312
00:17:20,000 --> 00:17:23,500
So if you have a large number of products,
imagine you had a huge product catalog.

313
00:17:23,866 --> 00:17:27,166
You wouldn't even be able to fit all of
the descriptions into the contacts window.

314
00:17:27,833 --> 00:17:30,933
And the final reason is that
including one of the product descriptions

315
00:17:30,933 --> 00:17:34,700
could be expensive as you pay by token
when using language models.

316
00:17:34,900 --> 00:17:36,866
So by selectively loading information,

317
00:17:36,866 --> 00:17:39,233
you can reduce the cost
of generating responses.

318
00:17:40,733 --> 00:17:43,666
In general, the time ending
when dynamically load information

319
00:17:43,666 --> 00:17:46,000
into the models context
and allowing the model to decide

320
00:17:46,000 --> 00:17:48,333
when it needs more information
is one of the best ways

321
00:17:48,333 --> 00:17:50,700
to augment the capabilities
of these models.

322
00:17:50,700 --> 00:17:51,766
And to reiterate,

323
00:17:51,766 --> 00:17:54,566
you should think of a language model
as a reasoning agent that requires

324
00:17:54,566 --> 00:17:58,200
the necessary context to draw useful
conclusions and perform useful tasks.

325
00:17:59,100 --> 00:18:02,366
And so in this case,
we had to give the model

326
00:18:02,366 --> 00:18:05,500
the product information
and then it was able to reason about that

327
00:18:05,500 --> 00:18:08,633
product information
to create a useful answer for the user.

328
00:18:09,333 --> 00:18:12,600
And in this example,
we only of a call to a specific function

329
00:18:12,600 --> 00:18:14,833
or functions
to get the product description by product

330
00:18:14,833 --> 00:18:17,833
name, want to get the category products
by category name,

331
00:18:18,266 --> 00:18:21,700
but the models are actually good
at deciding when to use a variety

332
00:18:21,700 --> 00:18:24,866
of different tools and can use them
properly with instructions.

333
00:18:25,200 --> 00:18:27,733
And this is the idea behind
chatting about plug ins.

334
00:18:28,033 --> 00:18:31,100
We tell the model what tools
it has access to and what they do,

335
00:18:31,266 --> 00:18:33,600
and it chooses to use them
when it needs information

336
00:18:33,600 --> 00:18:36,900
from a specific source or wants to take
some other appropriate action.

337
00:18:37,533 --> 00:18:38,533
In our example,

338
00:18:38,533 --> 00:18:42,500
we can only look up information
by exact product and category name match,

339
00:18:42,500 --> 00:18:45,300
but there are also more advanced
techniques for information retrieval.

340
00:18:45,833 --> 00:18:49,233
One of the most effective ways to retrieve
information is using text embeddings

341
00:18:49,666 --> 00:18:52,466
and embeddings can be used to implement
efficient knowledge retrieval

342
00:18:52,466 --> 00:18:56,100
over a large corpus to find information
related to a given query.

343
00:18:56,700 --> 00:18:59,400
One of the key advantages of using text
embeddings is that they

344
00:18:59,400 --> 00:19:02,700
enable fuzzy or semantic search,
which allows you

345
00:19:02,700 --> 00:19:05,700
to find relevant information
without using the exact keywords.

346
00:19:06,100 --> 00:19:09,500
So in our example, we wouldn't necessarily
need the exact name of the product,

347
00:19:09,900 --> 00:19:14,700
but we could do a more a search with a
more general query, like a mobile phone

348
00:19:16,133 --> 00:19:17,000
for planning to create a

349
00:19:17,000 --> 00:19:20,833
comprehensive course on how to use
embeddings for various applications soon.

350
00:19:20,933 --> 00:19:22,466
So stay tuned.

351
00:19:22,466 --> 00:19:24,066
And with that,
let's move on to the next video

352
00:19:24,066 --> 00:19:27,400
where we're going to talk about
how to evaluate the outputs from

353
00:19:27,966 --> 00:19:28,766
the language model.