Part2: #dailyreport #negativesampleing #sampling #llm #recsys
target word w and the negative samples
For binary Classification: Negative sampling transforms
the problem into a series of binary classification tasks,
where the model learns to distinguish between positive
and negative samples.
Example "The dog is playing with a bone," and assume a
window size of 2 positive samples for the target word
"dog" would include:
- ("dog", "The")
- ("dog", "is")
- ("dog", "playing")
- ("dog", "with")
- ("dog", "a")
- ("dog", "bone")
Negative Samples: ("dog", "car"), ("dog", "apple"),
("dog", "house"), ("dog", "tree")
calc: logσ(vdog⋅vbone) +
logσ(−vdog⋅vcar)+logσ(−vdog⋅vapple)+logσ(−vdog⋅vhouse)
蠡