• https://github.com/search?q=word2id+batch_size+seq_len+zip_longest&ref=opensearch&type=code

• https://github.com/hwchase17/langchain

• https://community.openai.com/t/foundational-must-read-gpt-llm-papers/197003

• https://github.com/THUDM/ChatGLM-6B

• https://github.com/karpathy/nanoGPT

• LoRA: Low-Rank Adaptation of Large Language Models

• https://platform.openai.com/docs/models/overview

• https://community.openai.com/t/foundational-must-read-gpt-llm-papers/197003

refs site

• https://github.com/mikaizhu/Notes/blob/f04d80456dde566abbd7e8cd1e3528cef1bc6f74/Deep_learning/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E5%9F%BA%E7%A1%80%E7%9F%A5%E8%AF%86/TencentAds.md
• https://www.sharetechnote.com/html/NN/Handbook_MachineLearning_Index.html
• https://pytorch.org/tutorials/
• https://pytorch.org/tutorials/beginner/basics/intro.html
• https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html#sphx-glr-beginner-nlp-word-embeddings-tutorial-py
• ai for begginer https://github.com/microsoft/AI-For-Beginners
• https://www.baeldung.com/cs/category/ai

To remember

• one-hot encoding
• torch.nn.Liner
• torch.nn.Embedding

torch.tensor([[1,2,4,5]])
weights = torch.rand(10, 3)
net = torch.nn.liner(10,2)
print(net,net.h)
torch.mm

AI > machine learning > deep learning

pytorch.nn.Embedding

• https://www.jianshu.com/p/63e7acc5e890
• https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html#sphx-glr-beginner-nlp-word-embeddings-tutorial-py
• https://discuss.pytorch.org/t/how-does-nn-embedding-work/88518/3

理解Epoch, Batch, and Mini-batch

• Gradient Descent
• Batch Gradient Descent
• Stochastic Gradient Descent
• Mini-Batch Gradient Descent
• 比如有个2000个sample
• 将2000个sample整体 一次性输入 (input net train&backward feedback) 就是Batch Gradient Descent
• 2000个数据one by one共2000次分别(input net train&backward feedback) 就是 stochastic gradient descent
• 2000个数据份 n组 每组2000/n个sample(input net train&backward feedback) 就是Mini-Batch Gradient Descent
• 上面三个方式，一次整体sample train完称为一个epoch
• 要经过多次epoch才能达到(Gradient Descent)剃度下降效果

similarity相似度 | diversity index多样性指数

• Cosine similarity #向量i和j之间的向量夹角大小 越小越相似
• Pearson Correlation Coefficient(皮尔逊相关系数) #与余弦相似度相比,加入了用户平均分对各独立评分进行修正,减小了用户评分偏置的影响
• Euclidean Distance #m维空间中两点之间的距离
• Jaccard Index(Jaccard similarity coefficient) #两个集合A和B的交集在其二者的并集中所占的比例
• Jaccard Distance #两个集合中不同元素占所有元素的比例来衡量两个集合的区分度