Transformers.jl

Julia implementation of Transformers models

This is the documentation of Transformers: The Julia solution for using Transformer models.

Installation

In the Julia REPL:

julia> ]add Transformers

Outline

• Get Started
• • Create a N-layered Transformer model
  • Create a N-layered Transformer Decoder model
  • Preprocessing Text
  • Using (HuggingFace) Pre-trained Models
  • GPU
• Tutorial
• • Transformer model
  • • Multi-Head Attention
    • Positional Embedding
  • Transformers.jl
  • • Example
    • Copy task
    • Defining the model
    • define the loss and training loop
    • Test our model
• Transformers.Layers
• • API Reference
• Transformers.TextEncoders
• • API Reference
• Transformers.HuggingFace
• • API Reference