API Reference

crossentropy(ŷ::AbstractArray, y::AbstractArray, m::AbstractSeqMask; ϵ)
crossentropy(sum, ŷ::AbstractArray, y::AbstractArray, m::AbstractSeqMask; ϵ)

Flux.crossentropy with an extra sequence mask for masking out non-needed token loss. y is the labels. By default it take the mean by dividing the number of valid tokens. This can be change to simply sum the valid losses by add the first argument sum. See also safe_crossentropy

logitcrossentropy(ŷ::AbstractArray, y::AbstractArray, m::AbstractSeqMask)
logitcrossentropy(sum, ŷ::AbstractArray, y::AbstractArray, m::AbstractSeqMask)

Flux.logitcrossentropy with an extra sequence mask for masking out non-needed token loss. y is the labels. By default it take the mean by dividing the number of valid tokens. This can be change to simply sum the valid losses by add the first argument sum. See also safe_logitcrossentropy

enable_gpu(t=true)

Enable gpu for todevice, disable with enable_gpu(false). The backend is selected by Flux.gpu_backend!. Should only be used in user scripts.

firsttoken(x, m::AbstractSeqMask)

Slice the first token from the hidden states. The "first" token is defined by the sequence mask.

firsttoken(x)

Slice the first tokens from the hidden states, normally equivalent to x[:, begin, :].

See also: lengthselect, skipboundarytoken

lasttoken(x, m::AbstractSeqMask)

Slice the last token from the hidden states. The "last" token is defined by the sequence mask.

lasttoken(x)

Slice the first tokens from the hidden states, normally equivalent to x[:, end, :].

See also: lengthselect, skipboundarytoken

lengthselect(x, i)

selectdim on the "length" dimension (2 for most array and 1 for integer array).

safe_crossentropy(ŷ::AbstractArray, y::AbstractArray, m::AbstractSeqMask; ϵ)
safe_crossentropy(sum, ŷ::AbstractArray, y::AbstractArray, m::AbstractSeqMask; ϵ)

crossentropy. If the label y is an integer array, then it would also call maximum on the label to make sure no label number is large then the first dimension of ŷ. See also unsafe_crossentropy.

safe_logitcrossentropy(ŷ::AbstractArray, y::AbstractArray, m::AbstractSeqMask)
safe_logitcrossentropy(sum, ŷ::AbstractArray, y::AbstractArray, m::AbstractSeqMask)

logitcrossentropy. If the label y is an integer array, then it would also call maximum on the label to make sure no label number is large then the first dimension of ŷ. See also unsafe_logitcrossentropy.

skipboundarytoken(x; first=1, last=1)

Select (lengthselect) the non-boundary tokens from the hidden states, normally equivalent to x[:, begin+first:end-last, :].

See also: lengthselect

skipfirsttoken(x)

Slice the non-first tokens from the hidden states, normally equivalent to x[:, 2:end, :].

See also: lengthselect, skipboundarytoken, skiplasttoken

skiplasttoken(x)

Slice the non-last tokens from the hidden states, normally equivalent to x[:, 1:end-1, :].

See also: lengthselect, skipboundarytoken, skipfirsttoken

todevice(x)

Move data to device, only when gpu is enable with enable_gpu, basically equal Flux.gpu. Otherwise just Flux.cpu.

togpudevice(x)

Move data to gpu device, backend selected by Flux.gpu_backend!.

unsafe_crossentropy(ŷ::AbstractArray, y::AbstractArray{<:Integer}, m::AbstractSeqMask; ϵ)
unsafe_crossentropy(sum, ŷ::AbstractArray, y::AbstractArray{<:Integer}, m::AbstractSeqMask; ϵ)

Compute crossentropy with integer labels. The prefix "unsafe" means that if y contain any number larger than the first dimension of ŷ, the behavior is undefined. See also safe_crossentropy.

unsafe_logitcrossentropy(ŷ::AbstractArray, y::AbstractArray{<:Integer}, m::AbstractSeqMask)
unsafe_logitcrossentropy(sum, ŷ::AbstractArray, y::AbstractArray{<:Integer}, m::AbstractSeqMask)

Compute logitcrossentropy with integer labels. The prefix "unsafe" means that if y contain any number larger than the first dimension of ŷ, the behavior is undefined. See also safe_logitcrossentropy.