Enormous Matrices

Fundamentally, $m$-mode analysis involves operations on block-diagonal matrices. Although the block-diagonal structure of these matrices makes it possible to store and perform computations on these matrices, they can still be enormous. That is, while each block individually may fit in the system memory, multiple blocks may not. Here we introduce some of the abstractions developed for working with block diagonal matrices that may each be multiple terabytes in size.

API

BPJSpec.AbstractBlockMatrix — Type.

abstract type AbstractBlockMatrix{B, N}

This type represents a (potentially enormous) block-diagonal matrix. This type is designed to be general enough to handle large matrices that fit in memory as well as enormous matrices that do not fit in memory. In principle this type can also be used to store small matrices, but it would be relatively inefficient compared to the standard Array{T, N}.

Type Parameters

B specifies the type of the blocks that compose the matrix
N specifies the number of indices used to index into the matrix blocks

Required Fields

storage contains instructions on how to read matrix blocks
cache is used if we want to read the matrix from disk and then keep it in memory for faster access.

source

BPJSpec.SimpleBlockVector — Type.

struct SimpleBlockVector <: AbstractBlockMatrix{Vector{Complex128}, 1}

This type represents a (potentially enormous) complex-valued vector that has been split into blocks. Each of these blocks is indexed by a number that varies from 1 to length.

Fields:

storage contains instructions on how to read the vector from disk
cache is used if we want to keep the vector in memory
length determines the number of blocks the vector is divided into

Usage:

julia> x = create(SimpleBlockVector, 10)
SimpleBlockVector(<no file>, cached=true, length=10)

julia> x[5] = Complex128[1, 2, 3, 4, 5];

julia> x[5]
5-element Array{Complex{Float64},1}:
 1.0+0.0im
 2.0+0.0im
 3.0+0.0im
 4.0+0.0im
 5.0+0.0im

source

BPJSpec.SimpleBlockMatrix — Type.

struct SimpleBlockMatrix <: AbstractBlockMatrix{Matrix{Complex128}, 1}

This type represents a (potentially enormous) complex-valued matrix that has been split into blocks. Each of these blocks is indexed by a number that varies from 1 to length.

Fields:

storage contains instructions on how to read the matrix from disk
cache is used if we want to keep the matrix in memory
length determines the number of blocks the matrix is divided into

Usage:

julia> x = create(SimpleBlockMatrix, 10)
SimpleBlockMatrix(<no file>, cached=true, length=10)

julia> x[5] = Complex128[1 2; 3 4];

julia> x[5]
2×2 Array{Complex{Float64},2}:
 1.0+0.0im  2.0+0.0im
 3.0+0.0im  4.0+0.0im

source

BPJSpec.MBlockVector — Type.

struct MBlockVector <: AbstractBlockMatrix{Vector{Complex128}, 1}

This type represents a (potentially enormous) complex-valued vector that has been split into blocks. Each of these blocks is indexed by its value of $m$ that varies from 0 to mmax.

Fields:

storage contains instructions on how to read the matrix from disk
cache is used if we want to keep the matrix in memory
mmax determines the largest value of the $m$ quantum number used by the matrix

Usage:

julia> x = create(MBlockVector, 10)
MBlockVector(<no file>, cached=true, mmax=10)

julia> x[0] = Complex128[1, 2, 3, 4, 5];

julia> x[0]
5-element Array{Complex{Float64},1}:
 1.0+0.0im
 2.0+0.0im
 3.0+0.0im
 4.0+0.0im
 5.0+0.0im

See also: MBlockMatrix, AbstractBlockMatrix

source

BPJSpec.MBlockMatrix — Type.

struct MBlockMatrix <: AbstractBlockMatrix{Matrix{Complex128}, 1}

This type represents a (potentially enormous) complex-valued matrix that has been split into blocks. Each of these blocks is indexed by its value of $m$ that varies from 0 to mmax.

Fields:

storage contains instructions on how to read the matrix from disk
cache is used if we want to keep the matrix in memory
mmax determines the largest value of the $m$ quantum number used by the matrix

Usage:

julia> x = create(MBlockMatrix, 10)
MBlockMatrix(<no file>, cached=true, mmax=10)

julia> x[0] = Complex128[1 2; 3 4];

julia> x[0]
2×2 Array{Complex{Float64},2}:
 1.0+0.0im  2.0+0.0im
 3.0+0.0im  4.0+0.0im

See also: MBlockVector, AbstractBlockMatrix

source

BPJSpec.FBlockVector — Type.

struct FBlockVector <: AbstractBlockMatrix{Vector{Complex128}, 1}

This type represents a (potentially enormous) complex-valued vector that has been split into blocks. Each of these blocks is indexed by the index of the corresponding frequency channel, which varies from 1 to length(frequencies).

Fields:

storage contains instructions on how to read the matrix from disk
cache is used if we want to keep the matrix in memory
frequencies is a list of the frequency channels represented by this matrix
bandwidth is a list of the corresponding bandwidth of each frequency channel

Usage:

julia> x = create(FBlockVector, [74u"MHz", 100u"MHz"], [24u"kHz", 24u"kHz"])
FBlockVector(<no file>, cached=true, frequencies=74.000 MHz…100.000 MHz, bandwidth~24 kHz)

julia> x[1] = Complex128[1, 2, 3, 4, 5];

julia> x[1]
5-element Array{Complex{Float64},1}:
 1.0+0.0im
 2.0+0.0im
 3.0+0.0im
 4.0+0.0im
 5.0+0.0im

See also: FBlockMatrix, AbstractBlockMatrix

source

BPJSpec.FBlockMatrix — Type.

struct FBlockMatrix <: AbstractBlockMatrix{Matrix{Complex128}, 1}

This type represents a (potentially enormous) complex-valued matrix that has been split into blocks. Each of these blocks is indexed by the index of the corresponding frequency channel, which varies from 1 to length(frequencies).

Fields:

storage contains instructions on how to read the matrix from disk
cache is used if we want to keep the matrix in memory
frequencies is a list of the frequency channels represented by this matrix
bandwidth is a list of the corresponding bandwidth of each frequency channel

Usage:

julia> x = create(FBlockMatrix, [74u"MHz", 100u"MHz"], [24u"kHz", 24u"kHz"])
FBlockMatrix(<no file>, cached=true, frequencies=74.000 MHz…100.000 MHz, bandwidth~24 kHz)

julia> x[1] = Complex128[1 2; 3 4];

julia> x[1]
2×2 Array{Complex{Float64},2}:
 1.0+0.0im  2.0+0.0im
 3.0+0.0im  4.0+0.0im

See also: FBlockVector, AbstractBlockMatrix

source

BPJSpec.MFBlockVector — Type.

struct MFBlockVector <: AbstractBlockMatrix{Vector{Complex128}, 2}

This type represents a (potentially enormous) complex-valued vector that has been split into blocks. Each of these blocks is indexed by its value of $m$, which varies from 0 to mmax, and the index of the corresponding frequency channel, which varies from 1 to length(frequencies).

Fields:

storage contains instructions on how to read the matrix from disk
cache is used if we want to keep the matrix in memory
mmax determines the largest value of the $m$ quantum number used by the matrix
frequencies is a list of the frequency channels represented by this matrix
bandwidth is a list of the corresponding bandwidth of each frequency channel

Usage:

julia> x = create(MFBlockVector, 2, [74u"MHz", 100u"MHz"], [24u"kHz", 24u"kHz"])
MFBlockVector(<no file>, cached=true, mmax=2, frequencies=74.000 MHz…100.000 MHz, bandwidth~24 kHz)

julia> x[0, 1] = Complex128[1, 2, 3, 4, 5];

julia> x[0, 1]
5-element Array{Complex{Float64},1}:
 1.0+0.0im
 2.0+0.0im
 3.0+0.0im
 4.0+0.0im
 5.0+0.0im

See also: MFBlockMatrix, AbstractBlockMatrix

source

BPJSpec.MFBlockMatrix — Type.

struct MFBlockMatrix <: AbstractBlockMatrix{Matrix{Complex128}, 2}

This type represents a (potentially enormous) complex-valued matrix that has been split into blocks. Each of these blocks is indexed by its value of $m$, which varies from 0 to mmax, and the index of the corresponding frequency channel, which varies from 1 to length(frequencies).

Fields:

storage contains instructions on how to read the matrix from disk
cache is used if we want to keep the matrix in memory
mmax determines the largest value of the $m$ quantum number used by the matrix
frequencies is a list of the frequency channels represented by this matrix
bandwidth is a list of the corresponding bandwidth of each frequency channel

Usage:

julia> x = create(MFBlockMatrix, 2, [74u"MHz", 100u"MHz"], [24u"kHz", 24u"kHz"])
MFBlockMatrix(<no file>, cached=true, mmax=2, frequencies=74.000 MHz…100.000 MHz, bandwidth~24 kHz)

julia> x[0, 1] = Complex128[1 2; 3 4];

julia> x[0, 1]
2×2 Array{Complex{Float64},2}:
 1.0+0.0im  2.0+0.0im
 3.0+0.0im  4.0+0.0im

See also: MFBlockVector, AbstractBlockMatrix

source

BPJSpec.LBlockMatrix — Type.

struct LBlockMatrix <: AbstractBlockMatrix{Matrix{Float64}, 1}

This type represents a (potentially enormous) complex-valued matrix that has been split into blocks. Each of these blocks is indexed by its value of $l$, which varies from 0 to lmax.

Fields:

storage contains instructions on how to read the matrix from disk
cache is used if we want to keep the matrix in memory
lmax determines the largest value of the $l$ quantum number used by the matrix
frequencies is a list of the frequency channels represented by this matrix
bandwidth is a list of the corresponding bandwidth of each frequency channel

Usage:

julia> x = create(LBlockMatrix, 2, [74u"MHz", 100u"MHz"], [24u"kHz", 24u"kHz"])
LBlockMatrix(<no file>, cached=true, lmax=2, frequencies=74.000 MHz…100.000 MHz, bandwidth~24 kHz)

julia> l = BPJSpec.L(0);

julia> x[l] = Float64[1 2; 3 4];

julia> x[l]
2×2 Array{Float64,2}:
 1.0  2.0
 3.0  4.0

See also: LMBlockVector, AbstractBlockMatrix

source

BPJSpec.LMBlockVector — Type.

struct LMBlockVector <: AbstractBlockMatrix{Vector{Complex128}, 2}

This type represents a (potentially enormous) complex-valued vector that has been split into blocks. Each of these blocks is indexed by its value of $l$, which varies from 0 to lmax, and $m$, which varies from 0 to mmax with the restriction that $m ≤ l$.

Fields:

storage contains instructions on how to read the matrix from disk
cache is used if we want to keep the matrix in memory
lmax determines the largest value of the $l$ quantum number used by the matrix
mmax determines the largest value of the $m$ quantum number used by the matrix
frequencies is a list of the frequency channels represented by this matrix
bandwidth is a list of the corresponding bandwidth of each frequency channel

Usage:

julia> x = create(LMBlockVector, 2, 2, [74u"MHz", 100u"MHz"], [24u"kHz", 24u"kHz"])
LMBlockVector(<no file>, cached=true, lmax=2, mmax=2, frequencies=74.000 MHz…100.000 MHz, bandwidth~24 kHz)

julia> x[0, 0] = Complex128[1, 2, 3, 4, 5];

julia> x[0, 0]
5-element Array{Complex{Float64},1}:
 1.0+0.0im
 2.0+0.0im
 3.0+0.0im
 4.0+0.0im
 5.0+0.0im

See also: LBlockMatrix, AbstractBlockMatrix

source