MPS ITensorQTT construction error (exponential scaling)

Hello,

I have made some numerical experiments (“Code1” and “Code2” below) using the ITensorQTT.jl package and I am getting some unexpected error scaling with respect to the system size n. I test the error by transforming a discrete function \vec f, i.e., a vector of size N=2^n, into an MPS representation \psi_f followed by the inverse transformation (see “Code1” below):

\vec f\coloneqq(f_1,f_2,\dots,f_N) \mapsto \psi_f\mapsto \vec {f'}.

My expectation is to see ||\vec f-\vec{f'}||_2 \approx \text{const.} \approx 10^{-16} (depending on machine precision) for all values of n, since the MPS construction is not truncated by cutoff nor maxdim arguments. This can be verified by the fact that maxlinkdim(ψ_f) = 2^(n/2), the maximum possible.

The MPS construction should be exact. However, I observe an exponential dependence

||\vec f-\vec{f'}||_2 \sim \exp(n).

This problem also appears, when one tries to reproduce (see “Code2” below) the truncation error bound by Verstraete and Cirac, here in the context of QTT, with the ITensorQTT.jl package. Where one expects

||\psi_f-{\psi_f}_D||_2 \sim \sqrt{n}.

\psi_f is the exact MPS construction and {\psi_f}_D is a truncated MPS with maxdim = D. Alternatively, one may use cutoff to truncate.

I have the following questions, I hope someone may be able to help:

  • Is machine precision to blame?
  • Are the MPS construction and the inverse not exact?
  • Am I missing something else?

Thank you in advance. Below you’ll find some code abstracts from the ITensorQTT.jl package along with minimal examples “Code1”, and “Code2”.

Source code abstract from ITensor/ITensorQTT.jl/src/function_to_mps.jl, lines 132-177:

#
# MPS to function conversion
#

function mps_to_discrete_function(ψ::MPS; kwargs...)
  return mps_to_discrete_function(Val(1), ψ; kwargs...)
end

function mps_to_discrete_function(::Val{ndims}, ψ::MPS) where {ndims}
  n = length(ψ)
  s = siteinds(ψ)
  s⃗ = ntuple(j -> s[j:ndims:n], Val(ndims))
  return reshape(Array(contract(ψ), vcat(reverse.(s⃗)...)), dim.(s⃗))
end

function mps_to_discrete_function(ψ::MPS, nbits::Integer; kwargs...)
  return mps_to_discrete_function(Val(1), ψ, nbits; kwargs...)
end

function mps_to_discrete_function(ψ::MPS, nbits::Tuple{Vararg{Integer}}; kwargs...)
  @assert allequal(nbits)
  ndims = length(nbits)
  nbits_original = ntuple(Returns(length(ψ) ÷ ndims), Val(ndims))
  nbits_retract = nbits_original .- nbits
  ψ = retract(ψ, nbits_retract; kwargs...)
  return mps_to_discrete_function(Val(ndims), ψ)
end

function mps_to_discrete_function(
  ::Val{ndims}, ψ::MPS, nbits::Integer; kwargs...
) where {ndims}
  return mps_to_discrete_function(ψ, ntuple(Returns(nbits), Val(ndims)))
end

#
# Vector/MPS conversion
#

function vec_to_tensor(v)
  n = Int(log2(length(v)))
  return permutedims(reshape(v, fill(2, n)...), n:-1:1)
end

function vec_to_mps(v, s; kwargs...)
  return MPS(vec_to_tensor(v), s; kwargs...)
end

Code1:

using ITensors
using ITensorsQTT
ITensors.disable_warn_order()

l2_err  = []

ns = [2, 5, 10, 15, 20]

xstart = -20
xstop = 20

for n in ns
  x = qtt_xrange(n, xstart, xstop)
  s = siteinds("Qubit", n)

  # discrete function (vector)
  f_vec = sin.(x)

  # vector to MPS
  ψ_f = vec_to_mps(f_vec, s)

  # MPS back to vector
  ψ_vec = mps_to_discrete_function(ψ_f)

  # calculate error
  push!(l2_err, norm(f_vec - ψ_vec))
end

Code2:

ITensors.disable_warn_order()

l2_err  = []

ns = [2, 5, 10, 15, 20]

xstart = -20
xstop = 20

for n in ns
  x = qtt_xrange(n, xstart, xstop)
  s = siteinds("Qubit", n)

  # discrete function
  f_vec = sin.(x)

  # function to MPS
  ψ_f = vec_to_mps(f_vec, s)
  truncate!(ψ_f, maxdim = 2^(n/2)) # or mindim = 2^(n/2)

  # truncate MPS
  ψ_tr = truncate(ψ_f, cutoff = 1e-16) # or maxdim = 2

  # calculate error
  push!(l2_err, norm(ψ_f - ψ_tr))
end

Now ploting l2_err with the y-axis in logarithmic scale shows an exponential scaling in n.

plot(ns, max.(l2_err, 1e-16), yscale = :log10)

Some comments:

  • [Shameless plug] Be sure to check out the more modern GitHub - JoeyT1994/ITensorNumericalAnalysis.jl
    • There we analytically construct sin(x) for any graph
  • sin(x) is exact with bond dimension 2. I suspect you are essentially measuring noise because you are asking for the exact truncation with two different roundings of svd
  • The paper you link deals with ground states. There, you have “psi^2”, i.e. you measure \langle \psi | O | \psi\rangle. Here, we only measure half a wave function \langle r|\psi\rangle, thus the SVD can be “half” of what you expect sometimes, as the singular values are related to |\psi|^2.

Also this paper might be helpful: [2311.12554] Multiscale interpolative construction of quantized tensor trains

I agree with the points @ryanlevy made. Also, for context, ITensorQTT.jl was more of a personal experiment for me to better understand QTTs, I haven’t worked on it in a long time and don’t plan on supporting it. As Ryan said you should probably try a more modern library like ITensorNumericalAnalysis.jl.

Thanks for the replies! I’ll be sure to take a look at ITensorNumericalAnalysis.jl.

@ryanlevy

  • I also see this scaling for other functions that have a much higher bond dimension. Functions which do not have an exact analytical MPS representation.

  • Is there an implementation to convert vector \leftrightarrow MPS and matrix \leftrightarrow MPO in the ITensorNumericalAnalysis.jl library? These are really helpful functions in the context of numerical analysis using QTTs.

  • On your last point regarding the paper, the scaling should not be exponential either way, even if the SVD is “half” of what I expect sometimes, I believe. And that is somehow what confused me when I made the numerics.

Thank you @mtfishman for the ITensorQTT.jl experiment. It helped me to learn some things about QTTs and about the ITensor library in general.

Glad to hear it! I learned a lot from it too. It was a helpful lesson in learning about operations we might need in ITensor that would be helpful for implementing that kind of functionality, and those lessons will be incorporated into ITensorNetworks.jl and ITensorNumericalAnalysis.jl.

I think you would need to carefully explain what you are trying to measure and what you are seeing. In general, as you increase the bond dimension you should be able to represent higher polynomials and should approach the same error as a (potentially overfitted) interpolation routine. As you increase the system size, then that’s a separate question as to the features of the function as the grid size shrinks.

Something that I forgot to mention is that some functions you may have rounding issues due to the high precision of the “QTT”. From our paper, we found that our sin(x) representation was off from the Julia one which rounded the last digit or two in Float64.

There is, it may or may not be in the repository at the moment (we’re trying to merge a summer project)
Take a look and if there’s not the right functionality you’d like please open an issue and we’ll try to add the feature!

Good point: with a fixed bond dimension, increasing the system size should reveal finer features, effectively higher modes. As a result, the approximation error grows with the system size when the bond dimension is held constant. However, we often assume the function is band-limited, so the error will eventually saturate.

I think the key point I was missing about why Verstraete and Cirac’s bound applies is that it assumes not only ground states (band-limited), but also that they are normalized states (as opposed to an unnormalized QTT representation of functions), as you mentioned earlier. After normalizing my QTTs before computing the error, I now observe the behavior I would expect: the error grows with system size but eventually saturates. This suggests that a bond dimension of two is sufficient to represent all points of sin(x) up to the saturation point, that is, within the precision limits of the QTT representation. So when applying this bound to QTTs, it’s crucial to normalize before evaluating the error. Or look at a similar error measure, like the MSE.

So the QTT representation of sin(x) was actually more accurate? That’s interesting. Sorry if this is a basic question, but how is that possible? As I understand it, extracting values from the QTT format still involves sums and multiplications of Float64 numbers, right? I’m guessing there are other factors in how Julia computes sin(x) and other functions that round the last few digits?

Thank you very much for your comments and the helpful discussion!

1 Like

I just mean that each tensor of the QTT we analytically know so its sort of like a funny series representation, but of course as you said when we multiply things out we lose the data that is below the Float64 bit precision. Then it becomes a question of “is the last digit a 2 or a 3” which is implementation specific.

Glad you have things working now! Note that the normalization is a very subtle point, as one can hide a hard problem’s hardness by removing an exponentially growing normalization

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.