PETERSSON’S TRACE FORMULA AND THE HECKE EIGENVALUES OF HILBERT MODULAR FORMS

ANDREW KNIGHTLY AND CHARLES LI

Abstract. Using an explicit relative trace formula, we obtain a Petersson trace formula for holomorphic Hilbert modular forms. Our main result expresses a sum (over a Hecke eigenbasis) of products of Fourier coefficients and Hecke eigenvalues in terms of generalized Kloosterman sums and Bessel functions. As an application we show that the normalized Hecke eigenvalues for a fixed prime 𝔭 have an asymptotic weighted equidistribution relative to a polynomial times the Sato-Tate measure, as the norm of the level goes to ∞.

1. Introduction

Let h ∈ S_{k}(Γ_{0}(N)) be a Hecke eigenform (k even), and for a prime p ∤ N define the normalized Hecke eigenvalue ν_{p}^{h} by

p^{− \frac{k−1}{2}} T_{p}h = ν_{p}^{h} h.

The Ramanujan-Petersson conjecture asserts that |ν_{p}^{h}| ≤ 2. This is a theorem of Deligne. Because we have assumed trivial central character, the operator T_{p} is self-adjoint, so its eigenvalues are real numbers, and thus

ν_{p}^{h} ∈ [−2, 2].

For a fixed non-CM newform h, the Sato-Tate conjecture predicts that the set {ν_{p}^{h} : p ∤ N} is equidistributed in [−2, 2] relative to the Sato-Tate measure

(1)  dμ_{∞}(x) = \{ \begin{matrix}\frac{1}{π} \sqrt{1 − \frac{x^{2}}{4}} dx & when x ∈ [−2, 2], \\ 0 & otherwise. \end{matrix}

Taylor has recently proven this in many cases when k = 2 [Ta].

When the prime p is fixed, the normalized eigenvalues of T_{p} on the space S_{k}(Γ_{0}(N)) are asymptotically equidistributed relative to the measure

dμ_{p}(x) = \frac{p + 1}{(p^{1/2} + p^{−1/2})^{2} − x^{2}} dμ_{∞}(x)

as k + N → ∞. This was proven in the late 1990’s by Serre [Se], and independently (for N = 1) by Conrey, Duke and Farmer [CDF]. An extension of the result (in the level aspect only) to Hilbert modular forms is given in [Li2]. These results have an antecedent in work of Bruggeman in 1978, who proved that the eigenvalues of T_{p} on Maass forms of level 1 exhibit the same distribution ([Br], Sect. 4; see also Sarnak [Sa]). Even earlier, in 1968 Birch proved a similar result for sizes of elliptic curves over F_{p} [Bi].

The first author was supported by NSA grant H98230-06-1-0039.

Document date: Jan. 26, 2008.

In this paper we prove a different result on the asymptotic distribution of Hecke eigenvalues, valid over a totally real number field F. The following is a special case of a more general result (cf. Theorem 6.6). Notation and terminology will be defined precisely later on.

Theorem 1.1. Let F be a totally real number field, and let m be a totally positive element of the inverse different 𝔡^{−1} ⊂ F. For a cusp form φ on GL_{2}(F)\ GL_{2}(A_{F}) with trivial central character, let W_{m}^{φ} denote its m^{th} Fourier coefficient (see (10)). Define a weight

w_{φ} = \frac{|W_{m}^{φ}(1)|^{2}}{∥φ∥^{2}} .

Fix a prime ideal 𝔭 ∤ m𝔡. Let {φ} be an orthogonal basis of eigenforms of prime-to-𝔭 level 𝔑 and weight (k_{1}, …, k_{r}), with all k_{j} > 2 and even. For such φ, let

ν_{𝔭}^{φ} ∈ [−2, 2]

denote the associated normalized eigenvalue of the Hecke operator T_{𝔭}. Then the w_{φ}-weighted distribution of the eigenvalues ν_{𝔭}^{φ} is asymptotically uniform relative to the Sato-Tate measure as the norm of 𝔑 goes to ∞. This means that for any continuous function f : R ⟶ C,

lim_{ℕ(𝔑)→∞} \frac{∑_{φ} f(ν_{𝔭}^{φ})w_{φ}}{∑_{φ} w_{φ}} = ∫_{R} f(x)dμ_{∞}(x).

Noteworthy is the fact that here the measure and the weights are independent of 𝔭, in contrast to Serre’s (unweighted) result. The Sato-Tate measure dμ_{∞} is nonzero on any subinterval of [−2, 2], so in particular the above illustrates the density of the Hecke eigenvalues in [−2, 2].

The proof of the above theorem involves a trace formula which may be of independent interest. In a previous paper [KL1], we detailed the way in which the classical 1932 Petersson trace formula can be realized as an explicit relative trace formula, as a conceptual alternative to the usual method using Poincaré series. Here we extend the technique and result to cusp forms on GL_{2}(A_{F}), where F is an arbitrary totally real number field. We work with spaces A_{k}(𝔑, ω) of holomorphic Hilbert cusp forms of weight k = (k_{1}, …, k_{r}), all strictly greater than 2, on the Hecke congruence subgroups Γ_{0}(𝔑). Our main result is Theorem 5.11 in which a sum (over a Hecke eigenbasis) of terms involving the associated Fourier coefficients, Petersson norms and eigenvalues of a Hecke operator T_{𝔫}, is expressed in terms of generalized Kloosterman sums and Bessel functions.

The incorporation of Hecke eigenvalues is a novel feature of this generalized sum formula. In Section 6 we prove a more general version of the above weighted distribution theorem by an argument based on bounding the terms of the sum formula in terms of ℕ(𝔑), following the technique for the F = Q case from [Li1]. This is analogous to the way in which Serre’s result follows from bounding the terms of the Eichler-Selberg formula for tr(T_{p^{m}}).

In Section 7 we give some variants of the generalized Petersson formula, including a simplified version for the case where F has narrow class number 1. In Corollary 7.3, setting 𝔫 = 1 for the trivial Hecke operator, we recover the classical extension of Petersson’s formula to Hilbert modular forms, as follows from a 1954 paper of Gundlach, [Gu]. His results are valid for cusp forms of uniform weight k ≥ 2. Refer to §2 of [Lu] for an overview of the classical derivation.

In 1980 Kuznetsov gave an analog of Petersson’s formula involving Maass forms and Eisenstein series on the spectral side, [Ku]. As an application, he used his formula to give estimates for sums of Kloosterman sums. His work was reformulated in a representation-theoretic setting by various authors, see [CPS] and its references, notably [MW] where the general rank-1 case is treated. More recently Bruggeman, Miatello and Pacharoni gave a general Kuznetsov trace formula for automorphic forms on SL_{2}(F_{∞}) of uniform even weight ([BMP], Theorem 2.7.1). An important application of their formula is an estimate for sums of Kloosterman sums.

In contrast to the above results, here we are concerned only with holomorphic cusp forms, i.e. those whose infinity types are discrete series. For our test function, we take discrete series matrix coefficients at the infinite components. This serves to isolate the holomorphic part of the cuspidal spectrum. At the finite places we take Hecke operators, which introduces Hecke eigenvalues into the final formula.

Thanks: We would like to thank Don Blasius for a helpful discussion about the Ramanujan conjecture. We also thank the referee for many insightful comments.

2. General setting

We recall the general setting of Jacquet’s relative trace formula [Ja]. Full details for the discussion below are given in §2 of [KL1]. Let F be a number field, with adele ring A. Let G be a reductive algebraic group defined over F. Let H be an F -subgroup of G × G, with H(A) unimodular. We assume that H(F)\H(A) is compact. Define a right action of H on G by g(x, y) = x^{−1}gy. For g ∈ G, let H_{g} be the stabilizer of g, i.e.

H_{g} = {(x, y) ∈ H| x^{−1}gy = g}.

For δ ∈ G(F), let [δ] be the H(F)-orbit of δ in G(F), i.e.

[δ] = {x^{−1}δy| (x, y) ∈ H(F)}.

Each element of [δ] can be expressed uniquely in the form u^{−1}δv for some (u, v) ∈ H_{δ}(F)\H(F).

Let f be a continuous function on G(A), and let

(2)  K(x, y) = ∑_{γ∈G(F)} f(x^{−1}γy) (x, y ∈ G(A))

be the associated kernel function. We assume that the above sum is uniformly absolutely convergent on compact subsets of H(A). In particular, K(x, y) is a continuous function on the compact set H(F)\H(A). Let χ(x, y) be a character of H(A), invariant under H(F). Consider the expression

(3)  ∫_{H(F)\H(A)} K(x, y)χ(x, y)d(x, y),

where d(x, y) is a Haar measure on H(A). A relative trace formula results from computing this integral using spectral and geometric expressions for K(x, y). Using the geometric expression (2), it is straightforward to show that the integral (3) is equal to ∑_{[δ]} I_{δ}(f), where

I_{δ}(f) = ∫_{H_{δ}(F)\H(A)} f(x^{−1}δy)χ(x, y)d(x, y).

We see that

I_{δ}(f) = ∫_{H_{δ}(A)\H(A)} f(x^{−1}δy) [∫_{H_{δ}(F)\H_{δ}(A)} χ(rx, sy)d(r, s)] d(x, y).

An orbit [δ] is relevant if χ is trivial on H_{δ}(A). From the above, we see that I_{δ} = 0 whenever [δ] is not relevant. Indeed the expression in the brackets is equal to χ(x_{0}, y_{0}) ∫_{H_{δ}(F)\H_{δ}(A)} χ(r, s)d(r, s), where (x_{0}, y_{0}) ∈ H(A) is any representative for (x, y), and this integral vanishes unless δ is relevant.

3. Preliminaries

3.1. Notation. Throughout this paper we work over a totally real number field F ≠ Q. Let

r = [F : Q],

and let σ_{1}, …, σ_{r} be the distinct embeddings F ↪ R. Let ∞_{1}, …, ∞_{r} denote the corresponding archimedean valuations. Let O be the ring of integers of F. We will generally use Gothic letters 𝔞, 𝔟 etc. to denote fractional ideals of F. We reserve 𝔭 for prime ideals. Let v = v_{𝔭} be the discrete valuation corresponding to 𝔭. Let O_{v} be the ring of integers in the local field F_{v}, and let ϖ_{v} ∈ O_{v} be a generator of the maximal ideal 𝔭_{v} = 𝔭O_{v}.

Let N : F → Q denote the norm map. For a nonzero ideal 𝔞 ⊂ O, let ℕ(𝔞) = |O/𝔞| denote the absolute norm. This extends by multiplicativity to the group of nonzero fractional ideals of F. For α ∈ F^{∗}, we define

ℕ(α) = ℕ(αO) = | N(α)|.

We also use the above norms in the local setting with the analogous meanings. We normalize the absolute value on F_{v} by |ϖ_{v}|_{v} = ℕ(𝔭)^{−1}.

By Dirichlet’s unit theorem, the unit group of F is

O^{∗} ≅ (Z/2Z) × Z^{r−1}.

Letting O^{∗2} denote the subgroup consisting of squares of units, we see that O^{∗}/O^{∗2} ≅ (Z/2Z)^{r} is a finite group. Let

(4)  U = {u_{1}, …, u_{2^{r}}} ⊂ O^{∗}

be a fixed set of representatives for O^{∗}/O^{∗2}.

For a fractional ideal 𝔞 ⊂ F, let ord_{v} 𝔞 (or ord_{𝔭} 𝔞) denote the order. Let 𝔞_{v} (or 𝔞_{𝔭}) denote its localization, so 𝔞_{v} = ϖ_{v}^{ord_{v}(𝔞)}O_{v}. Let A denote the adele ring of F, with finite adeles A_{fin}, so that

A = F_{∞} × A_{fin},

where F_{∞} = F ⊗ R ≅ R^{r}. Let Ô = ∏_{v<∞} O_{v}. Generally if 𝔞 is a fractional ideal of F, we write 𝔞̂ = 𝔞Ô = ∏_{v<∞} 𝔞_{v} ⊂ A_{fin}. Let Cl(F) be the class group of F, of cardinality h(F). For a fractional ideal 𝔞, let [𝔞] represent its image in the class group.

Let F^{+} denote the set of totally positive elements of F. We let F_{∞}^{+} denote the subset of F_{∞} of vectors whose entries are all positive. Let

𝔡^{−1} = {x ∈ F | tr_{Q}^{F}(xO) ⊂ Z}

denote the inverse different. We set 𝔡_{+}^{−1} = 𝔡^{−1} ∩ F^{+}.

We use the mathtt font to represent finite ideles. Thus we write

(5)  a ∈ A_{fin}^{∗}, 𝔞̂ = aÔ, 𝔞 = O ∩ 𝔞̂.

In the other direction, given a fractional ideal 𝔞 ⊂ F, there is an element a ∈ A_{fin}^{∗} such that (5) holds. Explicitly, we can take a_{v} = ϖ_{v}^{ord_{v} 𝔞}. The element a is unique up to Ô^{∗}. We define norms of ideles by taking the products of the local norms. For example, in the situation of (5), we have

|a|_{fin} = ∏_{v<∞} |a_{v}|_{v} = ∏_{v<∞} ℕ(a_{v})^{−1} = ℕ(a)^{−1} = ℕ(𝔞)^{−1}.

We use Roman or Greek letters for rational elements a, α ∈ F.

3.2. Haar measure. We use Lebesgue measure on R, and take the product measure on F_{∞} ≅ R^{r}. We normalize Haar measure on each non-archimedean completion F_{v} by taking meas(O_{v}) = 1. This choice induces a Haar measure on A_{fin} with meas(Ô) = 1, and because A = F + F_{∞} × Ô,

meas(F \A) = meas((F ∩ Ô)\(F_{∞} × Ô)) = meas((O\F_{∞}) × Ô)

= meas(O\F_{∞}) = d_{F}^{1/2}, (6)

where d_{F} is the discriminant of F. The resulting measure on A is not self-dual.

Let G = GL_{2}. We normalize Haar measure on G(R) as follows. Use Lebesgue measure dx to define a measure dn on the unipotent subgroup N(R) ≅ R. On R^{∗} we use the measure dx/|x|. We take the product measure dm on the diagonal subgroup M(R) ≅ R^{∗} × R^{∗}, and normalize dk on SO_{2}(R) to have total measure 1. On G(R) = M(R)N(R) SO_{2}(R) we take dg = dm dn dk.

Let

K_{fin} = ∏_{v<∞} K_{v} = ∏_{v<∞} G(O_{v}) = G(Ô)

be the standard maximal compact subgroup of G(A_{fin}). We normalize Haar measure on G(A_{fin}) by taking meas(K_{v}) = 1 for all v < ∞. We let Z denote the center of G, and write

\overline{G} = G/Z.

We normalize Haar measure on \overline{G}(F_{v}) by taking meas(\overline{K}_{v}) = 1.

3.3. Hilbert modular forms. Let 𝔑 ⊂ O be an integral ideal of F. Let k = (k_{1}, …, k_{r}) be an r-tuple of positive integers, each greater than or equal to 2. Let

ω : F^{∗}\A^{∗} → C^{∗}

be a unitary Hecke character. Write ω = ∏_{v} ω_{v}. We assume that:

(1) the conductor of ω divides 𝔑

(2) ω_{∞_{j}}(x) = sgn(x)^{k_{j}} for all 1 ≤ j ≤ r. The first condition means that ω_{v} is trivial on 1 + 𝔑_{v} for all v|𝔑, and unramified for all v ∤ 𝔑. In other words, ω_{fin} is trivial on the open set

Ô_{𝔑}^{∗} =^{def} Ô^{∗} ∩ (1 + 𝔑Ô) = ∏_{v|𝔑}(1 + 𝔑_{v}) ∏_{v∤𝔑} O_{v}^{∗}.

Thus ω can be viewed as a character of the ray class group mod 𝔑

Cl(𝔑) ≅ A^{∗}/[F^{∗}(F_{∞}^{+} × Ô_{𝔑}^{∗})].

We freely identify Z(A) with A^{∗} throughout this paper. For example if z ∈ Z(A) we write ω(z). Let G = GL_{2} and let

L^{2}(ω) = L^{2}(\overline{G}(F)\\overline{G}(A), ω)

be the space of left G(F)-invariant functions on G(A) which transform by ω under the center and which are square integrable over \overline{G}(F)\\overline{G}(A). Let R denote the right regular representation of G(A) on L^{2}(ω). Let L_{0}^{2}(ω) be the subspace of cuspidal functions. We know that the restriction of R to L_{0}^{2}(ω) decomposes as a discrete sum of irreducible representations. These are the cuspidal representations π. Every such π factorizes as a restricted tensor product of admissible local representations ⊗π_{v}.

Define groups

K_{0}(𝔑) = \{(\begin{matrix}a & b \\ c & d \end{matrix}) ∈ K_{fin}| c ∈ 𝔑Ô\}

and

K_{1}(𝔑) = \{(\begin{matrix}a & b \\ c & d \end{matrix}) ∈ K_{0}(𝔑)| d ∈ 1 + 𝔑Ô\}.

These are open compact subgroups of G(A_{fin}). Let

(7)  H_{k}(𝔑, ω) = ⨁ π,

where π runs through all cuspidal representations in L_{0}^{2}(ω) for which:

(1) π_{fin} = ⊗_{v<∞}π_{v} contains a nonzero K_{1}(𝔑)-fixed vector

(2) π_{∞_{j}} = π_{k_{j}} is the discrete series representation of G(R) of weight k_{j}, for

j = 1, …, r. The central character of π_{k_{j}} is given by

χ_{π_{k_{j}}} (x) = sgn(x)^{k_{j}} = ω_{∞_{j}}(x).

For a discrete series representation π_{∞_{j}}, let v_{π_{∞_{j}}} be a lowest weight vector, unique up to nonzero multiples. Define the subspace

(8)  A_{k}(𝔑, ω) = ⨁_{π} Cv_{π_{∞_{1}}} ⊗ ⋯ ⊗ v_{π_{∞r}} ⊗ π_{fin}^{K_{1}(𝔑)},

where π runs through all irreducible summands of H_{k}(𝔑, ω). Here π_{fin}^{K_{1}(𝔑)} is the subspace of K_{1}(𝔑)-fixed vectors in the space of π_{fin}.

Proposition 3.1. The space A_{k}(𝔑, ω) defined above is equal to the set of φ ∈ L_{0}^{2}(ω) satisfying:

(1) φ(gk_{fin}) = φ(g) for all k_{fin} ∈ K_{1}(𝔑)

(2) φ(gk_{∞}) = ∏_{j=1}^{r} e^{ik_{j}θ_{j}}φ(g) for all k_{∞} = ∏_{j} k_{θ_{j}} ∈ K_{∞} = SO_{2}(R)^{r}

(3) For any fixed x ∈ G(A) and 1 ≤ j ≤ r, the function g_{∞_{j}} ↦ φ(xg_{∞_{j}}) is

annihilated by R(E^{−}), where E^{−} = (\begin{matrix}1 & −i \\ −i & −1 \end{matrix}) ∈ 𝔤𝔩_{2}(C) and R denotes

the right regular action of G(F_{∞_{j}}). The elements of A_{k}(𝔑, ω) are continuous functions on G(A).

Proof. The proof is the same as for the case F = Q given in Theorem 12.6 of [KL2]. For the continuity, see Lemma 3.3 of [Li2]. □

By a general theorem of Harish-Chandra, the space A_{k}(𝔑, ω) is finite-dimensional (cf. [HC], [BJ]). In particular the set of π in (7) is finite.

3.4. Fourier coefficients. Let θ : A ⟶ C^{∗} be the standard character of A. Explicitly,

(1) θ_{∞}(x) = e^{−2πi(x_{1}+···+x_{r})} for x = (x_{1}, …, x_{r}) ∈ F_{∞}

(2) For v < ∞, θ_{v} is the composition

θ_{v} : F_{v} ⟶^{tr_{Qp}^{Fv}} Q_{p} → Q_{p}/Z_{p} ↪ Q/Z ⟶^{e^{(2πi·)}} C^{∗}.

Note that θ_{v} is trivial precisely on the local inverse different 𝔡_{v}^{−1} = 𝔡^{−1}O_{v} = {x ∈ F_{v}| tr_{Q_{p}}^{F_{v}}(x) ∈ Z_{p}}. Recall that θ is trivial on F, and that every character on F \A is of the form

θ_{m}(x) = θ(−mx)

for some m ∈ F. This identifies the discrete group F with the dual group { \atop F \A}.

Consider the unipotent subgroup N = {( _{1}^{1 ∗} )} of G. As topological groups, N(A) ≅ A, so characters of the two can be identified. For m ∈ F, we identify θ_{m} with a character on N(F)\N(A) in the obvious way.

For any smooth cusp form φ on G(A) and g ∈ G(A), the map n ↦ φ(ng) is a continuous function on N(F)\N(A), with a Fourier expansion

(9)  φ(\binom{1 x}{1} g) = \frac{1}{d_{F}^{1/2}} ∑_{m∈F} W_{m}^{φ}(g)θ_{m}(x).

The coefficients are Whittaker functions defined by

(10)  W_{m}^{φ}(g) = ∫_{F \A} φ(\binom{1 x}{1} g)θ(mx)dx.

The purpose of the factor of d_{F}^{−1/2} in (9) is to balance the fact that by our choice, meas(F \A) = d_{F}^{1/2}. This non-selfdual measure is convenient for the calculations later on.

Let y_{1}, …, y_{h} ∈ A_{fin}^{∗} be representatives for A_{fin}^{∗}/F^{∗}Ô^{∗}, the class group of F. Then

G(A) = ⋃_{i=1}^{h} G(F) [B(F_{∞})^{+}K_{∞} × (\begin{matrix}y_{i} \\ & 1 \end{matrix}) K_{1}(𝔑)]

([Hi], §9.1). Here B denotes the subgroup of invertible upper triangular matrices. It follows that an element φ ∈ A_{k}(𝔑, ω) is determined by the values

φ(( _{1}^{1 x} )_{∞}( \begin{matrix}y \\ & 1 \end{matrix} )_{∞}( \begin{matrix}y_{i} \\ & 1 \end{matrix} )_{fin})

for y ∈ F_{∞}^{+} and x ∈ F_{∞}. We define for y ∈ A^{∗} the notation

(11)  W_{m}^{φ}(y) = W_{m}^{φ}(( \begin{matrix}y \\ & 1 \end{matrix} )).

By the above remarks, φ is determined by the coefficients W_{m}^{φ}(y) for y ∈ A^{∗}.

Proposition 3.2. For φ ∈ A_{k}(𝔑, ω),

• W_{m}^{φ}(g) = W_{1}^{φ}(( \begin{matrix}m \\ & 1 \end{matrix} )g) for all m ∈ F^{∗} and g ∈ G(A)

• For any u ∈ Ô^{∗} and y ∈ A^{∗}, W_{m}^{φ}(y) = W_{m}^{φ}(uy) for all m ∈ F^{∗}

• For y ∈ A_{fin}^{∗}, W_{1}^{φ}(y) ≠ 0 only if y ∈ 𝔡̂^{−1}. (Here and throughout, we identify A_{fin}^{∗} with {1_{∞}} × A_{fin}^{∗} ⊂ A^{∗}.)

Proof. These are standard facts. The first follows by a change of variables in (10) plus the left G(F)-invariance of φ. The K_{1}(𝔑)-invariance of φ gives the second, and also implies that for u ∈ Ô

φ(( _{1}^{1 x} )( \begin{matrix}y \\ & 1 \end{matrix} )) = φ(( _{1}^{1 x} )( \begin{matrix}y \\ & 1 \end{matrix} )( _{1}^{1 u} )) = φ(( _{1}^{1 x+uy} )( \begin{matrix}y \\ & 1 \end{matrix} )).

This means that W_{m}^{φ}(y) = W_{m}^{φ}(y)θ(−muy) for all u ∈ Ô, which gives the third when m = 1. (See also §9.1 of [Hi].) □

Proposition 3.3. For any φ ∈ A_{k}(𝔑, ω) and any y ∈ F_{∞}^{+}, W_{m}^{φ}(y) = 0 unless m ∈ 𝔡_{+}^{−1} = 𝔡^{−1} ∩ F^{+}.

Proof. By the definition of cuspidality, the constant term φ_{N}(g) vanishes for a.e. g ∈ G(A). Because φ is actually continuous, it follows that φ_{N}(g) = 0 for all g. Therefore when m = 0,

(12)  W_{0}^{φ}(g) = φ_{N}(g) = 0.

To φ we can attach a holomorphic function on H^{r} by

(13)  h(x + iy) = φ(( _{1}^{y x} )_{∞} × 1_{fin}) ∏_{j=1}^{r} y_{j}^{−k_{j}/2}.

Then h is a Hilbert modular form for the group Γ_{1}(𝔑) = SL_{2}(F) ∩ K_{1}(𝔑), so it has a Fourier expansion of the form

h(x + iy) = a_{0}(h) + ∑_{m∈𝔡_{+}^{−1}} a_{m}(h)e^{−2π tr(my)}θ_{∞,m}(x),

where tr(my) = ∑_{j=1}^{r} σ_{j}(m)y_{j}. For any y ∈ F_{∞}^{+}, the Fourier coefficients a_{m}(h) and W_{m}^{φ}(y) are related by

(14)  W_{m}^{φ}(y) = d_{F}^{1/2}(∏_{j=1}^{r} y_{j}^{k_{j}/2})e^{−2π tr(my)}a_{m}(h).

This follows immediately by equating the classical and adelic Fourier expansions of φ(( \begin{matrix}y_{∞} & x_{∞} \\ 0 & 1 \end{matrix} )). Together with (12), this implies that W_{m}^{φ}(y) = 0 unless m ∈ 𝔡_{+}^{−1}. □

4. Construction of the test function

The right regular action of an element f ∈ L^{1}(G(A), ω^{−1}) on L^{2}(ω) is given by

(15)  R(f)ϕ(x) = ∫_{\overline{G}(A)} f(g)ϕ(xg)dg.

In this section we will construct a continuous integrable function f such that R(f) has finite rank (vanishing on A_{k}(𝔑, ω)^{⊥}), and acts like a Hecke operator on A_{k}(𝔑, ω). The function will be defined locally:

f = f_{∞}f_{fin} = ∏_{j=1}^{r} f_{∞_{j}} ∏_{v<∞} f_{v}.

4.1. Archimedean test functions. For j = 1, …, r let v_{k_{j}} be a lowest weight unit vector for π_{k_{j}} and let d_{k_{j}} be the formal degree of π_{k_{j}} relative to the measure on G(R) fixed in Sect. 3.2. We take f_{∞_{j}}(g) to be the normalized matrix coefficient d_{k_{j}} \overline{⟨π_{k_{j}}(g)v_{k_{j}}, v_{k_{j}}⟩}. Explicitly, if g = ( \begin{matrix}a & b \\ c & d \end{matrix} ), then (cf. [KL2], Theorem 14.5)

(16)  f_{∞_{j}}(g) = \{ \begin{matrix}\frac{(k_{j} − 1)}{4π} \frac{det(g)^{k_{j}/2}(2i)^{k_{j}}}{(−b + c + (a + d)i)^{k_{j}}} & if det(g) > 0 \\ 0 & otherwise. \end{matrix}

This function is integrable over \overline{G}(R) if and only if k_{j} > 2. Therefore in order for (15) to converge we must assume henceforth that

k_{j} > 2 (j = 1, …, r).

Proposition 4.1. Let f_{∞} = ∏_{j} f_{∞_{j}} be as above, and suppose f_{fin} is a bi-K_{1}(𝔑)- invariant function on G(A_{fin}) satisfying f_{fin}(zg) = ω_{fin}(z)^{−1}f_{fin}(g) and whose support is compact mod Z(A_{fin}). Then R(f) vanishes on A_{k}(𝔑, ω)^{⊥} (the orthogonal complement in L^{2}(ω)) and its image is a subspace of A_{k}(𝔑, ω).

Proof. The case F = Q is proven in Corollary 13.13 of [KL2], and the general case is no different. The main point is that ∫_{N(R)} f_{∞_{j}}(g_{1}ng_{2})dn = 0 for each j. Using this, one shows that R(f)ϕ is cuspidal for each ϕ ∈ L^{2}(ω). By the left K_{1}(𝔑)- invariance of f_{fin}, it is easy to see that R(f)ϕ is K_{1}(𝔑)-invariant, while the matrix coefficients project onto the span of the lowest weight vectors of the discrete series of the appropriate weight. Thus R(f)ϕ ∈ A_{k}(𝔑, ω). (See also [Li2], Prop. 2.2). □

4.2. Non-Archimedean test functions. We now specify the local factors of f more precisely. Fix a discrete valuation v of F, and let 𝔭 be the corresponding prime ideal of O. Let 𝔑 ⊂ O be the ideal fixed earlier, and let 𝔑_{v} = 𝔑O_{v} be its localization.

Let G_{v} = G(F_{v}), and similarly for its subgroups Z_{v} = Z(F_{v}), etc. Suppose f_{v} : G_{v} → C is a locally constant function whose support is compact modulo Z_{v}. Then the Hecke operator R(f_{v}) is defined by

R(f_{v})ϕ(x) = ∫_{\overline{G}_{v}} f_{v}(g)ϕ(xg)dg

for any continuous function ϕ on G_{v}. Note that the integrand is not always welldefined modulo Z_{v}. In our situation, ϕ will be a function satisfying ϕ(zg) = ω_{v}(z)ϕ(g) for all z ∈ Z_{v} and g ∈ G_{v}. Therefore we must require f_{v} to transform under the center by ω_{v}^{−1}.

The Hecke algebra of bi-K_{1}(𝔑)_{v}-invariant functions is generated by functions supported on sets of the form

Z_{v}K_{1}(𝔑)_{v} x_{v} K_{1}(𝔑)_{v}

for x_{v} ∈ G_{v}.

Fix an integral ideal 𝔫_{v} in O_{v}. We assume that 𝔫_{v} is coprime to 𝔑_{v}, i.e., that either 𝔫_{v} or 𝔑_{v} is equal to O_{v}. Define a set

M(𝔫_{v}, 𝔑_{v}) = \{(\begin{matrix}a & b \\ c & d \end{matrix}) ∈ M_{2}(O_{v})| c ∈ 𝔑_{v}, (ad − bc)O_{v} = 𝔫_{v}\}.

The determinant condition is equivalent to det g ∈ ϖ_{v}^{ord_{v}(𝔫_{v})}O_{v}^{∗}. By the Cartan decomposition of G_{v} we have

M(𝔫_{v}, 𝔑_{v}) = \{ \begin{matrix}⋃_{{i+j=ordv (𝔫v) \atop i≥j≥0}} K_{v} (\begin{matrix}ϖ_{v}^{i} \\ & ϖ_{v}^{j} \end{matrix}) K_{v} & if v ∤ 𝔑 \\ K_{0}(𝔑)_{v} & if v|𝔑. \end{matrix}

Clearly this is a compact set. We need to define a bi-K_{1}(𝔑)_{v}-invariant function f_{𝔫_{v}}, supported on

Z_{v}M(𝔫_{v}, 𝔑_{v}),

and with central character ω_{v}^{−1}. If v|𝔑, for k = ( \begin{matrix}a & b \\ c & d \end{matrix} ) ∈ K_{0}(𝔑)_{v} define

ω_{v}(k) = ω_{v}(d).

Because c ∈ 𝔑_{v}, one easily sees that this is a character of K_{0}(𝔑)_{v}. Now for z ∈ Z_{v} and m ∈ M(𝔫_{v}, 𝔑_{v}), define

(17)  f_{𝔫_{v}}(zm) = \{ \begin{matrix}ω_{v}(z)^{−1} & if v ∤ 𝔑 \\ ψ(𝔑_{v})ω_{v}(z)^{−1}ω_{v}(m)^{−1} & if v|𝔑. \end{matrix}

Here, when 𝔭|𝔑,

ψ(𝔑_{v}) = meas(\overline{K_{1}(𝔑)_{v}})^{−1} = [K_{v} : K_{0}(𝔑)_{v}] = ℕ(𝔭)^{ord_{v}(𝔑)}(1 + ℕ(𝔭)^{−1}).

It is straightforward to show that f_{𝔫_{v}} is well-defined.

Lemma 4.2. Suppose g = ( \begin{matrix}a & b \\ c & d \end{matrix} ) ∈ G(F_{v}) and that (det g) = 𝔫_{v}. Then f_{𝔫_{v}}(g) ≠ 0 if and only if g ∈ M_{2}(O_{v}) and c ∈ 𝔑_{v}.

Proof. Note that f_{𝔫_{v}}(g) ≠ 0 if and only if g = zm, with z ∈ Z(F_{v}), m ∈ M(𝔫_{v}, 𝔑_{v}). Taking determinants we see that z is a unit in O_{v} (identifying Z_{v} with F_{v}^{∗}). Thus z can be absorbed into m, so in fact g ∈ M(𝔫_{v}, 𝔑_{v}) as required. □

Proposition 4.3. The adjoint of the operator R(f_{𝔭_{v}^{ℓ}}) on L^{2}(G_{v}, ω_{v}) is given by

R(f_{𝔭_{v}^{ℓ}})^{∗} = ω_{v}(ϖ_{v}^{ℓ})^{−1}R(f_{𝔭_{v}^{ℓ}}).

Proof. We have R(f_{𝔭_{v}^{ℓ}})^{∗} = R(f_{𝔭_{v}^{ℓ}}^{∗}) where f_{𝔭_{v}^{ℓ}}^{∗}(g) = \overline{f_{𝔭_{v}^{ℓ}}(g^{−1})}. If g = zm, then

g^{−1} = z^{−1}m^{−1} = (z^{−1}ϖ_{v}^{−ℓ})(ϖ_{v}^{ℓ}m^{−1}) ∈ Z_{v}M(𝔫_{v}, 𝔑_{v}).

The proposition follows easily from this. (Note that ℓ = 0 if v|𝔑.) □

When v ∤ 𝔑, the functions f_{𝔫_{v}} defined above linearly span the spherical Hecke algebra of bi-K_{v}-invariant complex-valued functions on G_{v} with central character ω_{v}^{−1}.

Now suppose v ∤ 𝔑 and write 𝔫_{v} = 𝔭_{v}^{ℓ} for ℓ > 0. Let χ(( \begin{matrix}a & b \\ c & d \end{matrix} )) = χ_{1}(a)χ_{2}(d) be an unramified character of the Borel subgroup B(F_{v}), and let (π, V_{χ}) be the representation of G_{v} obtained from χ by normalized induction. We assume that

χ_{1}(z)χ_{2}(z) = ω_{v}(z)

for all z ∈ Z_{v}. Define a function ϕ_{0} ∈ V_{χ} by

(18)  ϕ_{0}(( \begin{matrix}a & b \\ 0 & d \end{matrix} )k) = |a/d|_{v}^{1/2}χ_{1}(a)χ_{2}(d).

Then ϕ_{0} spans the 1-dimensional space of K_{v}-fixed vectors in V_{χ}.

Proposition 4.4. Let q_{v} = ℕ(𝔭_{v}). Then π(f_{𝔭_{v}^{ℓ}})ϕ_{0} = λ_{𝔭_{v}^{ℓ}}ϕ_{0}, where

λ_{𝔭_{v}^{ℓ}} = q_{v}^{ℓ/2} ∑_{j=0}^{ℓ} χ_{1}(ϖ_{v})^{j} χ_{2}(ϖ_{v})^{ℓ−j}.

Proof. Because f_{𝔭_{v}^{ℓ}} is left K_{v}-invariant, R(f_{𝔭_{v}^{ℓ}})ϕ_{0} is again fixed by K_{v}, and hence R(f_{𝔭_{v}^{ℓ}})ϕ_{0} = λϕ_{0} for some λ ∈ C. The action of π is the same as the action of R, so it suffices to compute λ = R(f_{𝔭_{v}^{ℓ}})ϕ_{0}(1). Using the well-known left coset decomposition ([KL2], Lemma 13.4)

M(𝔭_{v}^{ℓ}, 𝔑_{v}) = ⋃_{j=0}^{ℓ} ⋃_{a∈O_{v}/𝔭_{v}^{j}} \binom{ϖ_{v}^{j} a}{ϖ_{v}^{ℓ−j}} K_{v},

we see that λ = ∫_{\overline{G}_{v}} f_{𝔭_{v}^{ℓ}}(g)ϕ_{0}(g)dg = ∫\overline{M(𝔭_{v}^{ℓ},𝔑_{v})} ϕ_{0}(g)dg

= ∑_{j=0}^{ℓ} ∑_{a∈O_{v}/𝔭_{v}^{j}} ϕ_{0}(( _{ϖ_{v}^{ℓ−j}}^{ϖ_{v}^{j} a} )) (meas(\overline{K}_{v}) = 1)

= ∑_{j=0}^{ℓ} q_{v}^{j} |ϖ_{v}^{j}/ϖ_{v}^{ℓ−j}|_{v}^{1/2} χ_{1}(ϖ_{v})^{j}χ_{2}(ϖ_{v})^{ℓ−j} = q_{v}^{ℓ/2} ∑_{j=0}^{ℓ} χ_{1}(ϖ_{v})^{j} χ_{2}(ϖ_{v})^{ℓ−j},

as claimed. □

Proposition 4.5. With notation as above,

(19)  q_{v}^{−ℓ/2}ω_{v}(ϖ_{v})^{−ℓ/2}λ_{𝔭_{v}^{ℓ}} = X_{ℓ}(q_{v}^{−1/2}ω_{v}(ϖ_{v})^{−1/2}λ_{𝔭_{v}})

where

X_{ℓ}(2 cos θ) = \frac{sin(ℓ + 1)θ}{sin θ} = e^{iℓθ} + e^{i(ℓ−2)θ} + ⋯ + e^{−iℓθ}

is the Chebyshev polynomial of degree ℓ.

Proof. Let α_{𝔭} = ω_{v}(ϖ_{v})^{−1/2}χ_{1}(ϖ_{v}) and β_{𝔭} = ω_{v}(ϖ_{v})^{−1/2}χ_{2}(ϖ_{v}). Note that α_{𝔭}β_{𝔭} = 1. Hence we may write α_{𝔭} = e^{iθ}, β_{𝔭} = e^{−iθ}, and α_{𝔭} + β_{𝔭} = 2 cos θ for some θ ∈ C. By the previous proposition, the left-hand side of (19) is

∑_{j=0}^{ℓ} α_{𝔭}^{j}β_{𝔭}^{ℓ−j} = ∑_{j=0}^{ℓ} e^{ijθ}e^{−i(ℓ−j)θ} = ∑_{j=0}^{ℓ} e^{i(2j−ℓ)θ} = X_{ℓ}(2 cos θ).

This proves the result since q_{v}^{−1/2}ω(ϖ_{v})^{−1/2}λ_{𝔭_{v}} = α_{𝔭} + β_{𝔭} = 2 cos θ. □

4.3. Global Hecke operators. Finally we define the global Hecke operator. Fix an ideal 𝔫 in O, relatively prime to 𝔑. Define a function on A_{fin} by

f_{𝔫} = ∏_{v} f_{𝔫_{v}},

where f_{𝔫_{v}} is defined as in the previous subsection. Then f_{𝔫} is bi-K_{1}(𝔑)-invariant, and supported on Z(A_{fin})M(𝔫, 𝔑), where

M(𝔫, 𝔑) = ∏_{v<∞} M(𝔫_{v}, 𝔑_{v}) = \{(\begin{matrix}a & b \\ c & d \end{matrix}) ∈ M_{2}(Ô)| c ∈ 𝔑̂, (ad − bc)Ô = 𝔫̂\} .

It is also clear that

f_{𝔫}(zg) = ω_{fin}(z)^{−1}f_{𝔫}(g) (z ∈ Z(A_{fin}), g ∈ G(A_{fin})).

We define the operator

T_{𝔫} = R(f_{∞} × f_{𝔫})

on L_{0}^{2}(ω), which we can view as an operator on A_{k}(𝔑, ω) by Proposition 4.1. The family of operators T_{𝔫} for (𝔫, 𝔑) = 1 is simultaneously diagonalizable (see Lemma 6.3 below).

The following proposition and its corollaries spell out the connection between Hecke eigenvalues and Fourier coefficients.

Proposition 4.6. Given 𝔫, choose n, d ∈ A_{fin}^{∗} such that nÔ = 𝔫̂ and dÔ = 𝔡̂. Then for any φ ∈ A_{k}(𝔑, ω),

W_{1}^{T_{𝔫}φ}(1/d) = ℕ(𝔫)W_{1}^{φ}(n/d).

Proof. We use the left coset decomposition

M(𝔫, 𝔑) = ⋃_{{r,s∈Ô/Ô^{∗} \atop rsÔ=𝔫̂}} ⋃_{t∈Ô/rÔ} (\begin{matrix}r & t \\ 0 & s \end{matrix}) K_{0}(𝔑),

which is proven as in [KL2], Lemma 13.5. We see that

T_{𝔫}φ(\binom{y x}{1}) = ∫_{\overline{G}(A_{fin})} f_{𝔫}(g)φ(\binom{y x}{1} g)dg

= ∑_{r,s} ∑_{t} φ(\binom{y x}{1} \binom{r t}{s}).

Note that

φ(\binom{y x}{1} \binom{r t}{s}) = φ(\binom{yr yt + xs}{s}) = ω_{fin}(s)φ(\binom{\frac{yr}{s} \frac{yt}{s} + x}{1})

= \frac{ω_{fin}(s)}{d_{F}^{1/2}} [W_{0}^{φ}(\frac{yr}{s}) + ∑_{m∈F^{∗}} W_{1}^{φ}(\frac{myr}{s})θ_{m}(\frac{yt}{s} + x)] .

Therefore

W_{1}^{T_{𝔫}φ}(y) = d_{F}^{1/2} · [coeff. of θ_{1}(x)] = ∑_{r,s} ω_{fin}(s)W_{1}^{φ}(\frac{yr}{s}) ∑_{t} θ(−\frac{yt}{s}).

Now suppose y_{∞} = 1 and identify y with y_{fin}. Then we can assume that yr/s ∈ 𝔡̂^{−1} since otherwise W_{1}^{φ}(\frac{yr}{s}) = 0. Then θ(−yt/s) is a well-defined character of t ∈ Ô/rÔ, and

∑_{t∈Ô/rÔ} θ(−yt/s) = \begin{cases}ℕ(r) & if − y/s ∈ 𝔡̂^{−1} \\ 0 & otherwise. \end{cases}

Thus

(20)  W_{1}^{T_{𝔫}(φ)}(y) = ∑_{{r,s \atop y/s∈𝔡̂^{−1}}} ω_{fin}(s)ℕ(r)W_{1}^{φ}(\frac{yr}{s})  (y ∈ A_{fin}^{∗}).

Now take y = 1/d. Then y/s ∈ 𝔡̂^{−1} only if s = 1 ∈ Ô/Ô^{∗}. We can therefore take r = n and s = 1, so

W_{1}^{T_{𝔫}φ}(1/d) = ℕ(𝔫)W_{1}^{φ}(n/d),

as claimed. □

Using the above, we can express Hecke eigenvalues in terms of Fourier coefficients and vice versa.

Corollary 4.7. Suppose φ is an eigenvector of T_{𝔫} with eigenvalue λ_{𝔫}. Then if W_{1}^{φ}(1/d) ≠ 0,

λ_{𝔫} = \frac{ℕ(𝔫)W_{1}^{φ}(n/d)}{W_{1}^{φ}(1/d)}.

Corollary 4.8. If (m𝔡, 𝔑) = 1, then for any T_{m𝔡}-eigenfunction φ ∈ A_{k}(𝔑, ω) with W_{1}^{φ}(1/d) = 1 and T_{m𝔡}φ = λ_{m𝔡}φ, we have

W_{m}^{φ}(1) = \frac{e^{2πr} ∏_{j=1}^{r} σ_{j}(m)^{k_{j}/2−1}}{d_{F} e^{2π tr(m)}} λ_{m𝔡}.

Proof. Apply Cor. 4.7 with 𝔫 = m𝔡 and n = md. We get

(21)  λ_{m𝔡} = ℕ(m𝔡)W_{1}^{φ}(1_{∞} × m_{fin}) = ℕ(m𝔡)W_{m}^{φ}(m_{∞}^{−1} × 1_{fin}).

Here m_{∞} = (σ_{1}(m), …, σ_{r}(m)) ∈ F_{∞}^{+}. Using (14), it is straightforward to show that

W_{m}^{φ}(m_{∞}^{−1}) = (∏_{j=1}^{r} σ_{j}(m)^{−k_{j}/2})e^{2π tr(m)}e^{−2π tr(1)}W_{m}^{φ}(1).

Substituting this into (21) and using ℕ(m𝔡) = d_{F} ℕ(m) gives the result. □

5. A Hilbert modular Petersson trace formula

Let f = f_{∞} × f_{𝔫} for an ideal (𝔫, 𝔑) = 1, and recall that T_{𝔫} = R(f) is the associated Hecke operator. Let F be any orthogonal basis for A_{k}(𝔑, ω). (Later we will require F to consist of eigenvectors of T_{𝔫}.) Then the kernel of R(f) is given by

K(x, y) = ∑_{φ∈F} \frac{R(f)φ(x)\overline{φ(y)}}{∥φ∥^{2}} = ∑_{γ∈\overline{G}(F)} f(x^{−1}γy),

The first expression is the spectral expansion of the kernel, and the second is the geometric expansion. The equality of the two hinges on the continuity (in x, y) of the geometric expansion. The proof of this continuity given in Prop 3.2. of [Li2] carries over easily to the case of nontrivial central character we consider here.

In this section we apply the technique of Section 2 to the kernel function given above, taking H = N×N. We need to fix a character on N(F)\N(A)×N(F)\N(A), which amounts to choosing two characters on F \A. As discussed earlier, every character on F \A is of the form

θ_{m}(x) = θ(−mx)

for some m ∈ F. Fix m_{1}, m_{2} ∈ F. Our goal is to obtain a trace formula by computing

(22)  ∫_{N(F)\N(A)} ∫_{N(F)\N(A)} K(n_{1}, n_{2})\overline{θ_{m_{1}}(n_{1})}θ_{m_{2}}(n_{2})dn_{1}dn_{2}

with the two expressions for the kernel.

5.1. The spectral side. Using the spectral expansion of the kernel, expression (22) is easily computed in terms of Hecke eigenvalues and Fourier coefficients of cusp forms. Suppose the basis F consists of eigenfunctions of T_{𝔫}. Then for φ ∈ F we have R(f)φ = λ_{𝔫}^{φ}φ for some scalar λ_{𝔫}^{φ} ∈ C. Hence (22) is

= ∑_{φ∈F} \frac{λ_{𝔫}^{φ}}{∥φ∥^{2}} ∫_{N(F)\N(A)} φ(n_{1})\overline{θ_{m_{1}}(n_{1})}dn_{1} ∫_{N(F)\N(A)} \overline{φ(n_{2})}θ_{m_{2}}(n_{2})dn_{2}

(23)  = ∑_{φ∈F} \frac{λ_{𝔫}^{φ} W_{m_{1}}^{φ}(1)\overline{W_{m_{2}}^{φ}(1)}}{∥φ∥^{2}},

as in (10). By Proposition 3.3, the above expression is nonzero only if

m_{1}, m_{2} ∈ 𝔡_{+}^{−1}.

We may assume that this holds from now on.

5.2. The geometric side. Here we use the method of Section 2 to compute (22) using the geometric expansion of the kernel. This gives a sum ∑_{[δ]} I_{δ}(f), where

I_{δ}(f) = ∫_{H_{δ}(F)\H(A)} f(n_{1}^{−1}δn_{2})\overline{θ_{m_{1}}(n_{1})}θ_{m_{2}}(n_{2})dn_{1}dn_{2}.

The orbits [δ] are in one-to-one correspondence with the double cosets

N(F)\\overline{G}(F)/N(F).

Let M be the group of invertible diagonal matrices. The Bruhat decomposition is the following partition of G(F) into two cells:

G(F) = N(F)M(F) ∪ N(F)M(F) (\begin{matrix}0 & 1 \\ 1 & 0 \end{matrix}) N(F).

We call these the first and second Bruhat cells respectively. This gives

N(F)\\overline{G}(F)/N(F) = \{[( \begin{matrix}γ & 0 \\ 0 & 1 \end{matrix} )]| γ ∈ F^{∗}\} ⋃\{[( \begin{matrix}0 & μ \\ 1 & 0 \end{matrix} )]| μ ∈ F^{∗}\}.

We need to determine which of these orbits are relevant in the sense of §2.

First let δ = ( \begin{matrix}γ & 0 \\ 0 & 1 \end{matrix} ) ∈ G(F). If (( _{1}^{1 t_{1}} ), ( _{1}^{1 t_{2}} )) ∈ H_{δ}(A), then

\binom{1 −t_{1}}{1} (\begin{matrix}γ & 0 \\ 0 & 1 \end{matrix}) \binom{1 t_{2}}{1} = z (\begin{matrix}γ & 0 \\ 0 & 1 \end{matrix}) ,

for some z ∈ Z(F). A simple calculation shows that z = 1 and t_{1} = γt_{2}, so

(24)  H_{δ}(A) = \{((\begin{matrix}1 & γt \\ 0 & 1 \end{matrix}) , (\begin{matrix}1 & t \\ 0 & 1 \end{matrix}))| t ∈ A\} .

Thus δ is relevant if and only if

θ((m_{1}γ − m_{2})t) = 1

for all t ∈ A, or equivalently, γ = m_{2}/m_{1} (since m_{1} ∈ 𝔡_{+}^{−1} is nonzero).

On the other hand, if δ = (\begin{matrix}0 & μ \\ 1 & 0 \end{matrix}) ∈ G(F), one sees easily that

H_{δ}(A) = {(e, e)},

so all of these matrices are relevant.

5.2.1. Computation of the first type of I_{δ}. Here we take m_{1}, m_{2} ∈ 𝔡_{+}^{−1}, and δ =

( \begin{matrix}γ \\ & 1 \end{matrix} ) where γ = m_{2}/m_{1}. By (24),

I_{δ}(f) = ∫_{\{(γt,t)∈F^{2}\}\A×A} f(\binom{1 −t_{1}}{1} (\begin{matrix}γ & 0 \\ 0 & 1 \end{matrix}) \binom{1 t_{2}}{1})θ(m_{1}t_{1} − m_{2}t_{2})dt_{1}dt_{2}

= ∫_{\{(γt,t)∈F^{2}\}\A×A} f((\begin{matrix}γ & γt_{2} − t_{1} \\ 0 & 1 \end{matrix}))θ(m_{1}t_{1} − m_{2}t_{2})dt_{1}dt_{2}.

Let t′_{1} = γt_{2} − t_{1} and t′_{2} = t_{2}. Then because m_{1}γ = m_{2}, we have m_{1}t_{1} − m_{2}t_{2} = −m_{1}t′_{1}, so

I_{δ} = ∫_{\{0\}×F \(A×A)} f((\begin{matrix}γ & t′_{1} \\ 0 & 1 \end{matrix}))θ(−m_{1}t′_{1})dt′_{1}dt′_{2}

= meas(F \A) ∫_{A} f((\begin{matrix}m_{2}/m_{1} & t \\ 0 & 1 \end{matrix}))θ(−m_{1}t)dt

= d_{F}^{1/2} ∫_{A} f(\binom{m_{2} m_{1}t}{m_{1}})θ(−m_{1}t)dt

= d_{F}^{1/2} ∫_{A} f((\begin{matrix}m_{2} & t \\ 0 & m_{1} \end{matrix}))θ(−t)dt.

Here we used (6) and the fact that f(zg) = f(g) for z ∈ Z(F).

We factorize the above integral into (I_{δ})_{∞}(I_{δ})_{fin}, and incorporate the coefficient d_{F}^{1/2} into (I_{δ})_{∞}. First we consider

(I_{δ})_{fin} = ∫_{A_{fin}} f_{𝔫}(\binom{m_{2} t}{m_{1}})θ_{fin}(−t)dt.

We shall compute this locally, as we may since f_{𝔫_{v}}(( _{m_{1}}^{m_{2} t} ))θ_{v}(−t)|_{t∈O_{v}} ≡ 1 for almost all v. For any finite place v we have

(I_{δ})_{v} = ∫_{F_{v}} f_{𝔫_{v}}(\binom{m_{2} t}{m_{1}})θ_{v}(−t)dt.

The integrand is nonzero only if

ord_{v}(\frac{m_{1}m_{2}}{s_{v}^{2}} ) = ord_{v}(𝔫_{v})

for some s_{v} ∈ F_{v}^{∗}. Supposing this is the case,

(I_{δ})_{v} = ∫_{F_{v}} f_{𝔫_{v}}((\begin{matrix}s_{v} \\ & s_{v} \end{matrix}) (\frac{m_{2}}{s_{v}} \frac{t}{s_{v}} ; \frac{m_{1}}{s_{v}} ))θ_{v}(−t)dt

is nonzero only if \frac{m_{2}}{s_{v}} , \frac{m_{1}}{s_{v}} , \frac{t}{s_{v}} ∈ O_{v} by Lemma 4.2. Assuming this, we have

(I_{δ})_{v} = ∫_{s_{v}O_{v}} ω_{v}(s_{v})^{−1}f_{𝔫_{v}}((\frac{m_{2}}{s_{v}} \frac{t}{s_{v}} ; \frac{m_{1}}{s_{v}} ))θ_{v}(−t)dt.

Suppose v|𝔑. Then by the definition of f_{𝔫_{v}} (see (17)),

(I_{δ})_{v} = ω_{v}(s_{v})^{−1}ω_{v}(\frac{m_{1}}{s_{v}})^{−1}ψ(𝔑_{v}) ∫_{s_{v}O_{v}} θ_{v}(−t)dt.

The integral vanishes unless s_{v} ∈ 𝔡_{v}^{−1}, in which case its value is |s_{v}|_{v}. Hence

(I_{δ})_{v} = |s_{v}|_{v} ω_{v}(s_{v})^{−1}ω_{v}(\frac{m_{1}}{s_{v}})^{−1}ψ(𝔑_{v}), v|𝔑.

16 ANDREW KNIGHTLY AND CHARLES LI

Now suppose v ∤ 𝔑. Then (I_{δ})_{v} = ω_{v}(s_{v})^{−1} ∫_{s_{v}O_{v}} θ_{v}(−t)dt,

which as before vanishes unless s_{v} ∈ 𝔡_{v}^{−1}. So in this case,

(I_{δ})_{v} = |s_{v}|_{v} ω_{v}(s_{v})^{−1}, v ∤ 𝔑.

Define

(25)  ω_{𝔑,v} = \begin{cases}ω_{v} & if v|𝔑 \\ 1 & if v ∤ 𝔑, \end{cases}

and let ω_{𝔑} = ∏ ω_{𝔑,v} = ∏_{v|𝔑} ω_{v}. Notice that ω_{𝔑} is a character on A_{fin}^{∗}. Multiplying the above local results together, we obtain the following.

Proposition 5.1. Suppose δ = (\begin{matrix}\frac{m_{2}}{m_{1}} & 0 \\ 0 & 1 \end{matrix}) ∈ G(F). The integral (I_{δ})_{fin} is nonzero if and only if there exists a finite idele s ∈ A_{fin}^{∗} such that

• \frac{m_{1}}{s} , \frac{m_{2}}{s} ∈ Ô

• ord_{v}(\frac{m_{1}m_{2}}{s_{v}^{2}} ) = ord_{v}(𝔫) for all v < ∞
• s ∈ 𝔡̂^{−1}.

Under these conditions,

(I_{δ})_{fin} = |s|_{fin}ω_{fin}(s)^{−1}ψ(𝔑)ω_{𝔑}(m_{1}/s)^{−1}

where ψ(𝔑) = [K_{fin} : K_{0}(𝔑)]. Remark: The expression is independent of the choice of s. Indeed if s′ = us for u ∈ Ô^{∗}, then

ω_{fin}(s′)^{−1}ω_{𝔑}(s′) = ∏_{v∤𝔑} ω_{v}(s′_{v}) = ∏_{v∤𝔑} ω_{v}(s_{v}) = ω_{fin}(s)^{−1}ω_{𝔑}(s)

since ω_{v} is unramified for v ∤ 𝔑.

For purposes of computation, the following lemma is helpful and easily verified. Lemma 5.2. With notation as above, (I_{δ})_{fin} is nonzero if and only if for every finite valuation v of F,

• ξ_{v} =^{def} \frac{1}{2} ord_{v}(m_{1}m_{2}𝔫^{−1}) ∈ Z

• ord_{v}(m_{i}) ≥ ξ_{v} ≥ − ord_{v}(𝔡_{v}) for i = 1, 2. If these conditions hold, then we can define s by taking s_{v} = ϖ_{v}^{ξ_{v}} for all v.

For the infinite part, we have the following.

Proposition 5.3. Let δ = (\begin{matrix}\frac{m_{2}}{m_{1}} & 0 \\ 0 & 1 \end{matrix}) with m_{1}, m_{2} ∈ F^{∗}. Then (I_{δ})_{∞} is nonzero if and only if m_{1}, m_{2} ∈ F^{+}. Under this condition,

(I_{δ})_{∞} = \frac{d_{F}^{1/2}}{e^{2π tr_{Q}^{F}(m_{1}+m_{2})}} ∏_{j=1}^{r} \frac{(4π)^{k_{j}−1}σ_{j}(m_{1}m_{2})^{k_{j}/2}}{(k_{j} − 2)!} .

Proof. We have

(I_{δ})_{∞} = d_{F}^{1/2} ∏_{j=1}^{r} ∫_{−∞}^{∞} f_{∞_{j}}(\binom{σ_{j}(m_{2}) t}{σ_{j}(m_{1})})θ_{∞_{j}}(−t)dt.

The integrand is nonzero only if σ_{j}(m_{1}m_{2}) > 0. By the formula for f_{∞_{j}},

∫_{−∞}^{∞} f_{∞_{j}}(\binom{σ_{j}(m_{2}) t}{σ_{j}(m_{1})})θ_{∞_{j}}(−t)dt

= \frac{k_{j} − 1}{4π}σ_{j}(m_{1}m_{2})^{k_{j}/2}(2i)^{k_{j}} ∫_{−∞}^{∞} \frac{e^{2πit}}{(−t + iσ_{j}(m_{1} + m_{2}))^{k_{j}}} dt.

Using a complex contour integral, this is nonzero only if σ_{j}(m_{1} + m_{2}) > 0, in which case it equals

\frac{(4π)^{k_{j}−1}}{(k_{j} − 2)!}σ_{j}(m_{1}m_{2})^{k_{j}/2}e^{−2πσ_{j}(m_{1}+m_{2})}.

Full details are given in [KL1], Proposition 3.4. Because σ_{j}(m_{1}) and σ_{j}(m_{2}) have the same sign, the condition σ_{j}(m_{1} + m_{2}) > 0 is equivalent to σ_{j}(m_{1}), σ_{j}(m_{2}) > 0. □

Proposition 5.4. Let δ = (\begin{matrix}\frac{m_{2}}{m_{1}} & 0 \\ 0 & 1 \end{matrix}) for m_{1}, m_{2} ∈ 𝔡_{+}^{−1}. Then I_{δ}(f) is nonzero if

and only if \frac{m_{1}m_{2}}{s^{2}} Ô = 𝔫̂ and \frac{m_{1}}{s} , \frac{m_{2}}{s} ∈ Ô for some s ∈ 𝔡̂^{−1}. Under this condition,

I_{δ} = [∏_{j=1}^{r} \frac{(4π\sqrt{σ_{j}(m_{1}m_{2})})^{k_{j}−1}}{(k_{j} − 2)!} ] \frac{d_{F}^{1/2}ℕ(𝔫)^{1/2}ψ(𝔑)}{e^{2π tr_{Q}^{F}(m_{1}+m_{2})}ω_{𝔑}(m_{1}/s)ω_{fin}(s)}.

Proof. By Lemma 5.2, we can take s_{v} = ϖ_{v}^{(ord_{v}(m_{1}m_{2})−ord_{v}(𝔫))/2}. Choose n ∈ A_{fin}^{∗} such that nÔ = 𝔫̂. Then

|s|_{fin} = (\frac{|m_{1}m_{2}|_{fin}}{|n|_{fin}} )^{1/2} .

Note that by the product formula and the fact that m_{1}, m_{2} ∈ F^{+},

|m_{1}m_{2}|_{fin} = ∏_{j=1}^{r} σ_{j}(m_{1}m_{2})^{−1}.

Hence

|s|_{fin} = ℕ(𝔫)^{1/2} ∏_{j=1}^{r} σ_{j}(m_{1}m_{2})^{−1/2}.

The proposition now follows immediately upon multiplying (I_{δ})_{fin} and (I_{δ})_{∞} together. □

5.2.2. Computation of the second type of I_{δ}. Let v < ∞ be a finite valuation of F. Let n_{v} ∈ O_{v} − {0} and let m_{1v}, m_{2v} ∈ 𝔡_{v}^{−1}. Recall that the conductor of ω_{v} divides 𝔑_{v}. For c_{v} ∈ 𝔑_{v} − {0}, define a generalized (local) Kloosterman sum by

S_{ω_{v}}(m_{1v}, m_{2v}; n_{v}; c_{v}) = ∑_{{s_{1},s_{2}∈Ov /cv Ov \atop s_{1}s_{2}≡nv mod cv Ov}} θ_{v}(\frac{m_{1v}s_{1} + m_{2v}s_{2}}{c_{v}})ω_{v}(s_{2})^{−1}.

The value of the sum is 1 if c_{v} ∈ O_{v}^{∗}. (The above notation is not quite consistent with [KL1], which has ω_{v}(s_{1})^{−1}.)

For n ∈ Ô ∩ A_{fin}^{∗}, c ∈ 𝔑̂ ∩ A_{fin}^{∗}, m_{1}, m_{2} ∈ 𝔡̂^{−1}, and ω_{𝔑} as in (25), we define

S_{ω_{𝔑}}(m_{1}, m_{2}; n; c) = ∑_{{s_{1},s_{2}∈Ô/cÔ \atop s_{1}s_{2}≡n mod cÔ}} θ_{fin}(\frac{m_{1}s_{1} + m_{2}s_{2}}{c} )ω_{𝔑}(s_{2})^{−1}.

Then

S_{ω_{𝔑}}(m_{1}, m_{2}; n; c) = ∏_{v<∞} S_{ω_{𝔑,v}}(m_{1v}, m_{2v}; n_{v}; c_{v}).

When δ = ( \begin{matrix}0 & μ \\ 1 & 0 \end{matrix} ), we have seen that H_{δ} = \{(e, e)\}, so

I_{δ}(f) = ∫∫_{N(A)×N(A)} f(n_{1}^{−1} (\begin{matrix}0 & μ \\ 1 & 0 \end{matrix}) n_{2}) \overline{θ_{m_{1}}(n_{1})} θ_{m_{2}}(n_{2})dn_{1}dn_{2}.

We will compute this locally. Write n_{i} = ( _{1}^{1 t_{i}} ), i = 1, 2. Then

(26)  n_{1}^{−1} (\begin{matrix}0 & μ \\ 1 & 0 \end{matrix}) n_{2} = (\begin{matrix}−t_{1} & μ − t_{1}t_{2} \\ 1 & t_{2} \end{matrix}) .

Proposition 5.5. Let m_{1}, m_{2} ∈ 𝔡^{−1}. If δ = (\begin{matrix}& μ \\ 1 \end{matrix}), then (I_{δ})_{fin} is nonzero only

if c^{2}μÔ = 𝔫̂ for some c ∈ 𝔑̂. Under this condition, let n = −c^{2}μ be a generator of 𝔫̂ (the negative sign is for convenience). Then

(I_{δ})_{fin} = [∏_{j=1}^{r}(−1)^{k_{j}} ] ψ(𝔑)ω_{fin}(c)S_{ω_{𝔑}}(m_{1}, m_{2}; n; c).

Proof. For a discrete place v,

(I_{δ})_{v} = ∫∫_{F_{v}×F_{v}} f_{𝔫_{v}}((\begin{matrix}−t_{1} & μ − t_{1}t_{2} \\ 1 & t_{2} \end{matrix}))θ_{v}(m_{1}t_{1} − m_{2}t_{2})dt_{1}dt_{2}.

The integrand is nonzero only if there exists c_{v} ∈ F_{v}^{∗} such that

(\begin{matrix}c_{v} \\ & c_{v} \end{matrix}) (\begin{matrix}−t_{1} & μ − t_{1}t_{2} \\ 1 & t_{2} \end{matrix}) ∈ M(𝔫_{v}, 𝔑_{v}).

This means (1) c_{v} ∈ 𝔑_{v}

(2) c_{v}t_{1}, c_{v}t_{2} ∈ O_{v}
(3) ord_{v}(c_{v}^{2}μ) = ord_{v}(𝔫_{v}).

Note that the first and third conditions determine c_{v} up to units. By the third condition, c_{v}^{2}μ = −n_{v} for some generator n_{v} ∈ 𝔫_{v}. Now make substitutions in the integral by replacing t_{1} and t_{2} by \frac{1}{c_{v}} t_{1} and \frac{1}{c_{v}} t_{2} respectively. Then

(I_{δ})_{v} = |c_{v}|_{v}^{−2} ∫∫_{O_{v}×O_{v}} f_{𝔫_{v}}(c_{v}^{−1} (\begin{matrix}−t_{1} & \frac{−n_{v}−t_{1}t_{2}}{c_{v}} \\ c_{v} & t_{2} \end{matrix} ))θ_{v}(\frac{m_{1}t_{1} − m_{2}t_{2}}{c_{v}})dt_{1}dt_{2}.

The integrand is nonzero if and only if (1) c_{v} ∈ 𝔑_{v}

(2) t_{1}t_{2} ≡ −n_{v} mod c_{v}O_{v}.

Assuming these hold, the value

f_{𝔫_{v}}(c_{v}^{−1} (\begin{matrix}−t_{1} & \frac{−n_{v}−t_{1}t_{2}}{c_{v}} \\ c_{v} & t_{2} \end{matrix} )) = \{ \begin{matrix}ω_{v}(c_{v})ψ(𝔑_{v})ω_{v}(t_{2})^{−1} & if v|𝔑 \\ ω_{v}(c_{v}) & otherwise \end{matrix}

depends only on the residue class of t_{2} modulo 𝔑_{v}. Furthermore, because θ_{v} is trivial on 𝔡_{v}^{−1} and m_{1}, m_{2} ∈ 𝔡^{−1}, the value θ_{v}(\frac{m_{1}t_{1}−m_{2}t_{2}}{c_{v}}) depends only on the cosets t_{1} + c_{v}O_{v} and t_{2} + c_{v}O_{v}. Thus the entire integrand is constant on cosets of c_{v}O_{v}. Each of these cosets has measure |c_{v}|_{v}, so these measures for t_{1} and t_{2} will cancel the coefficient |c_{v}|_{v}^{−2} in the integral. Therefore

(I_{δ})_{v} = \{ \begin{matrix}ψ(𝔑_{v})ω_{v}(c_{v}) ∑_{{s_{1},s_{2}∈Ov /cv Ov \atop s_{1}s_{2}≡−nv mod cv Ov}} ω_{v}(s_{2})^{−1}θ_{v}(\frac{m_{1}s_{1} − m_{2}s_{2}}{c_{v}}) & if v|𝔑 \\ ω_{v}(c_{v}) ∑_{{s_{1},s_{2}∈Ov /cv Ov \atop s_{1}s_{2}≡−nv mod cv Ov}} θ_{v}(\frac{m_{1}s_{1} − m_{2}s_{2}}{c_{v}}) & if v ∤ 𝔑. \end{matrix}

Replacing s_{2} by −s_{2}, in either case we see that

(I_{δ})_{v} = ψ(𝔑_{v})ω_{𝔑,v}(−1)ω_{v}(c_{v})S_{ω_{𝔑,v}}(m_{1}, m_{2}; n_{v}; c_{v}).

Multiplying the local results together, we obtain

(I_{δ})_{fin} = ψ(𝔑)ω_{𝔑}(−1)ω_{fin}(c)S_{ω_{𝔑}}(m_{1}, m_{2}; n; c).

The final point is that because ω(−1) = 1 and ω_{v} is unramified for v ∤ 𝔑,

ω_{𝔑}(−1) = ∏_{v|𝔑} ω_{v}(−1) = ∏_{v<∞} ω_{v}(−1) = ω_{∞}(−1)^{−1} = ∏_{j=1}^{r}(−1)^{k_{j}}.

□

Proposition 5.6. Let δ = (\begin{matrix}& μ \\ 1 \end{matrix}) ∈ G(F). Then

(27)  (I_{δ})_{∞} = ∫∫_{F_{∞}×F_{∞}} f_{∞}((\begin{matrix}−t_{1} & μ − t_{1}t_{2} \\ 1 & t_{2} \end{matrix}))θ_{∞}(m_{1}t_{1} − m_{2}t_{2})dt_{1}dt_{2}

is nonzero only if m_{1}, m_{2}, −μ are all totally positive. Under these conditions,

(I_{δ})_{∞} = \frac{ℕ(−μ)^{1/2}}{e^{2π tr_{Q}^{F}(m_{1}+m_{2})}} ∏_{j=1}^{r} \frac{(4πi)^{k_{j}} \sqrt{σ_{j}(m_{1}m_{2})}^{k_{j}−1}J_{k_{j}−1}(4π\sqrt{−σ_{j}(μm_{1}m_{2})})}{2(k_{j} − 2)!}

where J_{k} is the Bessel J-function.

Proof. The computation of (I_{δ})_{∞_{j}} is given in [KL1] Proposition 3.6, and the above is the product of these local computations. Note that since −μ ∈ F^{+}, ℕ(−μ) = N(−μ) = ∏_{j=1}^{r} σ_{j}(−μ). □

Multiplying the above results together, we obtain the following.

Proposition 5.7. Let δ = (\begin{matrix}& μ \\ 1 \end{matrix}) ∈ G(F), and m_{1}, m_{2} ∈ 𝔡_{+}^{−1}. Then I_{δ} is nonzero

only if (1) −μ is totally positive

(2) c^{2}μÔ = 𝔫̂ for some c ∈ 𝔑̂ ∩ A_{fin}^{∗}. Under these conditions, letting n = −c^{2}μ, we have

I_{δ} = ψ(𝔑)ω_{fin}(c)S_{ω_{𝔑}}(m_{1}, m_{2}; n; c)\frac{ℕ(−μ)^{1/2}}{2^{r}e^{2π tr_{Q}^{F}(m_{1}+m_{2})}}

× ∏_{j=1}^{r} \frac{(−4πi)^{k_{j}} \sqrt{σ_{j}(m_{1}m_{2})}^{k_{j}−1}}{(k_{j} − 2)!} J_{k_{j}−1}(4π\sqrt{−σ_{j}(μm_{1}m_{2})}).

5.2.3. Total contribution of the second Bruhat cell. The total contribution of the second type of δ is a summation of I_{δ} over all μ ∈ F^{∗} satisfying the conditions in Proposition 5.7. For this we need to give a more systematic description of the set of such μ.

Let c ∈ A_{fin}^{∗}, and let 𝔠 = cÔ ∩ F be the associated fractional ideal of F. Then condition (2) of Proposition 5.7 is equivalent to

(28)  𝔠^{2}(μ) = 𝔫

and

(29)  𝔑|𝔠.

We need to determine the elements μ which satisfy (28) for some 𝔠 as in (29). Consider the following equation in the ideal class group

(30)  1 = [𝔟]^{2}[𝔫].

If no solution 𝔟 exists, then the contribution of this Bruhat cell is 0. Some examples are given at the end of this section. Otherwise, let

[𝔟_{1}], [𝔟_{2}], …, [𝔟_{t}]

be the distinct solutions of (30). We take the 𝔟_{i} to be integral ideals. Because 𝔟_{i}^{2}𝔫 is principal, there exists a nonzero element η_{i} ∈ O such that

(31)  (η_{i}) = 𝔟_{i}^{2}𝔫.

We fix such generators η_{1}, …, η_{t} once and for all.

Suppose (28) holds. Then [𝔠]^{−1} = [𝔟_{i}] for some i, so 𝔠 = s𝔟_{i}^{−1} for some s ∈ F^{∗}. Substituting 𝔠 = s𝔟_{i}^{−1} into (28), we see that (μ) = (s^{−2}η_{i}), i.e. μ = \frac{−η_{i}u}{s^{2}} for some u ∈ O^{∗}. Conversely if μ = \frac{−η_{i}u}{s^{2}} for some s ∈ F^{∗}, then it satisfies (28) with 𝔠 = s𝔟_{i}^{−1}.

For each i = 1, …, t fix a generator b_{i} ∈ 𝔟̂_{i}. We obtain the following lemma:

Lemma 5.8. An element μ ∈ F^{∗} satisfies condition (2) of Proposition 5.7 if and only if

(32)  μ = \frac{−η_{i}u}{s^{2}}

for some i ∈ {1, …, t}, u ∈ O^{∗} and s ∈ 𝔟_{i}𝔑. If this condition holds, then we can take c = sb_{i}^{−1} and n = η_{i}ub_{i}^{−2} in Proposition 5.7.

Proof. The above discussion shows that μ satisfies (28) if and only if it is given by (32) where 𝔠 = s𝔟_{i}^{−1}. Then it is easy to check that 𝔑|𝔠 if and only if s ∈ 𝔟_{i}𝔑. The last part of the lemma follows by a simple calculation. □

Lemma 5.9. In the notation above, if

\frac{−η_{i}u}{s^{2}} = \frac{−η_{i′}u′}{s′^{2}} ,

then i = i′ and uO^{∗2} = u′O^{∗2}. Furthermore if we assume u, u′ ∈ U as in (4), then u = u′ and s = ±s′.

Proof. By (31), we have 𝔟_{i}^{2}𝔫s^{−2} = 𝔟_{i′}^{2}𝔫s′^{−2}. Therefore (𝔟_{i}𝔟_{i′}^{−1})^{2} = (ss^{′−1})^{2}. By unique factorization of ideals, we have 𝔟_{i}𝔟_{i′}^{−1} = (ss^{′−1}). Therefore [𝔟_{i}] = [𝔟_{i′}], and hence i = i′. With i = i′, we have uu^{′−1} = (ss^{′−1})^{2}. This implies that uu^{′−1} ∈ O^{∗} ∩ F^{∗2} = O^{∗2}. Then if u, u′ ∈ U, it is immediate that u = u′, and hence s = ±s′. □

Proposition 5.10. The total contribution of the second Bruhat cell is

(33)  \frac{ψ(𝔑)}{2^{r}e^{2π tr_{Q}^{F}(m_{1}+m_{2})}} ∏_{j=1}^{r} \frac{(−4πi)^{k_{j}} \sqrt{σ_{j}(m_{1}m_{2})}^{k_{j}−1}}{(k_{j} − 2)!}

× ∑_{i=1}^{t} ∑_{{u∈U \atop η_{i}u∈F^{+}}} ∑_{{s∈𝔟_{i}𝔑/± \atop s≠0}} \{ω_{fin}(sb_{i}^{−1})S_{ω_{𝔑}}(m_{1}, m_{2}; η_{i}ub_{i}^{−2}; sb_{i}^{−1})\frac{ℕ(η_{i}u)^{1/2}}{ℕ(s)}

× ∏_{j=1}^{r} J_{k_{j}−1}(4π \frac{\sqrt{σ_{j}(η_{i}um_{1}m_{2})}}{|σ_{j}(s)|})\} .

5.3. Main result. For φ ∈ A_{k}(𝔑, ω) and m ∈ 𝔡_{+}^{−1}, let W_{m}^{φ} denote the Fourier coefficient of φ as in §3.4. If φ is an eigenfunction of the Hecke operator T_{𝔫} = R(f), we write

T_{𝔫}φ = λ_{𝔫}^{φ}φ.

Equating the geometric and spectral computations of the previous sections, we obtain the following upon multiplying both sides by

\frac{e^{2π tr_{Q}^{F}(m_{1}+m_{2})}}{ψ(𝔑)} ∏_{j=1}^{r} \frac{(k_{j} − 2)!}{(4π\sqrt{σ_{j}(m_{1}m_{2})})^{k_{j}−1}} .

Theorem 5.11. Let 𝔫 and 𝔑 be integral ideals with (𝔫, 𝔑) = 1. Let k = (k_{1}, …, k_{r}) with all k_{j} > 2. Let F be an orthogonal basis for A_{k}(𝔑, ω) consisting of eigenfunctions for the Hecke operator T_{𝔫}. Then for any m_{1}, m_{2} ∈ 𝔡_{+}^{−1},

\frac{e^{2π tr_{Q}^{F}(m_{1}+m_{2})}}{ψ(𝔑)} [∏_{j=1}^{r} \frac{(k_{j} − 2)!}{(4π\sqrt{σ_{j}(m_{1}m_{2})})^{k_{j}−1}} ] ∑_{φ∈F} \frac{λ_{𝔫}^{φ}W_{m_{1}}^{φ}(1)\overline{W_{m_{2}}^{φ}(1)}}{∥φ∥^{2}}

= T̂(m_{1}, m_{2}, 𝔫)\frac{d_{F}^{1/2}ℕ(𝔫)^{1/2}}{ω_{𝔑}(m_{1}/s)ω_{fin}(s)}

+ ∑_{i=1}^{t} ∑_{{u∈U \atop η_{i}u∈F^{+}}} ∑_{{s∈𝔟_{i}𝔑/± \atop s≠0}} \{ω_{fin}(sb_{i}^{−1})S_{ω_{𝔑}}(m_{1}, m_{2}; η_{i}ub_{i}^{−2}; sb_{i}^{−1})\frac{ℕ(η_{i}u)^{1/2}}{ℕ(s)}

× ∏_{j=1}^{r} \frac{2π}{(\sqrt{−1})^{k_{j}}} J_{k_{j}−1}(4π \frac{\sqrt{σ_{j}(η_{i}um_{1}m_{2})}}{|σ_{j}(s)|})\} ,

where:

• T̂(m_{1}, m_{2}, 𝔫) ∈ {0, 1} is nonzero if and only if there exists s ∈ 𝔡̂^{−1} such that m_{1}, m_{2} ∈ sÔ and m_{1}m_{2}Ô = s^{2}𝔫̂ (see also Lemma 5.2)

• U is a set of representatives for O^{∗}/O^{∗2}

• b_{i}Ô = 𝔟̂_{i} for 𝔟_{i} as in (30) (for i = 1, …, t)

• η_{i} ∈ F generates the principal ideal 𝔟_{i}^{2}𝔫

• ω_{𝔑} = ∏_{v|𝔑} ω_{v} × ∏_{v∤𝔑} 1.

Remarks: (1) The above represents only part of a larger picture because we have treated just those Fourier coefficients coming from the identity component of \overline{G}(F)\\overline{G}(A)/K_{∞}K_{0}(𝔑). The general case would result from integrating

K(( \begin{matrix}y_{j} \\ & 1 \end{matrix} )n_{1}, ( \begin{matrix}y_{i} \\ & 1 \end{matrix} )n_{2}),

where y_{1}, …, y_{h} ∈ A_{fin}^{∗} are representatives for the class group of F.

(2) We can choose the basis so that each φ ∈ V_{π} for some cuspidal representation π. Then the eigenvalue λ_{𝔫}^{φ} depends only on π, and not on φ. This can be seen from (38) and (39) below.

Example 5.12. Suppose F = Q[\sqrt{d}] is a real quadratic field with narrow class number 1 (e.g. d = 2, 5).

For such fields, the fundamental unit ε has negative norm. We can take U = {±1, ±ε}, and 𝔫 = (η), with η ∈ F^{+}. See Theorem 7.2 below.

Example 5.13. Let F = Q[\sqrt{d}] be a real quadratic field with class number 1 and narrow class number 2 (e.g. d = 3).

In this case, the fundamental unit ε has positive norm. If 𝔫 does not have a totally positive generator (i.e. if 𝔫 = (η) is nontrivial in the narrow class group), then there is no u ∈ U for which ηu ∈ F^{+}. Thus for such 𝔫, the entire Kloosterman term in the above theorem vanishes.

For general F, η_{i} can only make a nontrivial contribution to the Kloosterman term if it satisfies the sign condition

(sgn η_{i}^{σ_{1}}, …, sgn η_{i}^{σ_{r}}) = ±(sgn ε^{σ_{1}}, …, sgn ε^{σ_{r}})

for some unit ε in a fundamental unit system. We now give some examples to illustrate various possibilities for the ideals 𝔟_{i}.

Example 5.14. F has class number 2.

In the class group Cl(F), the equation [𝔟]^{2}[𝔫] = 1 has solutions if and only if [𝔫] is trivial (since x^{2} = 1 for all x ∈ Cl(F)). Hence the Kloosterman term is nonzero only if 𝔫 = (η) is a principal ideal. In this case, we can take 𝔟_{1} to be any non-principal ideal and 𝔟_{2} = O.

Example 5.15. F has class number 3.

Now we have x^{2} = x^{−1} for all x ∈ Cl(F). Therefore the equation [𝔟]^{2}[𝔫] = 1 has a unique solution in Cl(F), and we can take 𝔟 = 𝔫.

Example 5.16. F has odd class number.

Suppose h = | Cl(F)| is odd. Then the equation [𝔟]^{2}[𝔫] = 1 has a unique solution. For example we can take 𝔟 = 𝔫^{(h−1)/2}.

6. Weighted distribution of Hecke eigenvalues

As an application of Theorem 5.11, we will show that relative to a certain measure, the (normalized) eigenvalues of the Hecke operator T_{𝔭} have a weighted equidistribution in the interval [−2, 2] as ℕ(𝔑) → ∞.

6.1. Estimates. As a function of 𝔑, the contribution of the first Bruhat cell (given in Proposition 5.4) has order

ψ(𝔑) = [K_{fin} : K_{0}(𝔑)] = ℕ(𝔑) ∏_{𝔭|𝔑}(1 + \frac{1}{ℕ𝔭}).

Here we will show that this is the dominant term in the Petersson trace formula as ℕ(𝔑) → ∞. For this, we need to show that the contribution (33) of the second Bruhat cell is small in comparison.

We start with the following naive estimate (essentially the triangle inequality) for the Kloosterman sums.

Lemma 6.1. For any m_{1}, m_{2} ∈ 𝔡^{−1}, nonzero n ∈ Ô and c ∈ 𝔑̂ ∩ A_{fin}^{∗},

|S_{ω_{𝔑}}(m_{1}, m_{2}; n; c)| ≤ ℕ(n)ℕ(c).

Proof. It suffices to prove the lemma locally. Note that

ℕ(c) = ∏_{v<∞} ℕ(c_{v}) = ∏_{v<∞} |O_{v}/c_{v}O_{v}|.

We have

|S_{ω_{𝔑,v}}(m_{1}, m_{2}; n_{v}; c_{v})|
≤ ∑_{{s_{1},s_{2}∈Ov /cv Ov \atop s_{1}s_{2}≡nv mod cv Ov}} |θ_{v}(\frac{m_{1}s_{1} + m_{2}s_{2}}{c_{v}})ω_{𝔑,v}(s_{2})^{−1}| = ∑_{{s_{1},s_{2}∈Ov /cv Ov \atop s_{1}s_{2}≡nv mod cv Ov}} 1.

In the case where ord_{v}(n_{v}) > ord_{v}(c_{v}), we can replace n_{v} by c_{v} without loss of generality. Thus assuming 0 ≤ ord_{v}(n_{v}) ≤ ord_{v}(c_{v}), we need to count the elements of the following set:

(34)  {(s_{1}, s_{2}) ∈ (O_{v}/c_{v}O_{v})^{2}| s_{1}s_{2} ≡ n_{v} mod c_{v}O_{v}}.

For a fixed s_{1} ∈ O_{v}/c_{v}O_{v} (we can assume that 0 ≤ ord_{v}(s_{1}) ≤ ord_{v}(c_{v})), there exists a solution s_{2} to the congruence if and only if ord_{v}(s_{1}) ≤ ord_{v}(n_{v}). If this condition holds, then the number of solutions s_{2} is

|\frac{c_{v}}{s_{1}} O_{v}/c_{v}O_{v}| = |O_{v}/s_{1}O_{v}| = ℕ(s_{1}).

Thus the cardinality of (34) is

∑_{s∈Ov /cv Ov ordv (s)≤ordv (nv)} ℕ(s) = ∑_{ℓ=0}^{ord_{v} n_{v}} ℕ(ϖ_{v}^{ℓ}) · |(O_{v}/\frac{c_{v}}{ϖ_{v}^{ℓ}} O_{v})^{∗}|

(35)  ≤ ∑_{ℓ=0}^{ord_{v} n_{v}} ℕ(ϖ_{v}^{ℓ})ℕ(c_{v}/ϖ_{v}^{ℓ}) = ℕ(c_{v})(ord_{v}(n_{v}) + 1) ≤ ℕ(c_{v})ℕ(n_{v})

as desired. □

Proposition 6.2. As a function of 𝔑, the contribution (33) of the second Bruhat cell is ≪ \frac{ψ(𝔑)}{ℕ(𝔑)^{2−ε}} for any 0 < ε < 1 as ℕ(𝔑) → ∞. Here the implied constant depends on 𝔫, k and ε.

Remark: The middle expression of (35) is ≪_{ε′} ℕ(c_{v})ℕ(n_{v})^{ε′} for any ε′ > 0. Using this, the dependence of (33) on 𝔫 can easily be shown to be ≪ ℕ(𝔫)^{3/2+ε′−ε/2} for 0 < ε < 1.

Proof. By Lemma 6.1, (33) is

≪ ψ(𝔑) ∑_{i=1}^{t} ∑_{{u∈U \atop η_{i}u∈F^{+}}} ∑_{{s∈𝔟_{i}𝔑/± \atop s≠0}} \frac{ℕ(η_{i}u)ℕ(s)}{ℕ(𝔟_{i})^{3}} \frac{ℕ(η_{i}u)^{1/2}}{ℕ(s)} ∏_{j=1}^{r} J_{k_{j}−1}(\frac{4π\sqrt{σ_{j}(η_{i}um_{1}m_{2})}}{|σ_{j}(s)|})

≪ ψ(𝔑) ∑_{i=1}^{t} ∑_{{u∈U \atop η_{i}u∈F^{+}}} ∑_{{s∈𝔟_{i}𝔑/O^{∗} \atop s≠0}} ∑_{a∈O^{∗}/\{±1\}} ∏_{j=1}^{r} J_{k_{j}−1}(\frac{4π\sqrt{σ_{j}(η_{i}um_{1}m_{2})}}{|σ_{j}(s)||σ_{j}(a)|}).

We remark that |σ_{j}(s)| is not well-defined for s ∈ 𝔟_{i}𝔑/O^{∗}, so the summands depend on a choice of representatives s, which we regard as fixed. By the units theorem, the set

{(log |σ_{1}(a)|, …, log |σ_{r}(a)|) : a ∈ O^{∗}}

is a full lattice Λ in the (r−1)-dimensional hyperplane x_{1}+⋯+x_{r} = 0 in R^{r}. Thus the summation over a ∈ O^{∗}/{±1} can be replaced by a sum over λ = (λ_{1}, …, λ_{r}) ∈ Λ. Recall that

J_{k_{j}−1}(x) ≪ min(|x|^{−1/2}, |x|^{k_{j}−1}) ≤ \frac{1}{max(|x|^{1/2}, |x|^{−2})} ≤ \frac{2}{|x|^{1/2} + |x|^{−2}} .

As a result, if we let τ_{j} = log |\frac{σ_{j}(s)}{4π\sqrt{σ_{j}(η_{i}um_{1}m_{2})}} | and λ_{j} = log |σ_{j}(a)|, we can bound

J_{k_{j}−1}(e^{−(λ_{j}+τ_{j})}) to give

∑_{a∈O^{∗}/\{±1\}} ∏_{j=1}^{r} J_{k_{j}−1}(\frac{4π\sqrt{σ_{j}(η_{i}um_{1}m_{2})}}{|σ_{j}(s)||σ_{j}(a)|}) ≪ ∑_{λ∈Λ} \frac{1}{∏_{j=1}^{r}(e^{−(λ_{j}+τ_{j})/2} + e^{2(λ_{j}+τ_{j})})}.

Let β = 2 − ε for 0 < ε < 1. Using ∑_{j} λ_{j} = 0, the right-hand sum is

= ∑_{λ∈Λ} \frac{1}{∏_{j=1}^{r} e^{β(λ_{j}+τ_{j})} ∏_{j=1}^{r}(e^{−(1/2+β)(λ_{j}+τ_{j})} + e^{ε(λ_{j}+τ_{j})})}

= ∑_{λ∈Λ} \frac{1}{e^{(∑_{j=1}^{r} τ_{j})β} ∏_{j=1}^{r}(e^{−(1/2+β)(λ_{j}+τ_{j})} + e^{ε(λ_{j}+τ_{j})})}

= |\frac{(4π)^{r}\sqrt{ℕ(η_{i}um_{1}m_{2})}}{ℕ(s)}|^{β} ∑_{λ∈Λ} \frac{1}{∏_{j=1}^{r}(e^{−(1/2+β)(λ_{j}+τ_{j})} + e^{ε(λ_{j}+τ_{j})})}.

We claim that this last sum over λ is bounded, independently of s. Let L_{0} be the hyperplane x_{1} + ⋯ + x_{r} = 0 containing Λ. Then

∑_{λ∈Λ} \frac{1}{∏_{j=1}^{r}(e^{−(1/2+β)(λ_{j}+τ_{j})} + e^{ε(λ_{j}+τ_{j})})} ≪ ∫_{L_{0}} \frac{dx_{1} ⋯ dx_{r−1}}{∏_{j=1}^{r}(e^{−(1/2+β)(x_{j}+τ_{j})} + e^{ε(x_{j}+τ_{j})})}.

The r^{th} factor of the integrand is bounded, so the above is

≪ ∫_{R^{r−1}} \frac{dx_{1} ⋯ dx_{r−1}}{∏_{j=1}^{r−1}(e^{−(1/2+β)(x_{j}+τ_{j})} + e^{ε(x_{j}+τ_{j})})} = ∏_{j=1}^{r−1} ∫_{R} \frac{dx}{e^{−(1/2+β)x} + e^{εx}} < ∞

as claimed. Therefore (33) is

≪ ψ(𝔑) ∑_{i=1}^{t} ∑_{{u∈U \atop η_{i}u∈F^{+}}} ∑_{{s∈𝔟_{i}𝔑/O^{∗} \atop s≠0}} |\frac{(4π)^{r}\sqrt{ℕ(η_{i}um_{1}m_{2})}}{ℕ(s)}|^{2−ε} .

The first two sums are taken over finite sets. The set of s ∈ 𝔟_{i}𝔑/O^{∗} is in 1-1 correspondence with the set of principal ideals (s) divisible by 𝔟_{i}𝔑. Hence we can replace the sum over s by a larger set: all integral ideals 𝔞 divisible by 𝔑. Therefore (33) is

≪ ψ(𝔑) ∑_{𝔑|𝔞} \frac{1}{ℕ(𝔞)^{2−ε}} ≪ \frac{ψ(𝔑)}{ℕ(𝔑)^{2−ε}} ∑_{𝔞′} \frac{1}{ℕ(𝔞′)^{2−ε}} ≪ \frac{ψ(𝔑)}{ℕ(𝔑)^{2−ε}} .

(We use the absolute convergence of the zeta function ζ_{F} (s) = ∑_{𝔞′} \frac{1}{ℕ(𝔞′)^{s}} for Re s > 1.) □

6.2. Weighted equidistribution. Let (X, μ) be a Borel measure space. For each i = 1, 2, … let F_{i} be a finite nonempty index set, and let S_{i} = {x_{ij}}_{j∈F_{i}} be a finite sequence of points of X. Suppose each j ∈ F_{i} is assigned a weight w_{ij} ∈ R^{+}. Define

dμ_{i} = \frac{∑_{j∈F_{i}} w_{ij}δ_{x_{ij}}}{∑_{j∈F_{i}} w_{ij}} ,

where δ_{x_{ij}} is the Dirac measure at x_{ij}. We say that the sequence {S_{i}} is wequidistributed with respect to the measure dμ if

lim_{i→∞} dμ_{i} = lim_{i→∞} \frac{∑_{j∈F_{i}} w_{ij}δ_{x_{ij}}}{∑_{j∈F_{i}} w_{ij}} = dμ.

This means that for any continuous function f : X → C, we have

lim_{i→∞} ∫_{X} f(x)dμ_{i}(x) = lim_{i→∞} \frac{∑_{j∈F_{i}} w_{ij}f(x_{ij})}{∑_{j∈F_{i}} w_{ij}} = ∫_{X} f(x)dμ(x).

If w_{ij} = 1 for all i, j, then this definition reduces to that of equidistribution given in §1 of [Se].

6.3. The distribution theorem. Let 𝔭 be a prime ideal, not dividing the level 𝔑. In this section we will apply the main theorem in the case where 𝔫 = 𝔭^{ℓ} (ℓ ≥ 0), and m = m_{1} = m_{2} ∈ 𝔡_{+}^{−1}.

For each irreducible summand π of H_{k}(𝔑, ω) (cf. (7)), let F_{π} be an orthogonal basis for the nonzero finite-dimensional subspace π ∩ A_{k}(𝔑, ω). Let

(36)  F = ⋃_{π} F_{π}

be the resulting orthogonal basis for A_{k}(𝔑, ω).

Lemma 6.3. For any φ ∈ F and any prime ideal 𝔭 ∤ 𝔑, the cuspidal representation (π, V) containing φ is unramified at 𝔭. Furthermore, φ is an eigenfunction of the global Hecke operator T_{𝔭^{ℓ}}, and the associated eigenvalue λ_{𝔭^{ℓ}}^{φ} coincides with the local eigenvalue λ_{𝔭_{v}^{ℓ}} attached to π_{𝔭} in Prop. 4.4.

Proof. Write π_{fin} = π_{𝔭}⊗π′, where π′ = ⊗_{{v<∞ \atop v≠v𝔭}} π_{v} is a representation of the restricted

direct product G′ = ∏_{{v<∞ \atop v≠v𝔭}}′G_{v}. Let K′ = G′ ∩ K_{1}(𝔑). Let f = f_{𝔫} with 𝔫 = (1). Then

π_{fin}(f) is the projection operator of V_{fin} onto π_{fin}^{K_{1}(𝔑)}, and

(37)  π_{fin}^{K_{1}(𝔑)} = π_{fin}(f)V_{fin} = π_{𝔭}(f_{v_{𝔭}})V_{𝔭} ⊗ π′(f′)V′ = π_{𝔭}^{K_{𝔭}} ⊗ π′^{K′}.

The middle equality holds e.g. by Prop. 13.17 of [KL2].

Now take 𝔫 = 𝔭^{ℓ} and f = f_{∞} × f_{𝔫}. By the definition of A_{k}(𝔑, ω) (eq. (8)) we can write φ = w_{∞} ⊗ w_{fin}, where w_{∞} = ⊗v_{π_{∞_{j}}} and w_{fin} ∈ π_{fin}^{K_{1}(𝔑)}. Because

0 ≠ w_{fin} ∈ π_{fin}^{K_{1}(𝔑)}, it follows immediately from (37) that π_{𝔭} is unramified. Because π_{𝔭} is also unitary, it follows that π_{𝔭} = π_{χ} is induced from some unramified character χ(( \begin{matrix}a & b \\ 0 & d \end{matrix} )) = χ_{1}(a)χ_{2}(d) of B(F_{v}). Therefore π_{𝔭}^{K_{𝔭}} = Cϕ_{0} is one-dimensional (for notation see (18)), and we can write w_{fin} = ϕ_{0} ⊗ w′ as in (37). Thus φ = w_{∞} ⊗ ϕ_{0} ⊗ w′, and

T_{𝔭^{ℓ}}φ = R(f)φ = π_{∞}(f_{∞})w_{∞} ⊗ π_{𝔭}(f_{v_{𝔭}})ϕ_{0} ⊗ π′(f′)w′
= w_{∞} ⊗ λ_{𝔭_{v}^{ℓ}}ϕ_{0} ⊗ w′ = λ_{𝔭_{v}^{ℓ}}φ.
□

The local Langlands class of π_{𝔭} is the GL_{2}(C)-conjugacy class of the matrix g(π_{𝔭}) = ( \begin{matrix}χ_{1}(ϖ_{𝔭}) \\ & χ_{2}(ϖ_{𝔭}) \end{matrix} ). Let q = ℕ(𝔭). By the above lemma and Prop. 4.4, the trace of g(π_{𝔭}) is given by

(38)  χ_{1}(ϖ_{𝔭}) + χ_{2}(ϖ_{𝔭}) = q^{−1/2}λ_{𝔭}^{φ}.

If χ is unitary (i.e. π_{𝔭} is not complementary series), then it is clear that |λ_{𝔭}^{φ}| ≤ 2q^{1/2}.

In fact by the Ramanujan conjecture this is always the case.^{1}

When 𝔫 = 𝔭^{ℓ}, the operator ω_{𝔭}(ϖ_{𝔭})^{−ℓ/2}R(f) is self-adjoint (cf. Prop. 4.3), so its eigenvalues are real numbers. We let

ν_{𝔭^{ℓ}}^{φ} =^{def} ω_{𝔭}(ϖ_{𝔭})^{−ℓ/2}q^{−ℓ/2}λ_{𝔭^{ℓ}}^{φ} ∈ R

denote this normalized Hecke eigenvalue. We sometimes write ν_{𝔭^{ℓ}}^{π} to emphasize that it depends only on the cuspidal representation π. Note that ν_{𝔭}^{φ} ∈ [−2, 2] by the Ramanujan conjecture. By Prop. 4.5,

(39)  ν_{𝔭^{ℓ}}^{φ} = X_{ℓ}(ν_{𝔭}^{φ}).

We will adapt the argument of [Li1] for finding the asymptotic weighted distribution of the set of ν_{𝔭}^{φ}.

Lemma 6.4. In the notation of Theorem 5.11, for any ℓ ≥ 0 and any m ∈ 𝔡_{+}^{−1}, T̂(m, m, 𝔭^{ℓ}) ≠ 0 if and only if both of the following hold:

(1) ℓ = 2ℓ′ is even

(2) 0 ≤ ℓ′ ≤ ord_{𝔭}(m𝔡).

Proof. The two conditions of Lemma 5.2 specialize respectively to the two conditions above in the special case where m_{1} = m_{2} = m and 𝔫 = 𝔭^{ℓ}. □

Proposition 6.5. Fix m ∈ 𝔡_{+}^{−1} and a prime ideal 𝔭 ∤ 𝔑. For each φ ∈ F, define a

weight w_{φ} = \frac{|W_{m}^{φ}(1)|^{2}}{∥φ∥^{2}} . Then for any ℓ ≥ 0 and 0 < ε < 1,

∑_{φ∈F} X_{ℓ}(ν_{𝔭}^{φ})w_{φ} = \begin{cases}Jψ(𝔑) + O(\frac{ψ(𝔑)}{ℕ(𝔑)^{2−ε}}) & if ℓ = 2ℓ′ with 0 ≤ ℓ′ ≤ ord_{v} m𝔡 \\ O(\frac{ψ(𝔑)}{ℕ(𝔑)^{2−ε}}) & otherwise, \end{cases}

where X_{ℓ} is the Chebyshev polynomial defined in Prop. 4.5, and

J = \frac{d_{F}^{1/2}}{e^{4π tr_{Q}^{F}(m)}} ∏_{j=1}^{r} \frac{(4πσ_{j}(m))^{k_{j}−1}}{(k_{j} − 2)!} .

The implied constant depends only on m, 𝔭, ℓ, k and ε.

Remark: This shows in particular that when ℕ(𝔑) is sufficiently large, (1) A_{k}(𝔑, ω) is nontrivial

(2) W_{m}^{φ}(1) is nonzero for some φ ∈ F.

Proof. This follows from the generalized Petersson trace formula. We use the form developed in the proof, rather than the final statement in Theorem 5.11. The spectral side (23) of the trace formula with 𝔫 = 𝔭^{ℓ} and m_{1} = m_{2} = m ∈ 𝔡_{+}^{−1} gives

∑_{φ∈F} \frac{λ_{𝔭^{ℓ}}^{φ} |W_{m}^{φ}(1)|^{2}}{∥φ∥^{2}} = ∑_{φ∈F} ω_{𝔭}(ϖ_{𝔭})^{ℓ/2}q^{ℓ/2}X_{ℓ}(ν_{𝔭}^{φ})w_{φ}.

1The Ramanujan conjecture was proven at all but a finite number of unspecified places for holomorphic cuspidal representations π of GL_{2}(A) with all weights ≥ 2 by Brylinski and Labesse ([BL], Theorem 3.4.6). Recently the full conjecture (at all places, when all weights are ≥ 2)

was proven by Blasius, with a parity condition on the weights [Bl]. The parity requirement was removed in the thesis of his student L. Nguyen [Ng]. However, as we remark after Theorem 6.6 below, our results do not actually depend on this deep theorem.

The above is equal to the geometric side, which by Prop. 5.4 and Prop. 6.2 is

= \frac{T̂(m, m, 𝔭^{ℓ})}{ω_{𝔑}(m/s)ω_{fin}(s)}q^{ℓ/2}Jψ(𝔑) + O(\frac{ψ(𝔑)}{ℕ(𝔑)^{2−ε}})

for J as above. By the above lemma, the first term is nonzero if and only if ℓ = 2ℓ′ with 0 ≤ ℓ′ ≤ ord_{𝔭}(m𝔡). In this case we can take

s = (…, m, m, mϖ_{𝔭}^{−ℓ′}^{𝔭^{th}}, m, m, …).

Note that ω_{𝔑}(m/s) = ∏_{𝔭|𝔑} ω_{𝔭}(1) = 1, so

ω_{𝔑}(m/s)ω_{fin}(s) = ω_{fin}(s) = ω_{fin}(m)ω_{𝔭}(ϖ_{𝔭})^{−ℓ′}.

Furthermore ω_{fin}(m) = ω_{∞}(m)^{−1} = 1 since m ∈ F^{+}. Thus in this case the geometric side is

= ω_{𝔭}(ϖ_{𝔭})^{ℓ/2}q^{ℓ/2}Jψ(𝔑) + O(\frac{ψ(𝔑)}{ℕ(𝔑)^{2−ε}}).

The proposition now follows by equating the spectral side with the geometric side and dividing by ω_{𝔭}(ϖ_{𝔭})^{ℓ/2}q^{ℓ/2}. □

Theorem 6.6. Let 𝔭 be a prime ideal of F, and let k = (k_{1}, …, k_{r}) be a weight vector with all k_{j} > 2. For each i = 1, 2, …

• let 𝔑_{i} be an ideal coprime to 𝔭, with lim_{i→∞} ℕ(𝔑_{i}) = ∞,

• let ω_{i} be a unitary character as in §3.3 relative to 𝔑_{i} and k,

• let F_{i} be an orthogonal basis for A_{k}(𝔑_{i}, ω_{i}) as in (36). Fix any m ∈ 𝔡_{+}^{−1}, and define weights w_{φ} = |W_{m}^{φ}(1)|^{2}/∥φ∥^{2} for φ ∈ F_{i}. For each i, define a sequence

S_{i} = {ν_{𝔭}^{φ}}_{φ∈F_{i}}

in the interval [−2, 2]. Then the sequence S_{i} is w_{φ}-equidistributed relative to the measure

dμ(x) = ∑_{ℓ′=0}^{ord_{𝔭}(𝔡m)} X_{2ℓ′}(x)dμ_{∞}(x),

where dμ_{∞}(x) is the Sato-Tate measure defined in the Introduction. In other words, for any continuous function h on R,

lim_{i→∞} \frac{∑_{φ∈F_{i}} h(ν_{𝔭}^{φ})w_{φ}}{∑_{φ∈F_{i}} w_{φ}} = ∫_{R} h(x)dμ(x).

Remarks: (1) When 𝔭 ∤ m𝔡, the measure μ coincides with μ_{∞} and is independent of 𝔭. Further taking all k_{j} even and ω_{i} trivial, we immediately obtain Theorem 1.1.

(2) The above result (and its proof) is actually independent of the Ramanujan conjecture. All we need is the existence of a finite interval I_{𝔭} which contains all of the eigenvalues ν_{𝔭}^{φ}. This is elementary ([Ro], Prop. 2.9).

(3) The theorem shows in particular that the Satake traces ν_{𝔭}^{π} are dense in the interval [−2, 2]. The referee has pointed out that this can also be seen directly by considering CM cusp forms.

Proof. Setting ℓ = 0 in the previous proposition, we have

∑_{φ∈F_{i}} w_{φ} = Jψ(𝔑_{i}) + O(\frac{ψ(𝔑_{i})}{ℕ(𝔑_{i})^{2−ε}}).

Therefore for any ℓ ≥ 0

lim_{i→∞} \frac{∑_{φ∈F_{i}} X_{ℓ}(ν_{𝔭}^{φ})w_{φ}}{∑_{φ∈F_{i}} w_{φ}} = \begin{cases}1 & when ℓ = 2ℓ′, with 0 ≤ ℓ′ ≤ ord_{v} m𝔡, \\ 0 & otherwise \end{cases}

= ∫_{R} X_{ℓ}(x)dμ(x). (40)

This last equality holds by the orthonormality of the polynomials X_{n}(x)

∫_{R} X_{i}(x)X_{j}(x)dμ_{∞}(x) = δ_{ij}

(cf. [Se]). Because deg X_{ℓ} = ℓ, the set {X_{ℓ}} spans the space of all polynomials. Because the space of polynomials is dense in L^{∞}([−2, 2]), we can replace X_{ℓ} by any continuous function h in (40) and thus obtain the result (see §29.3 of [KL2] for details). □

7. Variants and special cases

We have four corresponding types of parameters (under various hypotheses):

{Satake \atop ν_{𝔭}^{π}} ⟷ {Hecke \atop λ_{𝔫}^{φ}} ⟷ Whittaker ; W_{m}^{φ}(y) ⟷ Classical ; a_{m}(h).

For convenience, we give the correspondences explicitly here, so that when possible

anyone can rewrite the spectral terms \frac{λ_{𝔫}^{φ}W_{m_{1}}^{φ}(1)\overline{W_{m_{2}}^{φ}(1)}}{∥φ∥^{2}} in the main formula purely

in terms of their parameter of choice. Let nÔ = 𝔫̂ and dÔ = 𝔡̂. If W_{m}^{φ}(1/d) = 1, then by Corollaries 4.7 and 4.8,

λ_{𝔫}^{φ} = ℕ(𝔫)W_{1}^{φ}(n/d)  and  W_{m}^{φ}(1) = \frac{e^{2πr} ∏_{j=1}^{r} σ_{j}(m)^{k_{j}/2−1}}{d_{F} e^{2π tr(m)}} λ_{m𝔡},

either of which can be used if an orthogonal basis of such φ is given. Also, using (39) we have

(41)  λ_{𝔫}^{φ} = ω_{fin}(n)^{1/2}ℕ(𝔫)^{1/2} ∏_{𝔭|𝔫} X_{ord_{𝔭}(𝔫)}(ν_{𝔭}^{π}).

The classical picture is given explicitly at the end of this section.

The case of narrow class number 1. From now on we assume that F has narrow class number 1, i.e. that every fractional ideal in F has a totally positive generator. This implies that every totally positive unit is the square of a unit ([CH], Lemma 11.6). In this case Theorem 5.11 simplifies substantially, and a classical interpretation follows from (14).

Fix η, N, d ∈ F^{+} such that

(42)  (η) = 𝔫, (N) = 𝔑, (d) = 𝔡.

Proposition 7.1. Suppose F has narrow class number 1. For δ = ( \begin{matrix}m_{2}/m_{1} \\ & 1 \end{matrix} ),

I_{δ}(f) = \frac{T (dm_{1}, dm_{2}, η)d_{F}^{1/2}ℕ(η)^{1/2}ψ(𝔑)}{e^{2π tr_{Q}^{F}(m_{1}+m_{2})}ω_{𝔑}(\sqrt{m_{1}η/m_{2}})} [∏_{j=1}^{r} \frac{(4π\sqrt{σ_{j}(m_{1}m_{2})})^{k_{j}−1}}{(k_{j} − 2)! sgn(σ_{j}(\sqrt{\frac{m_{1}m_{2}}{η}}))^{k_{j}}} ] ,

where

T (a_{1}, a_{2}, a_{3}) = \begin{cases}1 & if a_{i}a_{j}/a_{k} is a square in O for all \\ & distinct i, j, k ∈ \{1, 2, 3\} \\ 0 & otherwise, \end{cases}

and the square roots are chosen compatibly so that \sqrt{m_{1}m_{2}/η}\sqrt{m_{1}η/m_{2}} = m_{1}.

Proof. By Prop. 5.4, we know that I_{δ} is nonzero only if • \frac{m_{1}}{s} , \frac{m_{2}}{s} ∈ Ô for some s ∈ 𝔡̂^{−1}

• \frac{m_{1}m_{2}}{s^{2}} Ô = 𝔫̂.

If these hold, then because the class number is 1, we can actually take s ∈ 𝔡^{−1}, so we change fonts to s as a reminder that s ∈ F. Furthermore the second condition is equivalent to m_{1}m_{2} = s^{2}uη for some unit u. Because m_{1}m_{2}, s^{2}, η ∈ F^{+}, it follows that u ∈ F^{+}, and hence u is the square of a unit. We can absorb this unit into s, so the condition becomes

(43)  m_{1}m_{2} = s^{2}η.

Write s = s′/d for d as in (42). Then the above conditions are equivalent to (1) \frac{dm_{1}}{s′} , \frac{dm_{2}}{s′} ∈ O for some s′ ∈ O

(2) \frac{(dm_{1})(dm_{2})}{s^{′2}} = η.

It is easy to check that these conditions hold if and only if T (dm_{1}, dm_{2}, η) = 1. Now suppose the above holds, so that by Prop. 5.4,

I_{δ} = [∏_{j=1}^{r} \frac{(4π\sqrt{σ_{j}(m_{1}m_{2})})^{k_{j}−1}}{(k_{j} − 2)!} ] \frac{d_{F}^{1/2}ℕ(𝔫)^{1/2}ψ(𝔑)}{e^{2π tr_{Q}^{F}(m_{1}+m_{2})}ω_{𝔑}(m_{1}/s)ω_{fin}(s)}.

By (43), we can write s = \sqrt{m_{1}m_{2}/η}. This element is defined up to ±1, and there is no canonical choice since we cannot guarantee that s ∈ F^{+}. (E.g. (1 + \sqrt{2})^{2} has no totally positive square root.) In any case, the final result is independent of this choice by the remark on page 16. Using this expression for s, the proposition follows by the fact that

ω_{fin}(s) = ω_{∞}(s)^{−1} = ∏_{j=1}^{r} sgn(σ_{j}(s))^{k_{j}}.

□

For the second type of I_{δ} (with δ = ( \begin{matrix}& μ \\ 1 \end{matrix} )), the condition on μ becomes μ = −ηu/c^{2} for some totally positive unit u ∈ U and nonzero c ∈ NO. As in the previous case, u is a square, so it can be absorbed into c. Thus

μ = −η/c^{2}

for some c ∈ 𝔑. Under this condition, by Prop. 5.7 the contribution is

I_{δ} =\frac{ψ(𝔑)}{2^{r}e^{2π tr_{Q}^{F}(m_{1}+m_{2})}} ω_{fin}(c)S_{ω_{𝔑}}(m_{1}, m_{2}; η; c)\frac{ℕ(η)^{1/2}}{ℕ(c)}

× ∏_{j=1}^{r} \frac{(−4πi)^{k_{j}} \sqrt{σ_{j}(m_{1}m_{2})}^{k_{j}−1}}{(k_{j} − 2)!} J_{k_{j}−1}(4π \frac{\sqrt{σ_{j}(ηm_{1}m_{2})}}{|σ_{j}(c)|}).

Once again, we cannot assume c ∈ F^{+} so ω_{fin}(c) may be nontrivial. In fact

ω_{fin}(c) = ∏_{j=1}^{r} sgn(σ_{j}(c))^{k_{j}}.

Summing over all μ amounts to summing over nonzero c ∈ 𝔑/±, and we obtain the following.

Theorem 7.2. With notation as above, suppose F has narrow class number 1, k = (k_{1}, …, k_{r}) with all k_{j} > 2, and F is an orthogonal basis for A_{k}(𝔑, ω) consisting of eigenfunctions for the Hecke operator T_{𝔫}. Then for any m_{1}, m_{2} ∈ 𝔡_{+}^{−1},

\frac{e^{2π tr_{Q}^{F}(m_{1}+m_{2})}}{d_{F}^{1/2}ℕ(η)^{1/2}ψ(𝔑)} [∏_{j=1}^{r} \frac{(k_{j} − 2)!}{(4π\sqrt{σ_{j}(m_{1}m_{2})})^{k_{j}−1}} ] ∑_{φ∈F} \frac{λ_{𝔫}^{φ} W_{m_{1}}^{φ}(1)\overline{W_{m_{2}}^{φ}(1)}}{∥φ∥^{2}}

= T (dm_{1}, dm_{2}, η) [∏_{j=1}^{r}(sgn σ_{j}(\sqrt{m_{1}m_{2}/η}))^{k_{j}} ] ω_{𝔑}(\sqrt{m_{1}η/m_{2}})^{−1}

+\frac{1}{d_{F}^{1/2}} ∑_{{c∈NO/± \atop c≠0}} S_{ω_{𝔑}}(m_{1}, m_{2}; η; c)\frac{(2π)^{r}}{ℕ(c)} ∏_{j=1}^{r} \frac{J_{k_{j}−1}(4π^{\frac{\sqrt{σ_{j}(ηm_{1}m_{2})}}{|σ_{j}(c)|}})}{(i sgn(σ_{j}(c))^{k_{j}}}.

Remarks: (1) Because Ô/cÔ ≅ O/cO, we see easily that for c ∈ 𝔑,

S_{ω_{𝔑}}(m_{1}, m_{2}; η; c) = ∑_{{t_{1},t_{2}∈O/cO \atop t_{1}t_{2}≡η mod cO}} ω_{𝔑}(t_{2})^{−1}θ_{fin}(\frac{m_{1}t_{1} + m_{2}t_{2}}{c}).

The formula can be further simplified if we replace c ∈ 𝔑 by c = Nuτ for u ∈ O^{∗}/{±1} and τ ∈ O/O^{∗}. Then substituting t′_{1} = u^{−1}t_{1}, t′_{2} = u^{−1}t_{2}, we have

S_{ω_{𝔑}}(m_{1}, m_{2}; η; c) = ω_{𝔑}(u)^{−1}S_{ω_{𝔑}}(m_{1}, m_{2}; u^{−2}η; Nτ),

so we can break the sum over c into sums over u and τ and group together the terms with the same τ.

(2) If F = Q, then r = 1. The sum over c ∈ NZ/± is simply a sum over c > 0, N|c, and we recover the generalized Petersson trace formula of [KL1].

Now take η = 1 so 𝔫 = O. Then λ_{𝔫}^{φ} = 1 for all φ. Because the narrow class number is 1, A_{k}(𝔑, ω) corresponds (isometrically) to a classical space S_{k}(𝔑, ω′) of cusp forms for Γ_{0}(𝔑) as in (13). Here ω′(( \begin{matrix}a & b \\ c & d \end{matrix} )) = ω′(d) = ω_{𝔑}(d)^{−1} for any ( \begin{matrix}a & b \\ c & d \end{matrix} ) ∈ Γ_{0}(𝔑) = SL_{2}(F) ∩ K_{0}(𝔑). Conversely, given a character ω′ : Γ_{0}(𝔑)/Γ_{1}(𝔑) → C^{∗}, the space S_{k}(𝔑, ω′) is isometric to A_{k}(𝔑, ω), where ω is the Hecke character determined by ω′ using strong approximation A^{∗} = F^{∗}(F_{∞}^{+} × Ô^{∗}) resulting from narrow class number 1. By (14), if φ ↔ h, then W_{m}^{φ}(1) = d_{F}^{1/2}e^{−2π tr(m)}a_{m}(h), and we obtain the following direct generalization of Petersson’s original formula and (8) of [Lu].

Corollary 7.3. For any orthogonal basis F for S_{k}(𝔑, ω′) and any m_{1}, m_{2} ∈ 𝔡_{+}^{−1} we have

\frac{d_{F}^{1/2}}{ψ(𝔑)} [∏_{j=1}^{r} \frac{(k_{j} − 2)!}{(4π\sqrt{σ_{j}(m_{1}m_{2})})^{k_{j}−1}} ] ∑_{h∈F} \frac{a_{m_{1}}(h)\overline{a_{m_{2}}(h)}}{∥h∥^{2}}

= χ(m_{1}, m_{2}) + \frac{1}{d_{F}^{1/2}} ∑_{{c∈NO/± \atop c≠0}} S_{ω′}(m_{1}, m_{2}; c)\frac{(2π)^{r}}{ℕ(c)} ∏_{j=1}^{r} \frac{J_{k_{j}−1}(4π^{\frac{\sqrt{σ_{j}(m_{1}m_{2})}}{|σ_{j}(c)|}})}{(i sgn(σ_{j}(c))^{k_{j}}},

where χ(m_{1}, m_{2}) ∈ {0, 1} is nonzero if and only if m_{2} = m_{1}u^{2} for some u ∈ O^{∗}

and S_{ω′}(m_{1}, m_{2}; c) = ∑_{s∈(O/cO)^{∗}, s\overline{s}=1} exp(2πi tr_{Q}^{F}(\frac{sm_{1}+\overline{s}m_{2}}{c}))ω′(s).

References

[Bi] B. J. Birch, How the number of points of an elliptic curve over a fixed prime field varies, J. London Math. Soc., 43 (1968), 57-60.

[Bl] D. Blasius, Hilbert modular forms and the Ramanujan conjecture, in “Noncommutative geometry and number theory,” Vieweg Verlag, (2006), 35-56.

[BJ] A. Borel and H. Jacquet, Automorphic forms and automorphic representations, Proc. Symp. Pure Math., 33, Part 1, Amer. Math. Soc., Providence, RI, (1979), 189-207.

[BL] J.-L. Brylinski and J.-P. Labesse, Cohomologie d’intersection et fonctions L de certaines variétés de Shimura, Ann. Sci. École Norm. Sup. (4) 17, no. 3, (1984), 361-412.

[BMP] R. Bruggeman, R. Miatello, and I. Pacharoni, Estimates for Kloosterman sums for totally real number fields, J. Reine Angew. Math. 535 (2001), 103–164.

[Br] R. Bruggeman, Fourier coefficients of cusp forms, Invent. Math. 45 (1978), no. 1, 1-18.

[Bu] D. Bump, Automorphic forms and representations, Cambridge Studies in Advanced Mathematics 55, Cambridge University Press, 1998.

[CDF] B. Conrey, W. Duke, and D. Farmer, The distribution of the eigenvalues of Hecke operators, Acta Arith. 78 (1997), no. 4, 405-409.

[CH] P. E. Conner and J. Hurrelbrink, Class number parity, Series in Pure Mathematics, 8. World Scientific Publishing Co., Singapore, 1988.

[CPS] J. Cogdell and I. Piatetski-Shapiro, The arithmetic and spectral analysis of Poincaré series, Perspectives in Mathematics, 13. Academic Press, Inc., Boston, MA, 1990.

[Gu] K.-B. Gundlach, Über die Darstellung der ganzen Spitzenformen zu den Idealstufen der Hilbertschen Modulgruppe und die Abschätzung ihrer Fourierkoeffizienten, Acta Math. 92, (1954), 309–345.

[HC] Harish-Chandra, Automorphic forms on semisimple Lie groups, Notes by J. G. M. Mars. Lecture Notes in Mathematics, No. 62, Springer-Verlag, Berlin-New York, 1968.

[Hi] H. Hida, Elementary theory of L-functions and Eisenstein series, London Math. Soc. Student Texts 26, Cambridge Univ. Press, Cambridge, 1993.

[Ja] H. Jacquet, A guide to the relative trace formula, in: Automorphic representations, Lfunctions and applications: progress and prospects, 257-272, Ohio State Univ. Math. Res. Inst. Publ., 11, de Gruyter, Berlin, 2005.

[KL1] A. Knightly and C. Li, A relative trace formula proof of the Petersson trace formula, Acta Arith., 122 (2006), no. 3, 297-313.

[KL2] ——, Traces of Hecke operators, Mathematical Surveys and Monographs, 133. Amer. Math. Soc., 2006.

[Ku] N. V. Kuznetsov, The Petersson conjecture for cusp forms of weight zero and the Linnik conjecture. Sums of Kloosterman sums, (Russian) Mat. Sb. (N.S.) 111 (153) (1980), no. 3, 334–383, 479. English translation: Math. USSR Sbornik, 39 (1981), 299-342.

[Li1] C. Li, Kuznietsov trace formula and weighted distribution of Hecke eigenvalues, J. Number Theory 104 (2004), no. 1, 177–192.

[Li2] ——, On the distribution of Satake parameters of GL_{2} holomorphic cuspidal representations, Israel J. Math., to appear.

[Lu] W. Luo, Poincaré series and Hilbert modular forms, Rankin memorial issue, Ramanujan J. 7 (2003), no. 1-3, 129–140.

[MW] R. Miatello and N. Wallach, Kuznetsov formulas for products of groups of R-rank one, Festschrift in honor of I. Piatetski-Shapiro on the occasion of his sixtieth birthday, Part II (Ramat Aviv, 1989), 305–320, Israel Math. Conf. Proc., 3, Weizmann, Jerusalem, 1990.

[Ng] L. Nguyen, The Ramanujan conjecture for Hilbert modular forms, Ph.D. thesis, UCLA, 2005.

[Ro] J. Rogawski, Modular forms, the Ramanujan conjecture and the Jacquet-Langlands correspondence, appendix in “Discrete Groups, Expanding Graphs and Invariant Measures,” by A. Lubotzky, Birkhäuser, Basel, 1994, pp. 135-176.

[Sa] Sarnak, P., Statistical properties of eigenvalues of the Hecke operators, in: Analytic number theory and Diophantine problems (Stillwater, OK, 1984), Progr. Math., 70, Birkhäuser, Boston, MA, 1987, 321–331.

[Se] J.-P. Serre, Répartition asymptotique des valeurs propres de l’opérateur de Hecke T_{p}, J. Amer. Math. Soc. 10 (1997), no. 1, 75–102.

[Ta] R. Taylor, Automorphy for some l-adic lifts of automorphic mod l Galois representations II, preprint, May 24, 2006.

Department of Mathematics & Statistics, University of Maine, Neville Hall, Orono, ME 04469-5752, USA

Department of Mathematics, The Chinese University of Hong Kong, Shatin, Hong Kong