# Fuchsian groups, finite simple groups and representation varietiesAcademic research paper on "Mathematics"

0
0
Share paper
Inventiones mathematicae
OECD Field of science
Keywords
{""}

## Academic research paper on topic "Fuchsian groups, finite simple groups and representation varieties"

﻿Invent. math. 159, 317-367 (2005) DOI: 10.1007/s00222-004-0390-3

Inventiones mathematicae

Fuchsian groups, finite simple groups and representation varieties

Martin W. Liebeck1, Aner Shalev2'*

1 Department of Mathematics, Imperial College, London SW7 2BZ, England

2 Institute of Mathematics, Hebrew University, Jerusalem 91904, Israel

Oblatum 25-VI-2003 & 4-V-2004

Published online: 2 September 2004 - © Springer-Verlag 2004

Abstract. Let r be a Fuchsian group of genus at least 2 (at least 3 if r is non-oriented). We study the spaces of homomorphisms from r to finite simple groups G, and derive a number of applications concerning random generation and representation varieties. Precise asymptotic estimates for |Hom(r, G)| are given, implying in particular that as the rank of G tends to infinity, this is of the form |G|»(г)+1+o(1), where fz(r) is the measure of r. We then prove that a randomly chosen homomorphism from r to G is surjective with probability tending to 1 as |G| ^ to. Combining our results with Lang-Weil estimates from algebraic geometry, we obtain the dimensions of the representation varieties Hom(r, G), where G is GLn (K) or a simple algebraic group over K, an algebraically closed field of arbitrary characteristic. A key ingredient of our approach is character theory, involving the study of the 'zeta function' ZG(s) = x(1)-s, where the sum is over all irreducible complex characters x of G.

Contents

1 Introduction.....................................317

2 Character degrees..................................326

3 Counting homomorphisms.............................332

4 Counting elements of given order..........................341

5 Maximal subgroups.................................348

6 Random homomorphisms..............................358

7 Representation varieties...............................362

1. Introduction

Results on random generation of finite simple groups often amount to saying that randomly chosen homomorphisms from certain infinite groups r

* The second author thanks EPSRC for its support and Imperial College for its hospitality while this work was carried out

to finite simple groups G are surjective with probability tending to 1 as |G | ^ to. For example, if r is the free group of rank two this is Dixon's conjecture on random generation of simple groups by two elements, which was proved in [6,16,27]; if r = Z2 * Z this is the Kantor-Lubotzky conjecture on random generation by an involution and another element proved in [29]; for r =PSL2(Z) this is random (2, 3)-generation obtained in [28, 14] (assuming G is not of type B2 or 2B2); and for r = Zr * Zs this is random (r, s) -generation, proved in [31] for G classical of large rank and r, s primes (not both 2).

All the groups r above are Fuchsian, and it is natural to ask whether results of this type can be obtained in the much more general context of Fuchsian groups. Some positive evidence is provided in [32], where various results on random Fuchsian generation of alternating and symmetric groups are established. In particular, we show in [32, 1.12] that if r is a Fuchsian group of genus at least 2 (3 if r is non-oriented), and G is an alternating group, then a random homomorphism from r to G is surjective with probability tending to 1 as |G One main goal of this paper is

to extend this result to all finite simple groups G (see Theorem 1.6 below).

To prove such a result one needs good estimates on |Hom(r, G)| and |Hom(r, M)| for finite simple groups G and their maximal subgroups M. Bounds for these numbers are given in terms of a character-theoretic 'zeta function' (see Lemma 3.3), and much of the proof involves analysis of these functions using character theory of finite groups of Lie type. Another ingredient is an estimate for the number of solutions to the equation xm = 1 in finite simple groups (see Sect. 4). These results seem to have some independent interest.

When G = G(q), a group of Lie type over Fq, the space Hom(r, G(q)) can essentially be regarded as the set of q-rational points of the representation variety Hom (r, G) ,where G is the simple algebraic group of the same type as G overthe algebraic closure of Fq .Combining our results on |Hom (r, G (q))| with Lang-Weil estimates for the number of q-rational points in algebraic varieties ([19]), we obtain the dimensions of these representation varieties, solving some open problems initiated in [42] (see 1.8-1.11 below).

Recall that a Fuchsian group is a finitely generated non-elementary discrete group of isometries of the hyperbolic plane H2. By classical work of Fricke and Klein, the orientation-preserving such groups r have a presentation of the following form:

(1.1) generators: a1; b1,...,ag, bg (hyperbolic)

x1,...,xd (elliptic)

y]_,... , ys (parabolic)

z1,... ,zt (hyperbolic boundary elements)

relations: xm1 = ••• = xmd = 1,

x1 •• • xd y1 ••• ys Z1 ••• Zt [a1, b1] - - - [ag, bg ] = 1,

where g, d, s, t > 0 and mi > 2 for all i. The number g is referred to as the genus of r. The measure i(F) of an orientation-preserving Fuchsian group r is defined by

,i{T)=2g-2 + +

It is well known that i(T) > 0.

We shall also study non-orientation-preserving Fuchsian groups; these have presentations as follows, with g > 0:

(1.2) generators: ai,... ,ag

yi,-- - , ys zi,... ,zt

relations: xf1 = ••• = xfd = 1,

xi •••xdyi ••• ysZi ■ •• Zt a2 •••a;; = 1.

In this case the measure i(r) is defined by

and again, i(r) > 0.

Note that r is a lattice in PSL2(R), and i(r) = -x(D, where x(r) is the Euler characteristic.

We call Fuchsian groups as in (1.1) oriented, and those as in (1.2) non-oriented. Define v = v(r) to be 2 if r is oriented and 1 otherwise. Define also d* = d*(r) to be the number of i such that mi is even.

If s + t > 0 then r is just a free product of cyclic groups. The most interesting Fuchsian groups are those with s = t = 0; these are co-compact (also termed proper in [32]) and are the main focus of this paper. Examples include surface groups (i.e. where d = 0) of genus g > 2(g > 3 in the non-oriented case); note that surface groups of smaller genus are not Fuchsian, and are in fact virtually abelian.

Many of our proofs will depend on the above-mentioned 'zeta function', which seems to have some independent interest. For a finite group G, let Irr(G) denote the set of irreducible complex characters of G, and for real s, define

ZG(s) = J2 x(D-s■

xeIrr(G)

For example, ZG(-2) = |G|, and ZG(0) = k(G), the number of conjugacy classes of G. The behaviour of ZG (s) for s > 0 is significant for many

applications: for instance, when G is a symmetric group, this was studied in [36,41,32], with applications to random walks, subgroup growth and coverings of Riemann surfaces.

Note that, if cn = cn (G) denotes the number of characters of degree n of G, then ZG(s) = 5Zn>1 cnn-s. A similar function can be defined for certain infinite groups, for which cn are all finite, and polynomially bounded, in which case ZG (s) converges for all large enough s. For example, this is the case when G is a compact connected semisimple Lie group and we count Lie group representations, as can be quickly deduced from the Weyl dimension formula. In this case the zeta function has great geometric significance, as shown for instance by Witten (see [53, (4.72)]). Note that for G = SU(2) we have cn = 1 for all n, and so in this case ZG coincides with the classical Riemann zeta function.

For our purpose here it is important to understand the asymptotic behaviour of ZG (s) where G is a finite quasisimple group whose order tends to infinity. Recall that G quasisimple means that G is perfect and G/Z(G) is simple. Note that with finitely many exceptions, quasisimple groups are central factor groups of either double covers of alternating groups, or of finite Chevalley groups of simply connected types (see [10, 6.1.4]).

We show the following.

Theorem 1.1 Let G be a finite quasisimple group.

(i) Ifs > 1, then ZG (s) ^ 1 as G | ^ to.

(ii) Ifs > | and G ^ L2{q) or SL2(q), then ZG(s) —► 1 as |G| oo.

The condition s > 1 in (i) is sharp: indeed, inspection of the character table of SL2(q) (see [7, p.228]) shows that ZSL2(q)(1) ^ 2 and for q odd, Z L2(q)(1) —> j iis tj —> oo, while Z is) —> oo for ,v < 1 for these groups (see Lemma 2.2). Likewise, the condition on s in (ii) is also sharp, since ZG(2/3) ^ 1 for G = L3(q) or U3(q). Excluding the latter groups, even sharper results can be obtained. These will appear in [33].

For G alternating, a stronger result holds, namely ZG (s) ^ 1as | G |^to for any s > 0 (see [32,2.7]). We show in [33] that this also holds for classical groups of rank tending to infinity.

Note that since the limit points of RG(1) : G finite simple} are 1, | and 2, it follows that with the above notation, cn (G) < Cn for all finite simple groups G, where C is an absolute constant, and sharper results follow for G = L2(q) - see Corollary 2.7.

Theorem 1.1 can be extended to the case where G is a nearly simple group - that is, F* (G) is quasisimple (see Theorem 2.8).

We shall apply Theorem 1.1 in the study of the space of homomorphisms from Fuchsian groups to finite quasisimple groups. In [41], the authors obtain good estimates for |Hom(r, G)| where r is a surface group and G = Sn. This is extended in [32] to the case where r is an arbitrary Fuchsian group and G = Sn or An. Hence our focus here is on the case where G is a quasisimple group of Lie type.

For a finite group G and a positive integer m, denote by jm(G) the number of solutions in G of the equation xm = 1. If r is a non co-compact Fuchsian group, then r decomposes as a free product of vg + s + t — 1 copies of Z and cyclic groups of orders m1,... ,md, and hence for any finite group G, we have |Hom(r, G )| = |G |vg+s+t—1 ■ nd=1 jm, (G). The following result shows that if G is a finite quasisimple group, a rather similar estimate holds also for co-compact groups (i.e. when s +1 = 0), provided the genus is not too small. In the statements below, o(1) refers to a quantity which tends to 0 as |G| ^ to.

Theorem 1.2 Let r be a co-compact Fuchsian group as in (1.1) or (1.2), and let G be a finite quasisimple group.

(i) If r is oriented of genus g > 2, we have

|Hom(r, G)| = (1 + o(1)) ■ |G|2g—1 ■ ["[ jmt(G).

(ii) Assume r is non-oriented of genus g > 3, and (G/Z(G), g) = (L2(q), 3). Then

|Hom(r, G)| = (1 + o(1)) ■ |G|g—1 ■ ["[ jm,(G).

(iii) Assume r is non-oriented of genus g = 3 and G = L 2(q). Then

|Hom(r, G)| = (h + o(1)) ■ |G|2 ■ ["[ jmt(G),

where h = 1 unless (/»,-, |G|) = 1 for all i, in which case h = \for q odd and h = 2 for q even.

(iv) Assume r is non-oriented of genus g = 3 and G = SL2(q) with q odd. Then

|Hom(r, G)\ = (h + o(1)) ■ |G|2 ^ jmi(G),

where h = 1 unless d* > 0 and (mi, G |) is 1 or 2 for all i, in which case h = 3 ■ 2d*—1.

The anomaly of L2 (q) and SL2(q) in the theorem is related to their exceptional behaviour in Theorem 1.1 and the remark following. This anomaly is also reflected in subsequent results on representation varieties (see 1.8,1.11). Note also that in part (iii) we have h = 1 if and only if all homomorphisms from r to G = L 2(q) factor through a non-oriented surface group of genus 3.

Theorem 1.2 takes a particularly simple form when d = 0, that is, r is a surface group:

Corollary 1.3 Let r be a surface group which is not virtually abelian, let g be the genus of r, let v = v(r), and let G be a finite quasisimple group. Then

|Hom(r, G)| = (h + o(1)) ■ |G|vg-1

where h = 1 unless v = 1, g = 3 and G = L2(q), in which case h = \for q odd, and h = 2 for q even.

In order to use Theorem 1.2 one requires information on the values of jm (G) for G quasisimple. Such information can be found in [32,52] for G = An, in [28,31] for G classical and m prime, and in [15] for G exceptional and m < 5. Recently, Lawther [20] has obtained tight estimates for the dimension of the variety Jm (X) = {x e X : xm = 1}, where X is any connected simple algebraic group. Using this we prove the following.

Theorem 1.4 Let G = G(q) be a finite quasisimple group of Lie type over Fq of rank r, and let m > 2 be an integer. Then

jm(G) = IGI1"^',

where |e(r) | = O(r-1).

We also obtain a number of more detailed estimates of jm (G) for G of Lie type, including GLn (q) (see Sect. 4).

Combining Theorems 1.2 and 1.4 gives the following.

Theorem 1.5 Let r be a Fuchsian group of genus g > 2 (g > 3 if G is non-oriented), and let G be a finite classical quasisimple group of rank r. Then

|Hom(r, G)| = |G

where |^(r) | = O(r-1).

Hence, if Gn is a sequence of finite quasisimple classical groups whose ranks tend to infinity, then

r log I Horn (T, G„) |

lim --——-= /i(D + 1.

log |G„ |

This also holds for alternating groups Gn = An, as shown in [32].

Our main result concerns random Fuchsian generation of finite simple groups, and uses some of the above theorems, as well as various new results on maximal subgroups of finite simple groups presented in Sect. 5.

Theorem 1.6 Let r be a Fuchsian group of genus g > 2 (g > 3 if G is non-oriented), and let G be a finite simple group. Then the probability that a randomly chosen homomorphism in Hom(T, G) is an epimorphism tends to 1 as |G| ^ to.

We make a few remarks concerning the strategy of the proof of Theorem 1.6. If a homomorphism in Hom(r, G) is not an epimorphism, then its image lies in a maximal subgroup M of G, and this happens with probability |Hom(r G)| • Hence the probability that a randomly chosen homomorphism in Hom(r, G) is not an epimorphism is bounded above by

^ |Hom(r, M)| Q{T,G)= V --——-.

|Hom(r, G)|

M max G 1 v '

It therefore suffices to show that Q(r, G) ^ 0as |G| ^ to.

To prove this we need not only lower bounds on |Hom(r, G) | provided by Theorem 1.2, but also upper bounds on |Hom(r, M)| for all maximal subgroups M of finite simple groups G. The latter are obtained via the inequality

|Hom(r, M)| < |M|vg—1 ■ ["[ jm(M) ■ ZM(vg — 2),

where v = v(r) (see Lemma 3.3). To apply this, a painstaking analysis of the function ZM (s) is required, leading to a bound on ZM (1) in terms of the index of M in G (see Theorem 5.1 below). This bound is combined with recent results on maximal subgroups [26] to complete the proof of Theorem 1.6.

Theorem 1.6 is new even for surface groups, where it takes the following form.

Corollary 1.7 Let r be a surface group which is not virtually abelian, and let G be a finite simple group. Then the probability that a randomly chosen homomorphism in Hom(r, G) is an epimorphism tends to 1 as G |^to.

Theorem 1.6 extends [32, 1.12], which yields the conclusion in the case where G = An . We note that some assumption on the genus is essential, since there are Fuchsian groups of genus 0 or 1 which do not have all large enough finite simple groups as quotients. Examples include triangle groups of genus 0 such as the Hurwitz (2, 3, 7)-group (see for example [4]), and genus 1 groups of the form (1.1) with d = 1 and m1 an odd prime (since there are infinitely many finite simple groups containing no element of order m1).

Still, it would be interesting to find partial extensions of Theorem 1.6 to Fuchsian groups of genus 0 or 1. We propose the following.

Conjecture For any Fuchsian group r there is an integer f(T), such that if G is a finite simple classical group of rank at least f(T), then the probability that a randomly chosen homomorphism from r to G is an epimorphism tends to 1 as G |^to.

We can show that the conjecture holds for non co-compact Fuchsian groups. Since the latter are free products of cyclic groups, some cases are already covered by results mentioned in the first paragraph of this paper; the proof is completed by establishing that the simple groups in question are also randomly (r, s)-generated when r, s are not both prime. This work will appear elsewhere.

The proof of the conjecture for co-compact Fuchsian groups (of small genus) seems to require strong bounds on character ratios |x(x)/x(1)| for X e Irr(G) and for elements x e G of given order (see Lemma 3.1 below). Some bounds on these ratios do exist (see for instance [11]), with many interesting applications, but these bounds are not sufficient to settle our conjecture, and substantial refinements will be required.

Finally, we apply our results on Hom(r, G) for G finite of Lie type to the study of representation varieties of r in reductive algebraic groups over algebraically closed fields. For a Fuchsian group r, an algebraically closed field K, and a positive integer n, define

This has a natural structure as an algebraic variety defined over the prime subfield of K, and has been extensively studied in the case where K has characteristic zero and r is a surface group (see [35,42,3,12]). However, not much seems to be known in positive characteristic. We make the following contribution.

Theorem 1.8 Let r be a surface group of genus g which is not virtually abelian, and let K be an algebraically closed field of characteristic p > 0.

(i) If r is oriented, then dim Rn,K(r) = (2g — 1)n2 + 1 and Rn,K(r) has a unique irreducible component ofhighest dimension.

(ii) If r is non-oriented, then dim Rn K(r) = (g — 1)n2 and Rn,K(r) has (2, p — 1) irreducible components of highest dimension unless (n, g) = (2, 3), in which case it has (2, p — 1) + 1 such components.

These dimensions agree with those given for the characteristic zero case in [42] for oriented groups and in [3] for non-oriented groups. In fact it is well known that the dimension of a variety in characteristic zero coincides with the dimension of its reduction modulo p for all large primes p, and so Theorem 1.8 provides an alternative proof of the characteristic zero dimension results in [42,3].

Our methods extend to give the values of dim Rn,K (r) for arbitrary Fuchsian groups. We need some notation. For positive integers n and m 1,... ,md, all at least 2, write n = kmi + k with 0 < k < mi, and m = (m1,... , md), and define

RnK(r) = Hom(r, GLn(K)).

Note that c(n, m) is bounded in terms of m only.

Theorem 1.9 Let r be a co-compact Fuchsian group as in (1.1) or (1.2), of genus g > 2 (g > 3 if r is non-oriented). Set E = {i : mi even}, I = /x(T), v = v(r), let n > 2, and let K be an algebraically closed field of arbitrary characteristic. Then

dim RnK(r) = (i + 1)n2 — c(n, m) + v — S,

where S = 1 unless v = 2, char(K) = 2, mi ^ for all i e E, and

HieE + 1) °dd, in which case S = 3.

It follows from the theorem that dim RnK(r) = (i(T) + 1)n2 + O(1); in particular,

dim^.jcCD

----> /x(r) + 1 as n oo.

As suggested in [42], it is of interest to extend these results to representation varieties Hom(r, G) for other algebraic groups G, the natural focus being on the case where G is a connected simple algebraic group. For a positive integer m, define

Jm(G) = {x e G : xm = 1},

Then Jm (G) is an algebraic variety. Information about its dimension can be found in Sect. 4 (see Theorem 4.1).

Theorem 1.10 Let r be a Fuchsian group of genus g > 2 (g > 3 if r is non-oriented), let v = v(r), and let G be a connected simple algebraic group over an algebraically closed field K of arbitrary characteristic. Then

(i) dim Hom(r, G) = (vg — 1) dim G + Y?i=i dim Jm ((5);

(ii) dimHom(T,G) ^ (r) + j as dim(5 ^ ^

For surface groups we obtain more detailed information. In the statement below, for a simple algebraic group GG we denote by n1 (G) the fundamental group of G, that is, the kernel of the canonical map from the simply connected cover of G onto G; and n1(Cr)2 denotes the subgroup generated by all squares in n1(Cr).

Corollary 1.11 Let r be a non-virtually abelian surface group of genus g, let v = v(r), and let G be a connected simple algebraic group over an algebraically closed field K of arbitrary characteristic.

(i) We have dim Hom(r, G) = (vg — 1) ■ dim G.

(ii) If r is oriented, then the number of irreducible components of highest dimension in Hom(r, G) is equal to ^^G)|; in particular if G is simply connected, this number is 1.

(iii) If r is non-oriented, then the number of irreducible components of highest dimension in Hom(r, G) is equal to ^^G)/n1(G)21, except when (g, G) = (3, PSL2), in which case the number is 1 +

|ni(G )/ni(G )2|.

It is interesting to note that while results for finite groups are frequently deduced from corresponding results for algebraic groups, in our case the deductions are in the reverse direction.

Finally, we note that when G is a compact Lie group and r a surface group, the space Hom(r, G) has geometric significance. For example, in [40] its volume is defined and studied using a representation-theoretic formula; this is closely related to Witten's celebrated volume formula for the moduli space of flat G-connections on the relevant surface [53].

Notation and layout

We shall freely use the notation already introduced, together with the following. We define F to be the class of all co-compact Fuchsian groups as in (1.1) with g > 2, or in (1.2) with g > 3. For functions f, g, we write f ~ g to mean that there are positive absolute constants c1, c2 such that c1 f < g < c2 f. Finally, if m1,... ,md are positive integers and G is a group, then, setting m = (m1,... ,md), we define

Im(G) = {(gi, ...,gd) : gi e G, gf = 1, f[ gi e G'}.

The layout of the paper is as follows. In Sect. 2 we study the function ZG (s) and prove Theorem 1.1. In Sect. 3 we recall two character-theoretic formulae, essentially dating back to Hurwitz, giving the size of certain homomorphism spaces from a Fuchsian group to a finite group, and use these, together with Theorem 1.1, to prove Theorem 1.2 and various related results. Section 4 is devoted to counting elements of given order in finite classical groups; this is where Theorems 1.4 and 1.5 are established. In Sect. 5 we study ZM (s) for maximal subgroups M of finite simple groups. The bound on ZM (1) in Theorem 5.1 is one of the main tools in our proof of Theorem 1.6, given in Sect. 6. Finally, representation varieties are discussed in Sect. 7, where the results 1.8-1.11 are proved.

2. Character degrees

In this section we prove Theorem 1.1 and various related results. For G = An, it is shown in [32, 2.7] that ZG (s) ^ 1 as n ^ to for any s > 0. Moreover, for the double cover An, it follows from [49] that every faithful irreducible character of An has degree at least ci, where c1 > iisan

absolute constant. Also k(An) < c~l for some absolute constant c2. Hence for s > 0,

J2 X(1)-s < cinc-sn — 0 as n —> to.

xeIrr( An) faithful

This establishes Theorem 1.1 in the case where G/Z(G) = An. Hence it remains to deal with simple groups of Lie type.

For a finite group G, let k(G) denote the number of conjugacy classes of elements in G. And for a simple group G of Lie type, not of type 2B2, 2G2 or 2F4, define the rank r = r(G) to be the untwisted Lie rank of G (that is, the rank of the ambient simple algebraic group G); for G of type 2B2, 2G2 or 2F4, define r(G) = 1, 1, 2 respectively.

Lemma 2.1 Let G = G (q) be a quasisimple group of Lie type of rank r = r(G) over Fq. Then there are positive absolute constants c1, c2 such that

(i) k(G) < c1 qr, and

(ii) x(1) > c2qr for any nontrivial irreducible complex character x of G.

Proof Part (i) follows from [23, Theorem 1] for groups of bounded rank, and from [9] for groups of unbounded rank (see also [8, 9.1]). Part (ii) is immediate from [18]. □

Proof of Theorem 1.1(i) This now follows quickly. By Lemma2.1, we have

ZG(s) < 1 + ciqr • (c2qr)-s = 1 + c3(s)q-(s-1)r,

which tends to 1 as |G | — to, assuming that s > 1. This completes the proof of Theorem 1.1(i).

To see that in general the condition s > 1 is necessary, we note the following.

Lemma 2.2 (i) For q odd, ZL2(q)(1) — 3/2 as q — to.

(ii) zSL2(q)(i) — 2 asq — to

(iii) For G = L2(q) or SL2(q) and s < 1,we have ZG(s) — to as q — to.

Proof Suppose first that q is odd. From the character table of SL2(q) given in [7, 38.1], we see that L2(q) has | + 0(1) irreducible characters of degree q + ior q — 1, and three other nontrivial irreducible characters, all of degree at least \ {q — 1). This yields part (i).

For (ii), observe that by [7, 38.1] (for q odd) and [7, 38.2] (for q even), SL2(q) has q + O(1) irreducible characters of degree q ± 1, and at most five other nontrivial irreducible characters, all of degree at least — 1). Part (ii) follows, and (iii) is immediate from the above information. □

To prove 1.1(ii), we need more detailed information about irreducible characters of small degree of groups of Lie type. First we handle exceptional groups. The following is taken from [18].

Proposition 2.3 Let G = G (q) be an exceptional quasisimple group of Lie type over Fq, and define h = h (G) as follows:

G es e7 ff fa 2f.4 3d4 g2 2G2 2b2

h 29 17 11 8 11 2 5 3 2 3 2

Then there is an absolute constant c > 0 such that every nontrivial irreducible character of G has degree greater than cqh .

We can now deduce the following.

Corollary 2.4 If G is an exceptional quasisimple group of Lie type and s > -pp then ZG(s) 1 as |G| oo.

Proof By Proposition 2.3, we have

ZG(s) < 1 + c1 qr • (cqh)-s = 1 + c3(s)qr-sh.

Note that j < yj- for all types except 2B2, G2 and 3Z)4, so the result follows, apart from these cases. The irreducible character degrees and their multiplicities for 2B2(q) and 3D4(q) can be found in [47,5], and for G2(q) a summary can be found in [43, Appendix]. Inspection of this data shows that ZG(s) 1 for s > so, where .y(1 = | according as G = 2B2(q), G2 (q), 3D4 (q) respectively. This completes the proof. □

For classical groups we shall need the following slightly more refined information than that in [18]. This result is taken from [48]; the case where G is orthogonal is not explicitly stated there, but it follows easily from the proofs of [48, Theorems 6.1, 7.6].

Proposition 2.5 Let G = G (q) be a classical quasisimple group over Fq, write H = G/Z(G), and let f = f(G) be as in Table 1 below. Then there is an absolute constant c > 0 such that G has at most q + 2 nontrivial irreducible characters of degree less than cq f .

Corollary 2.6 Let d be a family of classical finite quasisimple groups, not

L2(q) or SL2{q), and define So = limsupGs£) j^. Then for any s > so, we

have ZG(s) ^ 1 as with G e d.

In particular this holds when So = § and ¡D consists of all classical groups apart from L 2(q), SL 2(q).

Table 1

H Ll(q) (n > 4) L\(q) PSp2n(q) PQ.ln+l(q)(n> 3) Pilln(q)(n> 4)

f 2 n - 4 3 2 n - 1 2 n - 1 2 n - 2

Proof Using Lemma 2.1 and Proposition 2.5, we have

ZG(s) < 1 + (q + 2) • (c2qr)—s + (ciqr) • (cqf )—s = 1 + c3(s) • q1—rs + c4(s) • qr—fs.

As L2(q), SL2(q) are excluded, we have r > 2. The first assertion follows, provided s > max(s0, 7), which is equal to .y(1 (since j > \ in all cases).

T- < 2 / - 3

G = L3 (q) or P^8 (q)), yielding the last part. □

An easy check shows that j < | in all cases (with equality when

Proof of Theorem 1.1(ii) This now follows from Corollaries 2.4 and 2.6. This completes the proof of Theorem 1.1.

Combining the results 2.1, 2.3 and 2.5 easily yields the following.

Corollary 2.7 There exists an absolute constant c such that if G = L2(q) is a finite simple group and n is a positive integer, then G has at most cn2/3 irreducible characters of degree n.

Similar methods yield an analogue of Theorem 1.1 for nearly simple groups, that is, finite groups G such that F*(G) = G0 is quasisimple. For such a group G, the function ZG(s) — ZG/G0(s) is the sum x(1)—s over all irreducible characters x of G with ker x < Z(G0). Moreover, G/G0 < Out(G0/Z(Go)) is a soluble group with a transparent structure, and ZG/G°(s) can be easily computed. Note that ZG(s) > |G/G'|, and also ZG/G0(s) > |G/G'|, with equality if and only if G0 = G'.

Theorem 2.8 Let G be a finite nearly simple group with F*(G) = G0, and fix a real number s.

(i) Ifs > 1, then ZG(s) — ZG/G0(s) — 0 as |G| — to.

(ii) Ifs > 2/3 and G 0 = L 2(q) or SL 2(q), then ZG (s) — ZG/G0 (s) — 0 as G | — to.

(iii) In particular, ifG/G 0 is abelian then ZG (s) = |G/G' |+o(1) fors > 1, and the same holds for s > 2/3 if G0 = L2(q) or SL2(q).

Proof The case where G0 = An is covered by [32, 1.1], and that where G0 = An follows easily as at the beginning of this section.

So assume G0 is of Lie type, of rank r over Fq. By Lemma 2.1 together with Clifford's theorem, every irreducible character of G with kernel contained in Z(G0) has degree at least c2qr. It is well known that k(G) < |G : G0| • k(G0), so Lemma 2.1 gives k(G) < ciqr • |G : G0|. Moreover, |G : G01 < |Out(G0)| < c3r logq. Hence k(G) < c4rqr log q. It follows that

ZG(s) — ZG/G0(s) < c,rqr log q • (c2qr)~s < c5(s) • r log q • q—r(s—1).

If s > 1, the right hand side tends to zero as q or r tends to infinity, proving part (i).

For part (ii), in the case where G0 is an exceptional group, Proposition 2.3 shows that any irreducible character of G with kernel contained in Z(G0) has degree at least cqh, and hence

ZG (s) — ZG/G0 (s) < c4rqr log q • (cqh)—s.

Excluding type 3Z)4, we have < §, so the conclusion follows; for G of type 3D4 we use [5] as in the proof of 2.4.

Part (ii) in the case where G0 is classical is handled similarly, as in the proof of Corollary 2.6.

Finally, part (iii) follows immediately from (i) and (ii). □

Sometimes we shall also need to study certain variants of ZG involving only the real irreducible characters of G. For a finite group G, and a real number s, define

ZG (s) = J2 x(i)—s;

xeIrr(G), x real

and for an integer k, define

ZrG(k) = J2 i(x)kx(i)—k,

xeIrr(G)

where i(x) e {0, 1, —1} is the Schur indicator of x. Note that a finite group G has |G/G2| real linear characters, where G2 is the group generated by the squares in G. Hence

ZG(k) > |G/G21.

Lemma 2.9 Let G be a finite nearly simple group with F*(G) = G0. Suppose that either s > 1, or G0 = L2(q), SL2(q) and s > 2/3. Then

ZG(s) = ZG/G0(s) + o(1), and ZG(s) = ZrG/G0(s) + o(i).

If moreover G/G0 is abelian, then both zr (s) and Zf (s) have the form |G/G21+ o(1).

Proof Note that ZG(s) — zr/g0(s) = x(1)—s, the sum running over irreducible real x e Irr(G) such that ker x < Z(G0). This sum is bounded above by the sum x(1)—s over all x e Irr(G) such that ker x < Z(G0), which is of the form o(1) by Theorem 2.8. If G/G0 is abelian, then zr /G0 (s) = |G/G2|. This completes the proof for ZG, and a similar argument gives the conclusion for ZG. □

We conclude this section with a result on ZG and ZG for G = GLn (q) which is important for later applications. In the statement we fix n and let

q — to.

Proposition 2.10 Fix n > 2, and let G = GLn (q).

(i) For n > 3 and s > 2, we have ZG(s) = q — 1 + o(1).

(ii) Forn > 3 and k > 1,we have ZG(k) = (q — 1, 2) + o(1).

(iii) Forn = 2, we have ZG(2) = q + o(1), and ZG(s) = q — 1 + o(1) for s > 2.

(iv) For n = 2, we have ZG(1) = (q — 1, 2) + 1 + o(1), and ZG (k) = (q — 1, 2) + o(1) fork > 1.

Proof (i) First note that G has at most c1qn irreducible characters, by Lemma 2.1. There are q — 1 linear characters, contributing q — 1 to ZG (s). The remaining characters have degree at least c2 qn—1. Hence for s > 2,

q — 1 < ZG(s) < q — 1 + O(qn • q—2(n—1>),

giving the conclusion since n > 3.

(ii) The linear characters contribute (q — 1, 2) to ZG (k) for any k > 1. Of the remaining irreducible characters, Proposition 2.5 shows that there are at most c3q2 of degree less than c4q2n—4, and these have degree at least c5qn—1. Hence for k > 1,

(q — 1, 2) < ZG(k) < (q — 1, 2) + O(q2 • q—(n—1)) + O(qn • q^2"^).

This gives the conclusion for n > 5.

For n = 3 or 4, we use the information in [46]. Analysis of the character tables there shows that there are just O(q2) real irreducible characters of GL3(q) in all, and O(q) of these have degree less than c6q3, which yields

ZG(1) = (q — 1, 2) + O(q.q—2) + O(q2.q—) = (q — 1, 2) + o(1).

For n = 4, the number of irreducible characters of GL4(q) of degree less than c7q5 is O(q2), and hence

ZrG(1) = (q — 1, 2) + O(q2.q—3) + O(q4.q—5) = (q — 1, 2) + o(1).

(iii) As follows from [46], GL2 (q) has (1+o(1))q2 non-linear characters, and their degrees are q — 1, q, q + 1. The conclusion follows.

(iv) Again from [46], it can be checked that GL2(q) has q + O(1) real non-linear characters; their degrees are q, q — 1, q + 1, and Schur indicators are +1. The conclusion follows easily. □

Proposition 2.10 and its proof yield the following.

Corollary 2.11 Fix n > 2, and let G = GLn (q). Then for any integer k > 1,we have ZG (k) — ZG (k) = o(1) (as q ^ œ). Consequently

ZG (k) = (q — 1,2) + s + o(1),

where S = 0 unless n = 2 and k = 1, in which case S = 1.

3. Counting homomorphisms

In this section we prove Theorem 1.2 and related results. We shall use a well known formula expressing the sizes of certain spaces of homomorphisms in terms of characters. At this point the results of the previous section will come into play.

Throughout the section we assume that r is a Fuchsian group as in (1.1), (1.2) with s = t = 0.

Let G be a finite group, and C = (C1,... , Cd) be a d-tuple of conjugacy classes Ci of G with representatives gt. Set

Home(r, G) = { e Hom(r, G) : 0(xl) e Ci for i = 1,... , d}.

The next result, essentially dating back to Hurwitz, plays a key role in this paper; for a proof, see for example [32, 3.2].

Lemma 3.1 Let r be a co-compact Fuchsian group and G a finite group.

(i) If r is oriented, then

|Homc(T, G)| = |G|2g_1|Ci| • • • |Q| ^

xeIrr(G) x( )

(ii) If r is non-oriented, then

|Homc(T, G)| = |G|g_1|Ci| • • • |Q| J]

xeIrr(G) x( )

where i(x) e{0, 1, —1} is the Schur indicator of x.

We now estimate |Hom(r, G)| in terms of the functions ZG, ZG studied above. We start with the case of surface groups, which is already known [39], and follows from Lemma 3.1 by substituting d = 0.

Corollary 3.2 Let r be a surface group of genus g and let G be a finite group.

(i) If r is oriented, then |Hom(r, G)| = |G|2g—1ZG(2g — 2).

(ii) If r is non-oriented, then |Hom(r, G)| = |G|g—1ZrG(g — 2).

In order to deal with general Fuchsian groups, recall that we define jm(G) to be the number of solutions to the equation xm = 1 in G; also v = v(r) is 2 if r is oriented and 1 if not.

Lemma 3.3 For any finite group G, we have

2 - \$G(vg - 2) <-HO'nfr' G)l-< \$G(vg - 2).

|G|"g-i-n?=ii»,(G)

Proof First assume that r is oriented. Observe that for x s Irr(G),

x{\)d-2+2g -M >

X(gi) • ■■X(gd)

X(1)d-2+2g 1=xgIrr(G) AV 7

< ZG (2g - 2) - 1,

which yields

2 — Z°(2g — 2) < Y: ^^8-2). (1)

XS Irr(G)

|Hom(r, G)\=J2 |HomC(r, G)|,

where the sum ranges over d-tuples C of classes Ci of elements of order dividing mi .Also Ee |C1|--- \Cd \ = f[ f=1 jmt (G). Applying Lemma 3.1(i) and (1), we obtain

(2 - ZG(2g - 2))|G|2g-1n jmi(G) < |Hom(r, G)|

< ZG(2g - l^G^H jmt (G).

This completes the case where r is oriented.

Now assume r is non-oriented. We use 3.1(ii). Observe that for x s Irr(G), we have

, ,g X(gi) ■■■x(gd)

< X(1)-(g-2)-

X(1)d-2+g

The proof now follows as in the oriented case above. □

Lemma 3.3 is particularly useful when ZG (vg — 2) is close to 1, since the upper and lower bounds are then both close to 1.

Proof of Theorem 1.2 Let r be a Fuchsian group in F (recall that this means r has genus g > 2 (g > 3 in the non-oriented case)), and let G be a finite quasisimple group.

(i) Suppose r is oriented. Since 2g — 2 > 2, we have ZG (2g — 2) = 1 + o (1) by Theorem 1. 1 (i), and so Theorem 1.2(i) follows from Lemma 3.3.

(ii) Suppose now that r is non-oriented, and either G/Z(G) = L2(q) or g > 3. Then using both parts of Theorem 1.1, we have Z G (g — 2) = 1 + o(1), and Theorem 1.2(ii) follows again from Lemma 3.3.

(iii) Assume now r is non-oriented with g = 3 and G = L2(q). Write h = 3/2 if q is odd, and h = 2 if q is even. Let Ci be classes of elements of order dividing mi in G (1 < i < d). We claim that

|Home(T, G)| = (k + o(1)) -|G|2 • • • • |Cd|, (2)

where k = 1 unless Ci = {1} for all i, in which case k = h.

To prove the claim, suppose first that Ci = {1} for some i, and let gi e Ci. Inspection of the character table of G = L2(q) in [7, Sect. 38] shows that x(gi)/x(1)| < 2q—1/2 for all nontrivial x e Irr(G), and hence

< 2q—1/2 £ x(1)—1

1=xe Irr(G)

= 2q—1/2(ZG (1) — 1). Since ZG (1) is bounded (see Lemma 2.2), this shows that

E } =1+2<T1/2 • oa) = 1 + 0(1).

xeIrr(G) x( )

Hence by Lemma 3.1(ii), we have

|Home(r, G)| = (1 + o(1)) • |G|2 • |Ci| ••• |Cd|,

proving the claim (2) under the assumption that Ct ={1} for some i. Next, suppose Ct = {1} for all i. Then Lemma 3.1(ii) gives

|Home(T, G)| = |G|2 • £ i(x)x(1)—1 = |G|2 • ZG(1).

xe Irr(G)

It is easily checked from the character table that i(x) = +1 for all but at most two irreducible characters x of G = L2(q). Hence from Lemma 2.2 we have

ZG(1) = (1 + o(1)) • ZG(i) = h + o(1),

which completes the proof of (2).

Applying (2) and summing over all d-tuples C1;... ,Cd of classes of elements of orders dividing m ]_,... ,md respectively, we obtain

|Hom(r, G)| = (1 + o(1))|G|2(jmi(G) • • • jmd(G) — 1) + (h + o(1))|G|2.

If (mi, |G|) = 1 for some i the right hand side has the form

(1 + o(1))|G |2 jmi (G) ••• jmd (G),

while if (mi, G|) = 1 for all i, then it has the form (h + o(1))|G|2.

L »(/)

1=X€ lrr(G)

gX(gl)

X(1)d-2+g

This completes the proof of Theorem 1.2(iii).

(iv) Suppose r is non-oriented with g = 3, and G = SL2(q) with q odd. The proof is similar to the previous case, but a few modifications are necessary. Let Ci be classes of elements of order dividing mi in G (1 < i < d), and gi e Ci. Let z = —I be the non-identity central element of G. Recall that d* = d*(V) is the number of i such that mi is even. Denote by n(C) the number of i such that Ci = {z}. Note that n(C) < d*.

We claim that

|Homc (T, G )| = (k + o(1)) ■ G |2 ■|Cl|■■■ C U (3)

where k = 1 unless Ci | = 1for all i and n(C) is odd , in which case k = 2.

The proof of (3) in the case where Ci | = 1 for some i is essentially identical to the analogous proof for G = L2(q), so it will be omitted.

Assume then that the classes Ci are all central, and let n = n (C). Note that for an irreducible character x of G we have x(gi)/x(1) = 1 unless x is faithful and gi = z, in which case x(gi) = —1. Combining this with part (ii) of Lemma 3.1 we obtain

|Homc(r, G)| = |G|2(£ + £2), (4)

£1 = i(x)x(1)—1, £2 = E i(x)(—1)nx(1)—1.

xeIrr(G) nonfaithful xeIrr(G) faithful

Clearly £1 = ZrL2(q)(1), so as seen above,

£1 = 3/2 + o(1).

It is easily checked from the character table that i(x) = —1 for all but two

of the faithful irreducible characters / of G, of which there are f + 0(1)

of degree q ± 1. We conclude that

£2 = (-l)"+1Q + 0(l)

Hence £1 + £2 = k + o(1), where k is as in (3). Now (3) follows using (4).

Let Y (respectively Z) be the set of d-tuples (C1,... , Cd) such that Ci | = 1 for some i (respectively |Q | = 1 for all i). Note that | Z| = 2d*. Write

|Hom(r, G)| = A + B,

A = ^|HomC(r, G)|, B = |HomC(r, G)|.

CeY CeZ

By (3), we have

a = (i + o(i))\o\2 m m (g ) - n m (z (G » \i=i ¿=1 t

= (1 + o(1))\G\2(Y[jmi (G) - 2d*) . Also for d* > 0, (3) yields

B = \G\2 ■ I J2 1 + o(1) + J2 2 + o(1)

yCsZ, n(C) even CsZ, n(C) odd

= \G\2 ■ (3 ■ 2d*-1 + o(1)),

while B = \G\2(1 + o(1)) if d* = 0. In the latter case we have A + B = (1 + o(1))\G\2 nd=1 jm, (G). Assume now that d* > 0. If (mt, \G\) > 2 for some i ,then A is the dominant term and A+B = (1+o(1))\G\2 ]!?= jm (G). Finally, if (m,, \G\) = 1 or 2 for all i, then A = 0. Thus Theorem 1.2(iv) is proved in all cases.

This completes the proof of Theorem 1.2. □

Our methods also enable us to estimate \Hom(r, G) \ , where G is nearly simple, which is important for our applications to representation varieties. We first formulate the case where r is a surface group:

Proposition 3.4 Let r be a surface group of genus g, and let G be a finite nearly simple group with F* (G) = G0, a quasisimple group.

(i) If r is oriented with g > 2, then

\Hom(r, G)\ = \ G\ 2g-1 ■ (ZG/G0(2g - 2) + o(1)).

Inparticular, ifG0 = G' then \ Hom(r, G) \ = \G\ 2g-1 ■(\G/G'\ +o(1)).

(ii) If r is non-oriented with g > 3, and (g, G0/Z(G0)) = (3, L2(q)), then

\ Hom(r, G) \ = \G\ g-1 ■ (ZG/G0(g - 2) + o(1)). Inparticular, ifG0 = G' then \ Hom(r, G) \ = \ G \ g-1 ■ ( \ G/G2 \ +o(1)).

Proof Part (i) follows from Theorem 2.8 and Corollary 3.2(i). Similarly part (ii) follows from Lemma 2.9 and Corollary 3.2(ii). □

We next extend this result to general Fuchsian groups, assuming that G/G0 is abelian. This requires some preparation.

Recall from the Introduction that for a d-tuple m = (m 1,... ,md) of integers mi > 2, we define

Im(G) = {(g1, ...,gd) : gi s G, gm' = 1, [f gi s G'}.

Also, letting G2 denote the group generated by the squares in G, define

im (G) = {g,gd);: gt e G, g? = 1, g e G2}.

The following result is a variant of Lemma 3.3 which is useful in the case where G is not a perfect group.

Lemma 3.5 Let r be a Fuchsian group as in (1.1), (1.2), and let G be a finite group.

(i) If r is oriented, then

G |Hom(r, G)| G

21G/G \ - tg(2x - 2) < —-——— < Z (2e - 2).

1/1 4 K 8 IGI2«"1- |/m(G)| ;

(ii) If r is non-oriented, then

2 G |Hom(r, G)| G

2\GJG \ - (fa - 2) < |Gr,.|f,(J)| < tf(i - 2).

Proof The proof of this extends that of Lemma 3.3. For 1 < i < d let Ci = gG be a conjugacy class of G with g? = 1. Write

X(gi) ■ ■ ■ X(gd) _

Z^ yf\y-2+2g 1 + 2'

xeIrr(G) AV '

where £ and £2 are the sums over the linear and non-linear irreducible characters, respectively. Suppose HomC(r, G) = 0. Then the relation x1 ...xd [a1, b1 ]... [ag, bg] of r implies that g1... gd e G. Hence for every linear character x of G we have x(g1) ■ ■ ■ x(gd) = 1. This shows that £1 = |G/G'|. We also have < ZG(2g - 2) - |G/G'|. Now by Lemma 3.1 we have

|Homc(r, G)| = |G|2g-1|C1|... Cd| ■ (G/G'| + £2),

and so

i^i2^-^ n Ci WG/G'I - ZG(2g - 2)) < |Homc(r, G)|

< iG|2g-1["[ iCi iZG(2g - 2).

Summing over all Cs,... ,Cd such that ]"[i Ci £ G', and observing that En Ci ' |Cil ...\Cd \ = \Im(G )|, this yields (i).

The proof of (ii) is similar. For i < i < d let C¡ = gf be a conjugacy class of G with gf = 1. Write

y ^ y x(gi) ■■■ x(gd) = A+ A

xeIrr(G) AV '

where A1 and A2 are the sums over the linear and non-linear irreducible characters, respectively. Suppose HomC(r, G) = 0. Then the relation x1 . ..xdag ...a2g of r implies that g1 ...gd e Gg. Hence for every real linear character x of G we have x(g1) ■ ■ ■ x(gd) = 1. This shows that A1 = | G/Gg | . We also have | Ag | < zG(g - 2) - I G/Gg | . Now by Lemma 3.1 we have

|HomC(T, G)| = |G|g-1 |C11... |Cd| ■ (|G/Gg| + Ag). (5)

Summing over all C1,... ,Cd such that n 1 Ci c Gg, this yields the required conclusion. □

Theorem 3.6 Let r be a Fuchsian group in F, and let G be a finite nearly simple group with F*(G) = G0. Assume that G/G0 is abelian.

(i) If r is oriented, then

|Hom(r, G)| = |G|gg-1|Im(G)| ■ (|G/G'| + o(1)).

(ii) If r is non-oriented, and (g, G0/Z(G0)) = (3, Lg(q)), then

|Hom(r, G)| = |G|g-1|im(G)| ■ (|G/Gg|+ o(1)).

Proof (i) By Theorem 2.8(iii) we have ZG(gg - 2) = |G/G'| + o(1).

Substituting this in Lemma 3.5(i) gives the conclusion. (ii) By Lemma 3.5(ii),

|Hom(r, G)|

?-1 -j/m (g )i

- |G/G2j

< zG (g - 2) — |G/G2|.

The right hand side is of the form o(1) by Lemma 2.9, and the conclusion follows. □

In view of Theorem 3.6 it is important to estimate the sizes of the subsets 7m(G) and im(G). Trivially, we have

~[jmi (G') < |/m(G)j < |/m(G)| ^ m (G). (6)

i=i i=i

We shall show in Sect. 4 (see Corollary 4.4) that under some conditions, the upper and lower bounds in (6) are asymptotically the same, which will be crucial in the proof of Theorem 1.10 in Sect. 7.

For our applications on representation varieties we shall also need to estimate |Hom(r, G)| when G = GLn(q), where n is fixed and q ^ to (note that GLn (q) is not nearly simple). Again, we start with the easier case of surface groups.

Proposition 3.7 Let r be a surface group of genus g which is not virtually abelian, and fix n > 2.

(i) If r is oriented, then

|Hom(r, GLn(q))\ = (q - 1 + 5 + o(1)) ■ \GLn(q)\2g-1,

where 8 = 0 unless n = g = 2, in which case 8 = 1.

(ii) If r is non-oriented, then

!Hom(r, GLn(q))\ = ((q - 1, 2) + 8 + o(1)) ■ \GLn(q)\g-1, where 8 = 0 unless n = 2 and g = 3, in which case 8 = 1. Proof This follows by combining Corollary 3.2 with Proposition 2.10. □

Theorem 3.8 Let r be a Fuchsian group in F, and fix n > 2.

(i) If r is oriented, then

!Hom(r, GLn(q))\ = (1 + o(1))(q - 1) ■ \GLn(q)\2g-1 ■ \Im(GLn(q))\.

(ii) If r is non-oriented, then

\Hom(r, GLn (q))\ - \GLn (q)\g-1 ■ | I'm (GLn (q))|.

Proof (i) Set G = GLn(q). Then as g > 2, Lemma 2.10(i), (iii) shows that ZG(2g - 2) = q - 1 + 8 + o(1), where 8 = 0 unless n = g = 2, in which case 8 = 1. Substituting in Lemma 3.5(i), we obtain

!Hom(r, G)!

q- 1 - < ' , —— <q - l+5 + o(l).

1 ~ IGps-M/^G)! " 1

The conclusion follows.

(ii) By Corollary 2.11, ZG(g - 2) = (q - 1, 2) + 8 + o(1), where 8 = 0 unless n = 2, g = 3, in which case 8 = 1. Also \G/G2\ = (q -1, 2). Hence Lemma 3.5(ii) gives

!Hom(r, G)!

(q-l,2)-&- o(l) < —-, . | < (q - 1, 2) + 5 + o(l).

IG^"1 • |/5,(G)| "

The conclusion follows, unless n = 2, g = 3 and q is even. In this case, using the notation of the proof of Lemma 3.5(ii), and inspecting the character table of G = GL2(q), we see that if some Ci is non-central then A2 = O(q-1), and otherwise A2 = q/(q - 1). In either case, (5) shows that

!Homc(r, G) - \G\g-1\C1 \ ...\Cd

and the result follows by summing over all C1,...,Cd such that

n1 C« c G2. □

Note that the proof shows that the implied constants in part (ii) are between 1 + o(1) and 3 + o(1).

To apply Theorem 3.8 we need to estimate the sizes of the subsets Im(GLn(q)) and 4(GLn(q)). By (6), we have

Y\jm (SLn (q)) < \Im(GLn (q))| < | I'm (GLn (q))| < jm, (GLn (q)).

1=1 1=1

The next few lemmas show that under some extra conditions, tighter estimates can be obtained in terms of the numbers jm (GLn (q)) (which in turn will be studied in Sect. 4 - see Proposition 4.5).

Lemma 3.9 Let m = (m1,... , md), where the m1 > 2 are integers, let p be a prime, and let m i be the p' -part of mi. Let q be a power of p such that q = 1 mod mi for all i. Fix n > 2 such that (mi, n) = 1 for all i. Then as q ^ to, we have

\ Im(GLn (q))|~fl jm, (GLn (q)).

Proof In view of (7), it suffices to show that under the hypotheses of the lemma,

jm, (GLn(q)) = mi • jm, (SLn(q)). (8)

To see this, let m e Fq be an element of multiplicative order mi, and let z = mI e GLn (q). As (mi, n) = 1, det(z) also has order mi in Fq. Now let g e GLn (q) satisfy gmi = 1. Then det(g)m = 1, and so there is a unique power zj of z such that gzj e SLn (q). Moreover (gzj )mi = 1, and (8) follows. □

Lemma 3.10 Let m = (m1,... , md), where the mi > 2 are integers, and let q be a prime power. Ifq is odd, assume q = 1 mod 2a+1, where 2a is the maximal power of 2 dividing some mi. Then for any n > 2,

Im(GLn(q)) | =f] jm, (GLn (q)).

Proof Note first that GLn (q)2 consists of all elements of GLn (q) whose determinant is a square in Fq .If q is even then GLn (q)2 = GLn (q) and the result follows trivially from the definition of Imr (GLn(q)). So suppose q is odd and q = 1 mod 2a+1. We claim that for any g1,... , gd e GLn (q) satisfying gmi = 1, we have (g1, ... , gd) e Im (GLn (q)). Indeed, for 1 < i < d, write mi = 2aiki with ki odd and 0 < ai < a. Then 1 = det(gi)mi = det(gi)2a'ki. By our assumption on q, it follows that det(gi)ki is a square in Fq, and since ki is odd, det(gi) is therefore also a square. The claim follows, and with it the required conclusion. □

We remark that in general, |Im(GLn(q))\ and 1im(GLn(q))\ can be asymptotically smaller than d=1 jm (GLn (q)): for example, suppose d = 1, m 1 = n = 2 mod 4, q = 1 mod n and q = 3 mod 4. Then the largest class of elements of order dividing m 1 in GLn (q) contains g1 = diag(1, m, a2,... , a/"1-1), where a e Fq is a primitive mf root of 1; this has determinant -1, a non-square, hence does not lie in im(GLn (q)). It follows easily that in this example, fixing n and letting q ^ro, we have jm1 (GLn(q)) - q"2-n, while 1im(GLn(q))\- (GLn(q))\ - qn'-n-2.

Corollary 3.11 Let r be co-compact non-oriented Fuchsian group as in (1.2), of genus g > 3, and let q satisfy the assumptions of Lemma 3.10. Then for any fixed n > 2, we have

!Hom(r, GLn(q))\- \GL"(q)\g-1 ^ jm(GLn(q)).

Proof This is immediate from Theorem 3.8(ii) and Lemma 3.10. □

4. Counting elements of given order

In this section we obtain bounds on the numbers jm (G) of elements of order dividing m in G, where G is a quasisimple group of Lie type or a general linear group. We use these bounds to deduce Theorems 1.4 and 1.5.

One of our main tools is the following result of Lawther concerning the dimension of Jm (X), the variety of elements of order dividing m in a simple algebraic group X.

Theorem 4.1 (Lawther [20]) Let X be a simple algebraic group of rank r over an algebraically closed field, and let m be a positive integer. Then there is a constant c(m) depending only on m such that

(1 - - ) dimX — c{m) < dim Jm{X) < (1 - - ) dimX + —. \ m) \ m) m

Moreover, for X of adjoint type, dim Jm (X) is known and is given in [20].

The upper bound in this theorem is immediate from [20, Theorem 1]. The lower bound is an easy consequence of the work in [20] (see also the proof of (12) below).

Corollary 4.2 With notation as in Theorem 4.1, we have

dim Jm (X) 1

--> 1--as r —oo.

dim X m

Here is our main result on jm (G) for finite groups G of Lie type.

Theorem 4.3 Fix an integer m > 2, a prime p, let K = Fp, and let X be a simple algebraic group of rank r over K. For q a power of p, let G = G (q) be a quasisimple group of Lie type over Fq of the form X'a, where a = aq is a Frobenius endomorphism of X. Define t = j if G is of type 2B2, 2G2 or 2F4, and t = 1 otherwise. Then there are positive constants c1, c2, c3, c4 depending only on m, such that the following hold.

(i) jm (G) < jm (Xa) < c1mrqt dim Jm(X\

(ii) There exists q0 = pa such that if q is any power of q0, then jm (G) > c2qt d'm Jm ( x ).

(iii) For any power q of p, we have

q~C} ■ IGI1-"" < jm(G) < c4mr ■ q~»< ■ |G|1_»|.

Proof For G of type 2B2, 2G2 or 2F4, all the assertions are readily checked using the complete information on conjugacy classes of these groups found in [47,51,44]. So assume from now on that G is not of one of these types.

We first claim that if nm (X) denotes the number of conjugacy classes of elements of order dividing m in X, then

nm (X) < mr. (9)

To see this, write m = m1m2 where m1 is coprime to p and m2 is a power of p. Any element x e X of order dividing m is a commuting product x = x1 x2 , where x, has order dividing m^ for i = 1, 2. Here x1 is semisimple, hence lies in a maximal torus of X, and so there are at most mr1 possibilities for x1 up to X-conjugacy. Moreover, x2 is a unipotent element of D = CX(x1); by [45, II,4.4], D/D0 is a p'-group, and D0 is reductive. As in the proof of [23, 1.7(iii)], we see that the number of classes of unipotent elements of D is at most 6r. Hence nm (X) < (6m 1 )r. The claim (9) follows unless m2 < 5. In these cases we estimate the number of classes of unipotent elements of D0 of order m2 a little more carefully using the methods of [23, Sect. 1], and find that this number is at most mr2. This proves (9).

Now let x e Xa be an element of order dividing m, and consider the a-stable class xX. We bound the size of the fixed point set (xX)a in similar fashion to the proof of [31, Lemma 1]. Write x = x1 x2 as above. Then CX (x) is the centralizer of the unipotent element x2 in the reductive group D = CX (x1). Each of the following quantities is bounded by a function of m: \ D/D0\; the number of simple components of D0; and the rank of the torus Z(D0). Hence, writing C = CX(x), the corresponding statement is true for C/U = CD (x2)/U ,whereU is the unipotent radical of C.Moreover, \ Ua \ = qdimU by [21, 1.7]. By Lang's theorem [45, 3.4], (xX)a is a union of Xa-classes, the number of which is bounded in terms of \C/C0\, and the sizes of which are of the form \ Xa : (Cg)a \ for various a-stable conjugates Cg of C. Note also that dim(X/C) = dim xX. It follows that

c2(m) • qdimxX < \(xX)a \ < c1 (m) • qdimxX, (10)

where c1(m), c2(m) depend only on m. Part (i) follows; so does (ii), noting

that for suitable q0 we have Xa < X'a = G(q0), hence x e G(q0).

By Theorem 4.1, dim Jm{X) < (1 — dim X + so in particular, if x e Xa has order dividing m, then dimxA' < (1 — dim X + Since \G \ ~ qdimX, the upper bound of part (iii) now follows from (9) and (10).

We complete the proof by establishing the lower bound of part (iii). By taking c3 large enough, we may assume that the rank r of X is large. Let n be the dimension of the natural module V for G, and write n = km + t, where k, t are integers with k > 0,0 < t < 2m + 2 and k even for G unitary, symplectic or orthogonal. Then we may embed the cyclic group Cm = {x) in G in such a way that V | {x) = F © I, where F is free of dimension km and I is trivial of dimension s. Call such an embedding almost free. We shall show that

|xG| > q~C} ■ IGI1-™, (11)

where c3 depends only on m, which will establish the lower bound in (iii). Write G = X'a as above, where X is the corresponding classical algebraic group over Fq. Arguing as for (10), we see that

|(xX)ff| > ci (m) • qdimxX, and hence to prove (11) it is sufficient to show that

dimCx(x) <--hc2 (»?■). (12)

Write m = m 1m 2 and x = x1 x2 as above. Then x1 is semisimple, and on V has each eigenvalue different from 1 occuring with multiplicity km2, while the eigenvalue 1 has multiplicity km2 + t. Hence, if m1 is odd we see that CX(x1)0 is the image modulo scalars of

((GLkm2)m1-1 x GLkm2+t) n SLn, if X = PSLn (GLm)(mi-1)/2 x Spkm2+t, if X = PSpn

(GLkm2)(mi-1)/2 x SOkm2+t, if X = PSOn.

If m1 is even, the second line changes to (GLkm2)(m1-2)/2 x Spkm2+t x Spkm2, and similarly for the third line. A quick calculation with the dimensions of these groups yields

dim Cx (xi) <-+ c3 («/1) - (13)

Now x2 is a unipotent element in the semisimple group D = (CX(x1 )0)'. We shall show that

dim CD{x2) <--h c4{m). (14)

Together with (13), this will establish (12).

Now D is a product of at most m 1 + 1 simple factors, with x2 embedded almost freely in each, so it is sufficient to prove (14) for each simple factor of D. Let E be a simple factor, with natural module of dimension d. On this module the unipotent element x2 acts as ((Jm2)l,(J1 )u), where Ji is a unipotent Jordan block of size i, and we have d = lm2 + u with u < t. The dimension of CE (x2) can be read off from [50]: assuming E is not symplectic or orthogonal in characteristic 2, dim CE(x2) is as follows:

m2l2 + 2ul + u2 - 1, if E = SLd

\{m2l2+ 2ul + l + u2+ if E = Spd \{m2l2 + 2ul - I + u2 - u) if E = SOd.

A check of dimensions now yields (14). Finally, if E is symplectic or orthogonal in characteristic 2, then E has two classes of elements of type ((Jm2 )l, (J1)u); taking x2 in the larger of these, we have

dimCE(x2) = m2l2 + 2ul + u2 + u) or m2l2 + 2ul — l + u2 — u),

according as E = Spd or SOd, respectively. Again, (14) follows.

This completes the proof of the theorem. □

Proof of Theorem 1.4 Theorem 1.4 follows immediately from part (iii) of Theorem 4.3. □

Proof of Theorem 1.5 Let G be a finite simple classical group of rank r, and let r be a Fuchsian group as in (1.1) or (1.2).

Suppose first that r is co-compact of genus g > 2 (g > 3 if r is non-oriented). Recall that we defined v = v(r) to be 2 if r is oriented, and 1 otherwise. By Theorem 1.2, we have

|Hom(r, G)| - |G|vg"1 ^ jm(G).

Using Theorem 1.4, this yields

|Hom(r, G)| = |G|"g_1 • |G|E"1_^+ei(r),

where |e,(r)| < cim^r-1. Since vg - 1 + Ef=i(l " ¿) = ^ + the

conclusion of Theorem 1.5 follows, with S(r) = d=1 ^ (r).

Finally, if r is non co-compact (i.e. s + t > 0 in (1.1) or (1.2)), then from the preamble to Theorem 1.2 we have

|Hom(r, G)| = |G|vg+s+t-1 ^ jm(G),

and the same proof works, without any assumption on the genus of r. □

We conclude the section with a few further results concerning the functions jm, Jm, Im, im, which will be required later.

Corollary 4.4 Let m, K, X, t be as in Theorem 4.3, and write H = H(q) = Xa. Then the following hold.

(i) There exists q0 = pa such that for q a power of q0, we have jm (H') ~ jm (H) - qt dim Jm(X).

(ii) Let m = (m1;... , md) with all mi > 2. Then there exists q1 = pb such that for q a power of q1 , we have

\Im(H)|-|im(H)|- q^dimJmi(X

Proof Part (i) follows from Theorem 4.3(i),(ii), and part (ii) is immediate from (i) together with (6). □

In later applications we shall require the following version of Theorem 4.3 for jm(GLn(q)), which also gives an explicit formula for the dimension of Jm(GLn(K)) and Jm(SLn(K)).

Proposition 4.5 Fix integers m, n > 2, a prime p, and let K be an algebraically closed field.

(i) The conclusions of Theorem 4.3 hold for G = GLn (q) and X =

GLn (F p).

(ii) Writing n = km +1 with k, l e Z and 0 < l < m, we have

dim Jm(GL„(K)) = n2 (1 - - ) - / (1 - -

(iii) Ifm does not divide n, then given any mth root of unity X e K, there exists y e GLn(K) of determinant X such that dim Jm (GLn(K)) =

dim yGLn (KI

(iv) Ifm\n then there is a unique conjugacy class C = yGLn(K) such that dim Jm (GLn (K)) = dim C; moreover, y has determinant (-1)n(m+1)/m.

(v) We have dim Jm(SLn(K)) = dim Jm (GLn(K)), unless m is even, m divides n,n/mis odd and char(K) = 2,inwhichcasedim Jm(SLn(K)) = dim Jm (GLn (K)) - 2.

Proof (i) This follows from the proof of Theorem 4.3.

(ii) It is proved in [20,Theorem 1] that dim Jm{PGL„{K)) = n2{ — /(1 — If x e Jm{PGL„{K)), then as K is algebraically closed, x has a preimage in Jm(GLn(K)). It follows that dim Jm(GLn(K)) = dim Jm(PGLn(K)), proving (ii).

(iii, iv) We shall frequently use the following formula which can be found in [45, IV,1.8]: for X e K*, denote by Jv(X) the v x v Jordan block matrix with all eigenvalues X. If x e GLn(K) has Jordan form (Jj (X)ni)

(i.e. all eigenvalues X, and n is the multiplicity of the block of size i), and q = max{i : n > 0}, then

dim Cglh ( k )(x) = Y] (Hi + ... + Hq f. (15)

Note also that q=1 (Hi + ... + nq) = h.

Write m = pam 1, where p is the characteristic of K and m 1 is coprime to p (take a = 0if K has characteristic zero). Let h = um1+t withO < t < m1, and write u = kpa + s with 0 < s < pa. Then h = km + sm 1 + t and l = sm 1 + t. Let X1;... ,Xm1 be the mf roots of 1 in K, and define

m1 t m1

y = ® J pa X ))k e ® Js+1(Xi)) © ^ Js (Xi)). i=1 i=1 i=t+1

Calculating dimCGLn(K)(y) using (15), and using (ii), we see that

dimyGL"(K) = dim Jin(GLn(K)). Moreover, since n^ xi = (-1)m1+1 = (-1)m+1, we have

det( y) = (-1)u(m+1) . X1 ...Xt.

If t > 0 then we can choose X1;... ,Xt with product an arbitrary mf root of 1, giving the conclusion of (iii). So now assume that t = 0. If s > 0 and m1 > 1, define

Z = ^ Jpa (Xi ))k © Js+1(X1) © Js-1(X2) © (0 Js (Xi)). i=1 i=3

Using (15) we see that dim zGLn(K) = dim yGLn(K). Also det(z) = det(y) ■ X1X-1, so between them, y and z have determinant an arbitrary mf root of 1, and (iii) again holds.

Now assume s = 0 - that is, m |n and n = kpam1 = km. Here we have dim Cglh(k)(y) = k2m and det(y) = (-1)k(m+1). We claim that yGLn(K) is the unique class of highest dimension. To see this, let z e Jm (GLn (K )),and for 1 < i < m 1 let ri be the multiplicity of Xi as an eigenvalue of z. Write ri = kipa+li with0 < li < pa. By (15), for the given partition (r1,... , rm1) of h, the dimension of CGLn(K)(z) is minimal when z = z1 © ... © Zm1, where

Zi = ( Jpa (.Xi)k , Jk (Xi)).

For this z we have

dimCgl„(k)(z) = {pak2 + 2kili + h). i=1

This is equal to ^ >f\ — which is clearly at leasl/c2//'«/1. with

equality if and only if ri = kpa for all i - that is, if and only if z e yGLn(K). This establishes the claim. Part (iv) is now proved.

(v) If m does not divide n then the conclusion follows from (iii). So assume that m\n. It follows from (15) that all dimensions dim CGLn(K)(x) (x e Jm (GLn (K))) have constant parity, and moreover, if

X = J pa (Xi))k-i © J pa-i(Xi) © Ji(Ai ) © Q J pa (X, ),

then dimxGLn(K^ = dim yGLn(K) — 2. The conclusion of (v) follows. □

Corollary 4.6 Let m = (m 1,... , md), where mi > 2 for all i, let E = {i : mi even }, let K be an algebraically closed field, and let n > 2.

(i) Then

dim Im(GLn (K)) = £ dim Jm (GLn (K)) — e,

where e = 0 unless charf K) ^ 2, »;,\n for all i e E and ^('»i+l) is odd, in which case e = 2. (ii) Writing n = km + li with 0 < li < mi, we have

dimIm(GL„(K)) = n2 ■ J2 (l ~ —) ~ f1 " — ) "

i=i mi i=i mi

where e is as in part (i).

Proof (i) By Proposition 4.5, for each i we can choose yt e Jm (GLn(K)) such that dim yfLn (K} = dim Jm (GLn (K)) and det( yt) = ±i. If mi does not divide n for some i e E, then by 4.5(ii), both 1 and — 1 are possible for det(yi), so we can choose the yj so that nd =1 det(yj) = 1; then (yi,... , yd) e Im(GLn (K)), and hence

dim Im(GLn (K)) = £ dim Jm (GLn (K)). (16)

Now suppose mi\n for all i e E. Then by 4.5(iv), det(y) = (—1)n(mi+1)/mi for i e E, while det(yi) = 1 for i £ E, giving

Y[det( yi) = (-1)£ ieEn(mi+1)/mi. i=1

If this product is 1 then (16) holds again. If it is —1 (and char(K) = 2), then f"[ieE det(yi) = —1, so since —1 is not a product of mf roots of 1 for i e E, we see that (16) does not hold. In this case, choose i e E such that n(mi + 1)/mi is odd, and using 4.5(v), replace yi by zi e Jm(GLn(K)) of determinant 1 such that we have dim zGLn(K) = dim Jm (SLn(K)) = dim Jmi (GLn ( K )) — 2. Then ( yu ... , Zi, yd ) e Im (GLn ( K )), and hence we see that in this case

dim Im(GLn(K)) = £ dim Jm (GLn(K)) — 2. i=1

Part (i) is now proved.

Part (ii) follows from (i) together with the formula in Proposition 4.5(ii).

5. Maximal subgroups

Recall that for a finite group M and a real number s, we define

zM(s) = £ x(1)"s.

xeIrr( M)

In this section we prove the following result, which will be one of the main tools in our proof of Theorem 1.6.

Theorem 5.1 Let G be a finite simple group of Lie type of rank r over Fq, and let M be a maximal subgroup of G. Then there are absolute constants c, € > 0 such that

. c|G : M|

zM (1) <

unless G = L2(q) and M is a parabolic subgroup, in which case ZM(1) ~ |G : M|~ q.

We prove Theorem 5.1 in a series of lemmas.

Let G be a finite simple group of Lie type of rank r over Fq, and let M be a maximal subgroup of G.

Lemma 5.2 IfM is not a parabolic subgroup of G, then the conclusion of Theorem 5.1 holds.

Proof Obviously ZM(1) < k(M), the number of conjugacy classes of M, so we may assume that

k(M) ■|M|■q€r > c|G|, (17)

where € is an arbitrarily small, and c an arbitrarily large, positive constant.

Suppose G is classical, say G = Cln (q), with natural module V of dimension n over Fqu (where u = 2 if G is unitary, u = 1 otherwise). By Aschbacher's theorem [1], either M lies in one of the families C i (1 < i < 8) of subgroups of G ,or M lies in a family S of almost simple subgroups acting absolutely irreducibly on V; for explicit descriptions of the families Ci and full definition of S, see also [17].

If M e C1 then as M is not parabolic, it is of the form Clk (q) x Cln-k (q) (the stabilizer of a non-degenerate subspace of V). Now an easy check using Lemma 2.1(i) shows that (17) is violated: for example if G = Sp2r(q) and M = Sp2m (q) x Sp2r-2m (q) (so n = 2r, k = 2m), then 2.1(i) and (17) give

q(1+€)r . q2m2+m+2(r-m)2+r-m > c^2r2+r

leading to (1 + €)r > 4m (r — m), which is impossible.

For M e Ci with 2 < i < 8, the argument is similar, noting the following rough structure of such subgroups:

C2 : Clm(q) i Sk (mk = n), or GLn/2(qu) C3 : Clm(qk) (mk = n), or GUh/2(q) C5 : Cln(q1/k) (k > 2), or PSpn(q), PSOn(q) < G = Un(q) C8 : PSpn(q), PSOn(q), Un(ql/2) < Ln(q), or On(q) < Spn(q) (q even).

(Subgroups in the classes C4, C6, C7 are too small to concern us, having orders much less than |G|1/2.)

Finally, consider M e S. Here, for later use we establish the bound

k(M) < c|G : M|1/2, (18)

which is more than enough to violate (17). Now [22, 4.1] gives either |M| < q3un, or M e [AH+S, Sn+5} with 5 = 1 or 2. In the latter case (18) clearly holds. Next, observe that Lemma 2.1(i) implies that k(M) < c | M |1/2. Hence, in establishing (18) we may assume that |M|1/2 > c|G : M|1/2, which gives |G | < c1|M|2 < c1q6un. Inspection of the orders of the simple groups G = Cln (q) now shows that h < 12. For h < 12 we may assume that F*(M) e Lie(p) where q = pe (otherwise |M| is bounded), and it is simple to list the possible such groups having irreducible representations of dimension h < 12 (see [34] for example). In all cases (18) holds.

Now suppose that G is of exceptional type. By (17) we may assume that \M\ > |G|2~v for v arbitrarily small and positive. Hence Mis given by [24, Table 1], and inspection of this list, together with Lemma 2.1(i) shows that (17) is violated. □

In view of Lemma 5.2, we assume from now on that M is a parabolic subgroup, say M = QL, where Q is the unipotent radical and L a Levi subgroup.

We shall need some fairly crude information about k(M), the number of conjugacy classes of the maximal parabolic subgroup M. By [2], there is an L -invariant central series

1 = Qo< Qi <•••< Qi = Q

such that each factor Qi/ Qi-1 has the structure of an irreducible L-module over Fq (or possibly an extension field if G is twisted). Write Vi = Qi / Qi-1.

Lemma 5.3 Let C(L) be a set of conjugacy class representatives of the Levi subgroup L, and denote by o(L, Q) the number of orbits of L on Q. Then

k(M) < o(L, Q) + £ m \CVi(x)\J ■

1=xeC(L)\ i=1 )

Proof Every element of M is conjugate to an element of a coset Qx with x e C(L). The number of M-classes in the coset Q is at most o(L, Q). Now consider a coset Qx with 1 = x e C(L). For any u e Q we have

\Cq (ux)\<W\CVi (x)\, i=1

and hence \(ux)Q \ > \Q\/ (n \ CVi(x)\). Hence the number of Q-classes in Qx is at most f[ \ CVi (x) \. The conclusion follows. □

Here are our estimates for conjugacy class numbers of maximal parabolic subgroups.

Proposition 5.4 There is an absolute constant c > 0 such that if G is a classical group of rank r over Fq, and M = QL is a maximal parabolic subgroup of G, then

k(M) < c\Q\-q2r/3.

Proposition 5.5 There are absolute constants c,p > 0 such that if G is an exceptional group of Lie type over Fq, and M = QL is a maximal parabolic subgroup of G, then

k(M) < c\Q\ ■ q~p■

Proof of Proposition 5.4 Suppose first that G = SLn (q), and let M = Pm, the stabilizer of an m-space. Then M = QL with L = (GLm(q) x GLn-m (q)) n G and Q = Vm(q) ® Vn-m (q) as an FqL-module. Since Pm = Pn-m, we may assume that m < n/2.

If n = 2then M = P1 = AGL1 (q), and it is trivial to see that k(M) = q, giving the conclusion. So assume that n > 3.

Let C(L) be a set of class representatives for L. Then |C(L)| < cqn 1 by Lemma 2.1. Set C(L)* = C(L)\{1}, and define

C1 = C(L)*n (1 ® GLn—m (q)), C2 = C(L)* n (GLm(q) ® 1), C3 = C(L)*\(C1 U C2),

andfor1 < i < 3, let £i = £xeC. |Cq(x) |. By Lemma 5.3, we have

k(M) < o(L, Q) + £1 + £2 + S3. (19)

For x e C2 U C3 we have dim[Q, x] = dim Q — dim Cq(x) > h — m (see (22) below). Hence

£2 + £3 < cqn—1 ■ |Q| ■ q~(n—m = c|Q|- qm—1 < c|Q| ■ q2r/3, (20)

the last inequality since r = n — 1 and m < h/2.

To estimate £1, subdivide C1 into the following two subsets:

C'1 = {1 ® y e C1 : dim Cvn—m (y) > 2(h — m)/3}, C\ = C1\Ci.

For 1 ® y e C1 we have y e GL[2(„—m)/3](q) © 1 < GLn—m(q), so by Lemma 2.1,

|C1| < cq[2(n—m)/3].

Also |C'/| < |C11 < cqn—m, and for x = 1 ® y e C? we have dim Cq (x) < 2m(H—m)/3. Hence,defining £1 = £xeq |Cq(x)|, = Y_]XeCl |Cq(x)|, we have

£1 = £1 + £1' < cq[2(n—m)/3] ■ |Q|+ cqn—m ■ q2m(n—m)/3

<c\Q\ ■ (q^-'"^ + g(»-'">(i-f)) < C\Q\■ q2r/2 (21)

(recall that the rank r = n — 1 here).

Now observe that o(L, Q) < cm. Hence the conclusion follows from (19), (21) and (20). This completes the proof for G = Ln(q).

Next consider G = Spn (q) (with h = 2r). Take M = Pm, the stabilizer of a totally isotropic m-space. Here L = GLm (q) x Sp2r—2m (q) and Q has an L -invariant central series 1 < Q1 < Q, where as L -modules Q/Q1 = Vm (q) ® V2r—2m (q) and Q1 = S2(Vm(q)) (with trivial action of the factor Sp2r—2m (q) on Q1). If m < r we argue as above with the factor M/Q1 = (Vm(q) ® V2r—2m(q)).(GLm(q) x Sp2—m(q)), obtaining k(M/ Q1) < c| Q / Q11 ■ q2r/3. And if m = r then we have M = Pr = (S2(Vr(q)).GLr(q). A simple check shows that if 1 = x e L then dim[Q, x] > r/3, hence dim Cq(x) < dim Q — (r/3). Therefore, letting C(L) denote a set of class representatives from L (so that |C(L) | < cqr by Lemma 2.1), we have

J2 |Cq(x)| < cqr ■ |Q| ■ q—r/3 = c - |Q|- q2r/3.

1=xeC(L)

The conclusion now follows from Lemma 5.3.

Similar arguments yield a proof for the other classical groups Cln (q), noting that L = GLm (qu) x Cln-2m (q) and Q has an L-invariant series 1 < Q1 < Q, where Q1 and Q/ Q1 are the following L-modules:

G = SOn(q) : Q1 = A2(Vm(q)), Q/Q1 = Vm(q) ® Vn-2m (q) G = SUn(q) : Q1 = Vm(q2) ® Vm(q2)(q) (realised over Fq), Q/Q1 = Vm (q2) ® Vn-2m (q2)■

This completes the proof of Proposition 5.4. □

For the proof of Proposition 5.5, we require some results which classify the possibilities for the L-modules Vi = Qi/Qi-1 occurring within Q, and some information about the action of L on such modules. This is given in the next two lemmas. We use the standard notation for irreducible representations of groups of Lie type in the natural characteristic: thus V(X) = VG (X) denotes the irreducible G-module of high weight X in characteristic p. We often abbreviate V(X) by writing just X. Also for groups of small rank we write to represent the weight aX1 + bX2 + ■ ■■, where a, b, ■ ■■ are nonnegative integers and the Xi are the fundamental dominant weights. Finally, if V is a G-module in characteristic p, we write V = V(X)/V(X') /... or just X/X' ■■■ to indicate that the composition factors of V are V(X), V(X'), —

We need a definition, taken from [30]: if K is a field, and V a finite-dimensional vector space over K, set V = V ® K (where K is the algebraic closure of K), and for x e GL(V) define

v(x) = vV(x) = min {dim[V, ax] : a e K*}■

We shall also need the following elementary fact, taken from [30, 3.7]: if Va, Vb are K-vector spaces of dimensions a, b respectively, and x = x1 ® x2 e GL(Va) ® GL(Vb) is an element of prime order (acting in the obvious way on Va ® Vb), then

VVa®Vb(x) > max (avVbx), bvVax))■ (22)

Lemma 5.6 Let G be a finite group of Lie type over Fq, and let V = V(X) be an irreducible FqG-module of high weight X as in Table 2. Then for any semisimple element x e G\Z(G), we have vV(x) > nX, where nX is as specified in the table.

Moreover, for the entries in the table for G = Df5 (q) or C3 (q), the number of G-classes of semisimple elements x with vV (x) < 6 is at most c or cq, respectively.

Proof For G of type An, we have V(X) = a2(W) or A3(W) where W is the natural G-module, and finding a lower bound for v(x) with x e G semisimple is a routine calculation; in all cases the minimum value is attained by a diagonal matrix having an eigenspace on W of codimension 1 or 2. For G = E7 (q) note that if A7 (q) is a subgroup of maximal rank in G,

Table 2

G k dim ) n\

Ei(q) El(q) A*H(q)(n>3) K(q)(n> 5) Dj(q) Dl(q) D\(q) B3(q) C3(q), (q odd) h h k2 x3 Xtj k5 X-4 x3 k3 56 27 \n(n + 1) lzn(n2 - 1) 64 32 16 8 14 14 8 n n +3 8 8 5 4 4

we have V(X7) \ A7 (q) = VAl (X2) + VAl (X6) (see [25, 2.3]), and the bound follows from the A7 bound of l on VAl (X2) already observed. Likewise, for G of type E6, the restriction V(Xi) | Ai A5 = (1 ® Xi)/(0 ® X4) yields the bound 8 for v(x). The bounds for spin modules of Dn are obtained similarly by restricting to a Levi subgroup T1 An-1 using [25, 2.6]; the bound for B3 is clear; and the bound for C3 follows from the fact that the 14-dimensional module V(X3) is the wedge-cube of the natural module, factored out by a copy of the natural module. Finally, the last sentence of the lemma follows from the above considerations as well. □

We now complete the proof of Proposition 5.5 by using Lemma 5.6 together with Lemma 5.3, observing some relevant L-modules which occur as composition factors in Q. Here are the details.

Lemma 5.7 Let G be exceptional of Lie type over Fq, and let M = QL be a maximal parabolic subgroup of G. Table 3 below lists some of the high weights X of irreducible L-modules which occur as composition factors within Q. (The X's are listed only up to duals.)

Proof This is routine computation. Write L(G) for the adjoint module for G over Fq (so L (G ) is the restriction to G of the Lie algebra L (G), where G is the simple algebraic group corresponding to G). The composition factors of the restriction of the adjoint module L (G ) | L ' can be read off using [25, 2.1]. Further, if Q- denotes the unipotent radical of the parabolic opposite to M, then L (G ) = L ( Q ) + L ( L ) + L ( Q- ), all three subspaces fixed by L, and as L-modules, L( Q-) affords the dual of L( Q). It is therefore straightforward to compute the L'-composition factors of L ( Q), giving the conclusion. □

Table 3

type of G type of L'

Es Ei h

Ai Xi, X2, A3

AiA6 1 ® Ai, 1 ® A2, 0® Ai, 0® A3

AiA2A4 1 ® 10® m, 0® 01 ® M, 1 ® 00® A2, 0® 10 ® A2

A3A4 100 ® A2,000 ® X2, 100 ® A4, 010 ®

A2DS 10® 00® 10® Ai

AiE6 1 ® Ai,0® Ai

Di Ai, Xg

Ei D6 X5

A6 Ai, A3

AiA5 1 ® A2,O® A2

A1A2A3 1 ® 10® AI,0® 10® A2

a2a4 10® AI, 10® A2

A1D5 0 ® Ai, 1 ® A4

DS (e = +) X4

D4 (e = -) X3, X4

¿5 A3

A\A4 (e = +) 0® Ai, 1 ® A2

Al(cf)A2(q)(e = ~) 1 ® 1 ® 10, 0 ® 1 ® 10, 1 ® 0 ® 10

AIA2A2 (e = +) 1 ® 10® 10, 0® 10® 10

Ai(q)A2(q2)(€ = -) 1 ® 10® 10(«\0® 10® 10"?'

f4 c3 A3 (q odd) A3, Ai (q even)

AiÄ2 1® 02, 0®02(q odd) 1 ® 02, 0 ® 02, 1 ® 10, 0 ® 10 (q even)

AI A2 2® 10, 1® 10, 0 ©10(5 odd) 2 ® 10, 1 ® 10, (0 ® 10)2 (q even)

B3 X \, X3

2f4 2B2 102

AI 1, 2°, 1 ® 2°

g2 Ai l2

Äi 3(p#3) 3,Hp = 3)

3d4 A\(q) l4

Ai(q3) 1 ® l1«1 ® 11«2)

Proof of Proposition 5.5 Let G be an exceptional group of Lie type of rank r = r(G) over Fq, and let M = QL be a maximal parabolic subgroup. Assume for now that G is not of type 2B2, 2G2, G2 or 3D4; we handle these cases at the end of the proof.

Adopt the notation of the preamble to Lemma 5.3: so 1 = Q0 < Q1 < ••• < Qi = Q, and Vi = Qi/Qi-1 are the L-composition factors within Q. Let C(L) be a set of conjugacy class representatives of non-identity elements of L, and let C(L)t consist of those elements of C(L) whose semisimple part lies in Z(L). For x e L set vi(x) = vVi (x). Then Lemma 5.3 gives

k(M) < o(L, q) + J2 icq(X)i + iQi- J2 q"E1vi(x)■

xeC(L)t xeC(L)\C(L)t

Now | Z (L )| < cq (except for the cases where (G, L') = (2E6 (q), 2D4(q)) or (2E6(q), A1 (q2) A2(q)),when|Z(L)| < cq2). Also the number of unipotent classes in L is bounded by a constant, and hence |C(L)t| < c'q (or c'q2 in the exceptional cases). Since every non-identity element of Z(L) acts nontrivially on some composition factor V listed in Table 3, it follows that

o(L, Q) + J2 ICQ(x)l < c|Q| ■ q-1.

xeC( L )t

Now we know from Lemma 2.1 that |C(L) | < cqr. Hence it will suffice to show that for x e C(L)\C(L)t, we have

(x)> r(G). (23)

By definition of C(L)t, each element of C(L)\C(L)t has a power which is a non-identity semisimple element of L\Z(L). Hence (23) follows quickly from Lemmas 5.7 and 5.6 together with (22), except in the following cases: (G, L') = (E-, D-), (E6, D5) or (F4, C3). In the last two cases the extra refinement in the last sentence of Lemma 5.6 gives the conclusion. And in the first, we have vV(Xi)(x) > 2 for any non-identity semisimple x e D-(q) (where i = 1, 3 or 4); moreover, if vV1 (x) = 2 then vVi (x) > 2 for i = 3 or 4, and this yields (23).

We complete the proof by handling the postponed cases where G is of type 2B2, 2G2, G2 or 3D4. In the first two cases M = QL is a Borel subgroup, with | Q | = q2 or q3 and L cyclic of order q — 1. Moreover, |Z(Q)| = q, and L acts faithfully on both Z(Q) and Q/Z(Q). Hence Lemma 5.3 gives k(M) < cq or cq2 respectively, as required. Now consider G = G2 (q). Let T be a maximal torus of order (q — 1)2 in L. Then T lies in a maximal rank subgroup of G of type A1A1 (q), and (L(G2)/L(A1A1)) | A1A1 = 1 ® 3 (or 1 ® (3/1) if p = 3). Hence we can parametrise T by ordered pairs (c, d) e (F* )2, where (c, d) has eigenvalues c±1, d±3 on the

L-composition factors of Q listed in Table 3. It follows that

£ q-Ei W< cq-\

xeC(L)\C( L )t

and now the conclusion follows as above. The argument for G = 3D4(q) is similar, taking T of order (q — 1)(q3 — 1) in a maximal rank subgroup of type Ai(q)Ai(q3).

This completes the proof of Proposition 5.5. □

Proof of Theorem 5.1 We are now in aposition to complete the proof of 5.1. Let G be a finite simple group of Lie type of rank r over Fq, and let M be a maximal subgroup of G. Recall that

zM(i) = £ x(1)—1 •

xe lrr(M)

By Lemma 5.2, we may assume that M is a parabolic subgroup of G.

If G is of exceptional type, the result is immediate from Proposition 5.5, since clearly ZM(1) < k(M).

Now assume that G is classical. Write M = Pm = QL as above. Our estimation of ZM (1) requires one further ingredient.

Lemma 5.8 If x is an irreducible character of M such that Q < ker x, then x(1) > cqr—1, where c > 0 is an absolute constant; further, in the case where G = Ln (q) we have x(1) > cqr.

Proof First consider G = Ln (q). Here Q = Vm (q) ® Vn-m (q) and L is the image modulo scalars of (GLm (q) x GLn—m (q)) n SLn(q). By Clifford's theorem, x i Q = Q, where e is an integer and the 9t are L-conjugate linear characters of Q. Consequently, x(1) is at least the size of a nontrivial orbit of L on the linear characters of Q, and such an orbit has size at least (qm — 1)(qn—m — 1)/(q — 1) (see for example [17, 5.2.2]), whichis at least cqn—1 = cqr, as required.

Next consider G = PSp2r (q). This is a little more complicated. Again let M = Pm .As described in the proof of Proposition 5.4, we have L = GLm (q) x Sp2r—2m (q) (modulo scalars), and Q has an L-invariant central series 1 < Q1 < Q, where as L-modules Q/Q1 = Vm (q) ® V2r—2m (q) and Q1 = S2( Vm (q)) (with trivial action of the factor Sp2r—2m (q) on Q1). Note that GLm (q) acts irreducibly on Q1 if q is odd, while if q is even, Q1 has a unique GLm(q)-submodule Q0 = A2(Vm(q)), and Q1/Q0 is irreducible of dimension m.

Let x e Irr( M) with Q < ker x .If Q1 < ker x then x(1) is at least the size of a nontrivial orbit of L on Q/ Q1, hence is at least cq2r—m—1 (see [17,

5.2.2]), giving the conclusion. So suppose now that Q1 < ker x. Let S be the factor Sp2r-2m(q) of L, so QS< M. By Clifford's theorem, we can write

x I QS = e^ Xi

where e is a positive integer, and the xi are distinct, L-conjugate irreducible characters of QS.

Suppose first that m > 1 and the factor SLm (q) of L fixes xi for all i. Then Q n ker xi is SLm (q) -invariant for all i. Itis also S-invariant, and hence Q n ker xi = 1, Qo, Q1 or Q. Since QS/ker xi must have cyclic centre, this forces Q1 < ker xi, a contradiction. Hence either m = 1, or the factor SLm(q) permutes the xi nontrivially. In particular, t > cqm-1.

One checks (for example by matrix calculations) that the normal closure of S in QS is the whole group QS. Hence S < ker xi, and so xi (1) > qr-m by Lemma 2.1(ii). It follows that

x(1) = tXl (1)> cqm-1 • qr-m = cqr-1,

as required.

This completes the proof for G = PSp2r (q), and the proof for the other classical groups is similar, using the structure of the maximal parabolic subgroups given in the proof of Proposition 5.4. □

We now conclude the proof of Theorem 5.1. Write ZM(1) = £ + £2, where

£ = E x(1)-1, £2 = x(1)-1 -

xelrr( M ),Q <ker(x) xs lrr(M ),Q <ker(x)

Note that £1 = ZL(1). Now |L/L'\ < c1q2, and L' has a characteristic subgroup Lo of bounded index which is a central product of at most three quasisimple groups. By Theorem 1.1, ZLo (1) is bounded. Using the easy inequality ZL(1) < |L/L0|ZLo(1), it follows that

£1 < cq2. (24)

Further, by Propositions 5.4 and 5.8, we have

c|Q| • q2r/3

s2 < ' ,; (25)

(and the denominator can be improved to qr for G = Ln (q)).

Since |G : M| — | Q |, the conclusion of Theorem 5.1 follows from (24) and (25), except when r < 3. For these low rank groups, the conclusion is obtained by improving the bound in Proposition 5.4, tightening the argument slightly. We do this for G = L2(q), U3(q) and PSp4(q) and leave the remaining groups of rank at most 3 (viz. L3 (q), L4(q), U4(q), PSp6(q)) to be dealt with in similar fashion by the reader.

Suppose G = L2(q). Here M = P1 = (Fq).((q — 1)/S), where S = (q — 1, 2). Here L acts fixed point freely on Q, hence k(M) < q. Also M has (q — 1)/S linear characters, and the rest have degree at least cq. Consequently

(q — 1)/S < ZM (1) <(q — 1)/S + q/cq.

Thus ZM(1) : M| ~ q, giving the exceptional case in Theorem 5.1.

Next let G = U3(q). Here M = QL with Q = q1+2 and L = (q2 — 1)/S where S = (3, q + 1). Again L is fixed point free on Q/Z(Q) = q2, so k(M) < cq3. Irreducible characters x of M with Q < ker x have degree at least q. Hence we get

ZM(1) < c(q2 + q3/q).

Since \G : M\= q3 + 1 this shows that ZM(1) < c\G : M\q—1, giving the result.

Finally consider G = PSp4(q) with M = P1 = QL; here we have 1 < Q1 < Q with \ Q1 \ = q and Q/Q1 = Fq, and L = GL2(q)/{—1), acting naturally on Q/Q1 and as the determinant on Q1. Now L has only cq classes having nontrivial centralizer in Q/Q1, and hence Lemma 5.3 gives k(M) < cq3. Now we complete the argument in the usual way. The proof for M = P2 has no novel features.

This completes the proof of Theorem 5.1.

6. Random homomorphisms

In this section we prove Theorem 1.6. Let r be a Fuchsian group in F, let v = v(r), and let G be a finite simple group.

Theorem 1.6 is established in [32] for G alternating, so we assume that G is a simple group of Lie type of rank r over Fq .

Lemma 6.1 The probability that a randomly chosen homomorphism in Hom(r, G) is an epimorphism is at least

1 — (1 + o(1)) ZM(vg — 2) • \G : M\—(vg—1).

M max G

Proof Let Q be the complementary probability. Then

^ |Hom(r, M)| ^ |Hom(T, G)|

M max G 1 v ' M

By Theorem 1.2,

\Hom(r, G)\ > (1 + o(1))\G\vg—11 • ["[ jm(G).

By Lemma 3.3 we have

|Hom(r, M)| < |M|vg—1 • ["[ jm(M) • ZM(vg - 2).

It follows that

|Hom(r, G)|

giving the result. □

We shall also use the following result, taken from [27, Sect. 3] and [28, 2.1].

Lemma 6.2 Let G be a finite simple group. Then

(i) £MmaxG G : M|-2 ^ 0 as |G|^to;

(ii) if G is classical and s > 1 then^2Mmax G |G : M|-s ^ 0 as |G| ^ to.

Proof of Theorem 1.6 Note that by our assumptions on r we have vg — 2 > 1, and so ZM (vg — 2) < ZM (1). In view of Lemma 6.1, it suffices to show that

T(G) = J2 ZM(1) ■ \G : M\-{vg-l) ^ 0 as \G|^to. (26)

M max G

We distinguish between four cases. Case 1: (v, g) = (1, 3).

It follows from Theorem 5.1 that ZM(1) < c|G : M|. This yields

T(G) < J2 c|G : M| • |G : M|—(vg—= c G : M|—(vg—2).

M max G M max G

Our assumptions on (v, g) imply vg — 2 > 2, and so Lemma 6.2(i) yields T(G) ^ 0 as |G| ^ to.

Case 2: v = 1, g = 3, G is classical, and G = L2(q).

Let C denote the set of maximal subgroups of G belonging to the Aschbacher classes C 1;... , c8, and let S denote the remaining maximal

subgroups (see the proof of Lemma 5.2). Note that vg — 1 = 2. Set

T^G) = J2 ZM(1) -\G : M\-2,

T2(G) = J2 ZM(1) ■\G : M\-2.

Then T(G) = 7i(G) + 72(G), so it suffices to show that 7(G) ^ 0 as

\G\ ^ to.

By Theorem 5.1 (noting that G = L2(q) by assumption), we have ZM(1) < cq—er\G : M\ with e > 0, and this yields

Ti(G) < cq—erY^ \G : M\ — 1 = cq—erk(c),

where k(c) denotes the number of conjugacy classes of subgroups in e. By [13, 2.1], we have k(c) < c1(r + log log q). It follows that

Tl(G) < C2q—er ■ (r + loglog q) ^ 0 as \G\ ^ to.

Now, for M e s we have ZM(1) < k(M) < c\G : M\1/2 by (18), hence

T2(G) < ^ \G : M\—3/2,

which tends to 0 as \G \ ^ to by part (ii) of Lemma 6.2.

Case 3: v = 1, g = 3 and G = L2(q).

Suppose first that d > 0 and (mi, \G \) = 1for some i. It is easily checked for G = L2(q) that jm(M)/jm(G) < cq—1 for all maximal subgroups M of G. Fix a maximal subgroup M. By Lemma 3.3,

\Hom(r, M)\ < \M\2 •["[ jm(M) ■ ZM(1). i=1

Applying Theorem 1.2, this yields

^^id+od^lG^I-.li^.fd).

\Hom(r, G)\ i=1 jmi (G)

Since ZM(1) < c\G : M\ by Theorem 5.1, and jm(M)/jm(G) < cq—1 for some i, it follows that

|Hom(r, M)\ ^ ,

-< c\q ■ G : Ml .

|Hom(r, G)| ~ 1

Therefore

^ |Hom(r, M)| ^ ,

M max G M max G

which tends to 0 as q ^ro. This establishes Theorem 1.6 in this case.

To complete the proof in Case 3, suppose now that (mi, |G|) = 1 for all i. Then every homomorphism from r to G factors through a non-oriented surface group of genus 3, so we may assume that r is such a group, i.e. that d = 0. As usual, it is enough to show that £Mmav G ¡Hom(r G)| 0 as q ^ro. This sum over non-parabolic maximal subgroups can be shown to tend to 0 exactly as in Case 2 above, so it remains to show that the sum over parabolic subgroups also tends to 0.

Let M be a parabolic subgroup of G. By Corollary 3.2(ii), we have

|Hom(r, M)| = |M|2 ■ ZM(1)-

Now M = (Fq).((q - 1)/S), where S = (q - 1, 2), so M has at most 2 real linear characters. The non-linear characters have degree at least cq, and k(M) < q. Hence Z^(1) = £xeIr(M) i(x)x(1)-1 is bounded independently of q. Consequently

^ |Hom(r, M)| „ 2

V --——- < c2 V |G : MI"2,

^ |Hom(T, G)| ^

M parabolic M parabolic

which tends to 0 as q ^ ro.

Case 4: v = 1, g = 3 and G is exceptional.

For this case we require the following recent result on maximal subgroups of G taken from [26].

Lemma 6.3 Let G = G (q) be an exceptional simple group of Lie type over Fq, and let M be a maximal subgroup of G. Then there are absolute constants c1, c2 such that one of the following holds:

(i) M is a known subgroup, belonging to one of at most c1 log log q conju-gacy classes,

(ii) G is of type F4, Ef6, E7 or E8, M is almost simple, and | M| < c2.

Proof For G of type 2B2, 2G2, 3D4, G2, 2F4, all the maximal subgroups of G are known. For the remaining types, the result follows from [26, Corollary 4]. Note that the log log q term comes from the subfield subgroups G(q1/r), where r is a prime divisor of logp q. □

Denote by m1, m2 the sets of maximal subgroups as in parts (i), (ii) of Lemma 6.3 respectively, and for i = 1, 2 define

T(G) = J2 ZM(1) ' |G : M|-

Then T(G) = 71 (G) + T2(G), so it suffices to show that T{(G) — 0 as

\G\ — to.

By Theorem 5.1, ZM(1) < cq—er\G : M\ with e > 0, and this yields 71 (G) < cq—erJ2 \G : M\ — 1 = cq—er dloglog q

using Lemma 6.3(i). Thus T1(G) — 0 as \ G \ —^ ^o.

Finally, since for M e m2 we have ZM(1) < \M\ < c2, we conclude that

T2(G) < c2 J2 \G : M\-2,

M max G

which tends to 0 as \G\ — to by Lemma 6.2(i). This completes the proof of Theorem 1.6.

7. Representation varieties

In this section we apply results on Hom(r, G) for r a Fuchsian group and G a finite group of Lie type, and use them to study representation varieties of r in algebraic groups over algebraically closed fields.

Our main tool is a basic result from algebraic geometry on the number of q-rational points, which follows from the well known Lang-Weil estimate [19]. Here algebraic varieties are assumed to be affine or projective, but not necessarily irreducible; the dimension is defined to be the maximal dimension of an irreducible component.

Lemma 7.1 Let p be a prime, and let V be an algebraic variety over K = Fp. Suppose dim V = f, and that V has e components of dimension f. For a power q of p, let V(q) denote the set of q-rational points in V. Then there is a power q0 of p such that

\V(q)\ = (e + o(1))qf

for all powers q of q0.

Proof Write V = uh=1 Vi where Vi are the irreducible components. Suppose dim Vi = f for i < e and dim Vi < f — 1for i > e. Choose a p-power q0 such that all the varieties Vi are defined over the field Fq0. Then for q = qk the Lang-Weil estimate [19] yields

\Vi(q)\ = qf + O(qf—1/2) (1 < i < e),

\Vi(q)\ = O(qf—1) (e < i < h).

The conclusion follows.

We shall deduce our results on representation varieties by combining Lemma7.1 with our results on \Hom(r, G )\ for finite groups G ofLietype.

Proof of Theorem 1.8 Adopt the notation of the theorem, and write V = Hom(r, GLn(K)) = Rn,K(r). We may assume that K = Fp. Choose a power q0 of p such that all components of V are defined over Fq0. Let q be a power of q0, and a a Frobenius q-power endomorphism of GLn(K) with fixed point group GLn (q). Then a acts on V, with fixed points V(q) = Hom(r, GLn(q)). It follows from Proposition 3.7(i) that for r oriented, we have

\V(q)\ = (q — 1 + 5 + o(1)) ■ \GLn(q)\2g—1 = (1 + o(X))q{2g—1)"2+1.

On the other hand, Lemma 7.1 and the choice of q0 show that \V(q)\ = (e + o(1))qf, where f = dim V and e is the number of f -dimensional components of V. We see that e = 1 and f = (2g — 1)n2 + 1, proving part (i) of the theorem.

The proof of part (ii) is similar, using Proposition 3.7(ii). □

Proof of Theorem 1.9 We start with the following proposition.

Proposition 7.2 Let r e F be a Fuchsian group, n > 2, and K an algebraically closed field.

(i) If r is oriented, then

dim Rn,K (r) = 1 + (2g — 1)n2 + dim Im(GLn (K)).

(ii) If r is non-oriented, then

dim RnK (V) = (g — 1)n2 + J2 dim Jm (GLn (K)).

Proof It is well known that the dimension of a variety in characteristic zero coincides with the dimension of its reduction modulo p for all large primes p. Hence in the proof we may assume that the algebraically closed field K has characteristic p > 0.

Let V, V(q), K, q0 be as in the proof above. Then by Theorem 3.8(i) we have

\V(q)\ = (1 + o(1))q\GLn (q)\2g—1\Im(GLn (q))\.

Replacing q0 by a suitable power of it if needed, we may assume that all components of the variety U = Im(GLn(K)) are defined over Fq0. Let f1 = dim U and e1 the number of f1-dimensional components of U. Then for powers q of q0 we have

\ Im(GLn (q))\ = \U(q)\ = (e1 + o(1))qf1,

and this implies

\V(q)\ = (ei + o(1))q1+(2g—1)n2+f1.

Applying 7.1 again we see that dim V = 1 + (2g — 1)n2 + f1, proving part (i) of the proposition.

To prove part (ii) we apply Corollary 3.11. We first replace q0 by a suitable power of it which satisfies q = 1 mod 2a+1, where a is as in Lemma 3.10. It then follows that, with the above notation we have

\V(q)\ - \GLn(q)\g—1 • jm(GLn(q)).

Modifying q0 again if needed so that all components of Jm (GLn(K)) are defined over Fq0, we see that

\V(q)\-q(g—1)n2+E fi,

where fi = dim Jm (GLn (K)). Part (ii) now follows. □

We now complete the proof of Theorem 1.9. Suppose first that r is oriented. By Proposition 7.2(i) and Corollary 4.6(ii), we have

dim R„k (V) = 1 + (2g — 1)n2 + dim Im (GLn (K))

= \ + {2g- \)n2 + n2 • £f=1 (l-i)

- Ef=i h (! - ¿) -e

= 1 + (^ + 1)n2 — c(n, m) — €,

where e e {0, 2} is as in Corollary 4.6. Theorem 1.9 follows in the oriented case.

Finally, the non-oriented case follows by combining Propositions 7.2(ii) and 4.5(ii).

Proof of Theorem 1.10 Let r be a Fuchsian group in F, and let G be a simple algebraic group over the algebraically closed field K. As above, it suffices to consider the case where K has characteristic p > 0. For each power q of p let a = aq be a field morphism of G with fixed point group G = G(q) = Ga, a nearly simple group with G' quasisimple (of untwisted type). Write V = Hom(r, G),so that a acts naturally on V with fixed point space V(q) = Hom(r, G).

Assume now that r is oriented. By Theorem 3.6(i), we have

\V(q)\ = \G\2g—1\Im(G)\ • (\G/G'\ + o(1)). (27)

Note that \G/G'\ is bounded and \G\ - qdimG. By Corollary 4.4(ii), there exists q1 = pb such that for q a power of q1, we have

\Im(G)\ -qEdim Jmi(G\

It follows that for q a power of q1, we have

\V(q) \ ~ q(2g—1) dim dim Jmi (G)_

Part (i) of Theorem 1.10 in the oriented case now follows using Lemma 7.1.

The non-oriented case of Theorem 1.10(i) is similar, using part (ii) of Theorem 3.6, except when g = 3 and G = SL2 or PSL2. In the first case, G = SL2(q) and the result follows as above using Theorem 1.2(iv). In the second case, G = PGL2 (q), and we may assume q is odd (otherwise we are back in the first case). Inspection of the character table of G in [46] shows that ZG(1) = 3 + o(1). Also \G/G2\ = 2. Substituting in Lemma 3.5(ii) now gives

|Hom(T, G)|

The conclusion now follows as before.

This completes the proof of Theorem 1.10(i). Part (ii) now follows from (i) using Corollary 4.2. □

Proof of Corollary 1.11 Part (i) of the corollary is immediate from Theorem 1.10(i). To prove the other parts, we can assume as above that K has positive characteristic. Suppose first that r is oriented. Adopting the above notation, (27) gives

\V(q)\ = \G\2g—1(\G/G'\ + o(1)).

There exists q0 such that for all powers q of q0 we have \G/G'\ = \tt1((j) \. The conclusion of Corollary 1.11(ii) follows using Lemma 7.1.

Now suppose r is non-oriented. By Corollary 3.2(ii) we have \Hom(r, G)\ = \G\g—1ZG(g — 2). If g > 3 or G = SL2, PSL2 then Lemma 2.9 gives ZG(g — 2) = \G/G2\ + o(1), and since \G/G2\ = \n1(Cr)/n1(G)2\ for all powers q of a suitable q0, the conclusion of Corollary 1.11(iii) follows. So assume now that g = 3 and G = SL2 or PSL2. In the first case it is easily checked from the character table of G = SL2(q) that ZG(1) = 1 + o(1); and in the second case G = PGL2(q) and we have ZG(1) = (q — 1, 2) + 1 + o(1). The conclusion again follows. □

References

1. Aschbacher, M.: On the maximal subgroups of the finite classical groups. Invent. Math. 76, 469-514 (1984)

2. Azad, H., Barry, M., Seitz, G.M.: On the structure of parabolic subgroups. Commun. Algebra 18, 551-562 (1990)

3. Benyash-Krivets, V.V., Chernousov, V.I.: Varieties of representations of fundamental groups of compact nonoriented surfaces. (Russian) Mat. Sb. 188, 47-92 (1997); translation in Sb. Math. 188, 997-1039 (1997)

4. Conder, M.D.E.: Hurwitz groups: a brief survey. Bull. Am. Math. Soc. 23, 359-370 (1990)

5. Deriziotis, D.I., Michler, G.O.: Character table and blocks of finite simple triality groups 3D4(q). Trans. Am. Math. Soc. 303, 39-70 (1987)

6. Dixon, J.D.: The probability of generating the symmetric group. Math. Z. 110, 199-205 (1969)

7. Dornhoff, L.: Group Representation Theory, Part A. Marcel Dekker 1971

8. Fulman, J., Guralnick, R.: Derangements in simple and primitive groups. In: Ivanov, A., Liebeck, M.W., Saxl, J. (eds.), Groups, Combinatorics and Geometry: Durham, 2001. World Scientific 2003

9. Fulman, J., Guralnick, R.: The number and sizes of conjugacy classes in finite Chevalley groups with applications to derangements. Preprint

10. Gorenstein, D., Lyons, R., Solomon, R.: The classification of the finite simple groups, Vol. 3. Math. Surv. Monogr., Vol. 40, No. 3. Am. Math. Soc. 1998

11. Gluck, D.: Sharper character value estimates for groups of Lie type. J. Algebra 174, 229-266 (1995)

12. Goldman, W.M.: Topological components of spaces of representations. Invent. Math. 93, 557-607 (1988)

13. Guralnick, R., Kantor, W.M., Saxl, J.: The probability of generating a classical group. Commun. Algebra 22, 1395-1402 (1994)

14. Guralnick, R., Lübeck, F., Shalev, A.: Zero-one generation laws for Chevalley groups. To appear

15. Guralnick, R., Saxl, J., Liebeck, M.W., Shalev, A.: Random generation of finite simple groups. J. Algebra 219, 345-355 (1999)

16. Kantor, W.M., Lubotzky, A.: The probability of generating a finite classical group. Geom. Dedicata 36, 67-87 (1990)

17. Kleidman, P.B., Liebeck, M.W.: The Subgroup Structure of the Finite Classical Groups. Lond. Math. Soc. Lect. Note Ser. 129. Cambridge: Cambridge University Press 1990

18. Landazuri, V., Seitz, G.M.: On the minimal degrees of projective representations of the finite Chevalley groups. J. Algebra 32, 418-443 (1974)

19. Lang, S., Weil, A.: Number of points of varieties over finite fields. Am. J. Math. 76, 819-827 (1954)

20. Lawther, R.: Elements of specified order in simple algebraic groups. Trans. Am. Math. Soc. To appear

21. Lawther, R., Liebeck, M.W., Seitz, G.M.: Fixed point ratios in actions of finite exceptional groups of Lie type. Pac. J. Math. 205, 393-464 (2002)

22. Liebeck, M.W.: On the orders of maximal subgroups of the finite classical groups. Proc. Lond. Math. Soc. 50, 426-446 (1985)

23. Liebeck, M.W., Pyber, L.: Upper bounds for the number of conjugacy classes of a finite group. J. Algebra 198, 538-562 (1997)

24. Liebeck, M.W., Saxl, J.: On the orders of maximal subgroups of the finite exceptional groups of Lie type. Proc. Lond. Math. Soc. 55, 299-330 (1987)

25. Liebeck, M.W., Seitz, G.M.: Reductive subgroups of exceptional algebraic groups. Mem. Am. Math. Soc., Vol. 121, No. 580. Providence, RI: Am. Math. Soc. 1996

26. Liebeck, M.W., Seitz, G.M.: The maximal subgroups of positive dimension in exceptional algebraic groups. Mem. Am. Math. Soc., Vol. 169, No. 802, pp. 1-227. Providence, RI: Am. Math. Soc. 2004

27. Liebeck, M.W., Shalev, A.: The probability of generating a finite simple group. Geom. Dedicata 56, 103-113 (1995)

28. Liebeck, M.W., Shalev, A.: Classical groups, probabilistic methods, and the (2,3)-generation problem. Ann. Math. 144, 77-125 (1996)

29. Liebeck, M.W., Shalev, A.: Simple groups, probabilistic methods, and a conjecture of Kantor and Lubotzky. J. Algebra 184, 31-57 (1996)

30. Liebeck, M.W., Shalev, A.: Simple groups, permutation groups, and probability. J. Am. Math. Soc. 12, 497-520 (1999)

31. Liebeck, M.W., Shalev, A.: Random (r, s)-generation of finite classical groups. Bull. Lond. Math. Soc. 34, 185-188 (2002)

32. Liebeck, M.W., Shalev, A.: Fuchsian groups, coverings of Riemann surfaces, subgroups growth, random quotients and random walks. J. Algebra 276, 552-601 (2004)

33. Liebeck, M.W., Shalev, A.: Character degrees of finite Chevalley groups. Proc. Lond. Math. Soc. To appear

34. Lübeck, F.: Small degree representations of finite Chevalley groups in defining characteristic. LMS J. Comput. Math. 4, 22-63 (2001)

35. Lubotzky, A., Magid, A.R.: Varieties of representations of finitely generated groups. Mem. Am. Math. Soc., Vol. 58, No. 336, pp. 1-117. Providence, RI: Am. Math. Soc. 1985

36. Lulov, N.: Random walks on symmetric groups generated by conjugacy classes. Ph.D. Thesis, Harvard University 1996

37. Lyndon, R.C.: The equation a2b2 = c2 in free groups. Mich. Math. J. 6,155-164 (1959)

38. Malle, G., Saxl, J., Weigel, T.: Generation of classical groups. Geom. Dedicata 49, 85-116 (1994)

39. Mednykh, A.D.: On the number of subgroups in the fundamental group of a closed surface. Commun. Algebra 16, 2137-2148 (1988)

40. Mulase, M., Penkava, M.: Volume of representation varieties. Preprint

41. Müller, T.W., Puchta, J.-C.: Character theory of symmetric groups and subgroup growth of surface groups. J. Lond. Math. Soc. 66, 623-640 (2002)

42. Rapinchuk, A.S., Benyash-Krivetz, V.V., Chernousov, V.I.: Representation varieties of the fundamental groups of compact orientable surfaces. Isr. J. Math. 93, 29-71 (1996)

43. Shamash, J.: Blocks and Brauer trees for groups of type Gi(q). Proc. Symp. Pure Math. 47, 283-295 (1987)

44. Shinoda, K.: The conjugacy classes of the finite Ree groups of type (F4). J. Fac. Sci., Univ. Tokyo 22, 1-15 (1975)

45. Springer, T.A., Steinberg, R.: Conjugacy classes. In: Borel, A., et al. (eds.) Seminar on algebraic groups and related topics. Lecture Notes Math., Vol. 131, pp. 168-266. Berlin: Springer 1970

46. Steinberg, R.: The representations of GL(3, q), GL(4, q), PGL(3, q) and PGL(4, q). Can. J. Math. 3, 225-235 (1951)

47. Suzuki, M.: On a class of doubly transitive groups. Ann. Math. 75, 105-145 (1962)

48. Tiep, P.H., Zalesskii, A.E.: Minimal characters of the finite classical groups. Commun. Algebra 24, 2093-2167 (1996)

49. Wagner, A.: An observation on the degrees of projective representations of the symmetric and alternating groups over an arbitrary field. Arch. Math. 29, 583-589 (1977)

50. Wall, G.E.: On the conjugacy classes in the unitary, symplectic and orthogonal groups. J. Aust. Math. Soc. 3, 1-62 (1965)

51. Ward, H.N.: On Ree's series of simple groups. Trans. Am. Math. Soc. 121, 62-89 (1966)

52. Wilf, H.S.: The asymptotics of eP(z) and the number of elements of each order in Sn. Bull. Am. Math. Soc. 15, 228-232 (1986)

53. Witten, E.: On quantum gauge theories in two dimensions. Commun. Math. Phys. 141, 153-209 (1991)