7/29/2019 10.1.1.89.7871

1/7

Determining the Automorphism Group of a HyperellipticCurve

Tanush Shaska

Department of MathematicsUniversity of California at Irvine

103 MSTB, Irvine, CA 92697

ABSTRACT

In this note we discuss techniques for determining the auto-morphism group of a genus g hyperelliptic curve Xg definedover an algebraically closed field k of characteristic zero.The first technique uses the classical GL2(k)-invariants ofbinary forms. This is a practical method for curves of smallgenus, but has limitations as the genus increases, due to the

fact that such invariants are not known for large genus.The second approach, which uses dihedral invariants of

hyperelliptic curves, is a very convenient method and workswell in all genera. First we define the normal decompositionof a hyperelliptic curve with extra automorphisms. Thendihedral invariants are defined in terms of the coefficients ofthis normal decomposition. We define such invariants inde-pendently of the automorphism group Aut(Xg). However,to compute such invariants the curve is required to be in itsnormal form. This requires solving a nonlinear system ofequations.

We find conditions in terms of classical invariants of bi-nary forms for a curve to have reduced automorphism groupA4, S4, A5. As far as we are aware, such results have not

appeared before in the literature.

Categories and Subject Descriptors

I.1 [SYMBOLIC AND ALGEBRAIC MANIPULA-TION]: ALGORITHMS

General Terms

Algorithms, Theory

Keywords

Hyperelliptic curve, automorphism, moduli space

Useful suggestions of the anonymous referee are gratefullyacknowledged.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.ISSAC03, August 36, 2003, Philadelphia, Pennsylvania, USA.Copyright 2003 ACM 1-58113-641-2/03/0008 ...$5.00.

1. INTRODUCTIONLet Xg be an algebraic curve of genus g defined over an

algebraically closed field k of characteristic zero. We denoteby Aut(Xg) the group of analytic (equivalently, algebraic)automorphisms of Xg. Then Aut(Xg) acts on the finite setof Weierstrass points of Xg. This action is faithful unless

Xg is hyperelliptic, in which case its kernel is the group of

order 2 containing the hyperelliptic involution of Xg. Thusin any case, Aut(Xg) is a finite group. This was first provedby Schwartz. In 1893 Hurwitz discovered what is now calledthe Riemann-Hurwitz formula. From this he derived that

|Aut(Xg)| 84 (g 1)which is known as the Hurwitz bound. However, it is notan easy task to compute the automorphism group of a givenalgebraic curve. Even compiling a list of possible candidatesfor a small genus g is quite difficult. In [10] we provide analgorithm which computes such lists. We give a completelist for g = 3 and list large groups for g 10. This workis based on previous work of Breuer, among many others;see [10] for a complete list of references.

If Xg is hyperelliptic then Aut(Xg) is a degree 2 centralextension ofZn, Dn, A4, S4, A5. We will explain this brieflyin section 2. However, computing Aut(Xg) for a given Xg isstill difficult. Even sophisticated computer algebra packagesdo not have such capabilities for g 3. The case g = 2 hasrecently been implemented in Magma [9] and is based onmethods used in [15].

In this short note we will focus on determining Aut(Xg)for a given genus g hyperelliptic curve Xg. We will not proveany of the results. The interested reader can check [8], [11],[12], [13], [14], or [15] for details. Most of the papers abovehave focused on studying the locus of all hyperelliptic curvesof genus g whose automorphism group contains a subgroupG and inclusions between such loci. In this paper we com-

bine the above results to get a treatment for all hyperellipticcurves in all genera. We generalize the notion of dihedralinvariants of hyperelliptic curves with extra involutions dis-covered in [8] for all hyperelliptic curves with extra automor-phisms (cf. Theorem 5.1.). Using these dihedral invariantsand classical invariants of binary forms of degree 2g + 2 wediscover some nice necessary conditions for a curve to havereduced automorphism group A4, S4, A5 (cf. section 5).

Notation: We will use the term curve to mean a com-pact Riemann surface. Throughout this paper Xg denotes ahyperelliptic curve of genus g 2. Dn denotes the dihedralgroup of order 2n.

7/29/2019 10.1.1.89.7871

2/7

2. PRELIMINARIESLet k be an algebraically closed field of characteristic zero

and Xg be a genus g hyperelliptic curve given by the equa-tion Y2 = F(X), where deg(F) = 2g + 2. Denote the func-tion field ofXg by K := k(X, Y). Then, k(X) is the uniquedegree 2 genus zero subfield of K. K is a quadratic exten-sion field of k(X) ramified exactly at d = 2g + 2 places1, . . . , d of k(X). The corresponding places of K are

called the Weierstrass points of K. Let P := {1, . . . , d}and G = Aut(K/k). Since k(X) is the only genus 0 sub-field of degree 2 of K, then G fixes k(X). Thus, G0 :=Gal(K/k(X)) = z0, with z20 = 1, is central in G. We callthe reduced automorphism group ofK the group G := G/G0.By a theorem of Dickson, G is isomorphic to one of the fol-lowing:

Zn, Dn, A4, S4, A5,

with branching indices of the corresponding cover : P1 P1/G given respectively by

(n, n), (2, 2, n), (2, 3, 3), (2, 4, 4), (2, 3, 5). (1)

In [1] all subgroups of G are classified and in [3] all groups

that occur as full automorphism groups of hyperelliptic curvesare classified. We use the notation of [3] and define Vn, Hn,Gn, Un, W2, W3 as follows:

Vn := x, y | x4, yn, (xy)2, (x1y)2,Hn := x, y | x4, y2x2, (xy)n ,Gn := x, y | x2yn, y2n, x1yxy ,Un := x, y | x2, y2n,xyxyn+1,W2 := x, y | x4, y3, yx2y1x2, (xy)4,W3 := x, y | x4, y3, x2(xy)4, (xy)8

(2)

The following is proven in [3].

Theorem 2.1. The automorphism group of a hyperellip-tic curve is one of the following Dn, Zn, Vn, Hn, Gn, Un,GL2(3), W2, W3.

The reader should be careful when reading Theorem 3.1.,in [3]. It seems as the cases H1 and G1 (which are isomorphicto Z4) must be excluded. For example, for g = 2, accordingto Theorem 3.1., H1 =Z4 must occur as an automorphismgroup, but it is well known that this is not the case; see [15]among many others. It is safe to exclude these cases sincethe group is cyclic and corresponds to case 3 of Table 1.

Also, for g = 3 let N = 3 in the case 3.d, of Table 2 in[3]. This case is not excluded from Theorem 3.1., (pg. 273).In this case the group is D3 (dihedral group of order 6) andthis group does not occur as an automorphism group of agenus 3 hyperelliptic curve; see [10].

2.1 MODULI SPACES OF COVERSLet 0 : Xg P1 be the cover which corresponds to the

degree 2 extension K/k(X). Then, := 0 has mon-odromy group G := Aut(Xg). From basic covering theory,the group G is embedded in the group Sn, where n = deg .There is an r-tuple := (1, . . . , r), where i Sn suchthat 1, . . . , r generate G and 1 r = 1. The signatureof is an r-tuple of conjugacy classes C := (C1, . . . , C r) inSn such that Ci is the conjugacy class of 1. We use thenotation np to denote the conjugacy class of permutations

which are a product ofp cycles of length n. Using the signa-ture of : P1 P1 given in (1) and the Riemann-Hurwitzformula, one finds out the signature of : Xg P1 for anygiven g and G. A natural question is if a given group Goccurs as an automorphism group of a curve Xg with morethen one signature C (cf. Theorem 2.2).

For a fixed G, C the family of covers : Xg P1 is aHurwitz space H(G, C). This is a quasiprojective variety,not a priori connected. To show irreducibility of H(G, C)one has to show that there is only one braid orbit in the setof Nielsen classes N i(G, C).

There is a map

: H(G, C) Hgwhere Hg is the moduli space of genus g hyperelliptic curves.We denote by (G, C) the dimension in Hg of (H(G, C)).Further i(G) denotes the number of involutions of G.

Theorem 2.2. For each g 2, the groups G that occuras automorphism groups and their signatures C are givenin Table 1. Moreover; H(G, C) is an irreducible algebraicvariety of dimension (G, C) as given in Table 1.

Finding algebraic descriptions of Hurwitz spaces is in gen-eral a difficult problem. In [14] it is shown that each of thespaces H(G, C) is a rational variety. Further, the inclusionsbetween such loci are studied.

Let t be the order of an automorphism of an algebraiccurve Xg (not necessary hyperelliptic). Hurwitz [5] showedthat t 10(g 1). In 1895, Wiman improved this boundto be t 2(2g + 1) and showed that it is the best possible.Thus, if a cyclic group H occurs as an automorphism groupthen |H| 2(2g + 1). Indeed, this bound can be achievedfor any genus via a hyperelliptic curve. For example, thecurve

Y2 = X(X2g+1 1)has automorphism group the cyclic group of order 4g + 2.This is the second case in Table 1, when n = 2g + 1. Thefamily of such curves is 0-dimensional in Hg.

Now we turn our attention to determining if a given curveXg belongs to any of the families of Table 1. In other words,find conditions in terms of the coefficients of Xg such thatXg belong to a family in Table 1. This would determine theAut(Xg).

3. INVARIANTS OF BINARY FORMSIn this section we define the action of GL2(k) on binary

forms and discuss the basic notions of their invariants. Letk[X, Z] be the polynomial ring in two variables and let Vd

denote the (d+1)-dimensional subspace ofk[X, Z] consistingof homogeneous polynomials.

f(X, Z) = a0Xd + a1X

d1Z+ ... + adZd (3)

of degree d. Elements in Vd are called binary forms of de-gree d. We let GL2(k) act as a group of automorphisms onk[X, Z] as follows:

M =

a bc d

GL2(k), then M

XZ

=

aX+ bZcX + dZ

.

(4)This action of GL2(k) leaves Vd invariant and acts irre-ducibly on Vd. Let A0, A1, ... , Ad be coordinate functions

7/29/2019 10.1.1.89.7871

3/7

G G (G, C) , n, g C = (C1, . . . C r) : P1 P1 i(G)

Z2 Zn 2g+2n 1 n < g + 1 (n2, n2, 2n, . . . , 2n) 3Z2n Zn

2g+1n

1 (n2, 2n, 2n, . . . , 2n) (n, n) 1Z2n

2gn

1 n < g (2n, 2n, 2n, . . . , 2n) 1Z2 Dn g+1n (n4, 22n, . . . , 22n) 2n+3

Vng+1n

12 (n4, 4n, 22n, . . . , 22n) n+3D2n Dn gn ((2n)2, 22n, . . . , 22n) (2n, 2n, n2) n+1Hn

g+1n

1 n < g + 1 (4n, 4n, n4, 22n . . . , 22n) 3Un

g

n 1

2 g = 2 (4n, (2n)2, 22n, . . . , 22n) n+1Gn

g

n 1 n < g (4n, 4n, (2n)2, 22n, . . . , 22n) 1

Z2 A4 g+16 (38, 38, 212, . . . , 212)Z2 A4 g16 (38, 64, 212, . . . , 212) 7Z2 A4 A4 g36 = 0 (64, 64, 212, . . . , 212) (26, 34, 34)SL2(3)

g26 = 0 (46, 38, 38, 212, . . . , 212)

SL2(3)g46

(46, 38, 64, 212, . . . , 212) 1

SL2(3)g66

= 0 (46, 64, 64, 212, . . . , 212)Z2 S4 g+112 (316, 412, 224, . . . , 224)Z2 S4

g312 (68, 412, 224, . . . , 224) 19

GL2(3)g212 (3

16, 86, 224, . . . , 224)

GL2(3) S4g612 (6

8, 86, 224, . . . , 224) (212, 38, 46) 13

W2g512 (4

12, 412, 316, 224, . . . , 224) 7

W2g912 (4

12, 412, 68, 224, . . . , 224)

W3g812

(412, 316, 86, 224, . . . , 224) 1

W3g1212

(412, 68, 86, 224, . . . , 224)

Z2 A5 g+130 (340, 524, 260, . . . , 260)Z2 A5 g530 (340, 1012, 260, . . . , 260) 31Z2 A5 g1530 (620, 1012, 260, . . . , 260)Z2 A5 A5 g930 (620, 524, 260, . . . , 260) (230, 320, 512)

SL2(5)

g14

30 (4

30

, 3

40

, 5

24

, 2

60

, . . . , 2

60

)SL2(5)

g2030 (4

30, 340, 1012, 260, . . . , 260) 1

SL2(5)g2430

(430, 620, 524, 260, . . . , 260)

SL2(5)g3030

(430, 620, 1012, 260, . . . , 260)

Table 1: Automorphism groups of hyperelliptic curves

on Vd. Then the coordinate ring of Vd can be identified withk[A0,...,Ad]. For I k[A0,...,Ad] and M GL2(k), defineIM k[A0,...,Ad] as follows

IM(f) := I(M(f)) (5)

for all f Vd. Then IMN = (IM)N and Eq. (5) defines anaction of GL2(k) on k[A0,...,Ad]. A homogeneous polyno-mial I k[A0, . . . , Ad, X , Z ] is called a covariantof index sif

IM(f) = sI(f)

where = det(M). The homogeneous degree in A1, . . . , Anis called the degreeofI, and the homogeneous degree in X, Zis called the order of I. A covariant of order zero is calledinvariant. An invariant is a SL2(k)-invariant on Vd.

We will use the symbolic method of classical theory to

construct covariants of binary forms. Let

f(X, Z) :=

ni=0

ni

aiX

ni Zi,

g(X, Z) :=

mi=0

mi

biXni Zi

(6)

be binary forms of degree n and m respectively with coeffi-cients in k. We define the r-transvection

(f, g)r := ck r

k=0

(1)k

rk

rf

Xrk Yk

rg

Xk Yrk(7)

where ck =(mr)! (nr)!

n!m!. It is a homogeneous polynomial

in k[X, Z] and therefore a covariant of order m + n 2r anddegree 2. In general, the r-transvection of two covariants oforder m, n (resp., degree p, q) is a covariant of order m + n

7/29/2019 10.1.1.89.7871

4/7

2r (resp., degree p + q).For the rest of this paper F(X, Z) denotes a binary form

of order d := 2g + 2 as below

F(X, Z) =di=0

aiXiZdi =

di=0

ni

biX

iZni (8)

where bi =(ni)! i!

n!ai, for i = 0, . . . , d. We denote invariants

(resp., covariants) of binary forms by Is (resp., Js) where thesubscript s denotes the degree (resp., the order). We definethe following covariants and invariants:

I2 := (F, F)d,

J4j := (F, F)d2j, j = 1, . . . , g ,

I4 := (J4, J4)4,

I4 := (J8, J8)8,

I6 := ((F, J4)4, (F, J4)

4)d4,

I6 := ((F, J8)8, (F, J8)

8)d8,

I6 := ((F, J12)12, (F, J12)

12)d12,

I3 := (F, Jd)d,

M := ((F, J4)4, (F, J8)

8)d10,

I12 := (M, M)8

(9)

GL2(k)-invariants are called absolute invariants. We de-fine the following absolute invariants:

i1 :=I4I22

, i2 :=I23I32

, i3 :=I6I32

, j1 :=I

6

I23,

j2 :=I6I23

, s1 :=I26I12

, s2 :=(I

6)2

I12, v1 :=

I6I6

,

v2 :=(I

4)3

I43, v3 :=

I6

I

6

, v4 :=(I6 )

2

I34, v5 :=

I6I

6

(10)

For a given curve Xg we denote by I(Xg) or i(Xg) the cor-responding invariants. Two isomorphic hyperelliptic curveshave the same absolute invariants.

Remark 3.1. It is an open problem to determine the fieldof invariants of binary form of degree d 7.

4. EQUATIONS OF CURVESIn this section we state the equations of curves in each case

of Table 1. For a more detailed treatment of these spaces,including proofs, the reader can check results in [13], [14].The reader can also check [4] where equations for each familyare computed; however the main goal of the bo ok is to studyhyperelliptic Riemann surfaces with real structures. In this

section G denotes a group as in the first column of Table1, and LGg the locus of hyperelliptic genus g curves Xg suchthat G is embedded in Aut(Xg).

4.1 Aut(Xg) is isomorphic to ZnIf Aut(Xg) =Zn then Xg belongs to cases 1, 2, 3 in Table

1. These loci were studied in detail in [13]. The family ofcurves are given below:

Y2 =Xnt + + aiXn(ti) + . . . at1Xn + 1,Y2 =Xnt + + aiXn(ti) + . . . at1Xn + 1,Y2 =X(Xnt + + aiXn(ti) + . . . at1Xn + 1)

(11)

where t is respectively 2g+2n

, 2g+1n

, 2gn

. To classify these curves(up to isomorphism) we need to find invariants of the GL2(k)-action on k(a1, . . . , at1). The following

ui := ati1 ai + a

ti ati, f or 1 i (12)

are called dihedral invariants for the genus g and the tuple

u := (u1, . . . , u)

is called the tuple of dihedral invariants. It can be checkedthat u = 0 if and only if a1 = a = 0. In this case re-placing a1, a by a2, a1 in the formula above would givenew invariants. The next theorem shows that the dihedralinvariants generate k(LGg ).

Theorem 4.1. LGg is a -dimensional rational variety.Moreover, k(LGg ) = k(u1, . . . , u).If n = 2 then G is the Klein 4-group. Then LGg = Lg whereLg is the locus of hyperelliptic curves with extra involutions,see [8]. A nice necessary and sufficient condition is found in[12] in terms of the dihedral invariants for a curve to havemore than three involutions. More precisely, for such curves

the relation holds:2g1u21 ug+1g = 0.

4.2 Aut(Xg) is isomorphic to DnThe dihedral group is generated by

Dn = , | n = 2 = 1such that

(X) = n X, (X) =1

X.

Then fixes X = 0, and fixes X = 1 and permutes 0and . We let

G(X) :=

ti=1

(X2n + iXn + 1)

Then,

G(X) =X2nt + a1X2ntn + + atXnt+

at1X(n1)t + + a1Xn + 1

(13)

where ai, i = 1, . . . t are polynomials in terms of the symmet-ric polynomials s1, . . . , st ofi (i.e., a1 = s1, a2 = t+s2, a3 =

(t 1)s1 + s3, a4 :=

t

n/2

+ (t 2)s2 + s4, etc.).

Depending on whether 0, 1, and are Weierstrass pointswe get the equations Y2 = F(X) where

F(X) = G(X), (Xn

1) G(X),X G(X), (X2n 1) G(X)X(Xn 1) G(X), X(X2n 1) G(X)

(14)

where n is respectively as in cases 4-9 of Table 1.

Remark 4.2. Notice that in all cases n is even; see The-orem 2.1., in [3].

The case Y2 = G(X) corresponds to the group Z2 Dn. Ifn = 2, then this is a special case ofG =Z2 Zn. Indeed,

2g1u21 ug+1g = 0

7/29/2019 10.1.1.89.7871

5/7

as expected; see [12] for details. If n > 2 then

u = (u1, . . . , ug) = (0, . . . , 0)

where u = (u1, . . . , ug) is as defined in [8].

4.3 Aut(Xg) is isomorphic to A4This case is treated in detail in [13]. Let

Gi(X) = X12iX

1033X8+ 2iX

633X4iX

2 + 1, (15)

for 21 + 108 = 0. Denote by

G(X) :=

i=1

Gi(X)

Then, each family is parameterized as in Table 2. The fol-

G Equation Y2 =

Z2 A4 g+16 G(X)Z2 A4 g16 (X4 + 2i

3X2 + 1) G(X)

Z2 A4 g36 (X8 + 14X4 + 1) G(X)SL2(3)

g26

X(X4 1) G(X)SL2(3)

g46 X(X

4

1)(X4

+ 2i3X2

+ 1) G(X)SL2(3)

g66 X(X

4 1)(X8 + 14X4 + 1) G(X)

Table 2: Hyperelliptic curves with Aut(Xg) = A4

lowing lemma gives a necessary condition that a curve hasautomorphism group Z2 A4 or SL2(3).

Lemma 4.3. Let Xg be a hyperelliptic curve of genus gwith Aut(Xg) = A4. Then, I4 = 0. Moreover;

i) if g = 4 then I2 = I4 = I

4 = I

6 = 0ii) if g = 5, 9, 12 then I4 = I6 = 0

iii) if g = 7, 10 then I2 = I4 = I

4 = I6 = 0

iv) if g = 8 then I4 = 0.

4.4 Aut(Xg) is isomorphic to S4In this case the reduced automorphism group is generated

by

(X) = x 1x + 1

, (X) = iX.

We also denote

Gi(X) :=X24 + X20 + (759 4)X16 + 2(3+

1288)X12 + (759 4)X8 + X4 + 1,

R(X) :=X12

33X8

33X4 + 1,

S(X) :=X8 + 14X4 + 1,

T(X) :=X4 1.

(16)

Let

G(X) :=

i=1

Gi(X)

where is as in Table 1. Then, the equations of the curves ineach case are Y2 = F(X) where F is as below (we suppressX):

F = G, SG, TG, STG, RG, RSG, RTG, RSTG.

Similar conditions in terms of the classical invariants as inthe previous case can be obtained in this case also.

Lemma 4.4. Let Xg be a hyperelliptic curve of genus gwith Aut(Xg) = S4. Then, I4 = 0.

4.5 Aut(Xg) is isomorphic to A5We briefly state the equations here. We denote by Gi(X),

R(X), S(X), T(X) the following:

Gi(X) :=(i 1)X60

36(19i + 29)X55 + 6(26239i 42079)X

50

540(23199i 19343)X45

+ 105(737719i 953143)X40

72(1815127i 145087)X35

4(8302981i + 49913771)X30

+ 72(1815127i 145087)X25

+ 105(737719i 953143)X20

+ 540(23199i 19343)X15

+ 6(26239i 42079)X10

+ 36(19i + 29)X5

+ (i 1)

R(X) :=X30

+ 522X25

10005X20

10005X15

522X5

+ 1

S(X) :=X20

228X15

+ 494X10

+ 228X5

+ 1

T(X) :=X10 + 10X 1.

(17)

As above we let

G(X) :=

i=1

Gi(X).

In the order of Table 1 equations are given as Y2 = F(X)where F(X) is as given as (we suppress X):

F = G, SG, TG, STG, RG, RSG, RTG, RSTG

These curves can be expressed as Y2 = M(X2) or Y2 =X M(X2) where M is a polynomial in X2. This fact willbe used in the next section. The expressions are ratherlarge and we will not state them here. However, we get thefollowing useful fact:

Lemma 4.5. Let Xg be a hyperelliptic curve of genus gwith Aut(Xg) = A5. Then, I4 = I4 = I6 = I6 = I12 = 0.

5. DETERMINING THE AUTOMORPHISM

GROUP OF A GIVEN CURVELet Xg be given. We want to determine Aut(Xg). In order

to find an algorithm which would work for any g we wouldhave to check whether Xg can be written in any of the formsabove. Thus, we want to find if there is a coordinate change

X aX+ bcX + d

which transformsXg to one of the forms of section 4. This

would require solving a system of equations for each caseand therefore would not be efficient.

5.1 Using classical invariantsFor a fixed g we know the dimension of the locus LGg . We

compute enough absolute invariants to generate this locus.Thus, we determine the loci LGg for all G in Table 1 in termsof some invariants i1, . . . i+1. These loci are computed onlyonce for each g. Then, for a particular curve we simplycompute these invariants and check if they generate any ofthe loci LGg . These spaces were computed in detail in [14] forG = A4. We will illustrate with g 12 and G = A4, S4, A5.

7/29/2019 10.1.1.89.7871

6/7

We define p(Xg) as follows:

p(Xg) := (p1, p2) =

v1, if g = 4,

(i1, i2), if g = 5, 9, and I2 = 0v2, if g = 5, 9, and I2 = 0

(j1, j2), if g = 7, and I3 = 0v3, if g = 7, and I3 = 0

(i1, i3), if g = 8, 12, and I2 = 0v4, if g = 8, 12, and I2 = 0

(s2, s1), if g = 10, and I12 = 0v5, if g = 10, and I12 = 0

From Lemma 4.3. and results for cases G = S4, A5 one cancheck that p(Xg) is well defined. Moreover, the subvarietyLGg is 1-dimensional if G is isomorphic to A4, S4, A5. Foreach parametric curve Xg of the previous section we computep(Xg) in terms of the parameter . Eliminating gives anequation for LGg , see [14] for explicit equations.

The following algorithm determines if the automorphismgroup of a hyperelliptic genus g 12 curve is isomorphic toZ2 A4,Z2 S4,Z2 A5, SL2(3), SL2(5), GL2(3), W2, W3.Algorithm 1:

Input: A hyperelliptic curve Xg : Y2 = F(X, Z).Output: Determine if the automorphism group Aut(Xg)

is one ofZ2A4, Z2S4, Z2A5, SL2(3), SL2(5), GL2(3),W2, W3.

Step1: Compute I4(Xg). If I4 = 0 then Aut(Xg) is notisomorphic to any ofZ2A4,Z2S4, Z2A5, SL2(3), SL2(5),GL2(3), W2, W3. Otherwise go to Step 2.

Step 2: Compute p(Xg).Step 3: Find LGg which is satisfied by p(Xg) (equations

are given in [13]). Then, Aut(Xg) is isomorphic to G.The definition of p(Xg) is a little more elaborate for G =

Zn, Dn since the dimension of

LGg is > 1. Once the definition

of the moduli point is modified and the corresponding LGgare computed the following can be used:

Algorithm 2:

Input: A hyperelliptic curve Xg : Y2 = F(X, Z).Output: The automorphism group Aut(Xg).Step 1: Compute p(Xg).Step 2: Find LGg which is satisfied by p(Xg). Then,

Aut(Xg) is isomorphic to G.The above method of classical invariants is difficult to im-

plement for large g. Thats because finding enough absoluteinvariants is not an easy task for large g. Also the expres-sions of these invariants and the equations for the loci LGgget very large as g grows. In order to deal with these prob-lems we use the dihedral invariants which will be explainednext.

5.2 Using dihedral invariantsIn section 4.1., we introduced dihedral invariants for hy-

perelliptic curves Xg such that Aut(Xg) =Zn. In this sectionwe generalize this approach to all hyperelliptic curves withextra automorphisms. Theorem 5.1., makes this generaliza-tion possible.

Let Xg be an hyperelliptic curve with extra automor-phisms. The following lemma gives a general descriptionof how to write an equation for Xg.

Theorem 5.1. LetXg be a hyperelliptic curve with|Aut(Xg)| > 2.

Then, Xg can be written asY2 = F(Xn), or Y2 = X F(Xn), (18)

where n = 2 or n is odd and divides 2g + 2, 2g + 1, g. More-over, if n > 2 then Aut(

Xg) is a cyclic group.

Let Xg be a hyperelliptic curve with |Aut(Xg)| > 2 andwritten as in (18). We call this form a decompositionof Xg. Let s be the smallest n that such decomposition ispossible. Then,

Y2 = F(Xs), or Y 2 = X F(Xs) (19)is called the normal decomposition or the normal formof Xg and s is called the degree of the decomposition. Ifno such decomposition is possible then we say that s = 1.Let Xg be in its normal decomposition given below:

Y2 =Xnt + + aiXn(ti) + . . . at1Xn + 1,

Y2

=X(Xnt

+ + aiXn(ti)

+ . . . at1Xn

+ 1)

(20)

where nt = 2g + 2, 2g + 1, 2g.We define the following

ui := ati1 ai + a

ti ati, f or 1 i = t 1, (21)

which are called dihedral invariantsfor genus g and the tuple

U1 := (u1, . . . , u)

is called the tuple of dihedral invariants. It can be checkedthat u = 0 if and only if a1 = a = 0. Then, let (aj , aj+1)be the first nonzero tuple. Replacing a1, a by aj , aj+1in the formula above would give new invariants. Thus, wedefine

uji := ai+1j ai + a

i+1j ai+1, (22)

for 1 i , and 1 j [ +12

]. Then

Uj := (uj1, . . . , u

jm) (23)

where m = 2j.Algorithm 3:

Input: A hyperelliptic curve Xg : Y2 = F(X, Z).Output: The automorphism group Aut(Xg).Step 1: Check whether the curve has a normal decompo-

sition. If Yes then go to Step 2 otherwise Aut(Xg) = Z2Step 2: Compute the degree s of the normal decomposi-tion. If s is odd then Aut(Xg) =Z2s, otherwise go to Step

3.Step 3: Compute the dihedral invariants Uji of the normal

decomposition. Go to Step 4.Step 4: Find LGg which is satisfied by Uji . Then, Aut(Xg)

is isomorphic to G.

The above method was used in [15] and [8] to determinethe automorphism group of genus 2 and 3. It has the advan-tages that it can be used for any g no matter how large. Adisadvantage is that a nonlinear system of equations mustbe solved in order to determine the normal decomposition.

7/29/2019 10.1.1.89.7871

7/7

Example 5.2. For genus 2, the curve can be written as

Y2 = X6 + a1X4 + a2X

2 + 1

and its the dihedral invariants are

u1 = a31 + a

32, u2 = 2a1a2,

Then,a) G

= V6 if and only if (u1, u2) = (0, 0) or

(u1, u2) = (6750, 450).

b) G = GL2(3) if and only if (u1, u2) = (250, 50).c) G = D6 if and only if

u22 220u2 16u1 + 4500 = 0,for u2 = 18, 140 + 60

5, 50.

d) G = D4 if and only if2u21 u32 = 0,

for u2 = 2, 18, 0, 50, 450. Cases u2 = 0, 450, 50 are reducedto cases a),and b) respectively, see [15] for details.

Remark 5.3. The notation used in [15] to denote thegroups is different. V6 is this case has order 24 and in [15]is identified asZ3D4.

6. CLOSING REMARKSWe briefly described techniques of determining the auto-

morphism group of a hyperelliptic curve. A combination ofboth methods sometime produces better results. Our goalis to combine these methods and explicitly compute loci LGgfor reasonable g (i.e., g 60).

There are polynomial time algorithms to compute the de-composition of a polynomial F(X) up to an affine transfor-mation X aX + b, see [7]. However, this is not sufficientfor our purposes since we want to find such decompositionup to a liner fractional transformation X aX+b

cX+d. If a

polynomial time algorithm would be found in this case thiswould make the second method preferable to the first.

Besides computing the automorphism groups the abovetechniques can also be used to answer other questions onhyperelliptic curves. For example dihedral invariants canbe used to determine the field of moduli of a given curve.The reader can check [12] for details and open questionson the field of moduli and other computational aspects ofhyperelliptic curves.

7. REFERENCES

[1] R. Brandt and H. Stichtenoth, Die

Automorphismengruppen hyperelliptischer Kurven,Man. Math 55 (1986), 8392.

[2] Th. Breuer, Characters and automorphism groups ofcompact Riemann surfaces, London Math. Soc. Lect.Notes 280, Cambridge Univ. Press 2000.

[3] E. Bujalance, J. Gamboa, and G. Gromadzki, Thefull automorphism groups of hyperelliptic Riemannsurfaces, Manuscripta Math. 79 (1993), no. 3-4,267282.

[4] E. Bujalance, F. J. Cirre, J. M. Gamboa, G.Gromadzki, Symmetry types of hyperelliptic Riemannsurfaces. Mm. Soc. Math. Fr. (N.S.), No. 86 (2001).

[5] A. Hurwitz, Uber algebraische Gebilde miteindeutigen Transformationen in sich, Math. Ann. 41(1893), 403442.

[6] The GAP Group, GAP Groups, Algorithms, andProgramming, Version 4.2; 2000.(http://www.gap-system.org)

[7] J. Gutierrez, A polynomial decomposition algorithmover factorial domains, Comptes Rendues

Mathematiques, de Ac. de Sciences, 13 (1991), 81-86.[8] J. Gutierrez and T. Shaska, Hyperelliptic curves

with extra involutions, 2002 (submitted).

[9] The Magma Computational Algebra System,http://magma.maths.usyd.edu.au/magma/

[10] K. Magaard, T. Shaska, S. Shpectorov, and H.Volklein, The locus of curves with prescribedautomorphism group, RIMS Kyoto Series,Communications in Arithmetic Fundamental Groups,ed. H. Nakamura, 2002, 112-141.

[11] T. Shaska, Genus 2 curves with (3,3)-split Jacobianand large automorphism group, ANTS V (Sydney,2002), 205-218, Lect. Not. in Comp. Sci., 2369,Springer, Berlin, 2002.

[12] T. Shaska, Computational Aspects of HyperellipticCurves, Computer mathematics (Beijing, 2003),Lecture Notes Ser. Comput., 10, pg. 248-258, WorldSci. Publishing, River Edge, NJ, 2003.

[13] T. Shaska, Some special families of hyperellipticcurves, J. Algebra Appl., (submitted).

[14] T. Shaska, Subvarieties of the moduli space ofhyperelliptic curves (preprint).

[15] T. Shaska and H. Volklein, Elliptic subfields andautomorphisms of genus two fields, Algebra andAlgebraic Geometry with Applications, (WestLafayette, 2000), Lect. Not. in Comp. Sci., Springer,Berlin, 2002 (to appear).